This post means to describe the several problems i found when trying to run SQL commands on Sybase from Hudson.
Problem 1: Host key verification failed”
Problem: When trying to SSH to the server which holds the Sybase database I got "Host key verification failed" even thought they keys were correct.
Solution: The missing piece of the puzzle was a row in the ~/.ssh/known_hosts file.
By using another user, which uses the same private/public key, I could run a ssh to the sybase server and get a row in that users known_hosts file. Then I just copied that row to
/var/lib/hudson/.ssh/known_hosts and voila everything worked.
Learn more:
http://amath.colorado.edu/computing/unix/sshknownhosts.html
Problem 2: Shell profile not loaded when running one-line ssh
Problem: The script we need to run on the server needed a lot of magic that was set up by the shell profile scripts. When ssh:ing to the server the profiles scripts are run and everything is dandy. But from Jenkins I need to use one-line ssh commands like this.
ssh user@server "isql < lots_of_stuff.sql"
But when running one-line ssh the profile is NOT loaded and and I get a lot of errors since the magic is not set up.
Solution: Load the profile in the ssh one-liner before running the real command.
ssh user@server ". ./.profile; isql < lots_of_stuff.sql"
Problem 3: Incorrect Locale
After solving the two problems above I was sure everything would work fine, but no, of course it didn't.
Instead of the expected SQL response I got:
The context allocation routine failed when it tried to load localization files!!
One or more following problems may caused the failure
Your sybase home directory is /opt/sybase. Check the environment variable SYBASE if it is not the one you want!
Using locale name "POSIX" defined in environment variable LC_ALL
An error occurred when attempting to allocate localization-related structures.
Locale name "POSIX" doesn't exist in your /opt/sybase/locales/locales.dat file
POSIX? It could make a grown man cry :~(
Solution: Adding -v, for verbose, to the ssh command I could quickly see the difference between what Hudson was doing and what happened when running it from my own machine.
Hudson:
...
debug1: Sending environment.
debug1: Sending env LC_ALL = POSIX
debug1: Sending command: . ./.profile; isql < lots_of_stuff.sql
Locally:
...
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending command: . ./.profile; isql < lots_of_stuff.sql
No locale is sent from my local machine so I tried overriding it before sending the command in Hudson and it worked.
export LC_ALL=;
ssh sybase@inttestsbn.its.sec.intra '. ./.profile;isql < lots_of_stuff.sql'
And it works!
Random musings on tech, gaming, Australia, travel, testing and software development in general.
Friday, April 13, 2012
Thursday, January 12, 2012
Ultimate Password Solution using Dropbox and KeePassX
Today I found the ultimate password management solution using Dropbox and KeePassX.
I wanted a secure enough solution that can be used and managed on multiple computers. My old solution was a .txt file in Dropbox with a name that hopefully nobody would look in. But today I migrated the solution to KeePassX using a encrypted database saved as a file in Dropbox.
Another problem I had was that I couldn't download my entire Dropbox account to work since it takes to much space. But nowadays Dropbox for Linux can handle Selective Sync, i.e. only check out selected folders and not everything.
Here is how you can get the same set up step by step.
Written assuming you have a Dropbox account and the folder where you want to keep the password file is already created.
Step 1: Install Dropbox
https://www.dropbox.com/install
Step 2: Check out the folder where you want to save the password database
https://www.dropbox.com/help/175
Step 3: Install KeePassX
Use synaptic package manager or link below.
http://keepass.info/download.html
Step 4: Create new database in KeePassX
Generate new file and save it in your Dropbox folder
Add secure password ( http://www.generate-password.com/ )
Step 5: Add your entries to KeePassX
Step 6: Repeat step 1 - 3 on all your computers.
Instead of creating a new database connect to the file you created in your Dropbox folder.
That's It and now you are in password management heaven!
Enjoy!
I wanted a secure enough solution that can be used and managed on multiple computers. My old solution was a .txt file in Dropbox with a name that hopefully nobody would look in. But today I migrated the solution to KeePassX using a encrypted database saved as a file in Dropbox.
Another problem I had was that I couldn't download my entire Dropbox account to work since it takes to much space. But nowadays Dropbox for Linux can handle Selective Sync, i.e. only check out selected folders and not everything.
Here is how you can get the same set up step by step.
Written assuming you have a Dropbox account and the folder where you want to keep the password file is already created.
Step 1: Install Dropbox
https://www.dropbox.com/install
Step 2: Check out the folder where you want to save the password database
https://www.dropbox.com/help/175
Step 3: Install KeePassX
Use synaptic package manager or link below.
http://keepass.info/download.html
Step 4: Create new database in KeePassX
Generate new file and save it in your Dropbox folder
Add secure password ( http://www.generate-password.com/ )
Step 5: Add your entries to KeePassX
Step 6: Repeat step 1 - 3 on all your computers.
Instead of creating a new database connect to the file you created in your Dropbox folder.
That's It and now you are in password management heaven!
Enjoy!
Wednesday, December 28, 2011
Classes and Test Jars and Poms Oh My. Building a TestCoverage Test takes them all.
After spending a few days on getting a complicated coverage test working I really feel I should document it.
My setup is that I have 3 maven projects. One parent project and two subprojects, one for functional testing and one for coverage tests. The crux I had with this assignment was that the coverage project needed to be able to read classes and their annotations from it's sister project "xml-functional" to determine what it actually tested.
Overview of the maven project set up
Description
xmltest : This is the parent project, it does nothing in it self.
xmltest-functional : The functional tests
xmltest-coverage : The coverage test what I was building
Creating a Test Jar
First step was to get this working was to set up my xmltest-functional project so it would create a test-jar with all the test classes. For this I used maven-jar-plugin to add a new execution.
In /xmltest/xmltest-functional/pom.xml
Did it work?
Now, after running
So?
And now I can use that jar in our coverage project. First we need to add a dependency to the functional project in our pom. We want to use the "tests.jar" so we set classifier and scope to tests and test.
In /xmltest/xmltest-coverage/pom.xml
Adding the normal dependency
But that is not all, we also need to add the normal dependency so that when in create class objects of what we find we don't get errors.
In /xmltest/xmltest-coverage/pom.xml
Writing the Test
Now, what we need to do is to write the coverage test.
A typical test class
Coverage Test
Now, finally, I can write a test which scans in the classpath for classes with are annotated with @Test, loops through all that have the correct package, checks if they have the @Path annotation and get the annotation.
And so on and so forth
And you'll just have to imaging that we do the same for the controller classes, puts it all into two lists and compare the lists. But I'm sure you can figure out how to do this :)
Why don't you just filter on Path.class instead of Test.class
I am sure someone would ask this if I actually had any readers.
Good question and the reason we do this is because we want to find test classes which has missed being annotated with @Test. It happens and we want to catch those cases as well.
That's It
I hope you got something out of this, I sure learnt a lot while writing it.
Hope to see you again in the next blog post.
And remeber:
mvn versions:display-plugin-updates
Gives you a list of plugs you can update.
mvn versions:display-dependency-updates
Gives you a list of dependencies you can update.
My setup is that I have 3 maven projects. One parent project and two subprojects, one for functional testing and one for coverage tests. The crux I had with this assignment was that the coverage project needed to be able to read classes and their annotations from it's sister project "xml-functional" to determine what it actually tested.
Overview of the maven project set up
/xmltest
/xmltest-functional
/xmltest-coverage
Description
xmltest : This is the parent project, it does nothing in it self.
xmltest-functional : The functional tests
xmltest-coverage : The coverage test what I was building
Creating a Test Jar
First step was to get this working was to set up my xmltest-functional project so it would create a test-jar with all the test classes. For this I used maven-jar-plugin to add a new execution.
In /xmltest/xmltest-functional/pom.xml
<project>
...
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
...
Did it work?
Now, after running
mvn clean install, there is a tests.jar created which contain all the test classes in our functional project.~/.m2/repository/com/companyx/xmltest/xmltest-functional/2.0.0-SNAPSHOT$ ls -l
xmltest-functional-2.0.0-SNAPSHOT.jar
xmltest-functional-2.0.0-SNAPSHOT.pom
xmltest-functional-2.0.0-SNAPSHOT-tests.jar
So?
And now I can use that jar in our coverage project. First we need to add a dependency to the functional project in our pom. We want to use the "tests.jar" so we set classifier and scope to tests and test.
In /xmltest/xmltest-coverage/pom.xml
...
<dependency>
<groupId>com.companyx.xmltest</groupId>
<artifactId>xmltest-functional</artifactId>
<version>2.0.0-SNAPSHOT</version>
<classifier>tests</classifier>
<scope>test</scope>
</dependency>
Adding the normal dependency
But that is not all, we also need to add the normal dependency so that when in create class objects of what we find we don't get errors.
In /xmltest/xmltest-coverage/pom.xml
...
<dependency>
<groupId>com.companyx.xmltest</groupId>
<artifactId>xmltest-functional</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
Writing the Test
Now, what we need to do is to write the coverage test.
A typical test class
package com.companyx.xmltest.functional;
...
@Path("/image/{id}")
@Test
public class ImageControllerGetTest {
Coverage Test
Now, finally, I can write a test which scans in the classpath for classes with are annotated with @Test, loops through all that have the correct package, checks if they have the @Path annotation and get the annotation.
ClassPathScanningCandidateComponentProvider scanner = new ClassPathScanningCandidateComponentProvider(false);
scanner.addIncludeFilter(new AnnotationTypeFilter(Test.class));
for (BeanDefinition bd : scanner.findCandidateComponents("com.companyx.xmltest.functional")) {
try {
Class clazz = Class.forName(bd.getBeanClassName());
if( hasPathAnnotation(clazz) ){
Annotation annotation = clazz.getAnnotation(Path.class);
And so on and so forth
And you'll just have to imaging that we do the same for the controller classes, puts it all into two lists and compare the lists. But I'm sure you can figure out how to do this :)
Why don't you just filter on Path.class instead of Test.class
I am sure someone would ask this if I actually had any readers.
Good question and the reason we do this is because we want to find test classes which has missed being annotated with @Test. It happens and we want to catch those cases as well.
That's It
I hope you got something out of this, I sure learnt a lot while writing it.
Hope to see you again in the next blog post.
And remeber:
"To error is human, to test is divine"
Maven Ninja Moves
I'll round off with some good commands my Mentor taught me.mvn versions:display-plugin-updates
Gives you a list of plugs you can update.
mvn versions:display-dependency-updates
Gives you a list of dependencies you can update.
Monday, November 28, 2011
Recovering deleted files from an svn repository
Hi
This posts show how to recover a deleted file in Subversion.
Assuming the file is called "VeryImportantFile.java" and it was deleted by mistake by the sloppy developer John Doe.
1. Get svn log
Assuming the file is to big to work with in the terminal we save the output
>svn log --verbose > svn_log_verbose
2. Open file in text editor
>gedit svn_log_verbose
3. Find the commit that delete it
Ctrl+F "VeryImportantFile.java"
Fixed stuff in the system
------------------------------------------------------------------------
r53308 | john_doe | 2011-11-17 13:15:39 +0100 (Thu, 17 Nov 2011) | 1 line
Changed paths:
M /trunk/xmlapi-test/src/test/java/com/RegularFile.java
A /trunk/xmlapi-test/src/test/java/com/NewFreshFile.java
D /trunk/xmlapi-test/src/test/java/com/VeryImportantFile.java
4. Revert it
You need to use one revision BELOW the one that deleted it.
> svn up -r 53307 src/test/java/com/VeryImportatFile.java
A src/test/java/com/VeryImportantFile.java
Updated to revision 53307.
5. Add to repo
> svn cp -r 53307 src/test/java/com/VeryImportantFile.java svn://svn.project.com/java/trunk/xmlapi-test/src/test/java/com/VeryImportantFile.java
Now you are promted default text editor to enter commit message. Enter message, save file and Exit to commit.
( cp is short for copy )
6. Update Repo
> svn up
U src/test/java/com/VeryImportantFile.java
Updated to revision 53790.
Now the file is reverted and back in repo! Life could not be better!
This posts show how to recover a deleted file in Subversion.
Assuming the file is called "VeryImportantFile.java" and it was deleted by mistake by the sloppy developer John Doe.
1. Get svn log
Assuming the file is to big to work with in the terminal we save the output
>svn log --verbose > svn_log_verbose
2. Open file in text editor
>gedit svn_log_verbose
3. Find the commit that delete it
Ctrl+F "VeryImportantFile.java"
Fixed stuff in the system
------------------------------------------------------------------------
r53308 | john_doe | 2011-11-17 13:15:39 +0100 (Thu, 17 Nov 2011) | 1 line
Changed paths:
M /trunk/xmlapi-test/src/test/java/com/RegularFile.java
A /trunk/xmlapi-test/src/test/java/com/NewFreshFile.java
D /trunk/xmlapi-test/src/test/java/com/VeryImportantFile.java
4. Revert it
You need to use one revision BELOW the one that deleted it.
> svn up -r 53307 src/test/java/com/VeryImportatFile.java
A src/test/java/com/VeryImportantFile.java
Updated to revision 53307.
5. Add to repo
> svn cp -r 53307 src/test/java/com/VeryImportantFile.java svn://svn.project.com/java/trunk/xmlapi-test/src/test/java/com/VeryImportantFile.java
Now you are promted default text editor to enter commit message. Enter message, save file and Exit to commit.
( cp is short for copy )
6. Update Repo
> svn up
U src/test/java/com/VeryImportantFile.java
Updated to revision 53790.
Now the file is reverted and back in repo! Life could not be better!
Saturday, October 8, 2011
Ubuntu 11.04 Freeze On Startup using Current Nvidia Drivers
Yesterday coming home from a night of dining and board games I decided to update my Nvidia drivers. The reason I wanted to do this was because I just bought the Humble Frozen Synapse bundle and it was running slow on my machine, I thought that updating the Drivers would give me some sweet Hardware acceleration and kung fu graphics.
But instead of Gaming Heaven I entered Linux Hell.
After selecting "NVIDIA accelerated graphics driver (version current) Recommended" and rebooting I couldn't use my computer any more. Ubuntu and Unity would start. I was able to move the mouse but rest of the desktop was frozen. It was like it had frozen in mid launching.
I figured that changing back to the old driver would fix it. Likely by entering some kind of Graphics Safe Mode. But how to do this without a GUI was a not obvious to me.
This is the long version of how I fixed it. I have included some, but not all, of my trial and error. I am writing this both for my sake, otherwise I am likely to forget how it was fixed, and for your sake, hoping to help someone with similar problems.
Part 1: Grub
Since I only have one OS on my computer Grub was configured to not show a menu on start up. So first I needed to figure out how to enter Grub on boot. Holding down SHIFT like it said in the documentation didn't work so I after I while I figured out that if I pressed "Ctrl+Alt+Del" and held for a few seconds it would reboot and Grub menu would show up. Doing this all the time was annoying so I decided to enable Grub on boot.
Step by Step instructions:
* Reboot and hold "Ctrl+Alt+Del" to enter Grub.
* Select Recovery mode of latest kernel
* Select "Drop To Root Shell Prompt"
(Root shell did not ask for password which was kind of scary)
* Edit file /etc/default/grub
I changed "GRUB_HIDDEN_TIMEOUT=0" to "GRUB_HIDDEN_TIMEOUT="
This, according to Grub Documentation, means
- The menu will be displayed for the number of seconds designated by GRUB_TIMEOUT.
I changed "GRUB_HIDDEN_TIMEOUT_QUIET=true" to "GRUB_HIDDEN_TIMEOUT_QUIET=false"
This, according to Grub Documentation, means
- A counter will display on a blank screen for the duration of the GRUB_HIDDEN_TIMEOUT value.
* Restarting Computer
Now Grub menu is shown by default. Yay!
Part 2: Disable auto-login
Since I'm the only user on the system Ubuntu has been set up to Auto-Login with my User. I needed to turn this off since I figured I could select Safe Mode from login screen. I couldn't enter the system since it freezes so I needed to do this from the command prompt.
Step by Step instructions:
* Reboot to enter Grub.
* Select Recovery mode of latest kernel
* Select "Drop To Root Shell Prompt"
* Edit file /etc/gdm/custom.conf
I changed "AutomaticLoginEnable=false" to "AutomaticLoginEnable=true"
* Restart Computer, select latest Linux in Grub
Now I have a login screen instead of a Unity that hangs. Yay!
Part 3: Login Safe Mode
When login screen appears I stopped the timed login and selected my user.
At the bottom of the screen there is bar where you can choose window manager.
I selected "Ubuntu Classic" which means Gnome, and enter Ubuntu.
Won't Start! Same as in Unity, freezes Desktop but I can move the mouse. Damn!
* Restart and try again
This time I select "Ubuntu Safe Mode" and enter Ubuntu.
It starts and I can use my computer again!
I'm so close to victory that I can taste it.
Part 4: Disable nvidia-current
* From Gnome menu I select "System" -> "Administration" -> "Additional Drivers" and try to enable the old Nvidia drivers but is informed that I do not have permissions to do such a thing.
* Using "System" -> "Preferences" -> "Main Menu" I find out the command to start "Additional Drivers" which is "/usr/bin/jockey-gtk".
* Start terminal and enter "sudo /usr/bin/jockey-gtk"
* Dialogue opens and I can enable the old Nvidia drivers.
* Restart Computer and Select "Ubuntu" on login screen.
Now Unity starts without freezing!
After a few hours of fighting with Linux I am finally back to where I started.
Frozen Synapse still has bad FPS and lagging but I so much better than a Computer that freezes.
Have a great weekend!
But instead of Gaming Heaven I entered Linux Hell.
After selecting "NVIDIA accelerated graphics driver (version current) Recommended" and rebooting I couldn't use my computer any more. Ubuntu and Unity would start. I was able to move the mouse but rest of the desktop was frozen. It was like it had frozen in mid launching.
I figured that changing back to the old driver would fix it. Likely by entering some kind of Graphics Safe Mode. But how to do this without a GUI was a not obvious to me.
This is the long version of how I fixed it. I have included some, but not all, of my trial and error. I am writing this both for my sake, otherwise I am likely to forget how it was fixed, and for your sake, hoping to help someone with similar problems.
Part 1: Grub
Since I only have one OS on my computer Grub was configured to not show a menu on start up. So first I needed to figure out how to enter Grub on boot. Holding down SHIFT like it said in the documentation didn't work so I after I while I figured out that if I pressed "Ctrl+Alt+Del" and held for a few seconds it would reboot and Grub menu would show up. Doing this all the time was annoying so I decided to enable Grub on boot.
Step by Step instructions:
* Reboot and hold "Ctrl+Alt+Del" to enter Grub.
* Select Recovery mode of latest kernel
* Select "Drop To Root Shell Prompt"
(Root shell did not ask for password which was kind of scary)
* Edit file /etc/default/grub
I changed "GRUB_HIDDEN_TIMEOUT=0" to "GRUB_HIDDEN_TIMEOUT="
This, according to Grub Documentation, means
- The menu will be displayed for the number of seconds designated by GRUB_TIMEOUT.
I changed "GRUB_HIDDEN_TIMEOUT_QUIET=true" to "GRUB_HIDDEN_TIMEOUT_QUIET=false"
This, according to Grub Documentation, means
- A counter will display on a blank screen for the duration of the GRUB_HIDDEN_TIMEOUT value.
* Restarting Computer
Now Grub menu is shown by default. Yay!
Part 2: Disable auto-login
Since I'm the only user on the system Ubuntu has been set up to Auto-Login with my User. I needed to turn this off since I figured I could select Safe Mode from login screen. I couldn't enter the system since it freezes so I needed to do this from the command prompt.
Step by Step instructions:
* Reboot to enter Grub.
* Select Recovery mode of latest kernel
* Select "Drop To Root Shell Prompt"
* Edit file /etc/gdm/custom.conf
I changed "AutomaticLoginEnable=false" to "AutomaticLoginEnable=true"
* Restart Computer, select latest Linux in Grub
Now I have a login screen instead of a Unity that hangs. Yay!
Part 3: Login Safe Mode
When login screen appears I stopped the timed login and selected my user.
At the bottom of the screen there is bar where you can choose window manager.
I selected "Ubuntu Classic" which means Gnome, and enter Ubuntu.
Won't Start! Same as in Unity, freezes Desktop but I can move the mouse. Damn!
* Restart and try again
This time I select "Ubuntu Safe Mode" and enter Ubuntu.
It starts and I can use my computer again!
I'm so close to victory that I can taste it.
Part 4: Disable nvidia-current
* From Gnome menu I select "System" -> "Administration" -> "Additional Drivers" and try to enable the old Nvidia drivers but is informed that I do not have permissions to do such a thing.
* Using "System" -> "Preferences" -> "Main Menu" I find out the command to start "Additional Drivers" which is "/usr/bin/jockey-gtk".
* Start terminal and enter "sudo /usr/bin/jockey-gtk"
* Dialogue opens and I can enable the old Nvidia drivers.
* Restart Computer and Select "Ubuntu" on login screen.
Now Unity starts without freezing!
After a few hours of fighting with Linux I am finally back to where I started.
Frozen Synapse still has bad FPS and lagging but I so much better than a Computer that freezes.
Have a great weekend!
Sunday, August 7, 2011
VVVVV on Ubuntu 11.04
Today I bought The Humble Indie Bundle #3
It's a collection of Indie games and it's pay-what-you-want. They clamied that "All games in the Humble Indie Bundle work natively on Linux." and I am interested to see if they really do. I have heard of most of the games and was very keen on trying out VVVVV which seems really cool.
It's only 2 days left on the package so hurry if you want to try it out.
The sale of the bundle is over :(
http://www.humblebundle.com/
After paying $10 I got a link where I could download the games. After downloading and unzipping VVVVV I entered the terminal and tried to run it.
>sh VVVVV
But got an error
./VVVVVV_32: error while loading shared libraries: libSDL_mixer-1.2.so.0: cannot open shared object file: No such file or directory
A litle googleing told me that the SDL_mixer library is missing.
Let's install it.
>sudo apt-get install libsdl-mixer1.2
And it works!
I'll try the other games and write more about it later.
Now I am going to enjoy the game.
Have fun!
UPDATE:
Mini Review!
VVVVV is a fantastic game, it's loads of fun and extremly challenging. I died almost a thousand times in the 2½ hours it took me to complete the game. But it was worth every death. It's innovative and groundbraking.
VVVVV is like Meteroid on LSD!
Final Stats.
I still have some "trinktes" left to collect. They are optional but adds extra value to the game. I'm sure that collecting the last 6 take me as long time as to finish the regular game.
If you get chance to try VVVVV you should, it's a great game.
http://thelettervsixtim.es/
Have fun!
Progress on Player Levels
Player Levels I have finished so far:
* Line Warp ( fun and challanging )
* Roadtrip to the moon ( fun but short )
* Pyramid Of Doom ( great but to short )
* Variation Venture ( best so far! )
It's a collection of Indie games and it's pay-what-you-want. They clamied that "All games in the Humble Indie Bundle work natively on Linux." and I am interested to see if they really do. I have heard of most of the games and was very keen on trying out VVVVV which seems really cool.
The sale of the bundle is over :(
http://www.humblebundle.com/
After paying $10 I got a link where I could download the games. After downloading and unzipping VVVVV I entered the terminal and tried to run it.
But got an error
A litle googleing told me that the SDL_mixer library is missing.
Let's install it.
And it works!
I'll try the other games and write more about it later.
Now I am going to enjoy the game.
Have fun!
UPDATE:
Mini Review!
VVVVV is a fantastic game, it's loads of fun and extremly challenging. I died almost a thousand times in the 2½ hours it took me to complete the game. But it was worth every death. It's innovative and groundbraking.
VVVVV is like Meteroid on LSD!
Final Stats.
I still have some "trinktes" left to collect. They are optional but adds extra value to the game. I'm sure that collecting the last 6 take me as long time as to finish the regular game.
If you get chance to try VVVVV you should, it's a great game.
http://thelettervsixtim.es/
Have fun!
Progress on Player Levels
Player Levels I have finished so far:
* Line Warp ( fun and challanging )
* Roadtrip to the moon ( fun but short )
* Pyramid Of Doom ( great but to short )
* Variation Venture ( best so far! )
Friday, July 1, 2011
The true beauty of QTP
Ian Fraser has well said it -
“The true beauty of QTP is that the only limitation is your imagination and ability to code in VBScript.”
“The true beauty of QTP is that the only limitation is your imagination and ability to code in VBScript.”
Subscribe to:
Posts (Atom)



