Integrating Selenium tests with Hudson CI

For most folks the challenge is that their CI system lives in some Unix environment and Selenium tests need to run on a Windows machine.To get this working I did the following

My goal was to

a)Have the test drive and report selenium tests from a single interface

b)Ability to run it on different machines with various browsers

I picked Hudson for a) for the simple reason that it’s widely used in our organization as well as it had a Selenium Grid plugin which will probably be used for accomplishing b).It’s easily available and can display JUnit reports in graphical format with drilldown capabilities.In short the works and then some:)

Tools used are :Hudson,Selenium RC 1.0.1,FF 3.5,Ant 1.7,Selenium Grid plugin for hudson,Java,TestNG,SVN,Windows XP,Hudson JUnit plugin

Note:Selenium Grid is only introduced to run the tests on multiple machines and different browsers.If this is what you don’t need,you can avoid one more layer of complexity

Step 1

This step is crucial and can differ from org to org,but if your hudson master and slave are in same subnet this should generally work.Make sure of this by pinging each host to other.If not you might need to get help from your IT such that the master hudson can ping the slave hudson and viceversa.

Step 2 (skip if not running on multiple machines)

Install Selenium Grid plugin from Manage Hudson->Plugins->Install

Hudson restart will be required

Step 3

Create Hudson slave using following method

a)Go to Manage Hudson->Manage Nodes->Add Node(Dumb Slave)

b) Enter in Name field anything you wish.However hudson recommends giving same name as the host name of slave.So give the computer name of your windows machine where the tests will run

c)Num of executors is directly corresponding to number of RC’s you will run.For now keep it 1.If you want to run two browsers(e.g FF 3.5 and IE 7) this number needs to be bumped upto 2

d)Remote FS Root:c:\hudson (this is where hudson Workspace artifacts will be stored if needed).If you are going to use Selenium Grid plugin to run,you won’t need this

e)Labels:Give a unique label e.g(windowsxp_1).You will see it’s use later with Selenium Grid

f)Usage:Leave this machine for tied jobs only

g)Launch slave agents via JNLP

h)Availability keep this machines as much online as possible

Rest of the options keep blank.

Repeat the process for each Slave machine,but to keep things simple let’s just configure 1 machine at a time

Step 4

Go to the Slave machine and

a)Open your browser and point to Hudson Master

b)Go to the Manage Nodes page and you should see a launch JNLP jar icon.Click on it and it should start the process of downloading “remoting.jar” from Hudson Master(I had trouble with this and found out that somehow hudson master didn’t have the jar file at a specific location.With the help of my Hudson admin,I was able to copy this jar correctly).So if you repeatedly fail to launch the slave agent,first thing to check would be the location of remoting.jar.Also look at the java exception and it will give you clues in what could have gone wrong.

c)If all goes well you should see a small window with Hudson icon showing the message “Connected”.

The above step is very important and needs to happen in order to Hudson run selenium tests.If this doesn’t work look into hudson logs,slave logs found on hudson server and there should be some clue on what’s going on.

Step 5

Create a new job on hudson and the configure the option to “run it on the slave node” that you just created.

Publish the Junit report by pointing to the output directory xml file.This will be something like c:\hudson\workspace\xx\xx.xml

Depending on what your hudson job is doing the slave machine should have appropriate software.E.g if you are going to run ANT files,the slave machine needs to have ANT properly configured.Same thing goes with SVN.Essentially everything that your job would need to run on master should be available on slave as well.

Step 6

Start a selenium server on the slave machine as you would if it was your local machine.For this testing make sure that versions of browser that you used for your environment to run selenium test locally is same.This will eliminate unnecessary issues that are not due to this setup but browser configuration issues.

Step 7

Run the hudson job from your master and you should see the browser window popping up on the slave and running the tests.

The above should accomplish the task of running your Selenium tests on a single Windows machine.However many times you would need to run your tests in parallel as well as to optimize hardware use,make sure that Hudson can trigger test on same machine with different browser and report results back.In order to accomplish this we will use the Selenium Grid plugin which I will cover in next part if folks are interested.

Some resources that were helpful to me in this exercise were

Amit Easow’s writeup on Hudson integration


VN:F [1.9.10_1130]
Rating: 8.6/10 (22 votes cast)
Integrating Selenium tests with Hudson CI, 8.6 out of 10 based on 22 ratings

  

{ 3 trackbacks }

Blog post about Hudson Selenium Grid plugin and continuous integration
February 28, 2010 at 11:28 pm
How do I integrate my Selenium RC tests integrate with Hudson
May 7, 2010 at 1:36 am
continuous integration with Hudson | selenium automation testing
April 21, 2011 at 10:43 pm

{ 19 comments… read them below or add one }

Vinay February 10, 2010 at 7:01 am

Hi,

How do i configure the reporting to email the testng report which gets generated.

When i point it to the emailable-report.html , to the selenium report section, it fails currently

Awaiting your reply ..

Regards,
Vinay

rutvik February 10, 2010 at 2:01 pm

Hudson currently doesn’t support displaying testng reports,but if you have JUnit report viewer plugin installed,you can point to something like this
xxxprojectxxx/test-output/**/*.xml.

Some other options to get the report is add a ANT target which will email this html file to your account after the test runs.However within Hudson the first approach works nicely for me

Pankaj Nakhat February 11, 2010 at 1:27 pm

Hi,

Hudson can display TestNG reports, only thing you need to so it configure the job to Archieve the TestNg Results directory.

However, if you use reportng, Hudson will pick results out of the box.

rutvik February 11, 2010 at 11:22 pm

Pankaj,
Does reportng fit nicely into the Selenium continuous integration workflow as described above.I looked into reportNG but looks like it’s another report viewer unless there is a Hudson plugin for reportNG
Archiving the TestNG results is fine,but you can’t get the graphical report/trending between builds with that method.

Dan February 13, 2010 at 9:27 pm

ReportNG is primarily for generating HTML output from TestNG. It does also generate JUnit-format XML, which is what Hudson uses to keep track of test results. TestNG has its own JUnit-format XML output, but for Hudson you get better results if you use the ReportNG one.

anoop March 15, 2010 at 7:20 am

Hi
I am getting an error while running tests against a slave machine. It showing as unable to create directory. Is it due to bad ‘Remote FS root’ configurartion. Actually what this Remote FS root means?
Thanks in advance..

Rutvik March 15, 2010 at 12:59 pm

Anoop,
The remote FS root is a local directory on your slave machine.Can you please make sure that the FS root that you have specified in Hudson does not have any special restrictions.To do a quick test,make sure
a)The path is correct.e.g “C:\Anoop” in Hudson.sometimes the slashes work differently,so try both ways
b)Login to your slave machine as admin,so that we can eliminate any permissions issue

If this doesn’t work,give me some information about what kind of OS is the slave machine and is your Hudson running on Unix or Windows?

Peter Kreidermacher May 20, 2010 at 11:07 am

Hi,
Thanks for the great outline of integrating selenium tests in hudson. I am having a problem though. When I build the project that holds the test, the tests are repeated 3 times before the build completes. While it might be a nice thing to have 3 tests to make sure something didn’t pass because of a fluke, my tests run for approximately 20 minutes and it’s not efficient to wait about an hour for one build to complete. Is there something I’m missing (or need to remove) from my code/configuration to stop the tests from repeating? Thank You!

Peter Kreidermacher May 20, 2010 at 12:18 pm

Actually, ignore my previous comment/question. I found out that it only repeats the tests 3 times until there is a successful build. Once there has been at least one successful build, it only runs the tests once. Thanks!

Alex October 7, 2010 at 10:27 am

I still having problems on step 4. I am getting connection refused on execution of jnlp, anyone could help?

AA October 11, 2010 at 10:13 am

Hi,

when installing the selenium grid plugin and after the server’s restart process completed, I got the following exception

java.net.MalformedURLException: no protocol: 192.168.56.50:8080/hudson/
at java.net.URL.(URL.java:567)
at java.net.URL.(URL.java:464)
at java.net.URL.(URL.java:413)
at hudson.plugins.selenium.PluginImpl.getMasterHostName(PluginImpl.java:225)
at hudson.plugins.selenium.ComputerListenerImpl.onOnline(ComputerListenerImpl.java:59)
at hudson.model.Hudson.(Hudson.java:645)
at hudson.model.Hudson.(Hudson.java:554)
at hudson.WebAppMain$2.run(WebAppMain.java:220)

can anybody guide me through the steps needed to fix this exception.

Thanks in advance,
AA

rutvik October 13, 2010 at 1:14 pm

Alex-
Can you do a simple ping from the machine that you are trying to setup as Hudson Slave and your Hudson master.It’s important that both of them are able to communicate to each other.

If this passes-sometimes the jnlp file is not at the correct location on Hudson master.Check with your Hudson admin and see if the jnlp resource is present or not.

rutvik October 13, 2010 at 1:16 pm

Please check following
-Make sure Se Grid is not on the same port as Hudson(default is 4444)
-Does your Hudson work correctly if Se Grid plugin was not installed?

tom March 3, 2011 at 7:47 pm

This would be great if you could show exactly what has to go in the job configuration in Hudson to run Selenium tests standalone without running Ant.

Is this what would be used in a shell?

java -jar selenium-server.jar -htmlsuite *firefox http://aut_url “suite location”

rutvik March 5, 2011 at 3:13 pm

Hi Tom,
The example in the blog assumes that the tests were written in Selenium RC.From your example it looks like you are trying to run it with tests generated using Selenium IDE.However as long as you are able to run this through a simple command line ,you should be able to plug it into Hudson where instead of using the Ant file option in Hudson,you can choose “Invoke Windows batch file/or Shell ” where the batch file will call the command that you mentioned in your comment.Let me know if you need more help.

HansV2 April 5, 2011 at 4:22 am

and dont forget to set the M2_HOME wariable in your hudson job. Or else it will fail with an exception like
“FATAL: Couldn’t find any executable in…”

Took me quiet a while to figure that one out.

Deepa July 13, 2012 at 3:09 am

Hi,

I am stuck on step no. 5. I am unable to configure the new job to “run it on the slave node”, no such option is visible.

Hudson Master n Slave are hosted on the same machine and Hudson Slave Agent window says its connected.

Reply Awaited.

Thanks for the Help.

Regards,
Deepa

ccdijian.com April 16, 2013 at 12:24 pm

I am in fact thankful to the holder of this web page
who has shared this great post at here.

google plus account for business August 23, 2014 at 11:14 pm

Hello! This is my first visit to your blog! We are a group of volunteers and starting a
new project in a community in the same niche.
Your blog provided us useful information to work on. You have done
a wonderful job!

Leave a Comment