Posts tagged as:

Test Automation

A Manual Tester Becomes an Automation Engineer Using Fitnesse

We know Selenium IDE is one which is used by many manual testers and does not require knowledge of any programming language. But a manual tester can become an automation engineer using fitnesse. The developer need to create a framework using selenium rc/webdriver with any programming language like java, C# etc and integrate with fitnesse. There is no need to have knowledge of junit and testing framework.

I will show how this can be done step by steps

  1. Download fitnesse from http://fitnesse.org/FrontPage.FitNesseDevelopment.DownLoad

Unzip downloaded folder to any drive like C:\fitnesse

  1. Download selenium from http://seleniumhq.org/download/

Keep selenium-server-standalone-2.21.0.jar in C:\fitnesse

  1. Start fitnesse server clicking run.bat or from command prompt type java -jar fitnesse.jar
  2. Now open your browser and go to http://localhost. You will see the FitNesse home page.
  3. Click on Edit menu at left pane. Remove introduction note from home page and create project. We need to set path as shown below once our project is created.

!path selenium-server-standalone-2.21.0.jar

!path fitnesse.jar

!path lib/fitlibrary.jar

!path D:\Workplace\Fitenium\bin

Also configure SetUp and TearDown as if required.

 

  1. Make sure that you have created project Fitenium using eclipse in D drive in Workspace. Below I have created sample java class which utilizes java fixture wrappers around selenium call.

package com.sample.automation.framework;

import org.openqa.selenium.By;

import org.openqa.selenium.WebElement;

import com.sample.automation.browser.Browser;

import fitlibrary.SequenceFixture;

/*

* This class extends SequenceFixture so that we can create and use java style fixture in fitnesse*/

 

public class FitSeleniumFramework extends SequenceFixture

{

private static WebDriver driver;

public void initializeBrowser() {

driver = new FirefoxDriver();

}

public WebDriver getWebDriver() {

return driver;

}

public void stopBrowser() {

if(driver != null) {

driver.quit();

}

}

/*

* This is a fixture which we use to open home page url

*/

public void navigateToHomePage(String url) {

initializeBrowser();

try {

getWebDriver().navigate().to(url);

}catch (Exception e) {

System.out.println(“ERROR OPENING THE ” + url);

e.printStackTrace();

stopBrowser();

}

}

/*

* This fixture is used to check if particular element present on web page

*/

public boolean isElementPresent(String element) {

WebElement webElement = getWebDriver().findElement(By.xpath(element));

if(webElement != null)

return true;

else

return false;

}

  1. We have now two fixtures from above code

navigateToHomePage(String url)

isElementPresent(String element)

 

  1. Edit Project and create test suite namely HomePageSuite. Use Properties menu from left pane to change properties of page.

 

  1. Edit suite created in fitnesse and create test case namely “HomePageElements”. Write fixture in test case like

|navigateToHomePage|http://kayak.com|

|check|isElementPresent|//input[@id="origin"]|true|

|check|isElementPresent|//input[@id="destination"]|true|

|check|isElementPresent|//button[@id="fdimgbutton"]|true|

 

  1. Run test case clicking Test button from left pane.

Once tester got familiar with fixture, effectively he can start productively automating test cases without any programming experience. There is no need for the whole team to  know java and selenium or any other programming language. By just understanding wiki markup and xpath or css, non-technical person can write test cases using fitnesse.

VN:D [1.9.10_1130]
Rating: 8.7/10 (3 votes cast)

  

{ 1 comment }

Java – The Most Preferred Language for Web UI Test Automation using Selenium

The quality engineering has gained the popularity as same way as software design and development and sometime more than some of the development technologies; as it is required for most of the application /systems. As being most recognized company in Quality engineering, we observed that most of the companies will have quality engineering department of their own even-though the software development is out sourced to other vendors.

Now a days, software test engineering (QA) is not just limited to test case managements with tools like QMetry etc. and test execution manually. But there are many areas like test automation, performance / load testing, security testing, SOA testing and code review with testing etc. being introduced and widely used by each departments at different level. In all of this, the test automation is very much popular as it reduces the regression test duration plus easily integrated with continuous build systems etc. to result in best ROI over the time.

There are many automation tools available which can be used by any manual test engineer for regression tools like Selenium IDE, Sahi etc.. Still once you start the automation, there will be  some more demand  from your QA Manager / Test Lead / Project manager / CTO about test results, debugging, reusability, continuous integration, maintainability, execution from cloud etc. and that will required you to develop the framework using scripting language / programming language.

Amongst big group of automation tools, selenium is the only tool which supports many programming languages ranging from simple  like HTML to complex like Ruby/Java/C# to automate the test bed of your web application. As selenium supports variety of languages for automation, manier  get confused to select the suitable language for automation. Mainly the selection of laungage depends on many different crieterias as well. Therefore  to overcome from this delta situation, I tried to get input from different big players in automation world about the preferred language and based on which crieteria they select that etc. We had run the poll for same for quite a month long time and able to get good information as descried in following sections of this post.

Overall Result:

Language preferred for Selenium Test Automation Project? Write in comment for other languages.

Language preferred for Selenium Test Automation Project? Write in comment for other languages.

We can see that JAVA got around 77% votes from total 194 votes worldwide.  The HTML, PHP and Javascript etc. was not part of selection but few peoples are using this for their test automation.

We have got some language selection criteria based on the feedback during this poll are:

  1. SUT development environment
  2. Open Source Tool
  3. Material/libraries available for language
  4. Performance and Transaction Time
  5. Easy to code, IDE
  6. Language exposure to automation engineer

Etc.

If we have to rate these criteria’s, then the first point will win the race and most of has voted based on this criteria.

Brainstorming: Based on first preference and poll data, can we think that JAVA is most popular in development as well!!

The consolidated feedback and related information is as below.

1. SUT development environment

Many time, we (auto architects) think to use the language as same as the application under test development platform/language. Because it help us to use the same tests as unit tests and helps us for customization etc.

“I prefere Python because I like this language and I used if for wide range of tasks before I start automation with Selenium. Python has everything what I need for any purposes. But Current project based on PHP, so one of requirement was that automation will also based on PHP” – how can you leave your preference.

Helps in unit testing – “ Being able to write Selenium tests in JUnit allows me to do end-to-end testing of our server-side Java application and browser clients. So I can call into a backend Java application(handy for me when doing test setup and cleanup) and control the browser from the tests. I can also easily take automated screenshots of the browser view.”.

Further from other expert, “It really depends on what your goals are. Personally I prefere using the language the app is written in. Of the benefits its nice to be using the same language and tool chain the developers are using to make collaboration and integration points simpler.”

Manish says, “I’m working in a product that uses Selenium RC with C# and codedUI and I’m enjoying it and the reason of choosing c# is the same app written in .Net”

More specific quote “For your tests to be economically maintainable, they need to be in a Turing-complete language (e.g. not HTML-based). That said, assuming either: 1. There’s no dedicated QA team and testing is an integral function of Development or 2. there’s a dedicated QA team that works closely with development, the only responsible choice for your scripts is in the language of the application under test. Otherwise, cooperation with Development for test case issue resolution will be difficult as not all members may be familiar with the script language.”

2. Open source Tool

We all know that why JAVA won the race, because it is open source.

“Eclipse with JAVA and selenium is free, however .NET IDE is not free. There is an open source .net IDE called mono that could be used with selenium;”

Ratankumar adds “ I’d prefer JAVA, as it is also open source and easily available for download.”

3. Material/Libraries available for language

The availability of the documentation / samples / tutorials can matter the selection of the language for your selenium (others as well) automation projects very well because not all test automation engineers / architectures will have strong programming language experience. So this type of supporting material helps them for quick start. Selenium is the very best automation platform which gives us the wide variety of language selection for automation and which make ease for the new people to quick start with the help of this kind of supporting material.

“I favor Perl because I can use the martial from CPAN.org to build a framework. I have lot of modules that I can interface to the selenium api” – Perl selection for automation based on this criteria.

Kshitij describes in same line “ I prefer java/junit with Eclipse, due to easy available solution on web and most other plugins and technology for automation is available in Java.. So its easy to integrate with selenium.”

C# Documents seems ahead of all, “ C# has a ton of documentation and tools that can complement your script development efforts.”

Do you think same “ There’s much less user support for Perl, but having that extensibility opens up many avenues. I’m looking forward to checking out the Webdriver Perl bindings.”

4. Performance and Transaction times

Sometimes we think on this before starting the automation development.

Selenium can perform well with Java. Do you agree with “Selenium is built on JAVA and runs through a JVM. So I suggest using the language that selenium was created in.”

5. Easy to code, IDE

Definitely this is the easiest criteria as well, but test automation sometimes required customization in results presentation, looping based on data etc. So I am not sure that whatever language we will define as “easy to code” will be good selection for test automaton. Still we can find some other supporting interfacing language etc. like HTML is very easy so try to use Java Script extensions with this will definitely help you to keep it easy to code plus some level of customization. Let me describe same thing in users’ words: “I prefer html tests/cases with selenium server + js extentions + Hudson”.

Also, IDE does matter in the selection as it helps us to make our coding with easy due to context sensitive help, auto complete etc. C# IDE Visual Studio, Java IDE Eclipse etc. are good IDEs and is important point in language selection decision making process.

“If you have access to the VisualStudio IDE you cant go wrong with that”. I like this.

6. Language exposure to automation engineer

Sometimes we do not disclose or mention this criteria, but I think it is always get hidden consideration for language selection.

Are you thinking same? “I personally prefer Java because that’s the language I’m most familiar with. However, if the project and its timeline provided the opportunity for my learning curve, I would love to try out some other languages – it for nothing more than to see what differences there are.”

Anand is asking us with explanation, “Depends on which language you are already comfortable in, isn’t it? If you are comfortable in JAVA you don’t learn PHP to write selenium tests. As far as I know, most of the bindings provides the same features. And thanks to people behind Selenium for making this possible !”

In short, all points has their own pros and cons but this poll has created a platform to help someone to ask related queries to other who using same language.

At last, thanks to all for voting and commenting. We will come up with something similar and useful topic again!! 

courtesy:

  1. Selenium Automation User Group
  2. Infostretch Selenium Test Automation Framework
  3. All the linked in users who voted and commented on poll : “Language preferred for Selenium Test Automation Project? Write in comment for other languages.”

Thanks and Regards,

Akhilkumar Patel

Waiting for your suggestions and queries!!

 

 

VN:F [1.9.10_1130]
Rating: 9.0/10 (3 votes cast)

  

{ 0 comments }

“Why We Make Mistakes” in Test Automation

Test automation in general, and specifically scripted automation, is an essential component of any software testing effort. Advocating the appropriate use of automation in testing has been a major focus of mine since I started software testing over eight years ago.

Recently, I have been reading the book “Why We Make Mistakes” by Joseph T. Hallinan. It dives into the behavioral issues that humans experience in everyday life and how that behaviour influences the decisions we make. Many of these problems are not only applicable to software testing, but are the underpinnings of the successful test approaches that are in use today. Here are just a few of the important insights I have gained from this book that apply specifically to the use of automated scripts in software testing.

“Our expectations can shape the way we see the world …”

When we run an automated script suite and it comes back with all of the checks passed, we get a warm feeling of accomplishment - it shouldn’t! Scripts provide checks that indicate a low likelihood that regression defects exist and that is all. A successful automation run should be a starting gun to begin the real testing of the application, not provide false confidence that the application is defect free.

“Nearly eighty years of research on answer changing shows … that most people who change their answers usually improve their test scores.”

A major problem with automated checks is that they usually provide the first assessment of the software application quality. A “passing” automated assessment is often times misconstrued by uninformed managers and executives to imply passing quality. A technique I use in my automated scripts is to never report a “passing” run – only the execution failures. Also, when reporting the failure I always report the expected value and the measured value to encourage fast evaluation and remediation of the reported failure (many times in real time during the script execution). Going back and questioning the validity and correctness of “answers” is key to thorough testing.

“The miss-attribution of blame is one reason we make the same mistakes over and over again.”

Scenario: A fix is made to the application, you run a scripted test suite (which “passes”), and you then declare the application as ready for user testing. Five minutes into the test turns up a bug that was not caught during the scripted run. A typical solution is to expand the test suite to check for the bug the next time. Coverage complete again, job well done! The actual cause was probably the reliance on scripted tests as the only validation. A more realistic solution would be to investigate ways of providing effective, targeted exploratory testing in association with the scripted checks to provide better test coverage.

These are only a few of the tidbits I picked up in the first few pages of the book. Don’t take this blog at face value – read this book and others for yourself! I may have brought my own prejudices to reading it and glossed over the most important items of all. After all, I’m only human.  ;o)

VN:F [1.9.10_1130]
Rating: 9.7/10 (3 votes cast)

  

{ 0 comments }

Capture Screenshot with selenium remote web driver

I had been facing problems with the ‘capture screen-shot’ functionality in the remote webdriver implementation. There are many blogs on the web that try to address this problem, but I was unable to find any solution to the problem I was facing.  Finally, I found a solution, and here is what I did. I hope this will will help those stuck in a similar situation. I request you to kindly provide your feedback if it works for you, and also if it doesn’t.

Issue:

‘Capture screen-shot’ functionality works well with selenium2 standard implementation or webdriver implementation. When you try to do to the same with remote webdriver implementation, the ‘capture screen-shot’ functionality throws Class Cast Exception [java.lang.ClassCastException: org.openqa.selenium.remote.RemoteWebDriver cannot be cast to org.openqa.selenium.TakesScreenshot].

Reason:

The RemoteWebDriver class not implements TakesScreenshot interface.

Resolution:

One nice feature of the remote webdriver is that exceptions often have an attached screenshot, encoded as a Base64 PNG. In order to get this screenshot, you need to write code similar to:
if (e.getCause() instanceof ScreenshotException) {

String base64Str =

((ScreenshotException) e.getCause()).getBase64EncodedScreenshot();

…
}
This will provide you a screenshot when the WebdriverException exception occurs during action or event on browser. But what if you want to capture screenshot at particular step?

First, I tried unusual quick fix by utilizing WebdriverException. It worked for me, but it is not a standard way for sure. One more issue with the extracted screenshot from webdriver exception is, that it corresponds to the visible screen/desktop so sometimes the screenshot became meaningless when it does not contain the browser screen.

After analyzing the issue in greater detail I came up with a more standard solution. The main reason why screenshot in remote webdriver implementation is not supported is that the RemoteWebDriver class does not implement TakesScreenshot interface. So I created MyCustomRemoteWebDriver by extending RemoteWebDriver and implementing TakesScreenshot interface.

public
class MyCustomRemoteWebDriver extends RemoteWebDriver implements TakesScreenshot {

…

}

Finally I used MyCustomRemoteWebDriver in my implementation instead of standard RemoteWebDriver and it worked for capturing screenshot in the usual way!

VN:D [1.9.10_1130]
Rating: 9.5/10 (8 votes cast)

  

{ 19 comments }

Leveraging open source automation tools to reduce costs and optimize test process

With the shrinking development cycles, pressure of reducing the time to go-to-market and repetitive processes, automation is no longer optional. Also, there is no relaxation on the quality of the final products; companies just can’t afford to launch products that are not ready for the prime time. While test management tools like QMetry, Testopia, Quality center etc. provide a great way to manage and improve the overall test process, various test automation tools help further optimize it. Therefore, it is not surprising that the demand for test automation tools has increased drastically and more so for less expensive ones, given the current economic situation.
This has given rise to a number of open source test automation tools (See the list below). These open source tools offer an inexpensive way to automate the testing process. However, we still need to evaluate them against the commercially available tools that seem to offer greater predictability and support. Tools like QTP, Rational Robot have served the test automation needs of QA teams for a long time, but there have been many challenges and constraints that users have been facing with these tools. Some issues with proprietary tools that need to be addressed are:

- These tools are highly expensive, often times outside the budget of smaller QA teams
- As the application undergoes revision over time, maintenance costs of test scripts rises sharply
- Restrictions on licensing
- Lack of support for scripting language
- Dependency on add-ons

As far as the open source testing tool market is concerned, with so many players jumping onto the bandwagon, it is a herculean task for new entrants to differentiate and find a place for themselves The number of open source automation tools makes it hard for users to identify the best for their needs. At Infostretch, we evaluated almost all the tools out there on different parameters and discovered various strengths and weaknesses associated with them. (I’ll be publishing those findings in my next blog, stay tuned). We have been helping our clients in selecting the best tools and in developing various frameworks to address their specific needs. We spend most of our time in understanding the nature of application, business objectives, and the features that are required of the automation tools. This information is critical for selecting the right tool and developing robust frameworks.

In the later blogs on automation, I will be talking about our research findings, frameworks, testing on the cloud and open source test automation tools in general. In the meanwhile, if you have any questions or suggestions regarding any of these topics, please feel free to write to me at Durgesh.kaushik@infostretch.com.

List of Open source testing tools for functional testing (Source: http://www.opensourcetesting.org/)

Abbot Java GUI Test Framework
Anteater
Apodora
Arbiter
Autonet
AutoTestFlash
Avignon
Blerby Test Runner
Canoo WebTest
Capedit
Concordion
Crosscheck
csvdiff
CubicTest
Cucumber
DBFeeder
DbFit
DejaGnu
Dogtail
Doit: Simple Web Application Testing
Eclipse TPTP
EMOS Framework
Enterprise Web Test
Expect
expect-lite
Frankenstein
GITAK
GNU/Linux Desktop Testing Project
Harness
httest
IdMUnit
Imprimatur
ItIN – Infopath testing in .Net
ITP
ItsNat, Natural AJAX
ivalidator
Jacobie
Jameleon
jDiffChaser
Jemmy
JFunc: JUnit Functional Testing Extension
JSystem
JUnitum
jWebUnit
Latka
Linux Test Project
LogiTest
LReport
MActor
Marathon
MaxQ
miTester for SIP
Mockito
MozUnit
OLVER – Open Linux VERification
Om
Om
org.tigris.mbt
PAMIE
Pounder
pywinauto
QAliber
QAT (Quality Assurance Tests)
QMTest
Rasta
Robot Framework
Sahi
Samie
Sauce Labs: cloud-based Selenium service
Scalable Test Platform
Selenium
SharpRobo
soapui
Software Automation Framework Support
Software Testing Automation Framework (STAF)
Solex
SWAT (Simple Web Automation Toolkit)
SWTBot
SymbioTeam Lite
Systin
T-Plan Robot
tclwebtest
TestGen4Web
TestPlan
TextTest
Tomato
Toster – The Object-oriented Sofware Testing Environment
Watij
WatiN
Watir
Web Form Flooder
WebDriver
WebInject
WebTst
WET
Win32::IEAutomation
XML Test Suite

Other Resources:

http://www.slideshare.net/sebastian_bergmann/testing-with-phpunit-and-selenium

http://www.slideshare.net/hsplmkting/fitnesse-watir-and-ruby-based-test-automation-system

VN:F [1.9.10_1130]
Rating: 8.0/10 (10 votes cast)

  

{ 0 comments }

Selenium UI test automation with zero infrastructure cost!!

Preface:
There are several UI testing tools available in the market like SilkTest,QTP, Selenium etc. and test automation engineers are using those tools depending on the requirement. Among these tools, Selenium is a popular open source UI automation tool. Now for some projects, customer asks us to run the automation test against different browsers on different platform. So to do this we have to setup our test automation tool / server on different platform with different browsers and also the major challenge is to setup different versions of same browser on same machine. So to overcome these issues either we use virtualization technology or setup multiple systems for this testing, which results in increased cost and complexity in maintenance of systems for testing. To solve this kind of issues and tasks, Saucelabs offers one of the good cloud based solution called “Sauce On Demand” which facilitates us to test our application with different environments using their different models of pricing. Following sections I will describe about basics of sauce on demand.

Introduction of Sauce On demand:
Sauce on demand is the Cloud based solution for UI testing using selenium by Saucelabs. In this configuration we can run our tests on Cloud servers with different OS and different browsers and their versions. To run the test we have to purchase a plan and saucelabs will provide access id for that subscription. So to run the test we have to pass access id, user credentials and configurations (against which we want to test) in JSON format. Sample code is available in online documentation of sauce on demand along with currently supported configurations.
The online documentation is available @ http://saucelabs.com/products/docs/overview

Infrastructure cost=0 and other benefits:
As described above you get the idea that for our testing we can run test against different OS and different browsers and their version by just passing the configuration details so the main advantage of this tool that it will not require any effort to setup infrastructure to test our software against different OS and browsers.
The other advantage is, it stores video of whole test along with full log so it will be easier to find the issues and also it is easily accessible and downloadable from their server.
If we use threading or some other execution optimization concepts than we can test our application against different configurations simultaneously. I think same concepts can be applied for load testing.
There are so many advantages but for that I will suggest you to test a trial of sauce on demand.

Limitations:
Currently some combinations of OS and browser are not available. Check following link for available combinations : http://saucelabs.com/products/docs/sauce-ondemand/browsers
For parallel or load testing we have to write script in our tool, but it would be nice if such capability was available as part of Sauce On Demand

Conclusion:
If you really want to test against different OS, browser and their versions then give a thought about exploring Sauce on demand or we can help you for analyzing and preparing the framework for your this type of requirement.

At the end..
Please correct me if any information is incorrect. If you have any questions or comments on this topic you can send your queries to akhil.patel@infostretch.com or leave a comment below.

VN:D [1.9.10_1130]
Rating: 8.3/10 (10 votes cast)

  

{ 6 comments }