Posts tagged as:

automated testing

Automated Testing using BDT (Behavior Driven Testing)

Let’s go back to history first!

Behavior Driven Development is well-accepted and current methodology in agile projects.  Behavior Driven Testing (BDT) is a lesser known companion of Behavior Driven Development (BDD). BDT can be used without BDD.

In software engineering, behavior-driven development (abbreviated BDD) is a software development process based on test-driven development (TDD). Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software developers and business analysts with shared tools and a shared process to collaborate on software development.

Although BDD is principally an idea about how software development should be managed by business interests and technical insight, the practice of BDD does assume the use of specialized software tools to support the development process. Although these tools are often developed specifically for use in BDD projects, they can be seen as specialized forms of the tooling that supports test-driven development. The tools serve to add automation to the ubiquitous language that is a central theme of BDD.

Behavior-driven development was developed by Dan North as a response to the issues encountered teaching test-driven development:

  • Where to start in the process
  • What to test and what not to test
  • How much to test in one go
  • What to call the tests
  • How to understand why a test fails

At the heart of BDD is a rethinking of the approach to unit testing and acceptance testing that North came up with while dealing with these issues. For example, he found that unit test names should be whole sentences starting with the word “should“ and should be written in order of business value. Acceptance tests should be written using the standard agile framework of a User story: “As a [role] I want [feature] so that [benefit]“. Acceptance criteria should be written in terms of scenarios and implemented as classes: Given [initial context], when [event occurs], then [ensure some outcomes].

 

 

BDD & BDT Advantages

It involves getting stakeholders and delivery team with different perspectives onto the same page and ensuring that all have the same expectations.  BDD starts with a business goal and goal that translates to features and stories.

The goal of BDT is a business readable and domain specific language that allows you to describe a system’s behavior without detailing how that behavior is implemented.

In BDT, Tests in the form of plain text features descriptions with scenarios are typically written before anything else and verified by the non-technical stakeholders.

BDD Process

”"

Cucumber for BDD

Cucumber is optimized for BDD, by its support for particular set of interactions between team members and stakeholders.

Cucumber can execute plain-text functional (feature) specifications as automated tests. The language that Cucumber understands is called Gherkin.

Cucumber supports writing specifications in about 30 spoken languages, making it easy for teams outside of English-speaking territories or those working on internationally targeted software to deliver better.

Other salient features include:

  • Defining executable specifications in different ways like lists, prose and tabular data
  • The plain text files can be stored in any version control system
  • Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language
  • Can be integrated with all the most popular web testing libraries
  • Cucumber is both BDD in the form of feature descriptions and scenarios and TDD in the form of step definitions
  • Provides a layer of abstraction between the required behavior and the app implementation

The other tools that support BDD are 

  1. C - CSpec
  2. C++ - CppSpec Spec-CPP
  3. C# .Net - NSpec
  4. Net - NBehave, NSpecify & SpecFlow (for .NET, Mono & Silverlight)
  5. Groovy - GSpec
  6. Java - JBehave, JDave, beanSpec, Instinct, GivWenZen (Java and FitNesse), Cuke4Duke, accept4j.
  7. Javascript - JSSpec, jspec, Jasmine
  8. PHP - PHPSpec
  9. Python - Specipy, spec plugin for nose
  10. Ruby - RSpec, Shoulda, test-spec & bacon, Cucumber

”"

Behavior Driven Script Development

  • Specify the behavior looking at the system from the outside.
  • Collaborate and come up with a good and clear set of Acceptance Criteria.

Expected Functionality:

The customer should be able to withdraw money from his account.

Given the account has balance of INR 10000

When the customer requests INR 5000

Then the account is debited by INR 5000

 

Expected Behavior Driven Test:

It is an approach to testing. Work towards the big picture.

Given the account is in credit

And the dispenser contains cash

When the customer requests cash

Then ensure the account is debited

And ensure cash is dispensed

And ensure the card is returned

 

Behavior Driven Testing can be adopted for Mobile Test Automation

Most of the Enterprise clients are following scrum product development. Historically, test automation has not met with the level of success in the developmental phases (TDD & paired programming). Most often this is the result of inappropriate perceptions of the effort and resources necessary to implement a successful, long-lasting automation framework development. Why is this, we might ask? Well, realizing that we must develop reusable test strategies is no different than the reusability concerns of any good application development project. As we set out on our task of automating test, we must keep these past lessons upfront.

 “One test case for both Manual and Automation testing”

Here we try to get all the answers – BDT framework , we should focus our automation framework to deal with the common components that make up our unique automation framework that can useful in easy development of test cases as such as Manual tester can do with Library keywords.

Calabash + Cucumber

Nowadays, Calabash Open Source mobile automation testing tool is getting wide attention in market due to their automation capabilities for native & hybrid Android & iOS apps. User can write tests or features on top of BDT framework using Cucumber.

”"

 

VN:F [1.9.10_1130]
Rating: 0.0/10 (0 votes cast)

  

{ 0 comments }

API Automation for Internet Banking Platform

Background The program is about implementing new Internet Banking platform that aims to consolidate numerous internet instances across the Banking Group onto a shared platform and to implement new functionality for Group Businesses. The program is structured into a number of releases.

The Test Data is required for the different work streams

  •  Unit and Component Integration Test Data
  • System and System Integration Test Data
  • Automation Testing Team – Regression Test Data
  • Non Functional Test Data
  • Business Team – User Acceptance Test Data

Problem Statement

  • Numerous amount of test data request to be catered within given time frame.
  • Considerable amount of manual effort needed to create & tweak the test data using  Web applications, different desktop application, Mainframe systems
  •  Delivery slippage risk due to unavailability of the above said applications.
  • Manual data tweaking through mainframe systems was cumbersome & time consuming.

Solution

  • Investigation done to find out APIs responsible for data creation & tweaking through backend
  • Developed automation framework including combination of APIs, web applications &

Mainframe system in logical sequence, to do tweaking on the base accounts

API Automation

  • Directly can fire the APIs through Microsoft.XMLDOM and XMLHTTP objects for this we have to create a template XML file form the WSDL
  • Controller will call a specific API with the required data to be updated in existing XMLs. To update the XML for example <VBScript>

Variables details

strPath (The Directory Path)

srcCCXMLPath(Source XML File Path)

XMLNewFile(Updated XML will be saved on this Location)

destCCXMLPath(Response for the API will saved on this path)

strUpdate(A string with the Node Name and the Values with the separators i.e “;” and “=” )

LinkBUrl(API to be fired on this destination)

Response(Boolean Value for the result of API)

Sample Code

strPath=Environment.Value(“TestDir”)

Set fso = CreateObject(“Scripting.FileSystemObject”)

TestPath = fso.Getparentfoldername(strPath)

srcCCXMLPath = TestPath & “\XML_Template\” & CCXML

destCCXMLPath = TestPath & “\XML_Template\CreditCardXML_Resp.XML”

#And then we calls the reusable functions to update and sent the API on the URL

strUpdatedXML =  fnUpdateXML(srcLinkBXMLPath,strUpdate)

 

#Function body to update the XML File

‘********************************************************************’FUNCTION HEADER

********************************************************************’Name: fnUpdateXML

‘Description: This function is for updating the existing XML file with the passed parameters

‘Input Parameter:  XML file name, String in “XMLtag=XMLValue” format

‘ObjectName : XML DOM Object

‘Created By: Kuldeep Sharma ‘*****************************************

Public Function fnUpdateXML(ByVal XMLNewFile, ByVal strUpdateString)

arrSplitNode = split(strUpdateString,”;”)

For i=0 to ubound(arrSplitNode)

arrValue = split(arrSplitNode(i),”=”)

Set xmlDoc = CreateObject( “Microsoft.XMLDOM” )

xmlDoc.load(XMLNewFile)

Set objNodeList = xmlDoc.getElementsByTagName(arrValue(0))

set str = objNodeList.Item(0)

str.text = arrValue(1)

strUpdateXML = xmlDoc.text

xmlDoc.Save(XMLNewFile)

Next

Set xmlDoc = nothing

fnUpdateXML = strUpdateXML

End Function

 

# We can send this request to the URL with the XMLHTTP Objects

Response = SendXML(srcLinkBXMLPath,destLinkBXMLPath,LinkBUrl)

‘********************************************************************************* ‘FUNCTION HEADER

‘*********************************************************************************

‘Name: SendXML

‘Description: This function is to send the XML to the specified server

‘Input Parameter:  source XML file name, Destination XML file name, Server URL

‘ObjectName : XML DOM Object, XML HTTP Object

‘Created By: Kuldeep Sharma ‘*********************************************************************

Set xmlhttp = CreateObject(“Microsoft.XMLHTTP”)

xmlhttp.open “POST” ,url, False

Set xmldom = CreateObject(“Microsoft.XMLDOM”)

xmldom.load(srcXMLFile)

xmlhttp.send xmldom

xmlhttp.ResponseXML.Save(destXMLFile)

resp = FetchXML(destXMLFile, “SeverityCode”)

If resp = “0″ Then

SendXML = True

Else

SendXML = False

End If

Set xmlhttp = nothing

Set xmldom =nothing

End Function

”*******************************************************************

Function FetchXML is to ready the API Response if it is Passed

”*********************************************************************’FUNCTION HEADER ‘**********************************************************************

‘Name: FetchXML

‘Description: This function is for fetching XML tag value

‘Input Parameter:  XML file name, XML tag name

‘ObjectName : XML DOM Object

‘********************************************************

Public Function FetchXML(ByVal XMLNewFile, ByVal XMLTag)

On Error Resume next

Set xmlDoc1 = CreateObject( “Microsoft.XMLDOM” )

xmlDoc1.load(XMLNewFile)

Set objNodeList = xmlDoc1.getElementsByTagName(XMLTag)

set str = objNodeList.Item(0)

XMLTagValue = str.text

FetchXML = XMLTagValue

Set xmlDoc1 =nothing

End Function

‘****************************************************************************

Features & Benefits

Features

  • Factory Model Framework
  • Unique Solution of its Kind
  • Error Handling on code and functional level
  • Plug and Play framework (Completely Modular)
  • User Friendly

Benefits

  • High Effort Saving.
  • Less Manual intervention ensures less probability of error
  • Less dependency on UI
  • 50 % of effort saving.

 

VN:F [1.9.10_1130]
Rating: 0.0/10 (0 votes cast)

  

{ 0 comments }

Taking Testing to the Cloud

NASA decides to embrace cloud based load testing

This might be a stale topic to blog about, but I want to touch upon how NASA’s site was flocked last week to watch the landing of Mars rover Curiosity. What did NASA do to make sure it could handle the traffic and keep the stream going? It had to ensure that its application stack can handle this load without any interruptions.   It decided to partner with SOASTA , pioneers in the cloud based load testing service.

And my focus has been to see how cloud based testing can be easily leveraged for mobile applications which would help to circumvent some of the challenges currently experienced in the mobile space.

Introducing cloud based testing for mobile

The premise is based on using virtual servers to simulate real-world load on applications and test for various what if scenarios on several platforms – smartphones, tablets, social media sites the list is endless.  Cloud based testing has altered the testing game drastically by reducing the time to test, increasing the scale of test from hundreds to millions of virtual machines and slashing the cost of testing by removing the manual element out of testing.

Taking testing to cloud

The mobile market although lucrative, is extremely fragmented and dynamic.  There is a slew of devices – phones and tablets, OS’s, different versions of OS’S and zero standardization. Maintaining application quality in this kind of an environment is very challenging.

With cloud based testing, there is a web-based access to a huge number of devices that are connected to live networks that are spread across different geo-locations. This provides enterprises complete control over all their testing needs – manual and automated. Automated tests can be run on multiple devices, and can easily scale to support all types of intensive testing and the SaaS approach ensures that users always have access to the latest version of devices, platforms and OS’s.

There are huge cost savings with this approach obviously, because the ‘dedicated infrastructure and resources’ do not need to exist anymore. And not to mention the fringe benefits like on-demand flexibility, greater collaboration and reduced time to market for key applications.

The flip side

There are still a lot of issues and questions related to lack of standards and security practices especially in the public cloud.  Current encryption techniques are not considered sufficient. Some of the SLA, terms and conditions are sometimes difficult to comprehend. And of course, since public clouds are shared by multiple users, there might be some latency issues or disruption of service issues due to network outage.

Inspite of this, embracing the cloud especially for testing seems to be a no-brainer. It is very well suited from resource flexibility (commissioning and decommissioning) and security concerns point of view. So, it is time to take first-mover advantages which outweigh the risks significantly.

Next, we’ll talk about an execution strategy that needs to be in place before venturing into cloud based testing.

 

 

VN:F [1.9.10_1130]
Rating: 0.0/10 (0 votes cast)

  

{ 0 comments }

Automated Testing: Only for Happy Path !!!

Testing tools facilitate automate repetitive tasks with the use of software in tests execution and comparing actual and predicted outcomes. Automated testing helps in increasing the effectiveness, efficiency and coverage of software testing. Regression testing ensures that system do tasks for all the positive conditions and generate desired outcome. Now question arises does automated testing should perform only for Happy Path or negative scenarios should also share some space.

There are two schools of thoughts. One suggests that writing specific scripts to test negative conditions should not include in automated testing unless requested. Since exception handling usually covers all the negative scenarios before regression and it can be tackle manually. Inclusion of only positive conditions helps in bring down TCO even with fairly efficient system.

Second school of thought strongly recommends inclusion of negative scenarios in test automation. Negative tests are valid to automate as it is vital to make certain that application handles negative conditions during regression testing similar to positive conditions. However, such scenarios must be added if there is a risk that the system is incapable of error handling under some conditions. Practically there is lot of factors like budget, resource, time, and licenses etc. associated with development, so it’s always better to comprise such negative test cases which may cause or bring in a disastrous collapse of the application.

Automated testing is a big investment, and as company by now invested in automated testing tools to reduced long term costs, so it’s always recommend to include all test cases (positive or negative) which are repetitive in nature to make process efficient. All benefits of automation like it can be run simultaneously on different machines thus decreasing testing time, helps performing “compatibility testing” – testing the software on different configurations (also see Compatibility Testing – An Indispensable element of Testing) etc. will be leverage using automation.

If company have ample budget, resources and if team possess good skill set than one must go for testing all positive and negative test cases but it’s a staggering task in itself considering real life challenges. There is only one reason to include negative test cases is to minimize probability of crashing an application which could happen if testers will miss errors/faults in application.

Get more details on our Compatibility Testing.

VN:F [1.9.10_1130]
Rating: 5.5/10 (2 votes cast)

  

{ 1 comment }

Integrating Selenium tests with Hudson CI

For most folks the challenge is that their CI system lives in some Unix environment and Selenium tests need to run on a Windows machine.To get this working I did the following

My goal was to

a)Have the test drive and report selenium tests from a single interface

b)Ability to run it on different machines with various browsers

I picked Hudson for a) for the simple reason that it’s widely used in our organization as well as it had a Selenium Grid plugin which will probably be used for accomplishing b).It’s easily available and can display JUnit reports in graphical format with drilldown capabilities.In short the works and then some:)

Tools used are :Hudson,Selenium RC 1.0.1,FF 3.5,Ant 1.7,Selenium Grid plugin for hudson,Java,TestNG,SVN,Windows XP,Hudson JUnit plugin

Note:Selenium Grid is only introduced to run the tests on multiple machines and different browsers.If this is what you don’t need,you can avoid one more layer of complexity

Step 1

This step is crucial and can differ from org to org,but if your hudson master and slave are in same subnet this should generally work.Make sure of this by pinging each host to other.If not you might need to get help from your IT such that the master hudson can ping the slave hudson and viceversa.

Step 2 (skip if not running on multiple machines)

Install Selenium Grid plugin from Manage Hudson->Plugins->Install

Hudson restart will be required

Step 3

Create Hudson slave using following method

a)Go to Manage Hudson->Manage Nodes->Add Node(Dumb Slave)

b) Enter in Name field anything you wish.However hudson recommends giving same name as the host name of slave.So give the computer name of your windows machine where the tests will run

c)Num of executors is directly corresponding to number of RC’s you will run.For now keep it 1.If you want to run two browsers(e.g FF 3.5 and IE 7) this number needs to be bumped upto 2

d)Remote FS Root:c:\hudson (this is where hudson Workspace artifacts will be stored if needed).If you are going to use Selenium Grid plugin to run,you won’t need this

e)Labels:Give a unique label e.g(windowsxp_1).You will see it’s use later with Selenium Grid

f)Usage:Leave this machine for tied jobs only

g)Launch slave agents via JNLP

h)Availability keep this machines as much online as possible

Rest of the options keep blank.

Repeat the process for each Slave machine,but to keep things simple let’s just configure 1 machine at a time

Step 4

Go to the Slave machine and

a)Open your browser and point to Hudson Master

b)Go to the Manage Nodes page and you should see a launch JNLP jar icon.Click on it and it should start the process of downloading “remoting.jar” from Hudson Master(I had trouble with this and found out that somehow hudson master didn’t have the jar file at a specific location.With the help of my Hudson admin,I was able to copy this jar correctly).So if you repeatedly fail to launch the slave agent,first thing to check would be the location of remoting.jar.Also look at the java exception and it will give you clues in what could have gone wrong.

c)If all goes well you should see a small window with Hudson icon showing the message “Connected”.

The above step is very important and needs to happen in order to Hudson run selenium tests.If this doesn’t work look into hudson logs,slave logs found on hudson server and there should be some clue on what’s going on.

Step 5

Create a new job on hudson and the configure the option to “run it on the slave node” that you just created.

Publish the Junit report by pointing to the output directory xml file.This will be something like c:\hudson\workspace\xx\xx.xml

Depending on what your hudson job is doing the slave machine should have appropriate software.E.g if you are going to run ANT files,the slave machine needs to have ANT properly configured.Same thing goes with SVN.Essentially everything that your job would need to run on master should be available on slave as well.

Step 6

Start a selenium server on the slave machine as you would if it was your local machine.For this testing make sure that versions of browser that you used for your environment to run selenium test locally is same.This will eliminate unnecessary issues that are not due to this setup but browser configuration issues.

Step 7

Run the hudson job from your master and you should see the browser window popping up on the slave and running the tests.

The above should accomplish the task of running your Selenium tests on a single Windows machine.However many times you would need to run your tests in parallel as well as to optimize hardware use,make sure that Hudson can trigger test on same machine with different browser and report results back.In order to accomplish this we will use the Selenium Grid plugin which I will cover in next part if folks are interested.

Some resources that were helpful to me in this exercise were

Amit Easow’s writeup on Hudson integration


VN:D [1.9.10_1130]
Rating: 8.6/10 (22 votes cast)

  

{ 20 comments }