From the monthly archives:

April 2010

There are functions that are better suited for outsourcing than others. Traits that define a good candidate for outsourcing are as follows:
1.    Lack of ambiguity, clarity of specifications
2.    Difficult to hire and retail skilled human resources
3.    Easy to measure metrics
4.    Low requirement for domain expertise
5.    An activity that spans a time frame (is not short lived)
Among different activities in the QA function, automation passes well on the above tests.
1.    Test automation typically is done for regression testing, which means that tests to be automated are defined well already, with minimal ambiguity.  This results in well-defined projects with clear scope, deliverables and activities.
2.    Depending on where you are, it could be moderately to extremely difficult to hire and retain automation resources, an outsourcer on the other hand can leverage economies of scale and also have a cost arbitrage in hiring, training and retention of resources. In countries like India and China, it is relatively simpler to hire computer science engineers and train them for automation. An outsourcing vendor specializing in automation is likely to have (or get one that has) significant out of the box goodies that enable faster ROI. Such a framework enables them to write tests faster, not spend time writing common utilities and reports.
3.    Just like reduced ambiguity, with automation, there is easy measurement and metrics. The unit of work is a small deliverable (a test case), so you can easily measure number of tests automated in a day, per person, tell how much effort is being spent in maintenance and finally arrive at ROI decisions.
4.    It is often said that QA testers have to have a lot more domain expertise than their development counterparts. Automation engineers are synonymous to developers. Once a functional test has been written automating it requires only technical skill.
5.    Automation on long running projects span as long as the project does, which typically is many man years. Therefore the saving and value from automation is sustained over this period, this resulting in strong ROI.

As usual, it is not always as rosy as it seems. While automation is a strong candidate, here are some caveats that will help streamline the process:
1.    Manual tests are not often the same as automation tests: You and more importantly your vendor needs to understand this well.  Verbatim automation of manual tests often leads to less maintainable tests that are difficult to analyze and find bugs with. Before you start automating, the vendor may need to refactor the tests
2.    Automation framework is a must: Enough has been said about frameworks so I will not repeat it, but please put effort in understanding what framework is being proposed and how it helps new test development and maintenance.
3.    Technical skill level: Vendor’s thought leadership, credibility, partner ecosystem, skill sets, training processes, lead consultants, management and measurement processes are all very important characteristics that should be evaluated.
4.    Performance driven contracts: Can you execute performance bound contracts with the vendor? The contract can be dependent on automation metrics and have penalties for things not in your control such as vendor staff turn over. For automation these work well due to lack of ambiguity and nature of the projects. Check out this presentation from prof. Kaner  (www.kaner.com/pdfs/qaioutsourcing.pdf)

VN:F [1.9.10_1130]
Rating: 6.1/10 (10 votes cast)

  

{ 3 comments }

10 FitNesse questions answered in a simple language

If you are like me and have heard about FitNesse but never got your head wrapped around what exactly FitNesse does and who should use it.Here are some FAQ’s that I think will be useful for anyone starting on using or thinking about FitNesse.

1)What is FitNesse?

While here’s the official description of FitNesse,from testing perspective,it’s a automation tool that fits somewhere between unit testing and UI testing(manual or automation)

2)Why should I think about using FitNesse.?

If you are looking for a simple way to get quick feedback of the quality of your software,FitNesse can be of tremendous use.If your organization aims for quicker releases FitNesse can be an important tool along with unit testing and UI automation testing

3)Who should use FitNesse?

As the one minute description of  FitNesse suggests it’s for everyone.But Testers can use it to automate acceptance testing as well as test individual components.

4)What language do I write my tests in?

While initially this takes some time to get used to.The tests have to be written in Wiki language.Personally this is not the perfect way to write testcases,if you buy into the basic purpose of FitNesse you will understand the logic behind having Wiki as the language for test case writing

5)I looked at the documentation.What is this Slim and Fit business?

Here’s the official explanation through a diagram,but in simple terms they are two separate systems of writing FitNesse tests.If you are starting fresh,it’s recommended to go with Slim(However there is no complete escaping from Fit,the reasons for it will be discussed later)

6)What are Fixtures.Sounds intimidating.

Not to worry.Fixtures are nothing but simple Java classes that expose the applications/component’s functionality.

7)What is Table Styles.Is it necessary?

Yes it is necessary to understand and know what each of the Slim Table styles are.The testcases have to be written keeping this table styles in mind and it will also determine the Fixture code (class name,method name etc)

8)Where do I start?

Strongly recommend to read the official documentation.Because of it’s Wiki nature,it can get hard to have a linear narrative,but few hours spent and you will understand what’s important for you.Also read on Brett’s excellent tutorials on FitNesse.

9)Can I integrate FitNesse tests in my continuous integration cycle?

Ofcourse you can.Look at the FitNesse command line testrunner.There are also plugins available for Hudson.So check out your favorite CI and FitNesse might be available as a plugin

10)Give me the bottomline.What do I lose if I don’t use FitNesse?

Use FitNesse to fail fast.Discover and fix bugs earlier before it propagates to the UI.The learning curve is  a steep one,but the rewards do pay off handsomely once the basic structure is in place.Like any good tool you need to go through the pains of learning and optimizing for your organization’s needs.

There is much more to FitNesse than what’s mentioned above.To know more add your questions here or on twitter @infostretch.

VN:F [1.9.10_1130]
Rating: 5.5/10 (2 votes cast)

  

{ 1 comment }

Apple launched the most awaited and exciting update to its iPhone software – iPhone OS 4.0. Steve Jobs unveiled the new 100+ features to enhance the iPhone Experience.

iPhone OS 4.0
InfoStretch‘s Research Lab is impressed with the following cool features:

1. Multi-Tasking – This was one of the most sought after feature and Apple finally delivers ! Apple claims to multi-task activities without draining the batter power.

2. Enterprise – Apple iPhone wants to be a enterprise phone as well and challenge Blackberry. Some of the new Enterprise features include:
– Even better data protection
– Mobile Device Management
– Wireless app distribution
– Multiple Exchange accounts
– Exchange Server 2010
– SSL VPN support

iPhone OS 4.0 Features

3. iAd – Apple has introduced its own advertising platform – an ad plugin for app developers called iAd. People will watch ads while they continue to use iPhone or play games. It is definitely a vital tool and can challenge AdMob. The powerful advertising platform will 60% revenues to developers.

4. Game Center – Apple iPhone wants to be as cool as XBox as well. A lot of focus has been given to Game Center and features like inviting friends to play, competing with random players is available.

There are lot of other features including a new Mail app, ability to create folders, iBooks etc. It has also made the developer community happy by offering a new SDK with more than 1500 apps for creating powerful interesting apps.

The Smartphone market is hotting up !

VN:F [1.9.10_1130]
Rating: 8.7/10 (3 votes cast)

  

{ 3 comments }

In Second part of this article, we had discussion about testing methodology and how browser simulation will work. In this article, I will focus on some of browser simulation tools for IE and Firefox

Tools to simulate for IE

For browser simulation, there are couples of tools available. I compared two tools for Internet explorer:

1. Bayden UA Pick
2. Fiddler

Bayden UA Pick:

This tool accepts user-agent string and change behavior of your browser to given mobile device.
Installation and configuration for bayden UA Pick is very simple and straightforward. Setting user-agent and applying it to browser will change the behavior to desired phone.

Below screenshot is an example of how easy it is to configure and use

As per second article, user-agent string is not always enough. Sometimes it is needed to apply more parameters (e.g. accept, profile). For this situation I searched a couple of tools and came across fiddler, which gives flexibility to pass more parameters.

Fiddler :

Many of you must know use of fiddler for tracking traffic or for performance testing. In this article you will see another use of fiddler to change behavior of your application.

This tool has flexibility to change user-agent, profile, accept to change the browser behavior. Fiddler gives flexibility to add different rules for modifying request and response.

Fiddler also provides inspection of headers for inspection once you run your test. Please see below screen for iPhone headers, filters and logs.

Latest version of fiddler gives pre-defined phone or browser behavior that you can change it in single click.

I also compared two more tools for firefox

1. Modify Headers
2. User agent switcher

Both of those tools come as add-ons to firefox.

Modify Headers


This will also give you flexibility to add header information. Easy installation and configuration makes it more powerful. It has flexibility to import/export headers for further use.

User agent switcher

Easy installation and configuration makes it more powerful. This by default gives you some phones as well as browser user agent. By single click you can turn your browser to phone behavior. One can store multiple phone profile and later they can click and change browser behavior to that profile.

One advantage of user agent switcher over Modify Headers is, in agent switcher you can create multiple phones, browser profile and later you can change it on single click.

Tools to simulate browser for devices provide powerful, flexible and a cost effective option to work with automated testing. This tools can be really useful for anyone interested in automating mobile devices testing along with their existing tools for GUI automation.

~ Additional contribution by Shrujal

VN:F [1.9.10_1130]
Rating: 5.8/10 (5 votes cast)

  

{ 3 comments }

Read Part 1 for the introduction to the mobile web application test automation.

Mobile browsers have come a long way from being very basic browsers that displayed only static browsers to the present day advanced browsers that are virtually as equipped as the regular browsers. Mobile Websites rendering full blown content came hand in hand with the advancement in Mobile Browser Technology and the same brought in demand for high end mobile browser testing. But it was very impractical to test such stuff on hundred of handheld devices. It was the need of the hour for tools that would simulate a mobile browser type environment on the regular browser. Of course there were simulator tools, but each OEM had their own tool with limited functionalities.

There are multiple approaches to test web application

1. Actual Device
- Testing on it could be more perfect and effective, but on the other side, you must need to have large number of device lab or need subscription for this lab.
- It is also time consuming to check your test on each and every device manually.
One can perform automation for all those device and run script but you need subscription for those device lab for that.

2. Simulation
- In this case, you treat your browser as device and apply simulation for different phone
- This become cost-effective approach
- On the the other hand, this is simulation and not actual device, sometimes ajax, javascript which made to work for phone will not work for browser simulation

Let’s look at simulation approach.The question is how would simulation work. How would a web browser camouflage itself to appear as a mobile browser. The answer to that is user-agent headers/profile.

Each mobile device contains certain information which identify one phone behavior from other. That unique information is mentioned and explained below.

1. Accept : Contains list of MIME types accepted by user agent
2. User-Agent: Contains information used to identify device and detailed information for that device. Below is example of user-agent string and each of its artifacts.

A sample iPhone User Agent String and its understanding is mentioned below.

Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3

Mozilla                      :  It’s a Mozilla based browser
5.0                            :  The version of the browser
iPhone                      :  Apple iPhone
U                               :  Security Value. U denotes Strong Security
CPU like Mac OSX     :  Operation System iPhone OS based on Mac OS X
en                              :  Localized language for the Browser English
Apple Webkit            :  A set of core classes to display web content
420                           :  Version of the Webkit build
KHTML                     :  KHTML Engine like Gecko
Version                    :  Safari Version 3.0
Mobile                      :  Mobile Device
1A543                      :  a iPhone Firmware Version
Safari                       :  Name of the browser
419.3                        :  Browser Build

3. User-Agent Profile: User-Agent Profile is an xml document which describe device capabilities e.g screensize

All above user-agent headers and user-agent profile will change the browser behavior to mobile behavior.

Certain tools are available which changes user-agent header/Profile and simulate a similar environment on a different browser. Such tools help a lot in multiple environment mobile testing. Bayden UA Pick, Fiddler, etc are examples of such tools.

In next part of our article, you can find, detailed information about some simulation tools for IE and Firefox.

VN:F [1.9.10_1130]
Rating: 3.8/10 (5 votes cast)

  

{ 3 comments }

Today’s Internet users are more active on social networking websites such as FaceBook, MySpace, LinkedIn, Twitter and so on and this number is increasing exponentially. As these websites allows staying connected with colleagues, friends, clients to share ideas, activities, interests instantly across geographies, it has become a daily need for everyone. Apart from Internet, these social networking websites can be accessed by users using their feature phone/smartphone to remain connected, while on move.

Mostly all top leading mobile handset companies / OEMs offers native client apps preloaded to access these social networking websites.  Additionally, they do have their own mobile platform and/or development kits, helping developers to build all types of custom mobile apps including social networking service apps say Google Android, BlackBerry, Iphone etc.

All above are really good. But this year, Samsung has come up with a full fledged smartphone platform called Samsung Bada. It primarily focuses to deliver a true smartphone available for everyone by offering Rich user experience and applications. With Samsung’s TouchWiz user interface, it takes mobile user experience to next level.

In 2010, Samsung targets 1000 million full touch mobile phones (i.e 2.5 times than last year) and Samsung Bada will widely used in Samsung full touch mobile phones.

But what differentiate Samsung Bada with others, is its service oriented capabilities like social networking service, location based service, commerce service, device synchronization, content management etc, all supported by back-end Bada servers. With this, mobile developers can now implement these services to build service centric applications with ease. Social networking applications designed using Samsung Bada, can manage user profiles, including buddy information of application users. Additionally, it offers easy integration with widely popular social networking services.

Samsung’s partners demonstrated some cool and stunning applications at Mobile World Congress 2010, Barcelona, Spain. Attendees at MWC showed a great interest at Samsung booth.

It certainly looks Samsung Bada to be the best mobile platform for building service centric applications as of today. It’s yet to be seen, how Samsung Bada will withstand in 2010, the year of android as experts say.

VN:F [1.9.10_1130]
Rating: 7.0/10 (5 votes cast)

  

{ 7 comments }

Let’s say your organization has now a stable set of manual testcases that is used for regression every time your product releases. As you are aware this set keeps on increasing and soon you are seeing that the regression test cycle is weighing down on your product release timelines. In these days of Agile practices, the obvious next place you look at is to automate testing  such that regression cycle is shortened. This is not the only benefit of automation, but we will leave that discussion to some other time. Also for purpose of this discussion, we will concentrate on GUI automation tools. So if you are responsible for GUI automation, the very first question that will come up is

“What is the best GUI test automation tool for my application”?

If you had asked this question in any QA forum or just done a plain search, atleast one of the answer to above question would be Selenium. To keep this discussion in perspective, I would like to discuss merits/demerits of Selenium being the tool of choice.

Merits of using Selenium

–Choice of using language of your choice.

This is an advantage which has an indirect benefit of plugging into your engineering architecture with ease. For example if your organization is a Java shop, you already have a knowledge and technical expertise available with range of tools (e.g IDE, build systems, reporting tools etc.) that your development branch uses. It would be just a matter of transferring this know-how to your QA team which they might already be familiar with it.

–Script once, Run on multiple browsers

While this is not exactly true in simplest sense of speaking, the advantage of  running your tests on multiple browsers with the same set of scripts is an enormous advantage when you have to test your application on different browser version

–Inherent applicability to AJAX

Selenium works in browser, with the core technology that is driving your test case working along side your application under test (AUT). This provides incredible control over testing of AJAX web applications .

–A community supported Selenium Software suite

Selenium users can and should take advantage of  Selenium IDE,A Firefox plugin which assists you in recording and running your tests.Selenium Grid allows you to take your tests and deploy it in a clustered environment optimizing use of your resources. For details please refer to this Selenium documentation

—Support multiple test frameworks

Contrary to common thinking, selenium is not a testing tool. It is more like a GUI driving library.  This is a blessing.  Selenium allows users to wrap the test scripts in the framework of their  choice. So if you like JUnit vs TestNG,you have a choice and also taking advantage of reporting capabilities of this framework that your development team might be already using.

—Integration with the ecosystem

If your requirement is integration with different suite of tools like Hudson, QMetry etc,there are already solutions available out there which will make the integration easier.With almost no investment in purchasing tools,you can create a test automation infrastructure right on par with a commerical suite of tools. There are also multiple solutions available for you if you want to run tests in parallel. Check out selenium Grid and saucelabs

Having said all of the above,what are the areas where Selenium falls short

—Supporting non-web applications

Selenium is designed and growing keeping in mind that it would be used to automate web applications. So if you have a windows application or a Java applet or applications that use Flash, it’s not easy to use Selenium. However other tools also do not do a better job of supporting this applications. Technically there can be other solutions to take care of this shortcoming from selenium itself.

—Maintenance readiness is not inbuilt

If you look at commercial tools, there would always be a concept of storing UI object definitions separately from test code, test data management and other nice utilities built into the tool. In Selenium, this is left to individual’s implementation. While both approaches have it’s pros and cons, a well-written Selenium framework can go a long way in giving you a head start in writing Selenium test cases.

One last point…..

Often Selenium is offered as a low cost  (opensource)  alternative to commercial tools like QuickTest Pro or SilkTest. While this being true in terms of obtaining the actual software, i think it’s imperative to understand that since writing and maintaining various functionalities that would have been offered as part of the commercial tool package now falls on individuals, this cost should be taken into consideration while making the final decision

Where does all this information leave you in terms of answering the original question? The answer relies on number of factors, ranging from management goals to technical requirements but Selenium should be considered an important candidate along with the commercial tools which not only requires a huge upfront investment but also significant investment in learning a new suite of tools, a team dedicated to specific skillset etc.

Infostretch can help you determine whether Selenium is the right tool for you or not.Contact us at 408-727-1100 or follow us on twitter @infostretch

VN:D [1.9.10_1130]
Rating: 8.6/10 (32 votes cast)

  

{ 9 comments }

1. Communication at all levels:

    Challenge:

    • Communication plays a major role in any offshore software development and Testing and it is one of the biggest challenge if communication is not proper.

    My Solutions:

    • Effective Communication is the key to manage a project. Communication should be clear in between Product Management, Development and QA on what we are designing and what we are delivering.
    • Need to maintain good relationship with Customer, Project and Account Managers to know  the high level info about project progress.
    • The communication should be formulated as per the stakeholder needs and to manage the stakeholder expectation.
    • Communication should vertical , horizontal, with in the project and external to the project. There are various methods of communication(push, pull and interactive communication) and various tools are used in communication.
    • Last but not least, All peers should be on the same page w.r.t. goals, objective and beginning & ending of a product/project.

    2. Unrealistic Schedule:

      Challenge:

      • This is a worst thing and biggest problem in most of the STLC and it impacts on Quality of the product/project. Testers should get enough time to test the application.

      My Solution:

      • See the schedule and make a plan for deliverables, if time wont permits you to deliver then raise a flag and please don’t compromise here.
      • Make sure as everything is working as per plan or not, if not then take corrective actions rather than preventive.
      • The Responsibility assignment matrix should be used while assigning the task and identification of the resource.
      • The project risk should be identified continuously and throughout the project life cycle and the risk response owner should be identified while doing the risk planning.
      • It is always better to prevent the risk before coming otherwise it will increase the project cost and delays in completion of the project.

      3. Unclear Requirements / Changing Requirements:

        Challenge:

        • Most of the times client won’t give/share all the documents to QA Team, or sometimes requirements are not clear and changing it frequently.

        My Solution:

        • Client/BA should share all documents to QA team and should explain briefly about project and then only team will deliver quality output
        • Team should be clear on requirements, if not arrange a meeting and resolve it by discussing with BA/Client/Dev Lead.
        • Update the change requirements to team as it will cause many problems if team is not aware of it and should provide enough time to test the application for changed features and functionalities as well dependencies.
        • Requirement documentation and requirement mgmt plan should develop by the project team with the project lead

        4. Unclear Expectations / Outcome:

          Challenge:

          • Team should know the outcome of the product and met client expectations as well otherwise it will show big impact on end user expectations.

          My Solution:

          • Talk with the stakeholders of the project to collect the actual requirement to define the scope of the project.
          • Project objective is clear then requirement to execute the project should be identified by brainstorming and Delphi techniques and should develop the project charter and then the project mgmt plan
          • Always think as a end user while doing testing of an application and make a habit as why are you doing this project and outcome?
          • Tester should have ‘break the application attitude’ and found major issues in the early stage of the release.

          5. Utilization of Automation Tools:

            Challenge:

            • It is one of the most challenging stuff but definitely it will adds value to QA

            My Solution:

            • Need to add excellent experienced technical guys over here as experience and Should give a proper guidance and act like as a good mentor.

            Advantages:

            • It provides accuracy, reliable, faster, security as well better quality test results
            • Optimizes the process by reducing the turnaround time
            • It will gives you the better output and great ROI

            6. Delivering a product with good quality on time:

              Challenge:

              • If there is no proper plan in place then it is hard to deliver the product with in time with good quality.

              My Solution:

              • Understand the requirements properly and discuss with in team, Dev and Stake Holders to clear doubts/issues
              • Plan, Design and develop the test plan, test cases and test data … etc
              • Prepare Traceability Metrics  and Checklist for features coverage and dependency
              • Execute all test cases and make sure that we have covered 100% and re-verify it before delivering

              7. Managing Metrics to provide visibility to customer:

                Challenge:

                • It is little bit hard/difficult to manage, mainly in Product testing but it will gives a clear picture to everybody

                My Solution:

                • Identify no. of test cases identified/Automated and put it in Metrics w.r.t. Weekly & Yearly basis
                • Identify no. of bugs found  and put it in Metrics w.r.t. Weekly & Yearly basis
                • Track each resource achievements w.r.t. Weekly basis on Project and put it in Graphs
                • Prepare a Quarterly based report and give a small demo to Customer w.r.t achievements

                8. Focus more on Customer Business, domain and market:

                  Challenge:

                  • Need to be aware of customer’s customer and upgrade existing process or skills to ensure they meet their cusomter’s expectations.

                  My Solution:

                  • Add more Creativity and Innovation to existing work and go for Optimization
                  • Focus more on Customer needs and change the existing process
                  • Introduce new types of testing’s as per market trends
                  • Think ‘Out of the box’ and provides customer an optimal solution both technologically and meeting business needs.

                  9. Lack of Resources / Expertise’s:

                    Challenge:

                    • Experience and trained Resources would be ideal, if not then proper training programs should be in place.

                    My Solution:

                    • Add sufficient amount of resources to the project to balance work
                    • Sometimes resources might go on unplanned leaves or casual/personnel leaves, so it will show big impact on committed deliverables
                    • Need to create Training plans and Backup plans for resources/work/…etc
                    • Need to add expertise’s to the team w.r.t. domain knowledge, technologies, processes …etc

                    10.Last but not least, Internal Issues within the Team:

                    Challenge:

                    It will show big impact on Quality of Deliverables, need to resolve such type of issues as on high priority

                    My Solution:

                    • Communicate with the team and create friendly environment
                    • Motivate & Encourage team by giving more chances/responsibilities
                    • Provide development plans and Goals as objective
                    • Recognize excellence and Emphasize the balance

                    Please feel free to add a comment or suggestion, if any.

                    Thanks,

                    Baji Prasad CH

                    VN:F [1.9.10_1130]
                    Rating: 8.0/10 (12 votes cast)

                      

                    { 9 comments }