Cognizant 20-20 InsightsThe Case for Agile TestingBy applying principles of Agile software development, QA teams can helpbake in code quality as applications are being developed, ensuring morestable and business-aligned outcomes.Executive SummaryFor a variety of reasons (time, money, skills, etc.),the testing phase of software development sometimes gets short shrift from developers and ITmanagers. Yet testing is the only way to determinethe quality of software and whether an application will function properly in real time. It plays animportant role in the software development lifecycle (SDLC). Without an effective testing strategy,companies sometimes blindly take on significantrisks that may go well beyond simply havingpoorly functioning software. Organizational ramifications can be dire, including the risk of: Customer dissatisfaction.Brand injury.Competitive threat.Product revenue loss.At the development level, an oversight in testingcan trigger countless technical and projectoutcome challenges: Cost overruns.Poor resource estimations.Schedule overruns.Overlooked functionalities and scenariostriggered by poor developer/tester communication, resulting in compromised quality.cognizant 20-20 insights june 2014Agile testing mitigates these risks and ensuresthat IT organizations achieve the technical andbusiness goals of the development process. Inthis white paper, we provide an overview of howAgile testing works, why it is important, and thechallenges it poses.Agile Testing DefinedAgile testing is a software testing practice thatfollows the principles of Agile software development. Agile development integrates testing intothe development process, rather than keepingit as a separate and distinct SDLC phase. Agiletesting involves a cross-functional Agile teamactively relying on the special expertise contributed by testers.Agile teams use a “whole-team” approach to“bake in quality” to the software product. Becausetesting occurs in real time, this approach allowsthe team to collaborate actively with the development team, helping them identify and transferissues into executable specifications that guidecoding. Before the start of testing, the development team’s detailed design documents must beapproved so that test case creation can start. Bothcoding and testing are performed incrementallyand iteratively in Sprints (or iterations), buildingeach feature until it delivers enough stability andadds quality to the product (see Figure 1).
A More Flexible Approach to Software DevelopmentOffshore QA TeamOnsiteDevelopment TeamDesignExecuteWrapUser ReviewTechnology &Common ToolsVirtual QAAgile LifecycleUser ReviewSmallReleaseIteration 1User ReviewSmallReleaseSmallReleasePilotLaunchIteration 2ProductionLaunchFigure 1Sprinting AheadIn the Agile methodology, a Sprint is a set periodof time during which specific work must becompleted and made ready for review.Each Sprint begins with a planning meeting.During the meeting, the product owner (whorequested the work) and the development andtesting teams agree upon exactly which activitieswill be accomplished during the Sprint. The development team has the final say when it comes todetermining how much work can realistically beaccomplished during the Sprint, and the productowner has the final say on what criteria need tobe met for the work to be approved and accepted.The duration of a Sprint is determined by theScrum master, the team’s facilitator. Once theteam reaches a consensus on how many days aSprint should last, all future Sprints should be thesame. Traditionally, a Sprint lasts 30 days.After a Sprint begins, the product owner muststep back and let the development team doits work. During the Sprint, the team holds adaily stand-up meeting to discuss progress andbrainstorm solutions to challenges. The projectowner may attend these meetings as an observerbut is not allowed to participate unless it is toanswer questions. The project owner may notmake requests for changes during a Sprint andonly the Scrum master or project manager hasthe power to interrupt or stop the Sprint.cognizant 20-20 insightsAt the end of the Sprint, the team presents itscompleted work to the project owner and theproject owner uses the criteria established at theSprint planning meeting to either accept or rejectthe work.The Mechanics of Agile TestingThe combined team, including both developmentand testing, takes responsibility for analyzing thebusiness specifications. Together, they define theSprint goal.The QA team defines the testing scope (i.e., testplan). That is then validated and approved by theentire team and the client.Simultaneously, while the development teamstarts the implementation of modules (in thevery first Sprint), the QA team begins work on thetest case design. These are properly documentedeither in a testing tool or in an Excel spreadsheetthat is handed over to the development team andproject sponsor from the business side to review.This is to ensure that test coverage is as completeas possible.Once the test case review and any modificationsare completed for a particular Sprint, the QAteam then begins testing on the QA environment.Defects found during testing are logged properlyin a defect tracking tool. Depending on theseverity and priority of defects, fixing them canbe delayed but then is taken care of in upcoming2
Quick TakeAn Agile Principles ReviewThe Agile Manifesto1 is based on 12 principles: Face-to-face conversation is the best form ofcommunication (colocation). Customer satisfaction by rapid delivery ofuseful software. Welcome changes in requirements, even latein development.Projects are built around motivated individuals, who should be trusted. Working software is delivered frequently(weeks rather than months).Continuous attention to technical excellenceand good design. Working software is the principal measure ofprogress.Simplicity – the art of maximizing the amountof work not done – is essential.Self-organizing teams. Sustainable development – the ability tomaintain a constant pace. Close, daily cooperation between business professionals and developers.Sprints. At the end of each Sprint, the teamdetermines, along with the project sponsor, whichdefects are to be fixed in the current iteration.This iteration continues until all planned Sprintsare completed.QA, along with the development team andbusiness organization, defines which main flows(test cases) will be automated.When code is ready to test (after the end of eachSprint), QA works with development to executetest cases on the development environment,in order to identify the early stage defects sodevelopers can fix them during the next round,on a priority basis. This process is then repeatedthroughout the development process. Automatedtest cases are run daily throughout the SDLC.White-board/stand-up meetings are conducteddaily involving members of all teams associatedwith product development, support and testing.This helps to resolve the issues faced by teammembers and provides a clear picture of progressin both the coding and testing areas. Agilepromotes the introduction of requirements atall stages/iterations of the SDLC; however, thetesting team determines when to end this processto ensure product stability (see Figure 2).cognizant 20-20 insightsRegular adaptation to changing circumstances.Manual Testing: RegressionThe manual testing team plays an important rolein determining product quality.Once the majority of coding is completed (i.e., inthe last planned Sprint), the manual regressiontest cycle starts. This is key since it helps todetermine the stability of the application beforeit is pushed to preproduction deployment. Codeand defect fixes continue during this phase, butthis is to add value to the product. Preproductiondeployment is dependent on the approval of thetest manager’s testing report; deployment onlyoccurs if the report depicts green status for allapplication modules.Although a dedicated testing team exists in theAgile environment, unit testing by an individualdeveloper is beneficial to find early defects andminimize the rework of both the testing anddevelopment team.Automated Testing: RegressionAutomation is a critical component of Agiletesting. It would otherwise be impossible tokeep pace with the Agile development schedule.Automation is also used to run regression testing.The combined team (developers, product ownersand testers) usually predetermines, at the start3
Development TeamTypical QA Approach for Agile ProjectsInceptionIterative Sprints-Based TestingSprint 1Understandproject scopeeffort stRegressionExecutionReportTest AutomationWorkingproduct forreviewand feedbackProduct Product ProductafterafterafterSprint 1 Sprint 2 Sprint NSprint 2Product backlogUnderstanduser uirementsunderstandingdocument(To evolve over eportTest AutomationWorkingproduct for reviewand feedbackSprint NPrepare test planTest PlanIncrementallydevelop and testthe final rateUATTestTestRegressionExecutionReportTest AutomationSprint RegressionTest ExecutionTest ReportTest AutomationBacklogTest ClosureTest FigureExecution2Final ProductTest ReportTest Automationof the project, which parts of the software will betested using automation.Continuous integration/builds, unit, functional andintegration test execution as well as continuousor automated deployment are common areaswhere automation may work better than traditional tests.The entire project team agrees up front aboutwhich of the main flows will be automated.Automated tests consist of unit tests, capableof verifying even the most minute segment ofsoftware. Automation is required primarily todetermine the stability of modules developed ineach Sprint. Any defect found is reported andfixed, based on its priority. Promote the importance of testers andencourage continuous feedback sharing withthe programmers and the customer. Testers actively participate in meetings todefine the main business flows. Testers work on short iteration activitiesalongside developers. Encourage traceability between the requirements, test cases and bugs. Testers contribute to user Story improvements. Leverage automation testing as a key way to doregression testing.Leverage the specialized skills of test-drivendevelopment, including unit testing, continuousintegration and unit level.Furthermore, it is possible to execute the test setmultiple times per day, per hour or even morefrequently if needed.Agile Testing AdvantagesBest Practices in Agile Testing Testing requirements are discussed and refinedas a team (during stand-ups/Scrums) allowingthe combined team to better address thebusiness/technical aspects of the requirement.This enables overall alignment and preventsmisunderstandings. The Agile process often requires having entryand exit criteria for Stories (a compression ofThe benefits of Agile testing include:Effective Agile projects generally address thefollowing important elements: Include testers and QA managers as membersof the Agile development team. Include testers as active contributors inplanning and requirements analysis.cognizant 20-20 insights4
things to do in a particular release/iteration).Agile testing ensures that each requirement iswell defined and measurable, allowing QA specialists to determine whether the requirementwas actually completed or not. Addition of new user Stories into the current QA participates in the big-picture requirements-writing stage, thus ensuring that testingestimates aren’t overlooked.Looking Forward: GettingStarted with Agile Automated tests are fullyimplement regression testing. Quality becomes the combined team’s responsibility, rather than just solely that of thetesting team. The entire team agrees to thetesting strategies, test cases and defects prioritization plan.leveragedtoiteration. QA should be included in the additionof the new user Story, to build up appropriatecommitments and estimations in order to avoidmisalignment and protracted timeframes.Kicking off an Agile testing project is easy. Whereto dive in depends on organizational maturityand skills, among other factors. Regardless ofhow your organization stacks up, the followingguidelines can be applied: Tackle things one day at a time. A good firststep might be to start having a daily stand-upmeeting.Agile Testing Disadvantages andMitigations Agile testing proves to be the best testing methodology only if the requirements are clear to theproject sponsors. If the big-picture requirementsare unclear, the details can become muddied. Fornew products, the software architecture normallytakes a path based on the initial requirements. Ifthe requirements frequently change (as allowedfor in Agile), the following scenarios can occur:Attack it incrementally. Now that the teamis communicating a little better, start breakingthings into iterations. Your goal is to developeach feature, unit test it, QA it and accept itwithin the two weeks. Plan just as much as you need to. The Scrummaster, product owner and each developer, QAand writer should attend (along with usability,architects, etc.). The goal is to enumerate whatwill be accomplished in the iteration as well aswhat the acceptance criteria are. Review what you’ve accomplished. End theiteration with a demo and a retrospective. Integrate frequently. Continuous integrationwill likely be key. Integrate at least daily. Stayas releasable as possible. Focus on what is most important. Anotherkey area (which takes a while to master) isbacklog, or the ranked list of features/Storiesthat the team will address. These are rankedby priority (most important first). This is key —even if in some cases the order between twoitems is arbitrary. At least once, the organization should iterate and reevaluate the rankings.Adding, removing and shuffling based onknowledge gained is a good thing. Theteam struggles to adapt to changesbecause significant effort has already goneinto the initial requirements development andtesting process. Challenges in estimations and sizing require-ments. Sometimes QA gets short shrift sinceit’s logically the last task in completing the userStory. Therefore, any delay in the prior development task risks impacting QA timelines. QA is sometimes prevented from executing atest case for the whole iteration, leaving theteam struggling to finish the task. Not asking the right questions. It is very dan-gerous for QA not to ask questions, especiallyat the point where the user Story is picked upfor implementation. Daily team meetings canavoid this e/the-agile-manifesto.cognizant 20-20 insights5
References http://en.wikipedia.org/wiki/Agile testing. development.Lisa Cripian & Janet Gregory, “Agile Testing: A Practical Guide for Testers and Agile Teams,”Pearson Education Ltd., December efinition/Scrum-sprint.About the AuthorNaziya Iqbal Sayed is an Associate within Cognizant’s Quality Excellence and Assurance Business Unit.Over the last three-plus years, Naziya has provided quality analysis for banking and financial servicesclients. She earned a B.E. in information technology from Dr.B.A.M. University, Aurangabad, Maharashtra. Naziya can be reached at [email protected] CognizantCognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered inTeaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industryand business process expertise, and a global, collaborative workforce that embodies the future of work. With over 75development and delivery centers worldwide and approximately 178,600 employees as of March 31, 2014, Cognizantis a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked amongthe top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us onTwitter: Cognizant.World HeadquartersEuropean HeadquartersIndia Operations Headquarters500 Frank W. Burr Blvd.Teaneck, NJ 07666 USAPhone: 1 201 801 0233Fax: 1 201 801 0243Toll Free: 1 888 937 3277Email: [email protected] Kingdom StreetPaddington CentralLondon W2 6BDPhone: 44 (0) 20 7297 7600Fax: 44 (0) 20 7121 0102Email: [email protected]#5/535, Old Mahabalipuram RoadOkkiyam Pettai, ThoraipakkamChennai, 600 096 IndiaPhone: 91 (0) 44 4209 6000Fax: 91 (0) 44 4209 6060Email: [email protected] Copyright 2014, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by anymeans, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein issubject to change without notice. All other trademarks mentioned herein are the property of their respective owners.
Agile testing is a software testing practice that follows the principles of Agile software develop-ment. Agile development integrates testing into the development process, rather than keeping it as a separate and distinct SDLC phase. Agile testing involves a cross-functional Agile te