 |
 |
|
 |
 |
The handling of bug reports is often different in Agile projects (write them on a card; plan them
like any other potential change; etc.) It might be useful to tell a story of a bug as it goes from
being found to being fixed. It'd also be useful if the bug were inherently interesting, a
representative of a class of bugs useful to know about.
Although some of the "QA" testing is done in the dev environment and reported informally. Most
of the functional testing is done on deployed builds and reported using a bug tracking tool of one
type or another.
What about test cases?
An agile project team member gets involved with following categories of tests.
Unit tests- Developer facing tests : are written in executable format
Acceptance tests – Customer facing or business facing tests – written by testers and form
end to end business scenarios.
Test cases form the detailed requirements for a user story and use cases. The goal is always to
write test cases in EXECUTABLE format, meaning in a format usable by a test tool. If possible
avoid writing test cases twice.
For a typical medium-sized business application, the right unit test suite would be several
thousand test cases taking a few minutes to execute. Probably written in Java or C#. The right
system (acceptance) test suite for the same business application would be maybe a few dozen
end-to-end scenarios. Probably written in a higher-level language.
The right integration test suite would be somewhere in between, if at all necessary.
When and how UAT (User acceptance testing) happens
Iknow most agile projects don't do a single sitting of UAT because the customer is involved right
through the during thedevelopment cycle. In other words, UAT happens throughout the project.
However, if we have a very large system with many components we might needhave a “two”
week iteration at the end devoted to system testing. We sometimes have customers come in right
afterthatand sit with the testers to review results, do some of their own testing, etc.
From the measurement of quality standpoint, customer facing testing done by either customer or
any team that is separated from project team, the concept of traditional UAT be used. The bugs
found during such testing are considered as test escapes and need to be investigated upon to
see how to prevent them.
What happens to Test planning?
A Test plan is a design document of the test team. The test planning is an important task from the
high level view: what areas of features will be tested and how will resources be spent on creating
these tests. Unit tests should not be included in a test plan. However, the test plan should still
make it clear which functions are expected to be of high quality because unit tests were delivered,
and which functions will be tested by providing data, models and automation.
Working on traditional waterfall projects, typically it has been observed that nobody reads the test
plans anyway, and certainly nobody maintained them as changes occurred. High level test
cases/scenarios are written before or at the start of each iteration, then detailed test cases are
written as coding begins. Product owners and testers write the test cases, programmers
automate them, it's a collaborative effort.
When it is a client requirement to produce a test plan as a deliverable, spend minimum time on
this and mainly included the release plan, and refer the reader to the location of the actual test
cases.
What about Test Automation? Are current “state of the art” automation tools in-line with
agile principles?
Test automation/UI automation or automated regression tests as performed by current industry
standard– record and play back tools do not fall in line with Agile.
The trouble is these tools neither guide development (can't be done first) nor do they increase
agility (the tests are more brittle than the code).You can't record a script for a product that doesn't
exist yet!
So test automation in Agile terminologies – boils down to writing executable tests (both unit and
acceptance tests) Here you can write a test code for a piece of functionality that does not exist
yet and hence support Agility.
Traditional Test automation (read UI in most of the cases) assumes that requirements are nailed
down in advance, and that developers get the UI finished and locked down early. If the UI
changes, some capture/replay literature seem to say that this is a failing of the development
process, not that the tests are fragile.
Agile testing – role of tester
Be a customer’s representative with dev team and dev team’s representative with customer.
Help customer to think through the stated requirements, use cases and user stories.
Unfold user stories, ask intelligent questions, make sure that user stories are detailed and
exhaustive enough and Write acceptance tests.
Tester is no longer a sole quality advocate or final filter on quality– but shares the
responsibility with the whole team.
Help development team in executing unit tests
Work with developers to clarify the requirements where gaps exist caused by boundary
conditions and error conditions that were poorly understood or described in vague terms.
Offer feedback on the use cases to insure that the minimum success criteria is meaningful
and that all alternative paths are understood and meaningful
Help development team with develop features with minimum documentation that is available
and keep clarifying as iteration progresses.
Pair with the programmer to produce better software.
Support in customer facing testing– get feedback, engage development team and facilitate
constant communication.
A Typical Agile project – my experiences
This was project that we needed to build where in customer had a very vague idea of what they
wanted to build and were having problems in visualizing the whole thing. We quickly cobbled a
team of experienced developers, tester and project manager and started off what is known as
exploration into world of agile. Except one in the team for every body it is a new experience of
building something like this. We all were very committed and excited to make this project a
success.
We started off with some high-level design document that outlined core components and
suggested architecture. Both dev and test started banging off this stuff trying figure out what this
all about. We also had a set of use cases and user scenarios extracted from the users that depict
how the system could be used. Most of the project management and communication happened
using a scrum sheet that served as a single repository of requirements, development tasks and
dates of the code deliveries. The whole V1.0 project was broken down into 5-6 iteration of 2
weeks each with each iteration providing some set of features that could be demonstrated to the
user. We had daily meetings that allowed the whole team to sync up what that will done that day
and at the end of the to see whether we could achieve what we thought we could.
Scrum sheet contained a big laundry list of features/bug fixes and issues in the form of “Back log
sheet called “user Stories”. Each iteration started with a planning game or ceremony where whole
team gathers in a room with a prioritized list of stories that are lined up for that iteration. These
were pulled from backlog sheet based on priorities assigned to individual user stories. Each story
was broken down, unfolded with more details and there come the dev and test task that make up
iteration. Each team member that comes out with an estimate for completion of the task allocated
to them.
From project management’s perspective – it is SCRUM sheet all the way. Everybody at the end
of the day updated the tasks allocated to them by entering the time in hrs that task has left to be
done and completed task would have “0” as on that day’s time column. Depending upon the
progress made for a task, PM would then predict whether or not that piece of functionality would
make it iteration demo to customer.
There were customer demos at end of every iteration and customer was really excited about the
way we were delivering the stuff. There were changes at every iteration. Whole application was
built like a jig saw puzzle where we went on discovering different pieces of the app as we started
seeing the ready made piece. There was continuous communication in the team; every team
member knew everything about what is happening in the middle. Everybody contributed their
piece at every stage. There was little “complaining” from dev or test about changing requirements
as they were ready for it any stage of the project. So were the program managers – did not insist
on many mandatory sign off’s of the documents like, test plans, test cases. There was great
sense of faith in the capability of the team to deliver.
At every stage – the team demonstrated agile principles –
Test driven development
Simplified project management and communication.
Agility in response to changes.
Why not to go for Agile?
Requires a paradigm shift. As is the case with any changethere is resistance to change and
purists say agile is too ad-hoc. It tends to be more that way that when an agile team gets
stuck between agile v/s traditional.
In close knit teams where the functional barrier between Dev and test is fast thinning–
Testers runs into risk of loosing objectivity
Management blues/Budget issues – difficult to fund something that will be built over 5
iterations spread over 1-2 months without knowing which shape the end product takes.
There is still skeptism over the way agile team handles complex design and architecture
problems. It is difficult to convince management that Agile is the way to go and deliver such
complex applications in iterations. Traditional model handles such cases by building robust
and stable upfront design taking months of design and changes after design freeze are risky
and difficult to implement. There needs to be a change in the mind set to accept that even
these so called “Robust and stable designs” fall flat when it come to implementing something
that developer did not think of in the first place. This is something agile can handle.
Issues with continuous customer involvement – it is a bandwidth issue. As most of customer
feedback comes from group of selected end users and these people get involved with
feedback only as their part time work – they have their day’s work to do.
Road Ahead
Having seen what agile is about, what are challenges, how agile teams work, let us look what is
in stored for agilists?
Use Agile when -
Requirements are uncertain and volatile.
Start with teams that are smaller in size – Experiment and then try with bigger teams
As far possible, try with the teams that are geographically located at the same location.
Customer is easily accessible and likes the idea of agile dev.
Conclusion
Stable is the antithesis of agile. Agile makes it easier to change, because change IS a given.
This is precisely the problem that agile model is trying to solve. Agilists say “Requirement
changes– Welcome”.
In agile I see the distinction between tester and developer disintegrates as testing is another story
to be told. This future cross-role of developer-tester is difficult to articulate and develop and I
wonder where this path will lead. In an agile organization, does one have to choose between the
role of developer and the role of tester? It could be that one could play both roles as needed to
deliver what customer needs.
Every one in an agile team is playing roles that help the team to achieve the common goal of
delivering what customer wanted and at the same time not continuously complaining about “un
clear requirements”, “Changing requirements”, longer design and analysis cycles etc”.
Is this change for Good? If yes? for whom? Are we going back to historical days where there was
not dev test separation?
I would assert that history is not repeating. In the past, we used these practices in isolation,
without methodology or reinforcement. When projects started to fail, we were told that it was our
fault for not keeping an eye on quality, and then the solution was to increase project
management. Now, we are simply realizing that the solution was not better than the problem, but
that there is another way. Yes, we returned to our roots, but we then embarked on an entirely
different path. That is different. That is Agile. Welcome Agile.
Glossary
User story
It is a smallest unit of work that can be estimated for development and testing and one that
delivers a specific business value to the customer. The "story" is not a description of a feature in
a program, but the underlying real world problem that the software is designed to solve. An
important concept is that your project stakeholders write the user stories, not the developers.
User stories are simple enough that people can learn to write them in a few minutes, so it makes
sense that the domain experts (the stakeholders) write them. A user story carries a priority
assigned by the customer and has estimates. User stories should only provide enough detail to
make a reasonably low risk estimate of how long the story will take to implement. When the time
comes to implement the story developers will go to the customer and receive a detailed
description of the requirements face to face.
Iteration
An agile project is delivered in iterations which are nothing but small project cycles ranging from
two weeks to a month. Through the concept of iterations, project teams keep in pace with
customer requirements by planning the project iteration by iteration. Each iteration is pretty
independent and project team working on any given iteration is not aware of and does not bother
to know what is lined up for next iteration. In true agile style, a team member in agile team treats
every iteration as last one in the project so that team is ready to deliver few working code at the
end of each iteration.
SCRUM sheet
It is a single repository used by an agile team. It contains the details of iterations, backlog sheet –
list of user stories, project tasks. It acts as main source of information for project team – used
both as repository of features and project management tool giving details iteration dates etc.
Test driven development
It is one of the core practices of XP, where a developer developing a piece of functionality, writes
tests first before the code.
The steps:
Write a test that specifies a tiny bit of functionality
Ensure the test fails (you haven't built the functionality yet!)
Write only the code necessary to make the test pass
Refactor the code, ensuring that it has the simplest design possible for the functionality
built to date
The rules:
Test everything that can possibly break
Tests come first
All tests run at 100% all the time
Pair programming
Two programmers working side-by-side, collaborating on the same design, algorithm, code or
test. One programmer, the driver, has control of the keyboard/mouse and actively implements the
program. The other programmer, the observer, continuously observes the work of the driver to
identify tactical (syntactic, spelling, etc.) defects and also thinks strategically about the direction of
the work. On demand, the two programmers can brainstorm any challenging problem. Because
the two programmers periodically switch roles, they work together as equals to develop software.
SRUM Master
In the Scrum process, this is the person responsible for keeping the team following the process.
This is somewhat similar to a project manager role. However, because Scrum asks the team to
organize itself, this person may or may not be the perceived "leader" in a given Sprint.
The Scrum Master is the team member responsible to interface with the world outside the Scrum
iteration, and to remove/resolve blockages that impede the team's work within the Scrum
iteration.
Refactor
This is a coding activity where in a developer modifies or changes a piece of existing design or
code that works by removing redundancy, eliminating unused functionality, rejuvenate obsolete
designs. This helps to keep the code simple and tidy. The aim is to keep your code clean and
concise so it is easier to understand, modify, and extend. It ensures that everything is expressed
once and only once. In the end it takes less time to produce a system that is well groomed.
References
1.www.martinfowler.com/articles/newMethodology.html- “New methodology” by Martin
Flower
2.Extreme programming by Ron Jeffries
5.[Agile testing] – Yahoo group discussions http://groups.yahoo.com/group/agile-testing/
 |
|
 |
|