Sunday, June 3, 2007

Exploratory testing

"He is a skilled tester, hence he might be a good exploratory tester."

Exploratory software testing is a powerful and fun approach to testing .The plainest definition of exploratory testing is concurrent learning, design, execution and recording.Exploratory tests, unlike scripted tests, are not defined in advance and carried out precisely according to plan.

Exploratory testing is known as informal testing but it is formally informal method; analyze the outputs and give possible inputs to get that particular output. There are situations where exploratory testing is done without requirements document, rather test cases are written while exploratory testing and transformed into requirements document.

Exploratory testing is an approach that is extra suitable if requirements and specifications are incomplete, or if there is lack of time. This doesn’t mean that it cannot be used otherwise. It compliments or augment other; more formal testing. The main advantage of exploratory testing is that less preparation is needed, and finds serious bugs quickly which are not surfaced by the formal test cases. It is also known as “on the fly”.

The method can serve as a check on the test process, to help ensure that the most serious defects are found. It is common to perform a combination of exploratory and scripted testing where the choice is based on risk.

An example of exploratory testing in practice is Microsoft’s verification of windows compatibility.

Drawbacks:

1. Disadvantages are that the tests can't be reviewed in advance (and by that prevent errors in code and test cases), and that it can be difficult to show exactly which tests have been run.

2. When repeating exploratory tests, they will not be performed in the exact same manner, which can be an advantage if it is important to find new errors; or a disadvantage if it is more important to know that exact things are functional.

3. Exploratory tests are not the suitable candidates for automation.
While all exploratory testing is based on the knowledge of the tester.

The knowledge that the exploratory tester already has varies from person to person, just as with any sort of knowledge. Example, some testers have a great deal of familiarity with the application or type of application that they’re testing. Some testers base their tests on a sense of risk and their knowledge of how products can fail.

Bug taxonomies help the new and in experienced testers in doing exploratory testing efficiently.

Sources:

James Bach satisfice.com
Ashwin palaparthi
Cemkaner

2 comments:

Pradeep Soundararajan said...

@Ramani,

First, I am happy to encourage you to keep writing on testing. All that I plan to write below are not against you ( as an individual ) but your thoughts. If you are open to learnings, I am sure, you would love reading my comment.


Drawbacks:

The traditional mindset tries to draw a drawback of an approach. If dancing before the software can find bugs that appears to matter for some stake holders, it's a good idea to dance. It might not be a good idea to state : The drawback of dancing is, a) The tester gets tired b) The tester doesn't touches the computer...



1. Disadvantages are that the tests can't be reviewed in advance (and by that prevent errors in code and test cases), and that it can be difficult to show exactly which tests have been run.

James Bach and Michael Bolton's Rapid Software Testing teaches an important lesson: Think of cost v/s value when you are testing.

Assume, if I were to think of cost v/s value, that is: I think of the value that a test I am executing might produce and hence I do all the tests that I think add good value to my client.

To my knowledge, more than 97% of testers who review test case do not think cost v/s value, neither do they think of it when they write those test cases.

I think you should study about Session Based Test Management before you say "it is difficult to show what tests we ran"

There are FREE software that capture your work like Wink, have you tried those?

2. When repeating exploratory tests, they will not be performed in the exact same manner, which can be an advantage if it is important to find new errors; or a disadvantage if it is more important to know that exact things are functional.

Can you repeat a scripted test?

James Bach and Michael Bolton demonstrated the ambiguity in Scripted tests and you might want to listen to their conversation: http://www.developsense.com/audio/whatdoscriptstellus.mp3

Also, you might want to look at James Bach's mine field analogy.

3. Exploratory tests are not the suitable candidates for automation.
While all exploratory testing is based on the knowledge of the tester.


It's based on SKILLS of a tester to do a fantastic exploratory testing and that skills are hard to learn and practice and testers are lazy and not passionate to become a skilled tester.


The knowledge that the exploratory tester already has varies from person to person, just as with any sort of knowledge.

Again, it's not just knowledge, it's skill, too.

Example, some testers have a great deal of familiarity with the application or type of application that they’re testing. Some testers base their tests on a sense of risk and their knowledge of how products can fail.

Testing is never a one man show and there is nothing wrong if someone who has familiarity with an application tests it in his own way while others who are not so familiar also test in their own way.

Why not you wonder how the testers whom you claim to be the ones who are familiar got familiar with something?

Exploratory Testing - is simultaneous learning, test design and test execution - Dr Cem Kaner

There is a learning component associated and stressed and be careful with that.

Saying that - you or even someone who claims to do scripted testing does exploratory testing but not to an extent that you might see it as a dominant task you do.


“He is mad but there is method for his madness"—could be compared to exploratory testing.

I think you should consider saying:

"He is skilled and hence he might be a good exploratory tester"

I encourage you to read more literature on this and you did nothing wrong.

Look, finally I hope you have some good learning and interesting pointers.

ramani said...

Mrpradeep soundararajan,

Firstly thanks for encoraging an upcoming test engineer like me.Yes definitely I am open to learning and consider all the interesting points made.

@ramani