Today, I want to talk a little bit more about what exploratory testing is, and especially how you can do exploratory testing with the new Microsoft Test Manager 11.
Last month, I followed an ISTQB training where I got some lessons about Fundamentals of testing, Test design techniques and Test management. One part of the course was about exploratory testing, why use it, and mostly why not use it. I didn’t really agree about the fact that they discourage the use of exploratory testing, and definitely not agree if you use MTM11 to do exploratory testing. I’ll show you why.
What is exploratory testing?
According to Wikipedia:
Exploratory testing seeks to find out how the software actually works, and to ask questions about how it will handle difficult and easy cases. The quality of the testing is dependent on the tester’s skill of inventing test cases and finding defects. The more the tester knows about the product and different test methods, the better the testing will be.
Exploratory testing is all about simultaneous learning, test design and test execution. It’s a powerful and a fun approach to testing as you don’t have to follow a formal test plan, and you can use the system like an end customer should do. Like Anu Bharadwaj said in her ALM summit session, exploratory testing is about focusing on customer and business value. You have to ask yourself the questions: Does the software do what I want it to do? Does it act like it has to act? Those are the things a customer will also look at. They are not interested in how many bugs are fixed, which scripts you use,..
Microsoft Test Manager 11
Microsoft Test Manager 11 is the new to be released version in the ALM suite of Microsoft. MTM 11 will allow you to run exploratory tests on your software. The nice thing is that exploratory test sessions might be associated with user stories. That way, you can verify that the user story behaves correctly. It also provides an easy way to create manual test cases based on the action steps during your ET session. If you create a manual test based on the action steps that are recorded, the manual test case is automatically associated with your user story and all bugs that you file during this session will also be automatically linked to that story. The bug will also have some rich diagnostics attached.
Starting an exploratory test session
In this example, let’s start with creating new test plan in MTM
- Open the application and connect to your TFS11 environment (for tfs preview, use https://name.tfspreview.com , without the port 8080)
- In the testing center, click ‘Add’ and enter a name.
- Select the new plan and click ‘Select Plan’
- You’re now connected to the Exploratory Testing test plan.
- Now you can go to the ‘test’ tab and select the ‘Do Exploratory Testing’ functionality
- There you see an overview of all the work items that can be tested. You can also select ‘explore’ to start without a link to a work item.
Executing an exploratory test
- Click on the work item you want to test, and click the green arrow ‘Explore work item’
- You should see the MTM test-bar on the left side of the screen.
- If you want, you can enable Video and Audio recording by clicking on the settings button on the bottom-right corner and select the checkbox for video/audio recording
- Let’s start the test by clicking ‘Start’
- Now you can do your ‘test’ .. the one I selected was about getting an overview of the available session on the techdays 2012 website.
- We open IE, enter the URL
- When the website is loaded, we want to create a partial screenshot of the header
- Click on the ‘screenshot’ button.
- We select the piece we want
- Now you see the screen capture in the description box of the test-bar
- Double-click on the image, and Paint should open
- In Paint, you can add whatever you want to the screenshot
- When you are ready, just save it, and close Paint
- You see that the screenshot in your editor has the changes you just made
- Now you can continue with your test and do/test whatever you want
- Save your session by clicking on ‘End testing’
- You now see an overview of your test
Create test case from your exploratory test
In MTM11, there is a possibility to file a bug or create a test case while you are running your exploratory test.
- Let’s start a new session, not linked to any work item. Click on the ‘explore’ button
- Start your test and do what you want to do
- When you find a bug, click on the ‘Create bug button’
- You see a new ‘bug’ window
- The steps to reproduce screen is completed with all your steps
- You can also change the steps to be included in the bug by clicking on ‘Change steps’
- In the new dialog, move the slider to change the visible actions
- You can now save your bug, or click on ‘Save and create test’
- When you click on that, a new test case dialog opens. The steps from the test case are also completed
- You can save this test case.
- When you finish your current test session, you can see that there is one bug and one test case linked to this test.
Now you got an overview of how you can define and run exploratory tests with MTM11. In a next post, I will talk a little bit more about session based test management and testing tours. Thanks for reading!
Note: I also found a little issue when using MTM11 in combination with tfspreview. When you add some screenshots to your test, you see a ‘missing image’ error when you look at the test afterwards. More info here
