Alexander Vanwynsberghe

"There is nothing impossible to him who will try"

Exploratory testing with MTM 11

Today, I want to talk a little bit more about what exploratory testing is, and especially how you can do exploratory testing with the new Microsoft Test Manager 11.

Last month, I followed an ISTQB training where I got some lessons about Fundamentals of testing, Test design techniques and Test management. One part of the course was about exploratory testing, why use it, and mostly why not use it. I didn’t really agree about the fact that they discourage the use of exploratory testing, and definitely not agree if you use MTM11 to do exploratory testing. I’ll show you why.

What is exploratory testing?

According to Wikipedia:

Exploratory testing seeks to find out how the software actually works, and to ask questions about how it will handle difficult and easy cases. The quality of the testing is dependent on the tester’s skill of inventing test cases and finding defects. The more the tester knows about the product and different test methods, the better the testing will be.

Exploratory testing is all about simultaneous learning, test design and test execution. It’s a powerful and a fun approach to testing as you don’t have to follow a formal test plan, and you can use the system like an end customer should do. Like Anu Bharadwaj said in her ALM summit session, exploratory testing is about focusing on customer and business value. You have to ask yourself the questions: Does the software do what I want it to do? Does it act like it has to act? Those are the things a customer will also look at. They are not interested in how many bugs are fixed, which scripts you use,..

Logo

Microsoft Test Manager 11

Microsoft Test Manager 11 is the new to be released version in the ALM suite of Microsoft. MTM 11 will allow you to run exploratory tests on your software. The nice thing is that exploratory test sessions might be associated with user stories. That way, you can verify that the user story behaves correctly. It also provides an easy way to create manual test cases based on the action steps during your ET session. If you create a manual test based on the action steps that are recorded, the manual test case is automatically associated with your user story and all bugs that you file during this session will also be automatically linked to that story. The bug will also have some rich diagnostics attached.

Starting an exploratory test session

In this example, let’s start with creating new test plan in MTM

  • Open the application and connect to your TFS11 environment (for tfs preview, use https://name.tfspreview.com , without the port 8080)
  • In the testing center, click ‘Add’ and enter a name.
  • Select the new plan and click ‘Select Plan’
  • You’re now connected to the Exploratory Testing test plan.
  • Now you can go to the ‘test’ tab and select the ‘Do Exploratory Testing’ functionality
  • There you see an overview of all the work items that can be tested. You can also select ‘explore’ to start without a link to a work item.
01
02
03
04


Executing an exploratory test

  • Click on the work item you want to test, and click the green arrow ‘Explore work item’
  • You should see the MTM test-bar on the left side of the screen.
  • If you want, you can enable Video and Audio recording by clicking on the settings button on the bottom-right corner and select the checkbox for video/audio recording
  • Let’s start the test by clicking ‘Start’
  • Now you can do your ‘test’ .. the one I selected was about getting an overview of the available session on the techdays 2012 website.
  • We open IE, enter the URL
  • When the website is loaded, we want to create a partial screenshot of the header
  • Click on the ‘screenshot’ button.
  • We select the piece we want
  • Now you see the screen capture in the description box of the test-bar
  • Double-click on the image, and Paint should open
  • In Paint, you can add whatever you want to the screenshot
  • When you are ready, just save it, and close Paint
  • You see that the screenshot in your editor has the changes you just made
  • Now you can continue with your test and do/test whatever you want
  • Save your session by clicking on ‘End testing’
  • You now see an overview of your test
05
06
07
08
09
10

Create test case from your exploratory test

In MTM11, there is a possibility to file a bug or create a test case while you are running your exploratory test.

  • Let’s start a new session, not linked to any work item. Click on the ‘explore’ button
  • Start your test and do what you want to do
  • When you find a bug, click on the ‘Create bug button’
  • You see a new ‘bug’ window
  • The steps to reproduce screen is completed with all your steps
  • You can also change the steps to be included in the bug by clicking on ‘Change steps’
  • In the new dialog, move the slider to change the visible actions
  • You can now save your bug, or click on ‘Save and create test’
  • When you click on that, a new test case dialog opens. The steps from the test case are also completed
  • You can save this test case.
  • When you finish your current test session, you can see that there is one bug and one test case linked to this test.
11
12
13
14
15
16

Now you got an overview of how you can define and run exploratory tests with MTM11. In a next post, I will talk a little bit more about session based test management and testing tours. Thanks for reading!

Note: I also found a little issue when using MTM11 in combination with tfspreview. When you add some screenshots to your test, you see a ‘missing image’ error when you look at the test afterwards. More info here

Hosted TFS with TEE11 using Eclipse on Linux

A while ago, I made a blog post about how to access TFS2010 with Eclipse. Now the cool thing is that there is also a new version of Team Explorer  Everywhere 11 (TEE11) to be used with Visual Studio Team Foundation Server 11 Developer Preview, so this also includes the hosted TFS version (tfspreview.com, where I still have one invite left).

As I have an account on the hosted TFS preview, I definitely wanted to connect my brand new Ubuntu 11.10 distro with my brand new Eclipse 3.7 IDE. I will not explain each step of the installation process, as they are described in my previous post.

You can get the TEE11 download here, or using your TFS preview (Administration -> 3. Download software).  You will need the ‘TFSEclipsePlugin-UpdateSiteArchive-11.0.0-CTP1.zip‘ file.

Once the bits are downloaded, install the Eclipse plugin (Help -> Install New Software). After the plugin is installed, go to Window -> Show View -> Other -> Team Explorer and select the TFS specific windows you need.

The next thing you want to do off course is connect to your shiny hosted TFS. In the Team Explorer, click on plus-sign and on the server window, click on the add button. In this window, add the URL of you TFS server in the correct format. If you want to connect to your hosted TFS, enter ‘https://yourname.tfspreview.com’. This will automatically disable the connection details window. Next step, click on OK.

If you’re using Ubuntu 11.10, you will get a nice error window.. Auch..?

Tfs11_linux

I was not really sure what the problem was. Something about ‘XULRunner’, but I never heard about that before. So I posted an issue on the MSDN forum. Because I really wanted to know what was wrong and I started a quick wiki-search:

XULRunner is a runtime environment developed by the Mozilla Foundation to provide a common back-end for XUL-based applications. It replaced the Gecko Runtime Environment, a stalled project with a similar purpose. All XUL-based applications like Mozilla FirefoxMozilla ThunderbirdSongbird, …run on XULRunner.

So from what I understand is that XULRunner is a runtime environment where you can render things, like Firefox uses it to render webpages. The error I had was indeed when the plugin wants to open a windows to show the Windows Live login screen. Ok, but how to solve my problem, as Ubuntu 11.10 has XULRunner by default?

After some google work, I found out that TEE11 requires xulrunner-1.9.2, which is not installed anymore on Ubunutu 11.10. I will tell you how to install this, as it’s not just “apt-get install xulrunner-1.9.2, which will not work by default.

The first thing you have to do is adding a line to your /etc/apt/sources.list
Go to your terminal, and enter:

sudo gedit /etc/apt/sources.list

After entering your password, it will open the gedit texteditor. Just add the following line to the end of the file:
deb http://security.ubuntu.com/ubuntu lucid-security main  

Gedit

This line will allow you to apt-get on the security repositories of Ubuntu. Now you can open your terminal, and simply type:
apt-get install xulrunner-1.9.2

This will install the package on your machine. If this doesn’t work, you can also download the package as .deb file and install it by double clicking on the package, and follow the steps of the Ubuntu software center.

Swc

Next thing is restart your Eclipse IDE, a try to add a new TFS server. Now you should see the Authentication window like you have in VS2010 SP1 (with KB2581206)

Liveid

All right, mission accomplished! Now you can do whatever you want with TEE11 like source control, workitems, builds,..

Teexpl

Just wanted to say thanks to Shaw Terwilliger who gave me the following explanation about the issue:

I’m glad you figured this out quickly, I was just about to post the same work around.  We finished our TEE 11 CTP before Ubuntu 11.10 was released, and we depend on the Mozilla browser via xulrunner for this control.  Shortly after Ubuntu 11.10 was released, xulrunner wasn’t available in the repositories but the xulrunner-1.9.2 from 11.04 installed cleanly and works.

We’re investigating Ubuntu’s plans to continue shipping xulrunner in future releases, and may make some changes in our next release to work with the Webkit browser (the default in Eclipse 3.7 on Linux/GTK).

Accessing TFS2010 from Eclipse

One of the cool things about using Team Foundation Server is the fact that it’s not technology bounded. What I mean about that, is the fact that you’re not tight to Visual Studio as IDE and that TFS can also be used to build non-microsoft solutions like Java. In this blogpost, I would like to describe how you can use Eclipse to manage your TFS environment.

Tee

Last year, Microsoft released a plug-in for Eclipse, originally called TeamPrise but now it’s ‘Team Explorer  Everywhere‘ (TEE). Using this plug-in you have access to TFS 2010 from Eclipse, which is a very popular IDE. Projects using Java/PHP/.. can now also leverage the benefits like workitem tracking, source control, build automation, reporting and web-portals. 

This is a quick overview of how to install TEE. First of all, I will use Eclipe on a Ubuntu environment (just to show that it also work on Linux machine). I downloaded the latest Ubuntu release (which is really cool, btw) and created a Virtual Machine to install Ubuntu. After installation of the OS, you get this nice welcome screen:

Login_ubuntu

Next step is to install Eclipse. Very simple, just download the Eclipse (3.7-Indigo) environment you want (for Java, classic, PHP). In linux, this will be a tar.gz and in Windows a normal .zip file. All you have to do after downloading is unzip/untar the file and put it in C:/eclipse or /home/.. (whatever you want). The next thing that you have to check is that you have a JRE/JDK installed. (download on oracle.com). Now it’s time to run Eclipse:

Eclipse

The next thing we want to do is install TEE SP1 (SP1 is a full package, there is no need for the non SP1 version).You can download the connector using your MSDN subscription, or using the public download url. You can download all 3 files, but actually you only need the file TFSEclipsePlugin-UpdateSiteArchive-10.1.0.zip. When the download is finished, go to Eclipse and select ‘Help -> Install New Software’. This opens a dialog where you can install additional plugins/connectors.

Click on the ‘Add..’ button to add a new repository. Enter a name, like ‘TFS connector’ and click ‘Archive’ to select the downloaded zip file. Select the new item you’ve just downloaded and click next. Follow the wizard, accept the license and click on the ‘Restart Now’ button.

When Eclipse is restarted, you don’t see many changes. The TFS related windows are more or less hidden by default. You can show them by selecting ‘Window -> Show View -> Other -> Team Foundation Server‘. There you can select the windows you want.

Now we have TEE installed in Eclipse, we can start using it. Create new project and add some awesome codefiles to it (yes I use PHP in my examples :$). If you did that, we can add the project to our Team Foundation Server. Right-click on your project and select ‘Team -> Share Project‘. There you get a dialog asking you for the TFS settings (Url & credentials). Supply them correctly and click Next.

Note: If you use Active Directory to connect, enter the correct domain and complete the username with your windows username. Also be aware that if you check ‘Save password’, the password will be saved in plain text on your system!

Then you get a list with your team projects, select the team project you need and click Next. In the list you see now, click the Team Foundation Server workspace to which your project will be connected, and then click Next. (You can offcourse also change the default values from your workpace). The last step is to select the folder for you shared projects (the project folder path). Click Next. In the last dialog, you have an overview of your selected settings, and now just click Finish 

Now that the project is added, we can do a check-in using the ‘pending changes’ window. If you look there you will see that all files associated with your project are marked as ‘add’. Now add some comments and click ‘Check In‘.

Eclipse14

If you return to Visual Studio, and do a ‘Get latest version ‘ on your project, you should see your files from your Eclips project. 

Eclipse15

Ok, so now we can do (allmost) all things we can do with the ‘normal’ Visual Studio Team Explorer. We have our team explorer, we can manage our sourcecontrol, manage our build, our workitems, merge files on update conflicts,… The only restriction I found was that there is no possibility to use the braching/merging actions/overview like you can in the regular TE. Other than that, I think it’s quite similar to the Team Explorer from Visual Studio. Some screenshots:

This was an overview about how to install TEE on Eclipse. In a next blogpost, I’ll cover how you can build a JAVA project using ANT/MAVEN with the TFS buildserver (using the Microsoft Team Foundation Server 2010 Build Extensions). Stay tuned!

Some reading material:
Five Hidden Features in Team Explorer Everywhere 2010 SP1
Team Explorer Everywhere 2010 SP1 is Available

 

How to delete a Team Project in TFS 2010

If you are using Team Foundation Server, you definately have been in the situation that you want to delete a Team Project. Perhaps because it’s obsolete because you moved all branches to another Team Project.

To delete a Team project, you have two possibilities. The manual command-line way, or using the TFS Administration Console.

 

1. The command-line way

You can delete a Team Project using the TFSDeleteProject command. To use this command, open your Visual Studio Command Prompt, enter this: ‘TFSDeleteProject‘ and press enter. If you see the information about the command, you’re good!

First of all, keep the following in mind

  • Be sure you are a member of the Team Foundation Administrators security group or a member of the Project Administrators security group.
  • Know what you are doing! 
  • Take a back-up of all your important data as there is no way back (or you have to restore the latest backup from TFS)
  • /q = Optional: Use the quiet mode. Do not prompt the user for confirmation.
  • /excludewss = Optional. Specifies to not delete the SharePoint site that is associated with the team project. Specify this option to maintain the existing site so that other team projects can continue using it.
  • /collection:url =Required. Specifies the URI of the team project collection.
  • /force The program should continue even if some parts cannot be deleted.

Ok, let us assume that we want to delete ‘ProjectToRemove’.

Delete_project1

Enter the following command in the Visual Studio Command Prompt

In my case, the team project is located in the CogniStreamer collection. If you execute this action, you will be warned about the fact that this is an irrecoverable operation. Enter ‘Y’ to continue.

Delete_project2

 

2. TFS Administration Console

The next possibility is to use the TFS Administration Console. You can open the console using ‘Start – All Programs – Microsoft Team Foundation Server 2010’

Open the Application Tier node and select Team Project Collections. Select the correct collection, en select the team project you want to delete. Next step is to click ‘delete’. 

In the next dialog, you can select to delete all external artifacts (like SQL Server reporting services, labmanagement) and to delete all workspaces related to this team projects.

This should do the trick!

Ony one remark, from what I experienced, you can only delete the sharepoint site using the command line approach (or exclude the removal using /excludewss). But in the TFS Administrator console, you can only delete the external artifacts and workspaces, but NOT the Project Portals.. 

Getting started with Team Foundation Service

Logo

Introduction

At the Build conference this week, a new service for the upcoming version of TFS was announced. They call it Team Foundation Service. This is a Windows Azure hosted solution of a Team Foundation Server, which is pretty cool! The TFS team definitely did a great job to bring the Application Lifecycle Management features of Visual Studio and Team Foundation Server to a higher level. And from what I can say, it looks promising. In this post, I’ll describe some setup steps and a general overview of some new features. 

 

Setup

First of all you need an account for the Team Foundation Service on tfspreview.com. For now, you need an invitation code to create an account. (If you don’t have an invitation code, I have some left. Just post a comment..) Brian Harry already made a nice blogpost about the registration process and how to add and invite users to your Team Foundation Service environment. When you finish your registration (using your Windows Live Id), you’re in the adminstration mode of the Team Foundation Service.

One tip I can give is that the gray border on top indicates that you’re in administration mode. This is one thing I didn’t notice in the beginning. I was a little bit confused when I tried to open my team project when I was still in administration mode. But after a while I got whole point.

The first thing you can do is create your first team project. You can do this by clicking on ‘create team project’. There you get a dialog where you can enter a name and a project description. You can also select which process template you want to use. The Microsoft Visual Studio Scrum 2.0 Preview 1 template is selected by default. (More about this specific template will follow in a later post) Currently it is not yet possible to add extra process templates. But I think most of us will use the Scrum template or the Agile Software Development 6.0 template.

Creating the team project only takes about one minute or less. Actually it will queue a job in the background, and the service does some jobstatus polling to show the process status to the user.

 

Overview

After the team project is created, you can click on the ‘My Team home page’ and you are redirected. That’s the place where all the fun starts. On top of the page you see:

  • Home: where you have some shortcuts to your activites, and a link to some administration
  • Backlog: where you can add your userstories
  • Board: where you have a graphical overview of the running/sheduled/.. work items. There you can also change the status of the workitems.
  • Source: where you have the possibility to search in changesets, shelvesets. You can lookup code, compare code with previous versions.
  • Builds: where you get an overview of the build history. You can also manage the build qualities.

 

Conclusion

I spent some time on the Team Foundation Service now, and from what I experience it’s really good. It’s surprisingly easy to use and quite fast. I really like the ajaxified interface. The only things which I think will need some change are some usability issues. Apart from that, my first impressions are good.

In my next blog posts, I’ll cover some other things like the product backlog, work-items, the board, how to use Team Foundation Service with Visual Studio 2010,..   You can always share your personal experiences by adding a comment to this post.

Thanks for reading!

Automatic IIS7 Deployment using MSDeploy and AppCmd

In this blogpost, I will tell you a little bit more about how to deploy your web application to IIS using MSDeploy and AppCmd. I’ll give you some information about how to create a .bat file with all the parameters you will need create a new IIS entry for your application and sync your files using MSDeploy.

Iis7

First of all, what is MSDeploy.exe aka WebDeploy? Well, MSDeploy is a server technology that enables a publishing and deployment mechanism. It enables you to not only publish files, but also create database schemas, run database change scripts, set security and much more. If you want to know more about how to install webdeploy on IIS, just take a look at the great blogpost of Scott Guthrie. 

The other tool we use is AppCmd.exe. This is the single command line tool for managing IIS 7. It exposes all key server management functionality through a set of intuitive management objects that can be manipulated from the command line or from scripts. AppCmd.exe enables you to easily control the server without using a graphical administration tool and to quickly automate server management tasks without writing code. More information can be found here.

 

Allright, What we gonna do?

  • Create new IIS site
  • Assign an application pool
  • Add HTTP binding
  • Add HTTPS binding
  • Set require SSL
  • Enable custom error mode
  • Set HTTP to HTTPS redirect
  • Sync the files from the source to the new site using msdeploy
  • Set ACL on a specifc folder

 

Step 1: Create a new batch file called f.ex. ‘NewSite.bat’. On top of it, define your application variables which will be passed trough the commandline to the batch file

 

Step 2: Create a new batch file called f.ex. “PreSync.bat” where you define the AppCmd commands to perform the first 7 steps from the list above

 

Step 3: Continue in the ‘NewSite.bat’ file – As we want the ‘PreSync.bat’ have the correct parameters, we generate a new .bat file called ‘PreSyncGenerated.bat’. First we remove any existing files called like that, and create a new one with the correct parameters.

 

Step 4: Still in the ‘NewSite.bat’, find the MSDeploy.exe location and check if MSDeploy.exe exists

 

Step 5: Sync your application sources using MSDeply.exe. We use MSDeploy.exe to execute the AppCmd commands in the generated .bat file. Note that we skip some files and directories that we don’t want to be affected

 

Step 6: Add modify rights to a folder (in our case ‘AppData’) using MSDeploy.exe. The permission will be assigned to the ‘networkservice’

 

That’s it.. What we’ve done now is create three batch files. One called ‘NewSite.bat’ which handles the MSDeploy.exe, the second one is PreSync.bat which handles the AppCmd.exe command and a latest one which is generated from the previous file and contains the correct parameters.

Now you can just run the ‘NewSite.bat’ file using commandline with the correct parameters, and your site should be deployed on IIS. Also all files are synced from the source to the destination.

 

I hope you enjoyed this post! If you have any questions, just post a comment and I’ll be in touch.

 

Why scrum doesn’t work for us

In this blog post I’ll try to explain why using Scrum didn’t fit our needs. I’ll talk about the main reasons, my personal opinion and what the alternatives are for our team.

First of all, let me explain the team situation, the products we make and how we used to work.

As mentioned in my first blog post, I work as a development team manager at a software company. We develop a collaborative framework which is the base of some main products. The team working on the applications consists out of some developers, configurators and designers. They all work on our framework and the various products.

You get something like this:

Team_products

 

We did already work on an agile way of software development, but we needed a better way to manage the process, to create structure and overview. All this using a time-boxed iterative approach. So we did some research on Scrum. From what we’ve read and learned about it, this was the way to go. 

Scrum its use has focused on the management of software development projects, and it can be used to run software maintenance teams or as a general project/program management approach.

Scrum

On the exact moment that we wanted to implement Scrum, Visual Studio 2010 (our development software) released a Scrum process template, perfect! This process template is a great way to define a product backlog, create sprints, work-items, test cases, a burn down chart ..

So we had a decent tool, we had a great product backlog with lots of amazing new features to be implemented, we had a good team, we did standup-meetings in the morning, sprints were defined.. Off we were…  

 

But after a while, things did not work out as planned. That’s why:

  1. There are too many projects and custom client-solutions
  2. Our team is too small to focus on the sprint for each project
  3. Sprints take much more time to complete than planned.
  4. Every team member is working on all projects at random, it’s hard to ‘focus’ on the defined sprints.
  5. Sprints are influenced by clients. Client requests have a higher priority than the sprints
  6. There was not enough discipline for the daily standup-meetings
  7. In general, time boxed iterations do not work for our -type- of work
  8. Our sprints are mostly event driven, instead of time driven

My personal opinions:

  1. It’s definitely not the fault of the tool, which is awesome
  2. We have to combine big feature development with custom client-development, support and maintenance
  3. There are too many Work In Progress items at a time
  4. The best solution is to stop using Scrum

 

We’re currently looking into another solution which will suit us better. I’m talking about Kanban, but more about that in a later blogpost.

Any suggestions and experiences are welcome, just post a comment!

Pages:123