Automated Software Testing

So your boss told you that we have to automate this testing. What do you do? What do you need?

The answer really depends on the type of system that you need to test. The testing of a windows application will be very different than testing a web application. The Tools will be different; the approach will be different. Here are some general ideas that you will will want to follow for the various types of automated testing that you are going to need to perform.

If you are tesitng a windows application with a GUI; I recommend that you look into a 2 tools on the market:

• SilkTest (this is the tool I am most familiar with)
• TestRunner

These tools are all essentially the same. They operate on the idea that everything on the screen is an object and is tracked by the OS. Therefore a button in you application that contains the text OK can be found (the coordinates on the screen can be found via DLL calls and information sharing that the application and windows must do to play together). Custom controls (i.e. your own set of push buttons or grids, etc that are not included in windows are not supported out of the box, you will need to modify your application to provide a set of calls that the tool can perform to get the information from the application. This is typically not difficult and with the right software from the vendor you can do it in a couple of hours with basic knowledge of Windows programming)

The record feature will not doubt be sold to you (and company management) however it is not a clear cut as record and playback. Remember that record; records a lot of extra information that is not needed for the test. The tool record features do not create code that is easy to read or modify. I have always used the record feature to obtain the basic outline of what I want but rarely have I used more than 20% of what was actually recorded.

These tools tend to have baggage that the vendors will push you towards. TestRunner has a management package that integrates together so that you can manage and execute your test. Keep in mind that the idea of these management tools are to sell you something else and something that is sticky so that you will need to buy more stuff from the vendor.

Some different approaches to testing windows applications.

• Develop a command line interface to your application. This way you can can check the inputs against your outputs.

• Develop a command method to launch you application to a predetermined state in the application. Then use a GUI test tool to drive it; this can save significantly on the amount of test code

The Poor man’s tools:
There are someother tools outhere that are more like macro recorder from the original version of windows. Theses tools simply gather the keyboard and mouse events that you geneate and place them in a script. These do not work very well in the windows environment because:

1. Windows is a very dynamic environment.
2. Dialog boxes may pop up any time and any place
3. Windows may be placed in different spots by the application
4. Menus may have contextual changes. Therefore your menu item is not where it was last time.
5. The windows environment is a multitasking environment – timing can be affected.
6. Hardware may not be available during the test. (I.e. you removed the USB key required by the application or network drive is not shared correctly)

All of the above items can be handled in the professional tools.

There is a Perl GUI test module available. I have not had time to play with it but it works on the same basis as the professional tools. I cannot recommend it but it does look promising.