Posts in the “testing” category

Introduction: I first wrote this article about automated GUI testing many years ago, but I find that it still holds today.

I just wrote most of the following note on the Apple Mac Java-dev mailing list, and I'd like to share it here as well, because I think it captures my thoughts on the benefits of automated GUI testing and GUI testing software.

I ran automated GUI tests part-time (4-6 hours per week) on a project with 8-12 developers, and saw some good benefits. True, in the 80/20 rule, 80% of the problems were due to UI changes and communication, like “We forgot to tell you we split the Name field into First Name and Last Name,” but with a good automated GUI testing tool, one test may fail, but the rest of the automated GUI test suite keeps running (see Fowler’s continuous integration). Furthermore, with a good GUI testing tool, something like this is also a minor change to get the test running again.

The grand experiment has begun. The problem: I've been on a project developing a very deep application for four years now, and lately it's become so complex and intertwined that things are starting to break. Developers have been known to say "The application is smarter than I am." I'm just a wee bit concerned about our software quality.

Throughout all of this I started to notice that many of these bugs could be found if we had ... (drumroll) ... automated GUI tests.

Automated GUI testing tools FAQ: I've read that you've done a lot of work with automated GUI testing tools, can you share some "lessons learned" about your automated GUI testing tools experience?

I'll come back and update this article from time to time as I run into more "lessons learned", but after writing my last article (Seven benefits of automated GUI testing), I also wanted to share these ideas on "Automated GUI testing best practices."

[Note: This article was originally written in 2010.]

If you’re following along with my Automated GUI Testing (AGT) software progress, I’m showing my latest addition in another YouTube video. In short, if you supply a simple text file to describe your menus and menu items, I’ve created a new tool that does the following:

• Automatically generates menu “click” commands
• Automatically generates menu item “click” commands
• Automatically generates tests/demos for these items

Yesterday I created a short YouTube video demonstration of "GUI regression tests" against the Google Chrome browser, using my Agile GUI Testing software. This video is a little more than two minutes long, and demonstrates some simple GUI regression tests on the Chrome browser, in the format of a presentation.

As mentioned, the Chrome browser tests shown in that video are completely automated, using my Agile GUI Testing software (AGT).

Automated GUI Testing Software: I'm still debating on the name of my GUI testing software (I've been calling it XGT, for eXtreme GUI Testing software), and today I'm calling it Agile GUI Testing (or Agile GUI Tester). None of that matters at the moment, but after pounding away on it some more last night, I thought I'd share a sample script of how this software works.

To that end, here's a sample Agile GUI Testing software script that does the following:

Automated GUI Testing Software: If you happen to be looking for something to do this weekend, here's a very early release of our free (GPL) Agile GUI Testing software (AGT).

In short, just download the file below, and when you un-tar it, it will create a "scripts" folder. That folder contains our GUI testing software, along with two sample scripts. If you'd like to see how the sample scripts work, they are shown in this two-minute YouTube video (Agile GUI testing, basic demo).

Here's a "generic" version of a simple test plan I just wrote for testing one wizard in a GUI software application my team is currently developing. I wrote this for one specific wizard, then realized that many of these tests are generically-applicable to all wizards.

Without any further introduction, here is my sample test plan. Feel free to use it as a template for creating your own test plans.

In the biggest news of the week, well, at least for me, I'll finally be releasing my eXtreme GUI Testing software (XGT) to the public. I was hoping to get everything ready for release today, but other things came up, so bear with me.

As I mentioned in a blog post many years ago, I created XGT to see if I could push the envelope of GUI testing and software QA. Because XGT uses Ruby/JRuby as its scripting language, and because JRuby is backed by anything you want to do in Java, I think this makes for a very powerful, but also very simple, automated GUI testing framework.

I don't know how good of an idea it is, but in my Agile GUI Testing software, one thing I'm working on (debugging my old code, actually) is some very basic image recognition software. If/when it works, it will let the tester write a line of code like this to find an image on screen:

The folks at mockobjects.com have done a good job of creating MockObjects for the Java API. I'm not sure how to use some of their objects yet, but once I learn a few of the basic patterns I'll put a tutorial out here.

They also have links to several MockObject code-generation tools, which may be helpful. These include EasyMock, MockCreator, and MockMaker. Which one is the best or most popular? Dunno yet. That's often an interesting part of the open source world.