mediatribe.net -- Drupal and Web Development

Prerequisites

To get the best of this tutorial, it is best to know a bit about Simpletest, Drupal 7, the command line, and applying patches. Chrome and Firefox should be installed on your computer. I tested this on Mac OS but it should work on Linux. Going through it all might take you about an hour. You should do all of it on your laptop (using MAMP or Quickstart, rather than a remote server -- I have used MAMP).

Introduction

Drupal's core Testing (machine name: simpletest) module allows you to create functional tests which can be run from within any instance of Drupal. These tests ignore the current site and install a brand new testing instance of your Drupal site on which to perform tests; this sandbox is destroyed after the tests run. To learn this by example, please see the http://drupal.org/project/examples module.
The advantage of Simpletest is that all your tests are run in a sandbox: your test might want to make sure that it is possible to add a new node, and to do this it adds a new node, but it does it in a sandbox; your real live site is unchanged.
One limitation of Simpletest is its inability to use almost any Javascript, so what you're testing is your site's behavior with Javascript turned off.
Selenium itself deals well Javascript elements, and you can install a Selenium IDE plugin for Firefox to record and run tests on your Drupal (or any other) site. However, running these tests actually modifies whatever site you run them on, rather than in a sandbox.
To bridge this gap, the selenium (http://drupal.org/project/selenium) project uses Selenium to run tests on a brand new installation, not your current site.
Please note that as of this writing, the Selenium module for Drupal has many open issues and does not seem to be very widely used; so this is not meant for intensive use. The Drupal community is also looking at other options for Drupal 8 (see http://drupal.org/node/775050 and http://drupal.org/node/237566).
For one example of how Selenium is used for QA testing, see this blog post on The Economist's website.

Setup

For this tutorial we'll install a new instance of Drupal 7 and the selenium module, which can be found at http://drupal.org/project/selenium. I suggest you get the latest development version of both.
Because Drupal Selenium tests are run on specific real browsers, you will need to make sure you have Chrome and Firefox on your system.
Let's follow the instructions in selenium's INSTALL.txt file:
Get the Selenium server from http://seleniumhq.org/download/. This is a .jar file you can put anywhere on your computer (I suggest leaving it somewhere relatively permanent, not your Desktop or Downloads folder -- I put in my Mac's /Applications folder).
Now download the Chrome driver at http://code.google.com/p/chromedriver/downloads/list. Put it in the same folder as your Selenium server.
Now, in your command line, run the following command, using your actual path instead of /path/to/:
java -Dwebdriver.chrome.driver=/path/to/chromedriver -jar /path/to/selenium-server-standalone*jar
(you might want to set up an alias to this command to avoid typing it up each time you want to launch the selenium server. Here is how to do it on a mac: http://goo.gl/uOkW9)
Wait a few minutes, and the javascript app seems to hang after "Started org.openqa.jetty.jetty.Server@...". This is OK. Leave the terminal window open.
Notice that the Selenium module includes a patch for Drupal core. To avoid a "Hunk failed" error, check the issue http://drupal.org/node/1553258. If it has not been resolved, you might want to patch the selenium module and then use the updated patch in that module to patch core. (If you don't apply this patch to selenium before patching core, everything should still work except tests on Chrome.)
Patch Drupal core now following the instructions in Selenium's INSTALL.txt.

Running an example test to get the feel for it

Now, on your Drupal site, enable the selenium module, and its dependency, Testing (simpletest), and go to http://example.com/admin/config/development/testing. Note that you can launch your test from any browser; I use Safari.
Let's run a test that ships with the Selenium module for Drupal: In the Selenium test group, select the test "Filefield ajax upload". This is a javascript widget which is untestable with Simpletest alone. Click "Run tests".
This will open a Firefox window (you might have to dismiss a "safe mode" dialog box), and performs the Selenium tests in real time. Next, it does the same in Chrome (if you get an error, make sure the selenium server and the chrome driver are running and you performed the setup correctly).

Creating your own Selenium tests

That's all very neat. Now let's leverage this for our own modules and features.
Before creating your own Selenium tests on your module, disable Selenium, and visit http://example.com/admin/config/development/testing. If you get a Fatal error, you have run into the issue http://drupal.org/node/1553320.
This is ugly, but not critical, on your test site; but if your module contains a Selenium-based simpletest and you're deploying it, you don't want to cause Fatal errors on all sites where it's merely present (without even being enabled or installed).
My solution is to check for class_exists('DrupalSeleniumWebTestCase') before loading your test. Please see the patch at comment #1 of http://drupal.org/node/1553320 for details on this.
Another problem is Selenium's inability to select items from a Dropdown menu. Please see http://drupal.org/node/1553368 and http://drupal.org/node/1312184 to see the status of this problem. You might have to do some debugging to get this to work (I gave up for the moment!), or just avoid all dropdown menus in your tests.

Conclusion

Automated testing has greatly improved the stability of Drupal and contributed modules. With Javascript becoming more and more important for Drupal and on the web in general, we need to start systematic automated testing for our Javascript. For now the Selenium module is relatively functional, but the Drupal community is working on other options too.

More resources

• http://drupal.org/node/775050
• http://drupal.org/node/237566
• http://drupal.org/project/testswarm