Regression testing mods/themes for new releases

[kelvincool]

Whether you are a forum owner, a mod writer or a theme developer, regression testing is always a pain in the butt. I've recently dumped a bit of time into creating Selenium scripts to test my mods so I figured I'd share and hopefully benefit others besides myself.

The idea is pretty simple, when a new release of SMF comes out, I update my test forum and then install my mod. I crank up my Selenium IDE (I use the plugin for Firefox) and load up my test suite. Then it's a simple case of running the test suite, I go away and make a cup of tea. When I come back hopefully everything is green and if not I know I've got a incompatibility issue with the new release. Pretty nifty for testing new releases of SMF, mods and themes. Anyways I've attached my sample test suite so you can see how I've done it.

You need to include the action.js as a Selenium Core extension and you need to include var.js as a Selenium IDE extension. Both of these are found under Options -> Options in Selenium.

The var.js has all the user and passwords you want to use in your test and the action.js has a couple of simple custom actions I created. The SMF login one is quite useful to log into a test account just by specifying the index of the user and I added an action to select text for testing the BBC stuff. Please note it's set up to use localhost/smf/ as the SMF root. Just need to change action.js, the init test case and reset test case if you want to use a different path.

The test suite creates the users and deletes the users afterwards including all the test posts it's created.

Anyways if you've already been using Selenium, feel free to share some scripts and compare notes. If you aren't using Selenium, what are you using and is it any good? Would also be interested to hear what others are using for unit testing. I got nothing so far but was planning on doing it at some point.