Test Suite Timing Comparisons

Part of the release procedure for Apex is that I run the test suite on the compiled app. ‘The test suite’ is currently nine AppleScripts which run Apex through a lot of its paces — importing and exporting files, calculating a proxigram, saving an animation as a QuickTime movie, running the select particles in shell action from the isosurfaceOps plugin, and most recently I’ve added an rdf export.

I run the suite once on a PowerPC machine and once on an Intel-based machine. If the test succeeds, its a pretty good indication that the build is OK to go out the door. It doesn’t catch issues in the GUI, and it doesn’t run through all the functionality, but it gets a good deal. For example, the most recent problem I discovered was that one of the plugins was building for PowerPC only because of a glitch in the XCode config files.

Of course, one of the things that I monitor is how long each script takes, so I can verify that I haven’t screwed anything up too much in making changes for each release. And the other thing is that, having an automated test suite means its pretty easy to move the test to new machines. So I’ve done a little comparison of performance on a few different machines. From slowest to fastest:

G4 PowerBook , 1.25 GHz: 508 seconds
G4 PowerMac, Dual 1.25 GHz: 462 seconds
G5 PowerMac, Dual 2 GHz: 250 seconds
iMac, 1.83GHz Intel Core 2 Duo: 209 seconds
MacBook, 2 GHz Core Duo: 192 seconds
MacPro, Dual 2.66GHz Xeon: 133 seconds

These are all running 10.4.8. All in all, its about what you would expect, but I must say I’m very impressed by the MacBook.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: