It seems the arts plugin has some problems with Rails 2.2.2. Seems to be a simple enough fix. I simply added the following to the bottom of my arts.rb file.
It seems there was a new with_output_buffer method added in Rails 2.2.2. I'd guess it does something useful with regard to output buffering in the context of an actual request - but for testing purposes it seems to work fine if we add it to the Arts module and just have it call the block it's passed.
Hope this helps someone else.