So before I dug in to really work on Pastie I did a quick bump from 2.0 to 2.1. And I was instantly horrified at memory usage. "Memory out of bounds" e-mails were pouring into my inbox every hour two notifying me that Mongrels were getting killed and restarted by god. This was something entirely new.
Initially I thought perhaps Rails 2.1 was just more of a memory hog than 2.0. Perhaps I just needed to raise my memory ceiling and then things would be happy. Then I stumbled on to the real problem.
The Real Problem
Pastie makes extensive use of action caching to cache it's over 300,000 syntax highlighted Pasties. Rails 2.1 defaults to the memory store for caching rather than the file store (the previous default). And the file store for Pastie was easily over 10 gigabytes. Memory store, FAIL. :-)
Sometimes the problem isn't what we think. Sometimes it is. But sometimes the beginning of the solution should start with leaning back and asking yourself "What else could this be?"
Question your first assumption.