Pastie << self

Survive hard economic times in the Bahamas at RailsBizConf 2009

Looks like a potential new RailsBizConf in the works says Obie Fernandez in a message to the rails-business google group:

I am in preliminary stages of planning a business-oriented conference for principals of agile/rails consultancies. We will cover everything from sales and marketing, to financing, to keeping your clients happy and surviving in these tough economic times. [emphasis mine]

If you'd like to weigh in on RailsBizConf 2009 please do fill out the survey. The first question caught my eye:

My recommendation to Rails shops trying to trim some fat from the books: Fewer discretionary trips to exotic tropical island paradises.

Or if you absolutely must... vote Bahamas and fly coach. :-)

Arts plugin and Rails 2.2.2

It seems the arts plugin has some problems with Rails 2.2.2. Seems to be a simple enough fix. I simply added the following to the bottom of my arts.rb file.

Embedded pastie:

It seems there was a new with_output_buffer method added in Rails 2.2.2. I'd guess it does something useful with regard to output buffering in the context of an actual request - but for testing purposes it seems to work fine if we add it to the Arts module and just have it call the block it's passed.

Hope this helps someone else.

Pastie: Lesson 1 - Question your first assumption

So before I dug in to really work on Pastie I did a quick bump from 2.0 to 2.1. And I was instantly horrified at memory usage. "Memory out of bounds" e-mails were pouring into my inbox every hour two notifying me that Mongrels were getting killed and restarted by god. This was something entirely new.

Initially I thought perhaps Rails 2.1 was just more of a memory hog than 2.0. Perhaps I just needed to raise my memory ceiling and then things would be happy. Then I stumbled on to the real problem.

The Real Problem

Pastie makes extensive use of action caching to cache it's over 300,000 syntax highlighted Pasties. Rails 2.1 defaults to the memory store for caching rather than the file store (the previous default). And the file store for Pastie was easily over 10 gigabytes. Memory store, FAIL. :-)

Sometimes the problem isn't what we think. Sometimes it is. But sometimes the beginning of the solution should start with leaning back and asking yourself "What else could this be?"

Question your first assumption.

Re-factoring Pastie: Lessons Learned

It's been almost 2.5 years since Pastie #1. For the most part Pastie has changed little. I can only think of a few new features that have appeared since day one:

• true integration with TextMate (ie, the ability to paste any type of file in full color regardless of whether Pastie knows how to syntax highlight it or not)
• private pastes
• the switch from CodeRay to Ultraviolet leading to the support of many more languages

This lack of updates is largely due to my feeling that Pastie does everything it was designed to do - and does it amazingly well. I will concede though that Pastie has been a bit neglected recently. Other things have had my attention and Pastie has taken a back seat and also gotten a bit of a reputation (in some circles) for being finicky when it comes to reliability.

Recently my focus has returned to my beloved Pastie. Code has been re-factored, new features have been added while performance, responsiveness, and reliability have been restored.

Along the way I've learned a few interesting things that I'd like to share. I don't know if any of this will be astounding or rocket science. But maybe you'll learn something new or just find a new way to approach a problem you're having with your own app.

Lessons

• Lesson 1 - Question your first assumption

I'll add the other lessons here as I post them.

Rail's Dev Mode Performance Plugin

NOTICE: Don't even try this on 2.0... It's tested against 2.1.2 only but I plan to have it working on 2.2 soon.

Once upon a time there was dev_mode_performance_fixes. I was on a Powerbook at the time and almost crying from how slow Rails 1.2 was in development mode for any non-trivial app.

Then things changed...

Rails went 2.0, I got a Macbook Pro, and the plugin was all but forgotten. I think there was some minor interest in making it work with 2.1 but I didn't pay much attention because it really wasn't an issue for me anymore.

...and many moons came and went.

Today I'm proud to announce rails_dev_mode_performance. It's the original plugin reborn. I'm seeing an order of magnitude (12x) improvement in performance.

It's mostly a port of the old code to Rails 2.0 with all new hacks to deal with Rail's new TemplateFinder and it's dev mode performance issues. And now it's on GitHub... so if you want to contribute you can e-mail me bug fixes or fork the repository and send me a pull request.

How it Works

How all this works is that we're trying (very carefully) to track every dependency and relationship between objects back to the filesystem. When you edit a file in your application any objects that are somehow affected by that file are flushed and reloaded at the beginning of the next request.

This could mean reloading classes that inherit from a class that was defined in that file... or perhaps you've edited a module that you included in 12 classes... each of those classes have to be flushed before the next request so they can be reloaded to include the new version of your module.

Potential Gotchas

As you can imagine this requires a lot of hooks into dependency loading, modules inclusion, etc... the original plugin was patched numerous time to take this or that edge case into account. I'm sure we'll find more of these edge cases and I think there are probably solutions for most of them.

If you're doing something totally whacked and you know it (loading entire modules and classes from the database would be one thing that comes to mind) then this plugin may never work for you.

I think it can work for most of us though. Please check it out and let me know any bugs or issues you find.

NOTICE: Don't even try this on 2.0... It's tested against 2.1.2 only but I plan to have it working on 2.2 soon.