Open Sourcery

Daemon-kit has been getting a lot of TLC from me lately, and it’s not going to stop anytime soon. As I wander deeper and deeper into AMQP territory, I need to extend daemon-kit to cope with all kinds of new scenarios. One of those being unhandled exceptions.

The second thing I put on the TODO list was Rails-style exception handling. With version 0.1.6 there has been some progress made in that regard. Daemon-kit now sports a configurable safety net for dangerous code. By wrapping blocks of code in a “safety net”, unhandled exceptions are caught and logged, and optionally sent via email or to Hoptoad for review.

Hoptoad? In a Ruby daemon? Sure, inspiration came via these tweets.

And it only makes sense to do it. Now for some code:

safely do
  # do something silly
  silly.action!
end

safely is mixed into Object and can be used freely. It is important to note that you have to handle your daemon-specific applications on your own and rely on safely as a fall over mechanism.

To configure the safety net, you can edit your config/environment.rb file and add the following lines to the configure block:

  # for email notifications
  config.safety_net.handler = :mail
  config.safety_net.mail.recipients = ['you@gmail.com']
 
  # for hoptoad
  config.safety_net.handler = :hoptoad
  config.safety_net.hoptoad.api_key = 'your-hoptoad-key'

The documentation is very rough at the moment, but the files you want to explore are lib/daemon_kit/safety.rb and the error handlers in lib/daemon_kit/error_handlers.

NOTE: If you are upgrading from an earlier daemon-kit, please upgrade your daemons as well by running the following rake task in the root of your daemon projects:

$ rake daemon_kit:upgrade

In the coming days/weeks you can look forward to the following enhancements as well:

• Improved logging
• Improved backtrace cleanups
• Improved rdoc’s
• rack application generator (with rack-mount)

I’m patching things up as I go along, adding features as I need them (and stuff I recall from my first daemons). There is still a lot of things that need attention, but they’ll be addressed and hopefully daemon-kit grows to becoming the premier framework for writing daemon processes in our beloved Ruby.

It came down on me like a ton of bricks when I deployed the audit updates to our production PowerDNS on Rails setup, hoptoad isn’t installed and any errors would just be gone and forgotten.

The solution is pretty simple, if you use git. This quicky will show you how I’m currently doing it.

Multiple remote repos

All our projects have their origin on our own servers, and then either have “github” or “rubyforge” remotes for public changes. This works very well, especially allowing the team here to work on ideas for a couple of days and deploy to a staging/production server for review. Only once we’re happy do we push the code back to the public.

Basic idea is that our public master branches should always be deployable. Nothing sucks more than getting a OSS project and it doesn’t work out the box…

Enter the deploy branch

We all have a tracked “deploy” branch, which exist only in our private repos, not on Github/RubyForge. This deployment branch was used just for our capistrano recipes. As of this week I decided to through hoptoad in the mix as well, which great reward already.

The principle is so simple, hopefully the flow below explains the process.

Master is either your code, or a clone of someone else’s, and the deploy branch contains your modifications for your environment. In our case its capistrano recipes and hoptoad, but could include New Relic’s RPM as well (or anything else).

The deploy branch should have only additions, and no other changes are made to the deploy branch ever. If you accidently merge deploy back to master, you’ll have a tough time undoing the damage.

Capistrano config?

Well, if you maintain your own remote repo of another project, then make sure that repo has your deploy branch at configure capistrano like such:

set :branch, "deploy"

If you don’t want to maintain your own remote repo, but want a local deploy branch, you’ll have to switch to copy deployment recipe (something like this):

set :deploy_via, 'copy'
set :copy_strategy, :export
set :copy_cache, true
set :copy_exclude, ['.git']
set :repository, '.'

BIG FAT DISCLAIMER: I’ve never used the above copy strategy myself, so I cannot vouch for it!

Results?

Well, this neat feature of git allows us to deploy and run our OSS projects as if we were the only users. Sensitive information is not exposed to the world, but is still recorded by hoptoad. It allows us to get the real exceptions as they happen in the wild, without guessing what happened when some logs a “It doesn’t work!!” ticket.