Safely weekend updates

by Kenneth Kalmer on May 7, 2011

I took some time today and released three new versions of Safely in a row, 0.1.0, 0.2.0 and 0.3.0. Safely also has a nice wiki now which breaks down exactly how to use the different parts.

I’m just going to summarize here, you can find more information in the repo and on the wiki.

0.1.0 – Cleaned up Hoptoad support and added support for exception mails.

0.2.0 – Added a Log strategy for logging exceptions to the specified logger instance.

0.3.0 – Added support for logging all exceptions to a backtrace log when a Ruby application terminates abnormally.

Worth noting is the backtrace support in 0.3.0, which to me has been a super reliable feature of daemon-kit. When your program dies spontaneously (or from an unhandled exception), you’re usually left to wonder what just happened. With the new Backtrace class you will get the insight you require on a per-process basis in their own logs. Check out the Backtrace page on the wiki for more information.

I think for the time being I’m done with Safely, unless some bugs pop in. I’m contemplating a Rack middleware as well, but I fear for duplicating the efforts of others. If you’d like to have a Safely middleware, file an issue on Github (or vote it up if someone beat you to it).

I’m looking forward to ripping out this functionality from daemon-kit and bundling in Safely to do the dirty work. It is much better tested than daemon-kit’s implementation and feels cleaner overall.

Practicing safe Ruby, with safely

by Kenneth Kalmer on April 14, 2011

One of the things that makes daemon-kit such a great project (at least to me), is the built-in “safely” method wraps your code in a loving embrace of error reporting comfort. You could at any point do this:

safely do
  something_dangerous!
end

and your daemon would not die if something blew up. Instead the daemon would happily log the error to file (and possibly Hoptoad) and carry on working.

As part of resurrecting daemon-kit, I’ve extracted this functionality into a new gem called “safely“, available now on Github and Rubygems. At the moment it just supports Hoptoad via the Toadhopper gem, but soon will offer more!

I’ll expand on it in the coming days, and then tend to the daemon-kit fork queue while replacing the daemon-kit version with safely.

daemon-kit 0.1.8 released

by Kenneth Kalmer on August 4, 2010

I’m proud to announce that daemon-kit has finally made it to 0.1.8.1, almost a year after the last patch release. There has been a lot of changes since the last release, mostly cleanly up and staying with the times (so to speak). Github has an awesome compare view, detailing the historic moment, and I’ll go through it here in some detail as well.

First off I’d like to thank a few people who have quietly and vocally contributed to or promoted daemon-kit. In no particular order, thanks goes to:

The two biggest changes for me has been converting all the generators to use Thor, and using Bundler in the generated daemons. Both of these projects have received some flack, but they both continue to serve me very well and I’m sure they’ll serve daemon-kit well.

The logger got some minor updates, to be more compatible with the standard Ruby logger class. A brand new XMPP generator is included, and it uses Blather for some awesome evented-goodness. Argument handling got fixed up, email exceptions got removed from the project (I might consider implementing them after overhauling the error reporting features of dk). Hoptoad error reporting got upgraded to the use the newer API. The generated rake tasks are more forgiving when rspec/cucumber or other gems are missing. The cron daemon got some love, allowing easier exception logging/notification for when scheduled tasks fail.

I’ll continue chipping away at the project, tidying up loose ends as they are reported, and start developing a proper test suite for everything.

Over the long term I’m hoping to reach the following goals:

  • Always running the eventmachine reactor (think node.rb)
  • Support for more modular daemons, including ‘single-file’ daemons
  • Support for daemons inside Rails projects

Some things will probably get thrown out, like generating configs for god & monit. The more I use chef, the more I realize that those responsibilities lie with the infrastructure management (or devops), and not within the project itself. I’m aware this might cause an upset, however I firmly believe that a lighter and smaller daemon-kit will serve the greater community better.

Thanks for making daemon-kit the number 1 daemonizing project. Please share the love and send feedback, and more importantly use Github issues to log issues, feedback, and wishes.

Paginating documents with couchrest and will_paginate

by Kenneth Kalmer on February 8, 2010

CouchDB is hands down my favorite of the NoSQL variants and offers some pretty spectacular features, none of which I will bore you with in this post. I will however jot down how I (fairly easily) achieved pagination with couchrest & will_paginate in a fairly large Rails application recently.

John P Wood discussed some issues they faced with will_paginate and couchrest during the migration of TextMe to CouchDB, but l left out some code to work with. Couchrest itself had some pagination support that got pulled to some extent… This left me wanting, and wondering, since it was my turn to walk down this path.

CouchDB is a different beast, its aggressive use of indexes means that occasionally you loose some functionality that you’ve been accustomed to having in other persistence mechanisms, like the number of rows matching a query. Jan Lenhardt explains on the CouchDB issue tracker in more detail, but it boils down that you need a reduce function to calculate the number of rows. Sounds difficult? Not at all!

In my case I had a collection of announcements to deal with, and the announcement archives is a paginating collection of documents. Standard will_paginate stuff, nothing special.

Those reading in a reader would want to click through to the post to view the embedded gists further down, or view them directly at Github.

Below is a condensed version of the model from our paginating system:

I’ve included only one view and a corresponding class method, as it is enough to proof the principle. Lets dissect.

The map/reduce functions are extremely simplistic, they simply emit the announcer and the date the announcement was created. This allows for easy scoping and ordering of the announcements. The reduce simply counts our returned records. The magic is in the class method that sets up our WillPaginate::Collection with data from our views.

Line 30 creates a new WillPaginate::Collection instance, passing it the page number and total per page as parameters, it gives us back a pager that we can manipulate.

Lines 31 through 38 uses couchrest’s pagination support to pull out data from our view. The most important things to note here are that the page and per_page options are sent to the paginator and we skip the reduce step.

Once we have our records loaded, we ‘replace’  the pager’s collection with our results from our view (line 40).

The final step is to determine the total number of documents available to us, and for this we need the reduce function. On line 42 we call the same view, with the same arguments, except for requiring the reduce step to happen. We use the results (lines 43 to 47) to inform the pager (will_paginate’s pager) how many rows there are in total.

The controller and the views might look something like this:

It worked, and it shows that we don’t loose as much as we might think when moving away from ActiveRecord and the ton of plugins surrounding it.

Thanks to John for documenting the migration of TextMe, just knowing that it was possible to combine couchrest & will_paginate gave me the push I needed to figure this out.

Disclaimer: This code is extracted “as is” from a real life system and might contain idioms/phrases, and even code, that doesn’t make 100% when viewed in a gist. Please wear your thinking cap when applying this lesson to your own projects.