Code & Iron

Asynchronous Scheduling Options in a Rails Application

The stack behind torontofirecalls.com is not exciting, and that’s the way it should be.

There are no outlandish requirements for throughput, response time, or searching. The app will never serve thousands of users at once. Sharding, clustering, caching, eventual consistency; better left to start-uppy folks who are the next Twitter (or have deluded themselves as such).

Yes, it’s Rails.
Yes, it’s on Postgres.

Not much to talk about. So what to contribute?

Well, It’s been an absolute saga to schedule an asynchronous “every 5 minutes” job for this project. From that I ended up with a fairly good understanding of the options out there. Please learn from my pain. An alternate title to this article could be “Why Walled Gardens Eventually Suck”.

Req: Every 5 minutes the app shall update from the City of Toronto, parse the XML, and store the live events. If we miss an hour, that data is lost to us, possibly forever.

Scheduling Options on Heroku

1. Custom Rake Task and Heroku Scheduler
Pros – Free. Very easy setup. Create a custom rake task, and schedule/managed jobs in the web dashboard
Cons – Strictly limited to every 10 minutes, 1 Hour, 1 Day

heroku addons:add scheduler:standard

2. Clockwork and DelayedJob
Pros – Full, expressive DSL. Plays nice with Heroku Cedar. Excellent for applications deployed on multiple machines. Separates scheduling and performance of work (important). Works well when you have hundreds or thousands of jobs at once and need many workers.
Cons – $Expensive$ to run on Heroku, even for a few jobs. Requires one “clock” dyno for scheduling the work, and one “worker” dyno processing jobs. Adds $70 USD /mo to your app.

every(1.hour, 'feeds.refresh') { Feed.send_later(:refresh) }

3. Clockwork and DelayedJob with Workless
Pros – Workless Allows you to sleep a “worker” dyno until there’s something in the delayed_jobs table. Saves almost all the monthly costs of the worker dyno.
Cons – Unreliable. Uses heroku’s scaling API to sleep/awaken your worker(s). Often workers will die silently, or zombie with 0 jobs in the queue. Also, you still need a full-uptime clock dyno to schedule the delayed_jobs, so it still adds $35 USD/mo to your app.

In your production initializer:

config.after_initialize do
  Delayed::Job.scaler = :heroku_cedar
end

Then configure your scaler (to taste) with:

heroku config:add WORKLESS_MAX_WORKERS=1
heroku config:add WORKLESS_MIN_WORKERS=0
heroic config:add WORKLESS_WORKERS_RATIO=1

Scheduling Options off Heroku

* At this point, I got fed up with Heroku and it was time to move on. The Workless gem had failed to scale up/down multiple times. To stay reliable, I was running up an $70 USD/month bill just to process under 300 jobs a day. Not okay. I ported everything to WebFaction and within a few hours had a parallel instance running there.

4. Bare Cron
Pros – Expressive syntax. Full control. Literal decades of proven use. Available on most self hosted servers. Free as in beer.
Cons – Requirement of running the job is not stored alongside your code. Syntax can be intimidating for first time users. Heroku blocks it’s use. Is not ideal for multi-machine deployments/clusters.

Run crontab -e and:

17 8 * * * root echo "This command is run daily at 8:17 am"

5. Cron with Whenever
Pros – Simple, expressive DSL creates/updates jobs in your crontab with the “whenever” command. Requirement and changes to the jobs are stored inline with your code in SCM. Integration with Capistrano available for “update on deploy”. As available, sturdy, and free as cron (for Ruby apps).
Cons – … Heroku blocks its use. Is not ideal for multi-machine deployments/clusters.

In your schedule.rb file:

every :hour do
  runner "SomeModel.ladeeda"
end

Update your crontab using:

whenever -i

Solution #5 has been working very well for me, so the application will finish being ported to WebFaction, likely by the end of the week. It’s now costing ~$10 USD to host instead of ~$80 USD. Hopefully the above comparisons will help you make the right decision early by learning from my 4 mis-steps.

Next writeup will be a quick explanation of how I used the Google Maps for Rails gem and MapIcons by Nicolas Mollet to create the UI.

’till next time…

  Posted by Matt Holtom in Blog, Programming, Rails on August 27, 2013

Leave a Reply

Your email address will not be published. Required fields are marked *