play/type blog

We are creating Germany's juiciest event platform, boomloop.com. Because we love the Internet more than our own mothers. See for yourself. check out boomloop.com


10
Oct

Kick-ass Queuing for Ruby: using RabbitMQ over amqp

First off, you’ll need to install RabbitMQ. Instructions for OSX here. Once you’ve done that, install the pure ruby amqp library:

    gem sources -a http://gems.github.com
    gem install tmm1-amqp

You’re good to go. Now open up two IRB sessions. Paste the following code into the first session:

    require 'mq'

    EM.run {
      amq = MQ.new
      EM.add_periodic_timer(1) { amq.queue("noises").publish("moo") }
    }

Your publishing code has to run inside of an Event Machine loop. You can start it using EM.run. If you’re running inside of an Evented Container such as Thin or Evented Mongrel, you can skip this. The meat of it is just amq.queue("noises").publish("moo")that.

Now open a second irb session in another terminal tab, and paste this in:

    require 'mq'

    EM.run {
      amq = MQ.new
      amq.queue("noises").subscribe { |noise|
        puts noise
      }
    }

You’ll get the moos off the Queue “noises”. No polling required, which is very nice, you simply register a callback with amqp, and it’ll be called with the message when one becomes available.

Niceness!

Comments

There are 7 Comments for this post.  Write comment →

Excellent! The question is now : how can I integrate this in my Rails application to desynchronize heavy request?

@Julien just use workling: http://is.gd/6HcW

RabbitMQ part three.

Got it. See the Aman Gupta’s response on this posting on the RabbitMQ google group:

http://groups.google.com/group/ruby-amqp/browse_thread/thread/1e3226a666a69de6

Basically, Rabbit will only store strings. Complex objects pushed in to the queue need to be Marshaled and unmarshaled by the caller/receiver.

I think I see where to do this in the client code, so I may try and push you a patch shortly.

Todd,

Thanks for the quick reply. :-)

Actually, I just figured it out. I put in the marshalling calls in AmqpClient#subscribe and request and it works as expected now. That would probably be nice to put into the code base, although I wonder what kind of ramifications there are for doing marshalling. But it can certainly be a gotcha for users. Maybe make marshalling happen by default but expose a config to turn off marshalling.

Thanks again.

Sorry, my previous comment was in reference to a different blog post. But the same thing applies. :)

Hi,

I left a comment asking why I couldn’t get the AMQP stuff to work on another post – it looks like I hit the marshalling issue (i.e. it does not do any marshalling…)
How about using yaml to do this – then we’ve got human readable (and non-ruby readable) messages on the queues?

@Simon

YAML is slower than using JSON or Marshaling and Base64 encoding the marshaled object.

I’d recommend JSON if you have any non-Ruby processes touching the data, otherwise, Marshaling/Base64 encoding works like a champ.

Write a comment

Required in bold.