Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)

Posted by Eric Kidd Fri, 08 May 2009 18:06:00 GMT

Have you ever considered using instant messages to communicate between programs? You can do this using Jabber’s XMPP protocol, of course. But it’s also worth taking a look at AMQP, a distributed messaging protocol first used at JPMorgan Chase. AMQP is fast, easy to use, and implemented by at least 4 open source servers.

To try it out, install the excellent Ruby AMQP bindings, and set up the RabbitMQ server (which is written in Erlang using Mnesia). On a Mac, you might do something like this:

sudo gem install amqp
sudo port install python25 rabbitmq-server
sudo rabbitmq-server

Once your server is running, save the following code as chat.rb:

require 'rubygems'
gem 'amqp'
require 'mq'

unless ARGV.length == 2
  STDERR.puts "Usage: #{$0} <channel> <nick>"
  exit 1
end
$channel, $nick = ARGV

AMQP.start(:host => 'localhost') do
  $chat = MQ.topic('chat')

  # Print any messages on our channel.
  queue = MQ.queue($nick)
  queue.bind('chat', :key => $channel)
  queue.subscribe do |msg|
    if msg.index("#{$nick}:") != 0
      puts msg
    end
  end

  # Forward console input to our channel.
  module KeyboardInput
    include EM::Protocols::LineText2
    def receive_line data
      $chat.publish("#{$nick}: #{data}",
                    :routing_key => $channel)
    end
  end
  EM.open_keyboard(KeyboardInput)
end

Now, run copies in two different terminals:

ruby chat.rb channel_1 sarah
ruby chat.rb channel_1 joe

Everything you type into one terminal will be relayed to the other.

How it works

The following line creates a topic exchange named “chat”: 

$chat = MQ.topic('chat')

A topic exchange allows many-to-many communication. Here, we bind a listener to our exchange, and ask to receive all messages tagged with our channel name:

queue.bind('chat', :key => $channel)

Note that :key may be hierarchical, and it may contain wildcards. To write data to our topic exchange, we use publish:

$chat.publish("#{$nick}: #{data}",
              :routing_key => $channel)

Our keyboard input is processed using EventMachine, a Ruby library for writing high-performance, multi-protocol servers. It’s very similar to Python’s Twisted library, though it has less documentation and support for fewer protocols.

We use EventMachine’s EM.open_keyboard to create a asynchronous keyboard input channel, and we use EM::Protocols::LineText2 to treat the keyboard input as a line-oriented protocol.

Adding a Shoes GUI

Shoes is an eccentric, entertaining, and highly-portable GUI library by _why the lucky stiff. With a certain amount of grotesque kludging (and some pointers from “s1kx” on the #shoes IRC channel), I managed to get the Mac version of Shoes to talk to EventMachine. You may find that this code fails strangely on your computer. Honestly, I don’t know anything about Shoes. And I’m doing some pretty bad things with threads.

First, the pretty pictures:

Next, the code:

Shoes.setup { gem 'amqp' }
require 'mq'

$app = Shoes.app(:width => 256) do
  background(gradient('#CFF', '#FFF'))
  @output = stack(:margin => 10)

  def nick str
    span(str, :stroke => red)
  end

  def display text
    @output.append do
      if text =~ /^([^:]+): (.*)$/
        para nick("#{$1}: "), $2
      else
        para text
      end
    end
  end
end

Thread.new do
  begin
    AMQP.start(:host => 'localhost') do
      MQ.topic('chat')
      queue = MQ.queue('shoes')
      queue.bind('chat')
      queue.subscribe do |msg|
        $app.display(msg)
      end
    end
  rescue => e
    # Try to report at least _some_ errors
    # where we'll be able to see them.
    $app.display(e.to_s)
  end
end

Note that the GUI client listens to all channels simultaneously, because it doesn’t pass a :key to bind. And when writing code to run in a Shoes background thread, don’t expect to see any error messages.

Learning more about AMQP

The Ruby AMQP documentation page has a good list of papers, magazine articles, and other background material on AMQP.

Tags , , , ,

Comments

  1. Anon said about 12 hours later:

    Who is Sarah?

  2. Anon said about 15 hours later:

    What’s so great about AMQP?

  3. Astro said about 15 hours later:

    Very interesting, but doesn’t look thread-safe. Is it?

    I tried to do a Shoes XMPP PubSub client before, but threading hassle killed it. Shoes and especially EventMachine are not made for threading.

  4. Eric said about 15 hours later:

    Anon: The user names are made up.

    I’m not really the right person to ask about the relative merits of AMQP, XMPP, STOMP, ∅MQ, etc. But here’s my best explanation of the pros and cons of each:

    XMPP: Supports internet scale federation and addressing, and has a built-in presence service (the “buddies list”). Has only weak support for offline message delivery and complicated message routing.

    AMQP: Excellent support for offline message delivery, and sophisticated message routing. AMQP is quite fast for messages >= 100 bytes. The RabbitMQ server provides a distributed Erlang implementation, which should make it especially suited to clustering. I’m not sure what AMQP offers in the way of federation and global addressing.

    ∅MQ: Supports ~4 million messages per second per core, with extremely low latency. Excellent support for tiny messages, such as a stock ticker and price. Can interoperate with AMQP.

    STOMP: I know nothing at all about this protocol, but it seems to be widely used in the Ruby community.

    Basically, XMPP is optimized for people at the Internet scale, and AMQP is optimized for programs at a large organizational scale. ∅MQ was designed for use in high-performance stock-trading platforms, but to take full advantage of it, you’d certainly need to use a language other than Ruby.

    That was my massively uninformed summary of the major open source messaging queuing protocols. :-) If anybody has first-hand experience with any of the above, please feel free to correct my mistakes.

  5. Eric said about 16 hours later:

    Astro: EventMachine has built-in support for threads. In particular, see EventMachine.defer, which schedules blocking tasks using a thread pool, and the note in the FAQ on using EventMachine with other Ruby threads running.

    That said, I’m reasonably certain that my Shoes/EventMachine code isn’t thread-safe. Shoes is a very cute framework, but it has very little documentation, and it certainly isn’t meant for running multithreaded programs. In my case, I’m making GUI calls from two different green threads, which certainly might scramble the GUI state on certain platforms.

    If I were going to turn this into a real program, I’d try to use some simple locks to control access to the GUI, and I’d definitely test it on more than one platform. But without reading the Shoes source code, I’m basically just kludging around in the dark.

  6. riffraff said about 22 hours later:

    cool, I need to try out the shoes gui though :D

    PS: you can shave a few lines e.g. unless ARGV.length 2 STDERR.puts "Usage: #{$0} " exit 1 end with abort(msg) unless ARGV.size 2

    and if msg.index(”#{$nick}:”) != 0 puts msg end with puts(msg) unless msg[”^#{nick}:”]

    ;)

  7. John Apps said 1 day later:

    Here is a good comparison of XMPP and AMQP: http://www.opensourcery.co.za/2009/04/19/to-amqp-or-to-xmpp-that-is-the-question/ AMQP is well summed up with the term “enterprise messaging” as it has all the attributes required for that kind of application. Lots of clients available: Ruby, Python, Java, .NET, C/C++, high-level languages, etc.

    0MQ is a lot simpler with a great little API and blindingly fast. Very clean code – lean and mean.

  8. Eric said 2 days later:

    riffraff: Thanks! I didn’t know about abort.

    John Apps: Thank you for the link to To AMQP or to XMPP. That’s a nice comparison of the two protocols.

    I should mention, however, that RabbitMQ is ridiculously easy to install on any platform that has the appropriate packages. In fact, the RabbitMQ folks have a pretty admirable installation policy:

    Our goal is to streamline the broker installation process such that you can have RabbitMQ up and running within two minutes of completing your download. If this doesn’t happen to you, please let us know at legitimategrievance@rabbitmq.com!
  9. _why said 2 days later:

    Hey hello, nice bit of hacking you got there. I just wanted to apologize for the threading trouble, I try to tell experienced programmers to stay away from Shoes, since the GUI presently runs in the main thread and wrestles with Ruby for control. Yeah, bad stuff—Shoes was intended for beginners and I’ve had my hands full with all the platform code.

    At any rate, I’m working hard to switch us to Ruby 1.9 and changing the thread/memory situation in the process. Since the drawing primitives, music, animation and video are all cooked: now I can turn my attention to the robustness. This can’t happen soon enough.

    At any rate, very nice work, I may swap out Hackety Hack’s messaging code for AMQP, it looks very swift and featureful.

  10. TheCheshireCatalyst said 33 days later:

    Use

    PSYC

    And here

RSS feed for this post trackback uri

Comments are disabled