Random Hacks: Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by TheCheshireCatalyst

Thu, 11 Jun 2009 13:59:21 +0000

Use

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by _why

Mon, 11 May 2009 13:36:37 +0000

Hey hello, nice bit of hacking you got there. I just wanted to apologize for the threading trouble, I try to tell experienced programmers to stay away from Shoes, since the GUI presently runs in the main thread and wrestles with Ruby for control. Yeah, bad stuff—Shoes was intended for beginners and I’ve had my hands full with all the platform code.

At any rate, I’m working hard to switch us to Ruby 1.9 and changing the thread/memory situation in the process. Since the drawing primitives, music, animation and video are all cooked: now I can turn my attention to the robustness. This can’t happen soon enough.

At any rate, very nice work, I may swap out Hackety Hack’s messaging code for AMQP, it looks very swift and featureful.

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Eric

Sun, 10 May 2009 19:39:11 +0000

riffraff: Thanks! I didn’t know about abort.

John Apps: Thank you for the link to To AMQP or to XMPP. That’s a nice comparison of the two protocols.

I should mention, however, that RabbitMQ is ridiculously easy to install on any platform that has the appropriate packages. In fact, the RabbitMQ folks have a pretty admirable installation policy:

Our goal is to streamline the broker installation process such that you can have RabbitMQ up and running within two minutes of completing your download. If this doesn’t happen to you, please let us know at legitimategrievance@rabbitmq.com!

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by John Apps

Sun, 10 May 2009 10:16:57 +0000

Here is a good comparison of XMPP and AMQP: http://www.opensourcery.co.za/2009/04/19/to-amqp-or-to-xmpp-that-is-the-question/ AMQP is well summed up with the term “enterprise messaging” as it has all the attributes required for that kind of application. Lots of clients available: Ruby, Python, Java, .NET, C/C++, high-level languages, etc.

0MQ is a lot simpler with a great little API and blindingly fast. Very clean code – lean and mean.

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by riffraff

Sat, 09 May 2009 16:23:37 +0000

cool, I need to try out the shoes gui though :D

PS: you can shave a few lines e.g. unless ARGV.length 2 STDERR.puts "Usage: #{$0} " exit 1 end with abort(msg) unless ARGV.size 2

and if msg.index(”#{$nick}:”) != 0 puts msg end with puts(msg) unless msg[”^#{nick}:”]

;)

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Eric

Sat, 09 May 2009 09:41:30 +0000

Astro: EventMachine has built-in support for threads. In particular, see EventMachine.defer, which schedules blocking tasks using a thread pool, and the note in the FAQ on using EventMachine with other Ruby threads running.

That said, I’m reasonably certain that my Shoes/EventMachine code isn’t thread-safe. Shoes is a very cute framework, but it has very little documentation, and it certainly isn’t meant for running multithreaded programs. In my case, I’m making GUI calls from two different green threads, which certainly might scramble the GUI state on certain platforms.

If I were going to turn this into a real program, I’d try to use some simple locks to control access to the GUI, and I’d definitely test it on more than one platform. But without reading the Shoes source code, I’m basically just kludging around in the dark.

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Eric

Sat, 09 May 2009 09:31:09 +0000

Anon: The user names are made up.

I’m not really the right person to ask about the relative merits of AMQP, XMPP, STOMP, ∅MQ, etc. But here’s my best explanation of the pros and cons of each:

XMPP: Supports internet scale federation and addressing, and has a built-in presence service (the “buddies list”). Has only weak support for offline message delivery and complicated message routing.

AMQP: Excellent support for offline message delivery, and sophisticated message routing. AMQP is quite fast for messages >= 100 bytes. The RabbitMQ server provides a distributed Erlang implementation, which should make it especially suited to clustering. I’m not sure what AMQP offers in the way of federation and global addressing.

∅MQ: Supports ~4 million messages per second per core, with extremely low latency. Excellent support for tiny messages, such as a stock ticker and price. Can interoperate with AMQP.

STOMP: I know nothing at all about this protocol, but it seems to be widely used in the Ruby community.

Basically, XMPP is optimized for people at the Internet scale, and AMQP is optimized for programs at a large organizational scale. ∅MQ was designed for use in high-performance stock-trading platforms, but to take full advantage of it, you’d certainly need to use a language other than Ruby.

That was my massively uninformed summary of the major open source messaging queuing protocols. :-) If anybody has first-hand experience with any of the above, please feel free to correct my mistakes.

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Astro

Sat, 09 May 2009 08:50:23 +0000

Very interesting, but doesn’t look thread-safe. Is it?

I tried to do a Shoes XMPP PubSub client before, but threading hassle killed it. Shoes and especially EventMachine are not made for threading.

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Anon

Sat, 09 May 2009 08:48:54 +0000

What’s so great about AMQP?

"Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)" by Anon

Sat, 09 May 2009 05:45:09 +0000

Who is Sarah?

Write a 32-line chat client using Ruby, AMQP & EventMachine (and a GUI using Shoes)

Eric Kidd — Fri, 08 May 2009 18:06:00 +0000

Have you ever considered using instant messages to communicate between programs? You can do this using Jabber’s XMPP protocol, of course. But it’s also worth taking a look at AMQP, a distributed messaging protocol first used at JPMorgan Chase. AMQP is fast, easy to use, and implemented by at least 4 open source servers.

To try it out, install the excellent Ruby AMQP bindings, and set up the RabbitMQ server (which is written in Erlang using Mnesia). On a Mac, you might do something like this:

sudo gem install amqp
sudo port install python25 rabbitmq-server
sudo rabbitmq-server

Once your server is running, save the following code as chat.rb:

require 'rubygems'
gem 'amqp'
require 'mq'

unless ARGV.length == 2
  STDERR.puts "Usage: #{$0} <channel> <nick>"
  exit 1
end
$channel, $nick = ARGV

AMQP.start(:host => 'localhost') do
  $chat = MQ.topic('chat')

  # Print any messages on our channel.
  queue = MQ.queue($nick)
  queue.bind('chat', :key => $channel)
  queue.subscribe do |msg|
    if msg.index("#{$nick}:") != 0
      puts msg
    end
  end

  # Forward console input to our channel.
  module KeyboardInput
    include EM::Protocols::LineText2
    def receive_line data
      $chat.publish("#{$nick}: #{data}",
                    :routing_key => $channel)
    end
  end
  EM.open_keyboard(KeyboardInput)
end

Now, run copies in two different terminals:

ruby chat.rb channel_1 sarah
ruby chat.rb channel_1 joe

Everything you type into one terminal will be relayed to the other.

How it works

The following line creates a topic exchange named “chat”:

$chat = MQ.topic('chat')

A topic exchange allows many-to-many communication. Here, we bind a listener to our exchange, and ask to receive all messages tagged with our channel name:

queue.bind('chat', :key => $channel)

Note that :key may be hierarchical, and it may contain wildcards. To write data to our topic exchange, we use publish:

$chat.publish("#{$nick}: #{data}",
              :routing_key => $channel)

Our keyboard input is processed using EventMachine, a Ruby library for writing high-performance, multi-protocol servers. It’s very similar to Python’s Twisted library, though it has less documentation and support for fewer protocols.

We use EventMachine’s EM.open_keyboard to create a asynchronous keyboard input channel, and we use EM::Protocols::LineText2 to treat the keyboard input as a line-oriented protocol.

Adding a Shoes GUI

Shoes is an eccentric, entertaining, and highly-portable GUI library by _why the lucky stiff. With a certain amount of grotesque kludging (and some pointers from “s1kx” on the #shoes IRC channel), I managed to get the Mac version of Shoes to talk to EventMachine. You may find that this code fails strangely on your computer. Honestly, I don’t know anything about Shoes. And I’m doing some pretty bad things with threads.

First, the pretty pictures:

Next, the code:

Shoes.setup { gem 'amqp' }
require 'mq'

$app = Shoes.app(:width => 256) do
  background(gradient('#CFF', '#FFF'))
  @output = stack(:margin => 10)

  def nick str
    span(str, :stroke => red)
  end

  def display text
    @output.append do
      if text =~ /^([^:]+): (.*)$/
        para nick("#{$1}: "), $2
      else
        para text
      end
    end
  end
end

Thread.new do
  begin
    AMQP.start(:host => 'localhost') do
      MQ.topic('chat')
      queue = MQ.queue('shoes')
      queue.bind('chat')
      queue.subscribe do |msg|
        $app.display(msg)
      end
    end
  rescue => e
    # Try to report at least _some_ errors
    # where we'll be able to see them.
    $app.display(e.to_s)
  end
end

Note that the GUI client listens to all channels simultaneously, because it doesn’t pass a :key to bind. And when writing code to run in a Shoes background thread, don’t expect to see any error messages.

Learning more about AMQP

The Ruby AMQP documentation page has a good list of papers, magazine articles, and other background material on AMQP.