Posted by Eric Kidd
Fri, 08 May 2009 22:06:00 GMT
Have you ever considered using instant messages to communicate between programs? You can do this using Jabber’s XMPP protocol, of course. But it’s also worth taking a look at AMQP, a distributed messaging protocol first used at JPMorgan Chase. AMQP is fast, easy to use, and implemented by at least 4 open source servers.
To try it out, install the excellent Ruby AMQP bindings, and set up the RabbitMQ server (which is written in Erlang using Mnesia). On a Mac, you might do something like this:
sudo gem install amqp
sudo port install python25 rabbitmq-server
sudo rabbitmq-server
Once your server is running, save the following code as chat.rb:
require 'rubygems'
gem 'amqp'
require 'mq'
unless ARGV.length == 2
STDERR.puts "Usage: #{$0} <channel> <nick>"
exit 1
end
$channel, $nick = ARGV
AMQP.start(:host => 'localhost') do
$chat = MQ.topic('chat')
queue = MQ.queue($nick)
queue.bind('chat', :key => $channel)
queue.subscribe do |msg|
if msg.index("#{$nick}:") != 0
puts msg
end
end
module KeyboardInput
include EM::Protocols::LineText2
def receive_line data
$chat.publish("#{$nick}: #{data}",
:routing_key => $channel)
end
end
EM.open_keyboard(KeyboardInput)
end
Now, run copies in two different terminals:
ruby chat.rb channel_1 sarah
ruby chat.rb channel_1 joe
Everything you type into one terminal will be relayed to the other.
How it works
The following line creates a topic exchange named “chat”:
A topic exchange allows many-to-many communication. Here, we bind a listener to our exchange, and ask to receive all messages tagged with our channel name:
queue.bind('chat', :key => $channel)
Note that :key may be hierarchical, and it may contain wildcards. To write data to our topic exchange, we use publish:
$chat.publish("#{$nick}: #{data}",
:routing_key => $channel)
Our keyboard input is processed using EventMachine, a Ruby library for writing high-performance, multi-protocol servers. It’s very similar to Python’s Twisted library, though it has less documentation and support for fewer protocols.
We use EventMachine’s EM.open_keyboard to create a asynchronous keyboard input channel, and we use EM::Protocols::LineText2 to treat the keyboard input as a line-oriented protocol.
Adding a Shoes GUI
Shoes is an eccentric, entertaining, and highly-portable GUI library by _why the lucky stiff. With a certain amount of grotesque kludging (and some pointers from “s1kx” on the #shoes IRC channel), I managed to get the Mac version of Shoes to talk to EventMachine. You may find that this code fails strangely on your computer. Honestly, I don’t know anything about Shoes. And I’m doing some pretty bad things with threads.
First, the pretty pictures:
Next, the code:
Shoes.setup { gem 'amqp' }
require 'mq'
$app = Shoes.app(:width => 256) do
background(gradient('#CFF', '#FFF'))
@output = stack(:margin => 10)
def nick str
span(str, :stroke => red)
end
def display text
@output.append do
if text =~ /^([^:]+): (.*)$/
para nick("#{$1}: "), $2
else
para text
end
end
end
end
Thread.new do
begin
AMQP.start(:host => 'localhost') do
MQ.topic('chat')
queue = MQ.queue('shoes')
queue.bind('chat')
queue.subscribe do |msg|
$app.display(msg)
end
end
rescue => e
$app.display(e.to_s)
end
end
Note that the GUI client listens to all channels simultaneously, because it doesn’t pass a :key to bind. And when writing code to run in a Shoes background thread, don’t expect to see any error messages.
Learning more about AMQP
The Ruby AMQP documentation page has a good list of papers, magazine articles, and other background material on AMQP.
Tags AMQP, EventMachine, MQ, Ruby, Shoes | 10 comments
Posted by Eric Kidd
Tue, 05 May 2009 13:36:00 GMT
Charlie Munger is the long-time partner of Warren Buffet. Of the two, he’s the more politically conservative. Their company, Berkshire Hathaway, has recently bought big stakes in several of the better-off investment banks.
Recently, Munger sharply criticized the management of the investment banks, saying they’ve grown too politically powerful. The key quote:
“We need to remove from the investment banking and the commercial banking industries a lot of the practices and prerogatives that they have so lovingly possessed,” Munger said. “If they are too big to fail, they are too big to be allowed to be as gamey and venal as they’ve been – and as stupid as they’ve been.” (Bloomberg.com, via Baseline Scenario.)
What does the bankers’ stupidity have to do with the usual themes of this blog? Well, much of the crisis comes down to bad probability calculations: the big banks have been treating highly correlated events as though they were independent events.
Some good background material on the crisis:
Tags Finance, Politics, Probability | 2 comments
Posted by Eric Kidd
Thu, 30 Apr 2009 19:07:00 GMT
Over the last couple of years, I’ve occasionally written Ruby programs using RSpec and (more recently) Cucumber. These two tools are inspired by Test Driven Development (TDD), a school of thought which says you should write unit tests before implementing a feature.
When doing TDD, you work inwards from the interface to the implementation. You start by writing a test case against the interface you wish you had, and then you make that test case work. This is a subtle shift in how you approach a design problem, but it frequently results in beautiful APIs. (And you also get a fully automated test suite for your software, liberating you to make much larger changes without fear of breaking things.)
The problem with the word “test”
Unfortunately, the name “Test Driven Development” is misleading. Most folks think of “testing” as something you do after development is complete. But TDD is really more of a design activity—you’re specifying how your APIs should work before you actually start coding.
Dan North spent some time struggling to teach developers about TDD. After a while, he decided that the main barrier to understanding was the word “test.” He proposed replacing TDD with Behavior Driven Development (BDD), and he started referring to unit tests as “specifications.”
In the Ruby community, the most popular BDD tool is RSpec. Using RSpec, you might specify an API something like this:
describe "simplify_name" do
it "should convert all letters to lowercase" do
simplify_name("AbC").should == "abc"
end
it "should remove everything but letters and spaces" do
simplify_name(" Joe Smith 3 -+\n").should == "joe smith"
end
end
After writing this specification, you would then go ahead and implement simplify_name. And from then on, whenever you changed your program, you could automatically check it against this specification.
Using specifications to communicate with clients and users
By itself, RSpec is mostly useful for programmers. Sure, a specification looks a lot like English. But would you really want to show it to an end user?
Cucumber goes one step further. Instead of using code to specify how an API should work, it uses plain text to describe how a user interface should work. For example:
Feature: Log in and out
As an administrator
I want to restrict access to certain portions of my site
In order to prevent users from changing the content
Scenario: Logging in
Given I am not logged in as an administrator
When I go to the administrative page
And I fill in the fields
| Username | admin |
| Password | secret |
And I press "Log in"
Then I should be on the administrative page
And I should see "Log out"
Scenario: Logging out
...
Here’s the neat part: This specification is actually an executable program. Each line of text corresponds to a “step”, which is defined in another file. Here’s an example from the standard webrat_steps.rb file:
Then /^I should see "([^\"]*)"$/ do |text|
response.should contain(text)
end
Cucumber encourages you to think at a very high level, and to specify how different users will actually use your software. It’s particularly helpful if you need to communicate between programmers and end-users.
My experiences with RSpec and Cucumber
I’ve been using RSpec on and off for a couple of years now, and Cucumber since late last year. Initially, I found both tools fascinating, but also a bit frustrating. Both RSpec and Cucumber have very strong opinions about how you should write software. Now, I found those opinions very interesting, and I was quite happy to be influenced by the assumptions built into the tools. But every now and then, I would need to do something that the authors of RSpec and Cucumber hadn’t anticipated, and I would inevitably wind up struggling to make things work.
But recent versions of RSpec and Cucumber are richer and more flexible. They cover more important cases straight out of the box, and they’re easier to customize. So I can finally recommend both tools for real-world projects: They’ll still guide your thinking, but they should give you enough flexibility to handle the corner-cases.
The RSpec (and Cucumber) book
Unfortunately, the documentation for RSpec and Cucumber is scattered around the web, and there aren’t enough online guides showing the best way to solve common problems.
But the Pragmatic Press is working on The RSpec Book, which contains a large section on Cucumber, and a walkthrough of a typical development session using Cucumber and RSpec.
Currently, the RSpec book is available as a “beta book”. This is a downloadable, DRM-free PDF, with periodic updates throughout the publishing process. Right now, between one-third and one-half of the chapters have been roughed in, and the book is already very useful.
So if you’re curious about RSpec and Cucumber, have a look around the two web sites, and maybe watch some of the screencasts. If you decide to investigate further, pick up the beta book and dive in.
Tags Cucumber, RSpec, Ruby | 2 comments
Posted by Eric Kidd
Thu, 30 Apr 2009 16:57:00 GMT
Via LWN.
If you’re a Linux system administrator, you shouldn’t put your faith in security advisories. The kernelbof blog accuses Linux distributors of being too quick to label security bugs as “denial of service” attacks:
I’m wondering why kernel developers (or vendors?) continue to claim that kernel memory corruption are just Denial of Service. Most of the times they _are_ exploitable.
As an example, the author quotes Ubuntu Security Notice 751:
The SCTP stack did not correctly validate FORWARD-TSN packets. A remote attacker could send specially crafted SCTP traffic causing a system crash, leading to a denial of service.
(Emphasis added.)
The author claims, however, to have created an exploit for this bug. He says his exploit allows a remote attacker to gain root access, often on the first attempt. If this is true, it would give him a quick way to gain control over any Linux system which has a process listening to an SCTP socket.
Ubuntu’s security team is not doing system administrators any favors by labeling memory corruption as “denial of service” attacks. If you can corrupt memory, there are some terrifyingly clever ways to run code. And marking memory as non-executable won’t necessarily protect you.
If you administer a Linux system, you should probably aim to patch alleged “denial of service” bugs as quickly as you can.
Tags Linux, Security | 1 comment
Posted by Eric Kidd
Sun, 11 Jan 2009 14:55:00 GMT
On December 12th, Frank Li released TortoiseGit 0.1. When we downloaded this initial release at work, we were underwhelmed:
| git log | OK, but not as good as gitk |
| git commit | Broken |
On January 4th, however, Frank Li released TortoiseGit 0.2. He’d been extremely busy:
| git log | OK, but not as good as gitk |
| git commit | OK (except for add and rm) |
| git add | Broken (see Bug 6 for workaround) |
| git rm | Broken |
| git status | OK |
| git pull | OK |
| git push | OK |
| SSH | OK (tested with PuTTY) |
| git clone | Always clones to home directory (see Bug 8) |
| Clean merge | OK |
| Conflicted merge | Manual, as with command-line tool |
| Submodules | No support |
Basically, TortoiseGit 0.2 is almost usable, and the project is proceeding at a breakneck pace. If you have Windows users that you want to migrate to Git—and who don’t want to use the command-line tool—it’s worth a look.
Installation instructions follow.
Read more...
Tags Git | 1 comment
Posted by Eric Kidd
Mon, 01 Sep 2008 14:33:00 GMT
Ubiquity is an experimental Firefox plugin. It’s a “graphical command line” similar to QuickSilver on the Macintosh.
You can easily add your own commands to Ubiquity. The following article shows how to create a Hoogle search command that looks up Haskell functions by name or by type signature.
You can press Return or click on one the links in the preview.
Read more...
Tags Firefox, Haskell, JavaScript | 3 comments
Posted by Eric Kidd
Tue, 02 Oct 2007 11:50:00 GMT
From October 5-7, I’ll be at the Haskell Hackathon in Freiburg.
I’ll be working on probability monads, attempting to turn my various blog articles into a real Haskell library.
Some resources:
If you were a peer reviewer, or gave me feedback on the paper, my humble thanks–and apologies. I haven’t had a chance to revise the paper yet, and so your feedback is not yet included.
See you at the Hackathon!
Tags Haskell, Math, Monads, Probability | 5 comments
Posted by Eric Kidd
Tue, 18 Sep 2007 16:36:00 GMT
Good morning! I’ll be in Freiburg for several events this October, including CUFP and the Haskell Hackathon.
Commercial Users of Functional Programming (CUFP)
On October 4th, I’ll be speaking at CUFP ’07, describing the use of Scheme in a real-world multimedia engine. Some likely topics:
- How we switched to Scheme (and why refactoring is your friend).
- How our artists learned to program in Scheme (it’s all about the tools).
- The tension between functional programming and objects: Can we have both?
Dylan Beer Night
Once upon a time, I dreamt of generic functions and built RPMs for Gwydion Dylan.
Some current and former Dylan hackers are hoping to meet in Freiburg, most likely on October 4th. If you’re at ICFP or one of the workshops, we’d love to hear from you.
Haskell Hackathon
I’ll be at the Haskell Hackathon from Friday to Sunday.
Perhaps it’s time to whip Control.Monad.Probability into shape?
Tags Dylan, Haskell, Monads, Probability, Scheme | no comments
Posted by Eric Kidd
Tue, 18 Sep 2007 15:15:00 GMT
Just married! Our thanks and love to everyone,
for everything.
4 comments
Posted by Eric Kidd
Sun, 01 Jul 2007 23:00:00 GMT
Programming in Ruby makes me happy. It’s a lovable language, with a
pleasantly quirky syntax and lots of expressive power.
Programming in JavaScript, on the other hand, frustrates me to no end.
JavaScript could be a reasonable language, but it has all sorts of
ugly corner cases, and it forces me to roll everything from scratch.
I’ve been trying to make JavaScript a bit more like Ruby. In particular, I
want to support Ruby-style metaprogramming in JavaScript. This would make it possible to port over many advanced Ruby libraries.
You can
check out the interactive specification, or look at some examples
below. If the specification gives you any errors, please post them in the comment
thread, and let me know what browser you’re running!
Read more...
Tags JavaScript, Macros, Planetary, Ruby | 9 comments
