A long time ago in web years was written a blog post “Measure Anything, Measure Everything” by the devs at Etsy. It got me thinking about this issue, and it’s been really critical to me ever since. Years ago Seth Godin advised me that you have to measure in order to understand your effectiveness. And I’ve since found that whether that’s big or small, measuring removes anxiety. You know. You don’t have to guess. It’s either working or it’s not. And… it’s amazing how often people guess wrong. Also, people don’t believe that it is possible to measure a lot of things. But I’ve seen some great examples of making it possible to measure even really difficult things (like how you’re feeling) and reflect that data in a way that makes it clear.
In this specific case… I loved that fact that UDP keeps things loosely coupled, and that statsd rolls up measurements so that the raw data isn’t overwhelmingly large.
And last, I liked the fact that the folks at Metrics were friendly and helpful, even though they’re running pretty fast to get Metrics where they want it to be.
This is a brief guide to the steps I took to get a Rails 3 app sending notifications to statsd running on Ubuntu in the EC2 cloud, and pointed it at Metrics. I borrowed knowledge from here for the first steps, but I didn’t want some of the stuff that follows, and the recipes have grown out of date a bit, so realize that you’ll have some digging to do to get it to run. Generally, I didn’t find all that much on the topic of how to set this up so, maybe this will save another busy developer some time.
Note that this fires up a small EC2 instance, so you’ll have to pay for it…
download and install the EC2 API tools (e.g., into ~/tools)
generate a X.509 certificate (both public cert-…pem and private pk-…pem files) on the AWS Security Credentials page
generate a keypair (statsd_setup.pem) in the AWS Management Console
I stored the key files in ~/.ec2/ locally.
$ sudo chmod 600 ~/.ec2/statsd_setup.pem # set appropriate permissions on private key file
The EC2 API tools require certain environment variables to be set. I put the following in a local file (e.g., ~/bash/statsd_setup.sh):
export EC2_HOME=~/tools/ec2-api-tools-1.5.2.5
export EC2_PRIVATE_KEY=~/.ec2/pk-statsd_setup.pem
export EC2_CERT=~/.ec2/cert-statsd_setup.pem
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
export PATH=$PATH:$EC2_HOME/bin
$ source ~/bash/statsd_setup.sh
Finally, start the instances and modify some basic security settings:
$ ec2-run-instances ami-06ad526f -k statsd_setup # start a new instance with a recent Ubuntu 11 image
$ ec2-authorize default -p 22 # permit SSH
$ ec2-authorize default -p 80 # permit HTTP
$ ec2-authorize default -p 8125 -P udp # statsd will listen here
Then I logged into the instance once it was running: ssh -i ~/.ec2/statsd_setup.pem ubuntu@your_instance_here.amazonaws.com
Next, node and npm.
Install the dependencies:
sudo apt-get install g++ curl libssl-dev apache2-utils
sudo apt-get install git-core
sudo apt-get install make
When all done running…
mkdir dev
cd dev
git clone git://github.com/ry/node.git
cd node
./configure
make
sudo make install
If all went well node is installed. I smoke tested by simply typing the command “node”, and sure enough I got the prompt.
Since this is a test box, I took over /usr/local
sudo chown -R `whoami` /usr/local
and then installed npm in classic fashion: curl http://npmjs.org/install.sh | sh
next I moved back out to dev (cd ../)
And cloned statsd. Since I’m using Librato’s fork: git clone git://github.com/librato/statsd.git
I made a copy of the config example, and copied the settings I wanted in there.
I ran (as usual) npm install to get the dependencies installed, and then I fired it up: node stats.js myconfig.js
Win!
Locally, I fire up irb, where the gem statsd-ruby is already installed. I require ‘statsd’ (be warned, there’s more than one, and the namespaces collide), and fire off a few stats statsd = Statsd.new(your_host_here), and then statsd.increment ‘test’.
I already had my Metrics account, so I logged in and voilà, stats.
Next step wiring this into a Rails app. I create an initializer called notifications.rb. Inside is the code below cribbed mostly from here:
Which fires off all sorts of great info to statsd when you fire up the app and hit a few pages. Also, $statsd is available all over the place so you can measure pretty much anything going on in your app. I was surprised that there isn’t a simpler setup for this, but as I write this I tried Heroku, and Engineyard Cloud beta for node, and asked over at no.de and nodejitsu, and none of them support the standard port, or the UDP routing, or several of these requirements. It seems like this is an atypical scenario for now. Too bad, because it’s a great way to start measuring everything. A few months might change all of this, but for now that’s the state of things, so off to EC2 I went.
turnings :: daniel berlinger 