Samples for your viewing pleasure.
By day I'm a mild mannered software engineer. By night, I'm a code ninja. OK, I was a code ninja at night. Now I've got two daughters to photograph. ;-) If I have my way, I tend to write things in python because as I've been quoted on youtube (while at PyCon), it works the first time I write it and that's pretty damned cool.
Unfortunately, most of my code is locked up in the vaults of my company. But, I have written a fair amount of Free Software and generally open sourced code. I should add the disclaimer that some of this is poorly documented because it never intended to see the light of day outside of pheared labs, but professionally I'm a big proponent of documenting everything and test-driven development. To that end, a list of publicly available code I've written:
- I screen scraped the US Conference of Mayors stimulus wish-list and released a sqlite database. As an experiment in crowdsourcing, Peter Snyder and Jerry Brito then remixed it into a really slick web app called StimulusWatch.org. Take a peek at the lxml scraper, the database builder, and the MediaWiki stubber.
- ipfail.c and ipfail.h form a contribution to the Linux-HA project. Back in version 1.2, heartbeat would not initiatea failover if it lost connectivity, but could still talk to the other side of the cluster. We needed an HA solution for our corporate firewalls so I wrote this plugin to support our specific needs. It turned out that lots of people wanted this functionality and the plugin became quite popular (I even had NASA asking for suggestions on setting it up). The latest version of Linux-HA no longer uses ipfail because it has an entirely different architecture which makes it easy to solve the problem that ipfail solved.
- spider.py is a rather simple web spider that uses twisted to maximize throughput and concurrency. Be careful if you decide to let this one rip because it will do just that.
- Some years ago I took over the bidwatcher project and started hacking the code. I rewrote bidwatcher using python and glade but unfortunately haven't had time to work on it lately. It does work for watching auctions but not bidding yet. The code is much cleaner and shorter due to using python. I think it's also far more accessible to someone who might want to pick up the torch and resume hacking versus the old C++ code. My motivation to fix this has waned even further with the release of eBay's API (well, ever since it became free anyway). bidwatcher screen-scrapes eBay's pages which is a mess and no longer even necessary.
- My dealviewer is a deal aggregator. The dealer periodically pings several RSS feeds to see if new deals are available. When they are, some information is extracted and added to my postgres database. Simultaneously, the dealbot announces the deals to a channel on IRC: #deals on EFNet. On the client side, the dealviewer loads deals out of the database and then begins a polling AJAX loop to check for new deals. As new deals are found, it adds them to the list, and pops off excess deals without having to reload the entire page. Additionally, you may search through the entire database for deals matching a certain phrase.
- Things to do include a deal agent. This would be really useful because you could setup a stored query that checks for new results as deals are added to the database and notifies you of the results.
- The dealviewer is written using django. As a side-project I've been reimplementing it in Ruby on Rails just to see how things would be different.
- Another project of mine is a google mashup between google maps and my local police crime blotter. The Hyattsville Crime Map is kept up to date as I receive crime blotter e-mails from the city. I have hooked up mutt in such a way that I just press a key and it pipes the e-mail to my crimeParser.py. The crimeParser fills the database with crime events and also geocodes the locations. The GUI then searches for events and takes the geocoded coordinates and plots them on google's map. The GUI was written using quixote and python.
- sampler.py is a dirty little script to pack an mp3 player with songs much like the iPod shuffle.
- spamdrain.py is a neat utility for automatically trashing spam for an entire site's worth of users. It knows to dump spam that's older than a certain age and will leave people alone who don't get much spam (less than a threshold).
- wd-buttond.py is a proof-of-concept daemon to watch for events from my Western Digital External USB drive. The drive has several buttons on the front which are registered as a USB HID (Human Interface Device). Western Digital wasn't kind enough to provide any support at all (I tried to reason with them; I've got the e-mails) but I managed to reverse engineer the necessary details and hooked up the buttons so that they at least initiate a backup when I press one. Something that I noticed that's fairly interesting is that you can detect combinations of buttons, which is not supported by the provided WD Windows software, but could be used to implement any number of interesting features.