Egregious Thoughts

new photo galleries

Mon, 02 Jan 2012 02:05:20 GMT

I've been busy processing photo galleries from the last year (or two) and putting them online for your perusal.

2011 in review

Sun, 01 Jan 2012 06:46:25 GMT

2011 was the year that was pretty much defined by the earthquakes in Christchurch. Everything else seems insignificant, but it turns out I did other things too.

Christchurch was devastated by the earthquakes of 22 February. Unlike the big one last year, this one was almost directly under the city and destroyed lives, property, and livelihoods. Again we were lucky and did not sustain any major damage.
My parents visited in September. I took a couple of weeks off work and we did a big tour of the South Island and they had a great time.
Received a Microsoft MVP for C++ award for a second time. This is based on my continuing participation answering programming questions on Stack Overflow.
Amy and I officially became New Zealand citizens on 7 November.
Passed my NCEA Level 1 Chinese assessment. This involved a one minute spoken presentation where I talked about our trip with my parents.
In December, moved to Auckland. Amy will be attending school at AUT for a three-year programme for sign language interpreting for the Deaf. I'll be working remotely from home, which is new for me. We plan to return to Christchurch after Amy is done.

And, in the software section:

Published the Stack Overflow ebooks, which is a library of the top rated questions and answers on Stack Overflow for the most popular tags.
Contributed a number of improvements to Cppcheck, a static analyser for C and C++.
Started on a Javascript implementation of a Java VM. Others have done this too, I just wanted to try.
Added the NOAA Weather Radio tools to Github (I had written that code years ago, but had never released the source).
picomath is a collection of standalone mathematical functions in many different programming languages.

noaa weather radio tools

Sun, 05 Jun 2011 12:07:24 GMT

After many years (since 2003!) I have finally put the source code to my NOAA Weather Radio tools online at ghewgill/nwr on Github. Over the years I had quite a few people email me asking for the source code. I finally am able to reply to them all with a link to the code.

minilink performance improvement

Sun, 16 Jan 2011 01:08:18 GMT

My minilink/lnk.nu link shortener has been getting increasingly popular lately. There was a sharp jump up in activity in mid-October, which has now put the request rate at around 20k requests per day. There have been spikes up to 40k per day, too.

This increase in activity has taken a toll on my server, since minilink had been implemented as a simple Python CGI program. This was the original implementation I started with and I haven't needed to change it. However, CGI is only appropriate for low performance applications, and minilink is probably edging into medium performance at this point. A CGI program has to start up an entire Python runtime environment for every request.

Over the last two days, there have been big enough activity spikes that the load on my server goes through the roof and it can't really handle the requests anymore. This can cause things to crash and run out of memory and other impolite behaviours. This happened once last March, and twice in the last two days.

Today I did something I should have done long ago: I converted minilink from a CGI program to a WSGI program. WSGI improves performance because it keeps the Python runtime loaded in memory, and with Apache's mod_wsgi, the calls into Python code are much faster.

I also implemented a simple test suite so that I could refactor the implementation while ensuring that the behaviour stayed the same. That's something I should have done long ago, too.

You may see an increase in performance when using minilink starting today. If you see anything go wrong though, please let me know.

stack overflow questions in ebook format

Sun, 09 Jan 2011 05:32:08 GMT

In my quest to create useful reference ebooks for the Kindle, I've created ebooks of the top rated programming questions on Stack Overflow for the top 20 tags.

Stack Overflow ebooks

These files are created from the monthly Stack Overflow Creative Commons data dump. I've got a combination of Python, Java, and XSLT scripts that process the raw XML database dumps into something usable. Then the Amazon kindlegen program creates the ebook file (in Mobipocket format).

Last year I got a new computer with a fast CPU and lots of memory and disk space. While working on the XML processor, I realised that I was doing a lot of work seeking around a big XML file (it's over 4 GB) and collecting questions and answers together. This was taking quite some time because of the sheer size of the files. Since I am running a 64-bit OS (FreeBSD 8 amd64), I memory mapped the entire 4 GB XML file into memory and then didn't have to think about seeking anymore. Letting the OS manage the caching is a much better approach, and the improved performance really shows.

The preprocessing step (that needs to run once per data dump) creates all the HTML files for each question and its set of answers. I was originally storing all the files in one directory, but a million files in a single directory wasn't working very well. I ended up splitting the question number into groups of three digits, so 1234567.html is actually stored in 123/456/7.html. This step takes about two hours to run.

Creating each ebook file is then a single XSLT transformation (taking about a minute), plus the kindlegen step which can take several minutes depending on the number of questions. The performance of kindlegen isn't very impressive and appears to be O(n²) in the number of pages.

The source for all this is available on Github.

2010 in review

Sat, 01 Jan 2011 07:05:19 GMT

2010 was a pretty interesting year.

Went to Hawaii for three weeks with Amy. Didn't get to see flowing lava, but did get to swim with turtles! (Pictures coming when I get around to it, just got the underwater film roll developed a couple of weeks ago.)
Got a cat, "Toothless". I had never had a pet before, and he's now a constant source of smiles for us (eg. staring contest, chasing bees, big yawn).
Visited my grandmother in Canada for her 101½th birthday in a whirlwind weekend trip. She died a week before her 102nd birthday.
Repaired a hundred year old clock, which belonged to my great-great-great-grandfather.
Came through the 7.1 earthquake here in Christchurch with only superficial damage.
Got a Kindle, started reading a lot more.
Started a bathroom renovation project. Amy's doing the hard work (project management); I did the fun demolition.
Helped Amy with a mural on our bedroom wall.
Did an evening chocolate making class. 12 dozen chocolates is a lot!
New open source software: HP-15C calculator simulator, rss2mobi, Chinese-English dictionary for Kindle, pyqver updated for Python 3.

chinese-english dictionary for kindle

Thu, 30 Dec 2010 20:22:30 GMT

Further to my previous entry on Chinese text in the Kindle, I've put some more work into the Chinese-English dictionary I mentioned.

I used the CC-CEDICT dictionary as the basis for my dictionary. I also used the radical stroke count database from the Unihan Database. The result is a traditional radical-stroke order dictionary that seems reasonably usable on the Kindle.

One of the great features of the Kindle is the ability to set a default dictionary so that when you're reading text, you can simply point to a word and the Kindle will automatically look it up and show the definition in a small popup window. I've been able to create such dictionary files successfully for English, but have not been successful for Chinese. I currently suspect that the Kindle simply doesn't recognise the Chinese characters as something it can look up in a dictionary, so it doesn't even try. (If anybody has any ideas on this, feedback would be most welcome.)

The current result can be found here: Chinese-English Dictionary.

rss2mobi: google reader to mobipocket creator

Sun, 19 Dec 2010 01:47:45 GMT

I've been using my Kindle to read RSS feeds through Google Reader. Using the Kindle's built-in web browser, Google Reader works surprisingly well (as long as you enable keyboard shortcut commands). However, there are some annoyances:

The text is pretty small due to the layout of the Reader web page. It's possible to zoom in, but then you need to scroll left and right to read each line of text (which is awkward). The text is certainly not as readable as regular books on the Kindle.
Clicking on a link in Google Reader wants to open a new window, and the Kindle browser simply pops up a message box stating that multiple windows are not supported, with no option to continue.
You must be connected to the Internet to use Google Reader, and can't browse it offline.

In order to address these issues, I've created a program that creates proper e-books in Mobipocket format using kindlegen. I had to solve several problems to make this work:

I wanted to use the RSS feeds I've already configured in Google Reader. The Google Reader API is undocumented, so I had to rely on (out of date) descriptions of the API (eg. here and here).
Downloading just the text of feeds wasn't quite enough, so I had to download all the images and then rewrite the <img> tags in the HTML to refer to the included image files.
In order to tell the kindle each entry is a "chapter" and therefore allow navigation by skipping to the next entry, I had to create a table of contents in .ncx format.
Originally kindlegen refused to recognise my HTML files as UTF-8 encoded, and built Mobipocket files with the wrong encoding. Eventually I figured out through trial and error that the HTML file names must end in .html (I was using no extension).

Currently this rss2mobi program downloads all the new entries from Google Reader and packs them up into a file that I can manually copy to my Kindle. Next to do is:

Mark the downloaded entries as "read" in Google Reader so they aren't downloaded next time.
Automatically transfer the file to the Kindle. Amazon provides a service where I can email the document to a special email address, and then it will be automatically sent to the Kindle wirelessly.

Once all this is in place, I can set up a daily task that gets the new content from Google Reader, and transfers it to my Kindle without any manual interaction.

You can find the code for rss2mobi on Github.

Margaret Hunter Young (Gammie), 15 Dec 1908 - 8 Dec 2010

Wed, 15 Dec 2010 10:40:46 GMT

Today my grandmother would have been 102. She was born in Aberdeen, Scotland in 1908, and moved to Vancouver, Canada with her family a few years later. She was a teacher of French and Latin when she married my grandfather, but stopped teaching when they married (this was required by law at the time).

My grandmother never learned to drive a car, and I don't think she ever even rode a bicycle. However, she traveled a lot with my grandfather, both for his work (teacher training) and for leisure. During these travels, they made many lifelong friends. I remember at Christmas time they always had dozens (or was it hundreds?) of greeting cards hung on string around their dining room.

She had an astounding memory. Some 35ish years ago, they traveled to New Zealand (and Australia, Fiji, etc), but the trip was still so fresh in her mind that she could even recall specific roads they had driven here in New Zealand. We would tell her about a trip we took the week before, and she could probably remember details better than we could.

When Amy and I would call to chat with her, she was always glad to hear from us but she always tried to keep the call short. After, all, it was long distance and must have cost us a fortune! (Really, it doesn't add up very fast at Skype rates.) She never quite got the hang of the modern electronic world; the most advanced device she learned to use was probably a microwave oven.

Amy and I took a quick trip to Canada in June to celebrate her 101½th birthday (when you get to that age, you get to choose what time of year you want to celebrate your birthday). It was great to see her, she was doing very well all things considered.

Margaret passed quietly in her sleep last week. Today we had cup of tea with dinner in her honour, that's what she would have wanted.

kiwi pycon 2010 notes

Tue, 30 Nov 2010 09:24:30 GMT

Last week I went to Kiwi PyCon 2010 in Waitangi (New Zealand, of course). I didn't take my laptop and the Kindle isn't really built for notetaking, so I scribbled notes the old fashioned way on a sheet of venue-provided note paper. The following is a dump of what I have.

Consider Fabric for automating deployment/admin tasks
A different approach is Chef, which is a declarative language for describing what you want your deployment to look like
Wikkid is a new wiki, similar to ikiwiki but in Python. It uses repoze.who for authentication, which needs OpenID support.
lettuce is like cucumber for Python
Learn Hudson. Probably more well-supported than Buildbot.
When creating automated test code, prefer anonymous "stock" object creation functions instead of specifying the full content of the object all the time. With the full content, how can you easily identify what's special or important about that object?
Use an addCleanup style method to localise cleanup of test objects
Apparently, reduce() is almost always used to sum numbers (now we have sum(), of course). In my code, I seem to have also used it for max(), string.join(), and all().
Does my Psil deserve to be on PyPi? Maybe.
Prefer elementtree over xml.dom
Investigate Mox for mock objects
Python has a pure functional subset. Write something to identify pure functional code. Introduce a pattern matching decorator. Offer some Erlang-like features for Python (easy message passing, lots of parallel processes, etc).
Wikkid again. Support Git in addition to Bzr.
There exists a 3to2 converter in addition to 2to3
Embed Unicode characters by name: "\N{LATIN CAPITAL LETTER H}ello world" == "Hello world"
Unidecode claims to be able to convert Chinese characters to Pinyin
Chardet can auto-detect character encodings

Kiwi PyCon was great again this year! Lots of good presentations and worthwhile discussion. Plenty of excellent giveaways too, nobody went away empty-handed.

kindle 3 and chinese text

Tue, 09 Nov 2010 09:50:26 GMT

A few weeks ago I started a beginner level Chinese (Mandarin) evening class at the local community college. We're about four weeks in and I'm really enjoying it, will definitely do more.

I thought it would be an interesting project to load a Chinese-English dictionary onto my Kindle for reference. I've already played with kindlegen, which takes a collection of HTML files along with some additional metadata and creates a MobiPocket .mobi format file for the Kindle. (I've got several ideas for reference documents that could be loaded onto the Kindle, more on that later.)

Anyway, I've now got a prototype Chinese dictionary loaded onto the Kindle. However, although many of the characters are displayed correctly, quite a few are just empty boxes (in the form of I ⬜ Unicode). I couldn't find any particular pattern to the missing characters, for example 他 (he) displayed correctly but 她 (she) did not.

Some research with Google showed that some people had hacked the Kindle and found a way to load different font files onto the device. I didn't try doing that, becuase I thought there should be a better solution. After all, Chinese text is displayed correctly in the Kindle web browser! Furthermore, if I copy a UTF-8 format file directly to the Kindle for display, it will have the missing character problem. But if I email the file to Amazon where they automatically reformat it to a .azw book file and deliver it to the Kindle, the characters show up correctly. If I convert to a MobiPocket book file myself, missing characters.

After laboriously paging through dozens of threads on mobileread.com (typical internet "forum" software really is awful), I finally found a solution. It uses an undocumented debug feature of the Kindle. Press <Home> <Search> and enter the following commands:

    ;debugOn <Enter>
    ~setLocale zh-CN <Enter>

That's it, that is all that was required. I have no idea what this actually does or how it changes the Kindle's interpretation of UTF-8 documents (UTF-8 is supposed to be an unambiguous encoding of Unicode). But I now have a very basic Chinese-English dictionary in my Kindle.

my weekend in pictures

Mon, 25 Oct 2010 07:34:20 GMT

Saturday: Removed a tonne of bricks from the front path, piled them in the back. Sunday: Laid down weed mat and installed close to a tonne of gravel. The bricks had grass and weeds growing between them and were always damp and slippery.

Monday: Bathroom renovation begins. Removed grotty old shower door, bathtub, and shelves. Now we have an (almost) empty box inside the house which will (eventually) contain a new bathroom.

What I didn't get to:

bad math: pet ownership and lottery winners

Sun, 03 Oct 2010 20:21:35 GMT

A headline in the local paper this morning reads Lucky Lotto winnners like cats with the first two paragraphs:

HAVING A cat could improve your chances of winning Lotto - by a whisker.

A survey released today by New Zealand Lotteries shows 39 percent of big Lotto winners have cats, making felines the most popular pets among lucky Lotto winners. Dogs are second but way behind with only 27 percent of big winners preferring man's best friend to moggies.

My first reaction upon reading that was, "so what?". Surely the pet ownership statistics among the general population would be reflected in those statistics for the small number of lotto winners. The New Zealand Petfood Manufacturers Assocation claims the ownership numbers for New Zealand as a whole are 53% for cats and 35% for dogs.

What does this tell us? These numbers show that a greater proportion of non-pet owners win the lottery than there are non-pet-owners in the general population. Having a cat certainly doesn't "improve your chances" as the article claims. (These are just rough numbers because we don't know what proportion of either the general population or lottery winners own both a cat and a dog.)

They say the lottery is a tax on people who are bad at math. Apparently, the "bad at math" part extends to those writing about the lottery, too.

amazon kindle first impressions

Sat, 18 Sep 2010 04:10:52 GMT

I got my Amazon Kindle 3 earlier this week! I pre-ordered it around the end of August and seemed to receive it here in New Zealand before some US users received theirs. It was packed in an integrated protective/shipping box, and my first thought upon opening it was, "Amazon has been taking lessons from Apple." The device was sitting face up with clear plastic film on both sides that was easy to remove. Since the e-ink display doesn't take any power to hold the display, they had already displayed basic instructions on the screen ready to go (just where to plug it in and where the power switch is).

The display is fantastic! It's super high resolution, quite high contrast, and the brighter the ambient light is, the easier it is to read. The first day after I got it, I sat on a beach in full afternoon sun and it was brilliant. Any other LCD display would have been struggling.

I got the model with both WiFi and 3G, and both seem to work well enough. There was no trouble connecting to my home network, and the 3G works fine even if I'm on the bus. The price of 3G networking is included in the cost of the device itself and books you buy, so there's no contract and I need to have no relationship with whoever provides the 3G. The Kindle now has an experimental web browser which can browse any web site on WiFi but is limited to the Kindle store and Wikipedia on 3G. However, with Wikipedia on free 3G this device actually is the Hitchhiker's Guide to the Galaxy. All it needs is DON'T PANIC in large friendly letters on the front.

For the first few days, I didn't spend anything on content because Project Gutenberg (among others) provide free ready-to-download ".mobi" format files for the Kindle. Downloading books is a simple click and the book magically shows up on the Kindle home page. The first (and only, so far) book I bought was Erlang Programming which I bought on my way home on the bus the other day. The purchasing experience is seamless so I'm going to have to watch that I don't spend all my money on books.

The Kindle comes with two dictionaries built in, the "Oxford Dictionary of English" and "The New Oxford American Dictionary". Both these are books in their own right that can be read and browsed, but also when reading any other content you can point to a word and look it up in the dictionary (you can choose which one is your default). I often look up words to be sure I know what they mean (I'm a big user of wiktionary.org) so having the dictionary available everywhere is excellent.

There are some highlighting and note-taking features, and these seem to integrate into the social media space somehow so you can see what other people have highlighted and annotated. I haven't played with any of those features yet.

The web browser itself feels quite clunky and difficult to use (this is partly because the screen isn't a touch screen and you have to use the little navigation buttons to move a pointer around to point to things). I could log in and use some sites like Twitter, but the browser wouldn't let me type my name into the Gmail login screen for example (I tried both the regular and "mobile" sites). I look forward to upgrades on the browser, hopefully they will correct these problems. But the Kindle will never match an iPhone/iPod/iPad for general browsing capability.

I'm very happy with the Kindle and I believe I will find myself reading more because of it.

one hundred year old heirloom

Fri, 27 Aug 2010 11:10:23 GMT

When I was young, my grandparents had a fancy marble clock in their home. It always fascinated me, because it would chime every half hour (they would turn on the chime for me). At some point, a picture frame fell on the clock, which broke apart nearly all the marble pieces. The clock got put in a box where it stayed for 30 years. But my grandparents always said it was my clock.

Fast forward to earlier this year. Amy happened to be looking at antique clocks as she was thinking that something like that would go well on the mantel above the fireplace. I said, "Hey, did I ever tell you about my clock?" Apparently, I hadn't. The next day we called my parents, who found the box of clock parts under the stairs, and we managed to coordinate its transport to here in New Zealand with only minor damage.

Some of the marble bits were broken and one of the metal movement supports was bent, but everything was present and the movement (all the mechanical bits) worked. I researched its history and found that it was probably made in France in the late 19th or early 20th century, and the adhesive used at that time was roughly, plaster of Paris and egg white. This probably protected the marble pieces because the adhesive wasn't that strong and came apart first before breaking the marble. However, we decided to put it back together with a modern glue (not too strong, but not just white glue).

The more interesting part about this clock is its history as it relates to my family. The inscription reads:

Sons of Temperance Friendly Society
Birmingham District Grand Division
Presented to Bro. William Aucott J.P.
as a mark of esteem and appreciation
on completion of 44 years continuous service
as Trustee of the above Society
August 27th 1910

William Aucott was my great great great grandfather. He was born in 1830 in Leicestershire in England, was an early trade union leader when such things were being founded for the first time in 1863, became president of the National Association of Ironworkers in 1874, and served in a number of different roles in different labour organisations throughout his lifetime. He was involved in the Methodist church, helped establish the Sons of Temperance Society in Wednesbury, and was a magistrate on the Wednesbury bench.

Exactly one hundred years ago today, William Aucott was presented with this very clock as thanks for his service to the society. I'm sure he would be happy to know that the clock remains a treasured posession in our family.

web site content management using git

Tue, 24 Aug 2010 08:43:20 GMT

For the past several years, I've managed my web site content using a local copy (of everything) on my home machine, and a simple rsync script that transfers changed files up to my web site host. While this works, it's limited (there's only one "source" copy of my content) and fragile (if I need to change something on the server, I need to remember to make a local copy before syncing again).

I've always known it would be better to manage the content using a real source control system. When I first started managing my web site like this, the state of the art in source control was Subversion, but I never got around to actually setting it up. Today, my source control system of choice is Git, and it seems like a natural fit.

Importing all my content into a new repository is easy, and I pushed up the repository to my web host overnight. The compressed repo size is 2.6 GB and Git handles it just fine. It feels much better to have multiple mirror copies of the site content, and it's much nicer to be able to manage it using a powerful tool like Git.

HP-15C calculator simulator

Tue, 22 Jun 2010 10:55:17 GMT

I'm a bit of a calculator geek. My favourite calculator is the HP-15C, an advanced scientific programmable RPN calculator made in the 1980s. It's got a distinctive form factor:

I bought mine in 1987 and it got me through university, and still runs on the same set of batteries it came with 23 years ago. The HP-12C financial version of this calculator did so well that it continues to be sold to this day.

These days it's a bit awkward to actually carry a calculator around (since there are computers everywhere), so I've implemented a (nearly complete) simulator for the HP-15C in Javascript. You can find it at http://hp15c.com along with some more information about its capabilities and current known issues.

If you're unfamiliar with this calculator, you can find a scanned copy of the Owner's Handbook (and the Advanced Functions Handbook) online. There is even a Petition to Bring Back the 15C that you can sign.

If you've ever used this calculator, or one like it, I hope you find this simulator entertaining. Feedback, bug reports, and even contributions are welcome!

cat

Sat, 17 Apr 2010 23:59:51 GMT

Toothless

Yesterday Amy and I adopted a young black male cat from the SPCA. We had been talking about getting a cat for a while, and while we were out yesterday decided to go have a look. They had quite a few cats available, and we both quite liked the look of a small grey female with a very distinguished narrow face. However, this black one seemed much friendlier and purred a lot. It was a toss-up for a while but we were both won over by the black one.

We had just finished watching the movie "How to Train Your Dragon", which features a rather feline black dragon named "Toothless". That would be a fitting name for a black cat, we thought. While we were doing the paperwork, they noted that he was due for his deworming pill, so they gave him the pill right there. But while he was taking the pill, we noticed one of his teeth didn't look right. Sure enough, it was askew and fell out in Amy's hand! They suspected that it may have been like that for a while and that he would be fine without it. We realised right then that the name "Toothless" would stick.

He's very friendly and purrs almost continuously when sitting near us (right now I'm slouching on the futon with my laptop propped up on my knees, with Toothless sitting on my chest purring). He knows where his food and litter box is, but we're definitely going to have to get him a scratching post to save our furniture. For now he's got full reign over the living room and kitchen, and we'll introduce him to the rest of the house in time. Ultimately he will be an outdoor cat too, because we regularly leave windows and doors open to let the air through the house. Hopefully he won't get too tangled up with the other cats in the neighbourhood (of course he's been fixed).

Very happy with him.

sunrise/sunset update, new google chart api features

Thu, 18 Feb 2010 09:41:36 GMT

Just a small update today: I updated the Sunrise and Sunset graph page to show a green line on the graph to indicate the current date.

In related news, I noticed that the Google Chart API has been significantly upgraded recently, with interactive charts (a fully Javascript solution, no plugins such as Flash), plus new types of static charts including Forumlas using TeX.

careless people

Tue, 16 Feb 2010 07:49:46 GMT

Today when I went to get my bike from the bike shed at work on the way home, I noticed that it had been shifted from one side of its stall to the other. As soon as I got on, I noticed the right handgrip was tweaked to the side, and as soon as I shifted gears I noticed that the derailleur was wonky. I could still shift but the gears were all out of alignment.

It looks like somebody knocked my bike over at some point today, which scuffed up the right handgrip and bent the derailleur hanger. I'm annoyed because of careless people who knock over other people's bikes, and doubly annoyed because it's going to cost me a new derailleur hanger to fix this.

twig - twitter irc gateway

Wed, 10 Feb 2010 08:24:52 GMT

My primary "instant" communications medium is IRC. I've used IRC for roughly 15 years and it's an integral part of my communications toolkit.

In order to follow Twitter, rather than having to reload the Twitter home page all the time or use a separate Twitter client, I was looking for a way to make tweets show up in an IRC channel. Originally I wrote an IRC "bot", which reloaded my Twitter page periodically and posted new tweets to an IRC channel. This was a bit fragile and wasn't a great solution, since it depends on connecting to some existing IRC network.

There is a project called BitlBee that is an IRC server that connects to various different IM networks (AIM, Jabber, etc). You connect your IRC client to your own local BitlBee server, which then connects to the IM networks of your choice. BitlBee is written in C and is therefore less easy to adapt and modify than if it were implemented in a more modern language.

I wanted to play with the Twitter Streaming API, so I wrote a Twitter-IRC gateway in Python called twig. Right now it does the simplest thing possible - it forwards tweets from the people you follow on Twitter into an IRC channel. It's a select-loop based single threaded implementation which should be easy to extend and modify.

If you use IRC and want to follow Twitter, check it out on Github.

the state of the livejournal

Sun, 24 Jan 2010 18:34:12 GMT

My LiveJournal "friends" page looks pretty sad today. It's got 25 entries covering four days (back in the heyday of LJ, my friends list would typically roll over 25 entries in a day). There's one of my own entries, but as for the rest of them:

Number of new posts in the last 24 hours: 0 (except for status telling everybody the site is still up)
Number of posts by people I've actually met in real life: 0
12 entries by jwz (occasionally amusing but mostly videos I don't watch)
2 annoying "loudtwitter" crossposts
1 LJ news update from news
1 post where a mutual friend actually posts words they've written instead of just pictures.

LiveJournal = dead.

two dead mp3 players, what next?

Sat, 23 Jan 2010 01:11:29 GMT

I usually only use my MP3 player (Creative Zen Stone Plus) to listen to podcasts when I take the bus to work. After the Christmas holiday break, I went to plug in my player to copy the latest podcasts to it, and it didn't power up. I tried the reset button on it, I tried "charging" it for a whole day, I tried everything suggested on the Creative troubleshooting site, and didn't get so much as a blink out of its little LED. It's possible that the battery was drained to a level below some threshold where it refuses to charge again (apparently that sometimes happens with Li-ion batteries).

I liked the player, so I found another identical one on Trademe (New Zealand's own version of Ebay). I charged it up, copied my current files to it, and it worked like a charm. For just two days. Then, as I was copying some more files to it, I got some odd I/O errors during copying, followed by generally slow performance (and USB timeout errors in my Macbook system.log file), followed by a complete loss of files on the device and and inability to boot it up (turning it on shows the "Creative" logo screen and nothing further). That was an unfortunate waste of $60.

I'm now on the hunt for a new player. One of the features I use on the player is the ability to delete a file while I'm listening to it. I do this whenever I'm done listening to a podcast, as unlike music the exception is wanting to keep the file. Anyway, a file delete operation is a must, and I never realised that this was apparently an uncommon feature. Fortunately, today virtually all manufacturers provide the full user manuals for their devices online for download. Because deleting files seems to be an obscure feature, it generally doesn't seem to be mentioned in the short blurbs.

Sony Walkman - no delete function
Philips GoGear - no delete function
Samsung U4 - delete function, but not current file!
Creative Zen Stone - delete function, including current file

I'm sort of having trouble finding other models because I know I don't want an iPod (I've written my own program to download and manage podcast feeds, so I need an ordinary filesystem interface). I also don't care about: video, photos, hackability (ie. I don't need to load Rockbox), more than about 4GB of space, or anything over about NZ$100.

I'm currently undecided between trying my luck with another Zen Stone from Trademe (as they don't seem to be sold at retail here anymore), or getting a Samsung U4 at retail so I can return/replace it if it's not working for me. Does anybody have any other ideas?

y2010 bugs found via google code search

Thu, 07 Jan 2010 10:45:24 GMT

Last week the news started coming about regarding various computing system failures caused by the rollover to the year 2010. I wondered how easy it would be to identify such bugs in open source software, using Google Code Search. What kind of bug would be would be easy to identify? A common error in the last century was to use a C printf format string "19%d", which would roll over from 1999 to 19100 at the turn of the century.

What if people used "200%d" as a format string? That would roll over from 2009 to 20010 in the year 2010. But surely nobody would actually do that, right? Wrong. Some of those hits are false hits and not relevant to dates, but I found about 10 open source projects with such date-related format strings. Some of them are:

I've sent suggested patches to fix the bug(s) to each project that I could find.

I'm pretty sure this technique of using Google Code Search has been used to locate unsafe coding practices related to software security vulnerabilities, but I wonder whether anybody has successfully applied it to other types of software bugs.

2009 in review

Fri, 01 Jan 2010 03:22:05 GMT

2009 seems to have had even fewer changes than previous years. But it still had lots going on!

Circumnavigated the globe for the first time. Amy had already done this once, and it's good to catch up.
As a result of the above trip, I visited Germany, Ireland, Iceland (with the furthest from the equator I've ever been, past 66°N), London, and Tokyo for the first time.
Amy and I had a big wedding reception (two years later!) with my extended family in Vancouver. Saw lots of people I hadn't seen for years.
Attended SXSW as a participant for the first time. Plus, the seventh annual Nuclear Tacos at SXSW!
Met Andy Hopper (the head of the Computer Technology Department at Cambridge University) through a string of coincidences, plus some work I did on VNC back in 1998.
Presented a lightning talk on Psil at Kiwi PyCon.
New open source software: pyqver (identify required version for Python code), emulino (Arduino emulator), mandelbrot set viewer, relight.
Works in progress: psil, xearth.