Below are the 20 most recent journal entries recorded in Greg Hewgill's LiveJournal:

[ << Previous 20 ]

wednesday, january 1, 2014
7:01 pm	2013 in review 2013 is the year when everything changed. The biggest event was the birth of our daughter Lily. She was born prematurely in Shanghai while we were on a short stopover on our way home from Europe in July. Due to the generosity of thousands of people in New Zealand and around the world, we were able to fly her home by private jet in September. The logistics of carrying such precious cargo by air is insanely complicated. We thank everybody who played a part, big or small, in her safe trip home. Being a dad for the first time has revealed challenges and emotions that I was not previously aware of. But no matter what temporary difficulties I might have, it's all worth it when I get a spontaneous smile from my little girl. Other things happened too, I guess. Amy and I visited France, Italy, and Switzerland. I haven’t yet done anything with the photos we took. Both my parents and Amy's mom visited shortly Lily came home from hospital. Everybody enjoyed meeting Lily and exploring the Auckland area. New software this year: Ported Simon Tatham's Portable Puzzle Collection to iOS. (2 Comments |Comment on this)
tuesday, january 1, 2013
9:34 am	2012 in review 2012 has been fairly quiet. Maybe it just seems that way because I haven't actually written anything new in this blog since last year's annual summary. Spent an entire year working from home (telecommuting). I miss the social aspect of the workplace, but the zero commute sure is nice. Since I'm no longer cycling to work, I started running regularly. Logged 1052 km last year. I even ran 20 km (half marathon) before breakfast one day. We tried to buy a house in Auckland, but were immediately priced out of the market at auction. Now renting a nice flat in a great location. Quote of the year: "You don't stop learning because you get old; you get old because you stop learning." New software this year: NZSL Dictionary is a dictionary for New Zealand Sign Language. Auckland Trains is a train timetable app for Auckland, New Zealand. Free 15C is an HP-15C calculator simulator, which is now available for iOS. (1 Comment |Comment on this)
monday, january 2, 2012
3:05 pm	new photo galleries I've been busy processing photo galleries from the last year (or two) and putting them online for your perusal. Vancouver 2010 Northland 2010 South Island 2011 Snow in Christchurch Toothless the cat (Comment on this)
sunday, january 1, 2012
7:46 pm	2011 in review 2011 was the year that was pretty much defined by the earthquakes in Christchurch. Everything else seems insignificant, but it turns out I did other things too. Christchurch was devastated by the earthquakes of 22 February. Unlike the big one last year, this one was almost directly under the city and destroyed lives, property, and livelihoods. Again we were lucky and did not sustain any major damage. My parents visited in September. I took a couple of weeks off work and we did a big tour of the South Island and they had a great time. Received a Microsoft MVP for C++ award for a second time. This is based on my continuing participation answering programming questions on Stack Overflow. Amy and I officially became New Zealand citizens on 7 November. Passed my NCEA Level 1 Chinese assessment. This involved a one minute spoken presentation where I talked about our trip with my parents. In December, moved to Auckland. Amy will be attending school at AUT for a three-year programme for sign language interpreting for the Deaf. I'll be working remotely from home, which is new for me. We plan to return to Christchurch after Amy is done. And, in the software section: Published the Stack Overflow ebooks, which is a library of the top rated questions and answers on Stack Overflow for the most popular tags. Contributed a number of improvements to Cppcheck, a static analyser for C and C++. Started on a Javascript implementation of a Java VM. Others have done this too, I just wanted to try. Added the NOAA Weather Radio tools to Github (I had written that code years ago, but had never released the source). picomath is a collection of standalone mathematical functions in many different programming languages. (1 Comment |Comment on this)
sunday, june 5, 2011
12:07 pm	noaa weather radio tools After many years (since 2003!) I have finally put the source code to my NOAA Weather Radio tools online at ghewgill/nwr on Github. Over the years I had quite a few people email me asking for the source code. I finally am able to reply to them all with a link to the code. (Comment on this)
sunday, january 16, 2011
2:07 pm	minilink performance improvement My minilink/lnk.nu link shortener has been getting increasingly popular lately. There was a sharp jump up in activity in mid-October, which has now put the request rate at around 20k requests per day. There have been spikes up to 40k per day, too. This increase in activity has taken a toll on my server, since minilink had been implemented as a simple Python CGI program. This was the original implementation I started with and I haven't needed to change it. However, CGI is only appropriate for low performance applications, and minilink is probably edging into medium performance at this point. A CGI program has to start up an entire Python runtime environment for every request. Over the last two days, there have been big enough activity spikes that the load on my server goes through the roof and it can't really handle the requests anymore. This can cause things to crash and run out of memory and other impolite behaviours. This happened once last March, and twice in the last two days. Today I did something I should have done long ago: I converted minilink from a CGI program to a WSGI program. WSGI improves performance because it keeps the Python runtime loaded in memory, and with Apache's mod_wsgi, the calls into Python code are much faster. I also implemented a simple test suite so that I could refactor the implementation while ensuring that the behaviour stayed the same. That's something I should have done long ago, too. You may see an increase in performance when using minilink starting today. If you see anything go wrong though, please let me know. (3 Comments |Comment on this)
sunday, january 9, 2011
6:32 pm	stack overflow questions in ebook format In my quest to create useful reference ebooks for the Kindle, I've created ebooks of the top rated programming questions on Stack Overflow for the top 20 tags. Stack Overflow ebooks These files are created from the monthly Stack Overflow Creative Commons data dump. I've got a combination of Python, Java, and XSLT scripts that process the raw XML database dumps into something usable. Then the Amazon kindlegen program creates the ebook file (in Mobipocket format). Last year I got a new computer with a fast CPU and lots of memory and disk space. While working on the XML processor, I realised that I was doing a lot of work seeking around a big XML file (it's over 4 GB) and collecting questions and answers together. This was taking quite some time because of the sheer size of the files. Since I am running a 64-bit OS (FreeBSD 8 amd64), I memory mapped the entire 4 GB XML file into memory and then didn't have to think about seeking anymore. Letting the OS manage the caching is a much better approach, and the improved performance really shows. The preprocessing step (that needs to run once per data dump) creates all the HTML files for each question and its set of answers. I was originally storing all the files in one directory, but a million files in a single directory wasn't working very well. I ended up splitting the question number into groups of three digits, so 1234567.html is actually stored in 123/456/7.html. This step takes about two hours to run. Creating each ebook file is then a single XSLT transformation (taking about a minute), plus the kindlegen step which can take several minutes depending on the number of questions. The performance of kindlegen isn't very impressive and appears to be O(n2) in the number of pages. The source for all this is available on Github. (2 Comments |Comment on this)
saturday, january 1, 2011
7:05 am	2010 in review 2010 was a pretty interesting year. Went to Hawaii for three weeks with Amy. Didn't get to see flowing lava, but did get to swim with turtles! (Pictures coming when I get around to it, just got the underwater film roll developed a couple of weeks ago.) Got a cat, "Toothless". I had never had a pet before, and he's now a constant source of smiles for us (eg. staring contest, chasing bees, big yawn). Visited my grandmother in Canada for her 101½th birthday in a whirlwind weekend trip. She died a week before her 102nd birthday. Repaired a hundred year old clock, which belonged to my great-great-great-grandfather. Came through the 7.1 earthquake here in Christchurch with only superficial damage. Got a Kindle, started reading a lot more. Started a bathroom renovation project. Amy's doing the hard work (project management); I did the fun demolition. Helped Amy with a mural on our bedroom wall. Did an evening chocolate making class. 12 dozen chocolates is a lot! New open source software: HP-15C calculator simulator, rss2mobi, Chinese-English dictionary for Kindle, pyqver updated for Python 3. (Comment on this)
friday, december 31, 2010
9:22 am	chinese-english dictionary for kindle Further to my previous entry on Chinese text in the Kindle, I've put some more work into the Chinese-English dictionary I mentioned. I used the CC-CEDICT dictionary as the basis for my dictionary. I also used the radical stroke count database from the Unihan Database. The result is a traditional radical-stroke order dictionary that seems reasonably usable on the Kindle. One of the great features of the Kindle is the ability to set a default dictionary so that when you're reading text, you can simply point to a word and the Kindle will automatically look it up and show the definition in a small popup window. I've been able to create such dictionary files successfully for English, but have not been successful for Chinese. I currently suspect that the Kindle simply doesn't recognise the Chinese characters as something it can look up in a dictionary, so it doesn't even try. (If anybody has any ideas on this, feedback would be most welcome.) The current result can be found here: Chinese-English Dictionary. (Comment on this)
sunday, december 19, 2010
2:47 pm	rss2mobi: google reader to mobipocket creator I've been using my Kindle to read RSS feeds through Google Reader. Using the Kindle's built-in web browser, Google Reader works surprisingly well (as long as you enable keyboard shortcut commands). However, there are some annoyances: The text is pretty small due to the layout of the Reader web page. It's possible to zoom in, but then you need to scroll left and right to read each line of text (which is awkward). The text is certainly not as readable as regular books on the Kindle. Clicking on a link in Google Reader wants to open a new window, and the Kindle browser simply pops up a message box stating that multiple windows are not supported, with no option to continue. You must be connected to the Internet to use Google Reader, and can't browse it offline. In order to address these issues, I've created a program that creates proper e-books in Mobipocket format using kindlegen. I had to solve several problems to make this work: I wanted to use the RSS feeds I've already configured in Google Reader. The Google Reader API is undocumented, so I had to rely on (out of date) descriptions of the API (eg. here and here). Downloading just the text of feeds wasn't quite enough, so I had to download all the images and then rewrite the <img> tags in the HTML to refer to the included image files. In order to tell the kindle each entry is a "chapter" and therefore allow navigation by skipping to the next entry, I had to create a table of contents in .ncx format. Originally kindlegen refused to recognise my HTML files as UTF-8 encoded, and built Mobipocket files with the wrong encoding. Eventually I figured out through trial and error that the HTML file names must end in .html (I was using no extension). Currently this rss2mobi program downloads all the new entries from Google Reader and packs them up into a file that I can manually copy to my Kindle. Next to do is: Mark the downloaded entries as "read" in Google Reader so they aren't downloaded next time. Automatically transfer the file to the Kindle. Amazon provides a service where I can email the document to a special email address, and then it will be automatically sent to the Kindle wirelessly. Once all this is in place, I can set up a daily task that gets the new content from Google Reader, and transfers it to my Kindle without any manual interaction. You can find the code for rss2mobi on Github. (1 Comment |Comment on this)
wednesday, december 15, 2010
11:40 pm	Margaret Hunter Young (Gammie), 15 Dec 1908 - 8 Dec 2010 Today my grandmother would have been 102. She was born in Aberdeen, Scotland in 1908, and moved to Vancouver, Canada with her family a few years later. She was a teacher of French and Latin when she married my grandfather, but stopped teaching when they married (this was required by law at the time). My grandmother never learned to drive a car, and I don't think she ever even rode a bicycle. However, she traveled a lot with my grandfather, both for his work (teacher training) and for leisure. During these travels, they made many lifelong friends. I remember at Christmas time they always had dozens (or was it hundreds?) of greeting cards hung on string around their dining room. She had an astounding memory. Some 35ish years ago, they traveled to New Zealand (and Australia, Fiji, etc), but the trip was still so fresh in her mind that she could even recall specific roads they had driven here in New Zealand. We would tell her about a trip we took the week before, and she could probably remember details better than we could. When Amy and I would call to chat with her, she was always glad to hear from us but she always tried to keep the call short. After, all, it was long distance and must have cost us a fortune! (Really, it doesn't add up very fast at Skype rates.) She never quite got the hang of the modern electronic world; the most advanced device she learned to use was probably a microwave oven. Amy and I took a quick trip to Canada in June to celebrate her 101½th birthday (when you get to that age, you get to choose what time of year you want to celebrate your birthday). It was great to see her, she was doing very well all things considered. Margaret passed quietly in her sleep last week. Today we had cup of tea with dinner in her honour, that's what she would have wanted. (1 Comment |Comment on this)
tuesday, november 30, 2010
10:24 pm	kiwi pycon 2010 notes Last week I went to Kiwi PyCon 2010 in Waitangi (New Zealand, of course). I didn't take my laptop and the Kindle isn't really built for notetaking, so I scribbled notes the old fashioned way on a sheet of venue-provided note paper. The following is a dump of what I have. Consider Fabric for automating deployment/admin tasks A different approach is Chef, which is a declarative language for describing what you want your deployment to look like Wikkid is a new wiki, similar to ikiwiki but in Python. It uses repoze.who for authentication, which needs OpenID support. lettuce is like cucumber for Python Learn Hudson. Probably more well-supported than Buildbot. When creating automated test code, prefer anonymous "stock" object creation functions instead of specifying the full content of the object all the time. With the full content, how can you easily identify what's special or important about that object? Use an addCleanup style method to localise cleanup of test objects Apparently, reduce() is almost always used to sum numbers (now we have sum(), of course). In my code, I seem to have also used it for max(), string.join(), and all(). Does my Psil deserve to be on PyPi? Maybe. Prefer elementtree over xml.dom Investigate Mox for mock objects Python has a pure functional subset. Write something to identify pure functional code. Introduce a pattern matching decorator. Offer some Erlang-like features for Python (easy message passing, lots of parallel processes, etc). Wikkid again. Support Git in addition to Bzr. There exists a 3to2 converter in addition to 2to3 Embed Unicode characters by name: "\N{LATIN CAPITAL LETTER H}ello world" == "Hello world" Unidecode claims to be able to convert Chinese characters to Pinyin Chardet can auto-detect character encodings Kiwi PyCon was great again this year! Lots of good presentations and worthwhile discussion. Plenty of excellent giveaways too, nobody went away empty-handed. (8 Comments |Comment on this)
tuesday, november 9, 2010
10:50 pm	kindle 3 and chinese text A few weeks ago I started a beginner level Chinese (Mandarin) evening class at the local community college. We're about four weeks in and I'm really enjoying it, will definitely do more. I thought it would be an interesting project to load a Chinese-English dictionary onto my Kindle for reference. I've already played with kindlegen, which takes a collection of HTML files along with some additional metadata and creates a MobiPocket .mobi format file for the Kindle. (I've got several ideas for reference documents that could be loaded onto the Kindle, more on that later.) Anyway, I've now got a prototype Chinese dictionary loaded onto the Kindle. However, although many of the characters are displayed correctly, quite a few are just empty boxes (in the form of I ⬜ Unicode). I couldn't find any particular pattern to the missing characters, for example 他 (he) displayed correctly but 她 (she) did not. Some research with Google showed that some people had hacked the Kindle and found a way to load different font files onto the device. I didn't try doing that, becuase I thought there should be a better solution. After all, Chinese text is displayed correctly in the Kindle web browser! Furthermore, if I copy a UTF-8 format file directly to the Kindle for display, it will have the missing character problem. But if I email the file to Amazon where they automatically reformat it to a .azw book file and deliver it to the Kindle, the characters show up correctly. If I convert to a MobiPocket book file myself, missing characters. After laboriously paging through dozens of threads on mobileread.com (typical internet "forum" software really is awful), I finally found a solution. It uses an undocumented debug feature of the Kindle. Press <Home> <Search> and enter the following commands: ;debugOn <Enter> ~setLocale zh-CN <Enter> That's it, that is all that was required. I have no idea what this actually does or how it changes the Kindle's interpretation of UTF-8 documents (UTF-8 is supposed to be an unambiguous encoding of Unicode). But I now have a very basic Chinese-English dictionary in my Kindle. (9 Comments |Comment on this)
monday, october 25, 2010
8:34 pm	my weekend in pictures Saturday: Removed a tonne of bricks from the front path, piled them in the back. Sunday: Laid down weed mat and installed close to a tonne of gravel. The bricks had grass and weeds growing between them and were always damp and slippery. Monday: Bathroom renovation begins. Removed grotty old shower door, bathtub, and shelves. Now we have an (almost) empty box inside the house which will (eventually) contain a new bathroom. What I didn't get to: (1 Comment |Comment on this)
monday, october 4, 2010
9:21 am	bad math: pet ownership and lottery winners A headline in the local paper this morning reads Lucky Lotto winnners like cats with the first two paragraphs: HAVING A cat could improve your chances of winning Lotto - by a whisker. A survey released today by New Zealand Lotteries shows 39 percent of big Lotto winners have cats, making felines the most popular pets among lucky Lotto winners. Dogs are second but way behind with only 27 percent of big winners preferring man's best friend to moggies. My first reaction upon reading that was, "so what?". Surely the pet ownership statistics among the general population would be reflected in those statistics for the small number of lotto winners. The New Zealand Petfood Manufacturers Assocation claims the ownership numbers for New Zealand as a whole are 53% for cats and 35% for dogs. What does this tell us? These numbers show that a greater proportion of non-pet owners win the lottery than there are non-pet-owners in the general population. Having a cat certainly doesn't "improve your chances" as the article claims. (These are just rough numbers because we don't know what proportion of either the general population or lottery winners own both a cat and a dog.) They say the lottery is a tax on people who are bad at math. Apparently, the "bad at math" part extends to those writing about the lottery, too. (Comment on this)
saturday, september 18, 2010
4:10 pm	amazon kindle first impressions I got my Amazon Kindle 3 earlier this week! I pre-ordered it around the end of August and seemed to receive it here in New Zealand before some US users received theirs. It was packed in an integrated protective/shipping box, and my first thought upon opening it was, "Amazon has been taking lessons from Apple." The device was sitting face up with clear plastic film on both sides that was easy to remove. Since the e-ink display doesn't take any power to hold the display, they had already displayed basic instructions on the screen ready to go (just where to plug it in and where the power switch is). The display is fantastic! It's super high resolution, quite high contrast, and the brighter the ambient light is, the easier it is to read. The first day after I got it, I sat on a beach in full afternoon sun and it was brilliant. Any other LCD display would have been struggling. I got the model with both WiFi and 3G, and both seem to work well enough. There was no trouble connecting to my home network, and the 3G works fine even if I'm on the bus. The price of 3G networking is included in the cost of the device itself and books you buy, so there's no contract and I need to have no relationship with whoever provides the 3G. The Kindle now has an experimental web browser which can browse any web site on WiFi but is limited to the Kindle store and Wikipedia on 3G. However, with Wikipedia on free 3G this device actually is the Hitchhiker's Guide to the Galaxy. All it needs is DON'T PANIC in large friendly letters on the front. For the first few days, I didn't spend anything on content because Project Gutenberg (among others) provide free ready-to-download ".mobi" format files for the Kindle. Downloading books is a simple click and the book magically shows up on the Kindle home page. The first (and only, so far) book I bought was Erlang Programming which I bought on my way home on the bus the other day. The purchasing experience is seamless so I'm going to have to watch that I don't spend all my money on books. The Kindle comes with two dictionaries built in, the "Oxford Dictionary of English" and "The New Oxford American Dictionary". Both these are books in their own right that can be read and browsed, but also when reading any other content you can point to a word and look it up in the dictionary (you can choose which one is your default). I often look up words to be sure I know what they mean (I'm a big user of wiktionary.org) so having the dictionary available everywhere is excellent. There are some highlighting and note-taking features, and these seem to integrate into the social media space somehow so you can see what other people have highlighted and annotated. I haven't played with any of those features yet. The web browser itself feels quite clunky and difficult to use (this is partly because the screen isn't a touch screen and you have to use the little navigation buttons to move a pointer around to point to things). I could log in and use some sites like Twitter, but the browser wouldn't let me type my name into the Gmail login screen for example (I tried both the regular and "mobile" sites). I look forward to upgrades on the browser, hopefully they will correct these problems. But the Kindle will never match an iPhone/iPod/iPad for general browsing capability. I'm very happy with the Kindle and I believe I will find myself reading more because of it. (5 Comments |Comment on this)
friday, august 27, 2010
11:10 pm	one hundred year old heirloom When I was young, my grandparents had a fancy marble clock in their home. It always fascinated me, because it would chime every half hour (they would turn on the chime for me). At some point, a picture frame fell on the clock, which broke apart nearly all the marble pieces. The clock got put in a box where it stayed for 30 years. But my grandparents always said it was my clock. Fast forward to earlier this year. Amy happened to be looking at antique clocks as she was thinking that something like that would go well on the mantel above the fireplace. I said, "Hey, did I ever tell you about my clock?" Apparently, I hadn't. The next day we called my parents, who found the box of clock parts under the stairs, and we managed to coordinate its transport to here in New Zealand with only minor damage. Some of the marble bits were broken and one of the metal movement supports was bent, but everything was present and the movement (all the mechanical bits) worked. I researched its history and found that it was probably made in France in the late 19th or early 20th century, and the adhesive used at that time was roughly, plaster of Paris and egg white. This probably protected the marble pieces because the adhesive wasn't that strong and came apart first before breaking the marble. However, we decided to put it back together with a modern glue (not too strong, but not just white glue). The more interesting part about this clock is its history as it relates to my family. The inscription reads: Sons of Temperance Friendly Society Birmingham District Grand Division Presented to Bro. William Aucott J.P. as a mark of esteem and appreciation on completion of 44 years continuous service as Trustee of the above Society August 27th 1910 William Aucott was my great great great grandfather. He was born in 1830 in Leicestershire in England, was an early trade union leader when such things were being founded for the first time in 1863, became president of the National Association of Ironworkers in 1874, and served in a number of different roles in different labour organisations throughout his lifetime. He was involved in the Methodist church, helped establish the Sons of Temperance Society in Wednesbury, and was a magistrate on the Wednesbury bench. Exactly one hundred years ago today, William Aucott was presented with this very clock as thanks for his service to the society. I'm sure he would be happy to know that the clock remains a treasured posession in our family. (3 Comments |Comment on this)
tuesday, august 24, 2010
8:43 pm	web site content management using git For the past several years, I've managed my web site content using a local copy (of everything) on my home machine, and a simple rsync script that transfers changed files up to my web site host. While this works, it's limited (there's only one "source" copy of my content) and fragile (if I need to change something on the server, I need to remember to make a local copy before syncing again). I've always known it would be better to manage the content using a real source control system. When I first started managing my web site like this, the state of the art in source control was Subversion, but I never got around to actually setting it up. Today, my source control system of choice is Git, and it seems like a natural fit. Importing all my content into a new repository is easy, and I pushed up the repository to my web host overnight. The compressed repo size is 2.6 GB and Git handles it just fine. It feels much better to have multiple mirror copies of the site content, and it's much nicer to be able to manage it using a powerful tool like Git. (Comment on this)
tuesday, june 22, 2010
10:55 pm	HP-15C calculator simulator I'm a bit of a calculator geek. My favourite calculator is the HP-15C, an advanced scientific programmable RPN calculator made in the 1980s. It's got a distinctive form factor: I bought mine in 1987 and it got me through university, and still runs on the same set of batteries it came with 23 years ago. The HP-12C financial version of this calculator did so well that it continues to be sold to this day. These days it's a bit awkward to actually carry a calculator around (since there are computers everywhere), so I've implemented a (nearly complete) simulator for the HP-15C in Javascript. You can find it at http://hp15c.com along with some more information about its capabilities and current known issues. If you're unfamiliar with this calculator, you can find a scanned copy of the Owner's Handbook (and the Advanced Functions Handbook) online. There is even a Petition to Bring Back the 15C that you can sign. If you've ever used this calculator, or one like it, I hope you find this simulator entertaining. Feedback, bug reports, and even contributions are welcome! (2 Comments |Comment on this)
sunday, april 18, 2010
11:59 am	cat Toothless Yesterday Amy and I adopted a young black male cat from the SPCA. We had been talking about getting a cat for a while, and while we were out yesterday decided to go have a look. They had quite a few cats available, and we both quite liked the look of a small grey female with a very distinguished narrow face. However, this black one seemed much friendlier and purred a lot. It was a toss-up for a while but we were both won over by the black one. We had just finished watching the movie "How to Train Your Dragon", which features a rather feline black dragon named "Toothless". That would be a fitting name for a black cat, we thought. While we were doing the paperwork, they noted that he was due for his deworming pill, so they gave him the pill right there. But while he was taking the pill, we noticed one of his teeth didn't look right. Sure enough, it was askew and fell out in Amy's hand! They suspected that it may have been like that for a while and that he would be fine without it. We realised right then that the name "Toothless" would stick. He's very friendly and purrs almost continuously when sitting near us (right now I'm slouching on the futon with my laptop propped up on my knees, with Toothless sitting on my chest purring). He knows where his food and litter box is, but we're definitely going to have to get him a scratching post to save our furniture. For now he's got full reign over the living room and kitchen, and we'll introduce him to the rest of the house in time. Ultimately he will be an outdoor cat too, because we regularly leave windows and doors open to let the air through the house. Hopefully he won't get too tangled up with the other cats in the neighbourhood (of course he's been fixed). Very happy with him. (6 Comments |Comment on this)