Greg Hewgill (ghewgill) wrote,
Greg Hewgill

minilink performance improvement

My minilink/ link shortener has been getting increasingly popular lately. There was a sharp jump up in activity in mid-October, which has now put the request rate at around 20k requests per day. There have been spikes up to 40k per day, too.

This increase in activity has taken a toll on my server, since minilink had been implemented as a simple Python CGI program. This was the original implementation I started with and I haven't needed to change it. However, CGI is only appropriate for low performance applications, and minilink is probably edging into medium performance at this point. A CGI program has to start up an entire Python runtime environment for every request.

Over the last two days, there have been big enough activity spikes that the load on my server goes through the roof and it can't really handle the requests anymore. This can cause things to crash and run out of memory and other impolite behaviours. This happened once last March, and twice in the last two days.

Today I did something I should have done long ago: I converted minilink from a CGI program to a WSGI program. WSGI improves performance because it keeps the Python runtime loaded in memory, and with Apache's mod_wsgi, the calls into Python code are much faster.

I also implemented a simple test suite so that I could refactor the implementation while ensuring that the behaviour stayed the same. That's something I should have done long ago, too.

You may see an increase in performance when using minilink starting today. If you see anything go wrong though, please let me know.
Tags: minilink, python

  • 2013 in review

    2013 is the year when everything changed. The biggest event was the birth of our daughter Lily. She was born prematurely in Shanghai while we…

  • 2012 in review

    2012 has been fairly quiet. Maybe it just seems that way because I haven't actually written anything new in this blog since last year's annual…

  • new photo galleries

    I've been busy processing photo galleries from the last year (or two) and putting them online for your perusal. Vancouver 2010 Northland…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded