Greg Hewgill (ghewgill) wrote,
Greg Hewgill

minilink performance improvement

My minilink/ link shortener has been getting increasingly popular lately. There was a sharp jump up in activity in mid-October, which has now put the request rate at around 20k requests per day. There have been spikes up to 40k per day, too.

This increase in activity has taken a toll on my server, since minilink had been implemented as a simple Python CGI program. This was the original implementation I started with and I haven't needed to change it. However, CGI is only appropriate for low performance applications, and minilink is probably edging into medium performance at this point. A CGI program has to start up an entire Python runtime environment for every request.

Over the last two days, there have been big enough activity spikes that the load on my server goes through the roof and it can't really handle the requests anymore. This can cause things to crash and run out of memory and other impolite behaviours. This happened once last March, and twice in the last two days.

Today I did something I should have done long ago: I converted minilink from a CGI program to a WSGI program. WSGI improves performance because it keeps the Python runtime loaded in memory, and with Apache's mod_wsgi, the calls into Python code are much faster.

I also implemented a simple test suite so that I could refactor the implementation while ensuring that the behaviour stayed the same. That's something I should have done long ago, too.

You may see an increase in performance when using minilink starting today. If you see anything go wrong though, please let me know.
Tags: minilink, python

  • fault line flyers video online

    A couple of weeks ago, Fox 7 News went out to Fault Line Flyers (the soaring club where I used to fly in Texas) and filmed a bunch of footage for a…

  • pumpkin drop!

    Tomorrow is the annual Pumpkin Drop at the glider field! A pilot and a bombadier go up in a two-seat glider. The bombadier sits in the back with a…

  • private pilot checkride

    This year has been an especially disappointing year for soaring. Between uncooperative weather, my summer vacation to Spain, occasional lack of…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded