Greg Hewgill (ghewgill) wrote,
Greg Hewgill

psil presentation and compiler details

As promised last week, I've been working on making Psil work in a Python 3 environment. But first, a small but entirely relevant digression which includes a video of me giving a presentation:

A few weeks ago, I attended Kiwi PyCon here in Christchurch. I had thought about maybe possibly presenting something when I signed up, but I didn't actually prepare anything ahead of time. Immediately after lunch on the first day, there was a "lightning talk" session where people get 5 minutes to present something, no questions from the audience. During lunch, I decided that my opportunity was right then, and that if I didn't try to present something then I'd probably be annoyed at myself for not having tried.

Anyway, I hurriedly prepared a presentation (PDF, 8 slides) in the 30 minutes before the presentation, and wondered whether I would even be able to find 5 minutes worth of stuff to say. It turns out I did, because although it felt like I had zipped through my presentation in like 3 minutes, I actually took slightly over 5 minutes. There is a video recording of my (terrible) presentation that you can watch online (skip forward to about 52:00 using the slider).

Right, now that you're back, I'll continue with what I was going to write about.

The Psil compiler (as opposed to the interpreter) compiles Psil code into Python code, which is then executed directly by the Python runtime system. For example, here is a simple function and its use:

    Psil                            Python
    ----                            ------

    (define (sq x) (* x x))         def sq(x):
                                        return x * x

    (print (sq (+ 2 3)))            print sq(2 + 3)

Previously (before the current Python 3 migration), I did this by generating literal Python source code as shown in the right hand column above. But instead of generating the source code directly, I generated a Python AST (Abstract Syntax Tree) representation, then implemented a "de-parser" that creates equivalent Python source code from an AST. The original idea was to have Python compile and execte the AST directly (rather than going through the source code step), but I think the version of Python 2.x that I was using didn't support passing an AST to the compile() function.

Since Psil now requires Python 3, the compile() function supports direct compilation of an AST that represents Python code. So now Psil can generate the AST, pass it to compile() without going through source code, and execute the resulting code object directly. One of the difficulties with this was that the internal Python AST changed completely from version 2 to 3, so I had to rework most of the compiler code. Fortunately, it turned out to be reasonably straightforward. I also maintained the de-parser because debugging actual source code is a lot easier than debugging with only an AST.

Right now I've got this all working for simple examples. I still need to re-implement the "lambda lift" which promotes a Psil lambda expression (which may contain an arbitrary number of forms) to a Python function (because Python's lambdas can only contain a single expression).

The next interesting thing I can do with an AST is annotate the nodes with source line and column numbers. This means that if a Python exception happens at runtime, the traceback will actually point to the location in the Psil source code that caused it! That's a pretty cool feature and I'm looking forward to playing with that.

Tags: psil

  • 2013 in review

    2013 is the year when everything changed. The biggest event was the birth of our daughter Lily. She was born prematurely in Shanghai while we…

  • 2012 in review

    2012 has been fairly quiet. Maybe it just seems that way because I haven't actually written anything new in this blog since last year's annual…

  • new photo galleries

    I've been busy processing photo galleries from the last year (or two) and putting them online for your perusal. Vancouver 2010 Northland…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded