New Toys

So what new toys have been occupying my time? Clojure. It's a functional, compiled lisp for the Java Virtual Machine (JVM). Like any good lisp, it has macros and first-class functions; plus it also has many nice features for concurrent programming, like software transactional memory and agents. It has fast, read-only data structures designed to be used in a functional language. So far, I've had a lot of fun with it. I've mainly been using it to work through Programming Collective Intelligence by Toby Segaran (PCI, for the rest of this posting). Whenever I'm working through a programming book, I like to re-code the examples so I understand them better. And often it's better to use a language that's not at all like the one the author uses in the book. Since the code in PCI is in Python, most popular languages would be too similar to really help me understand the algorithms Segaran's describing. But since Clojure is a functional language, I have to step back, look at what he's trying to do, and think about how to accomplish the same thing with a different, functional approach. It's been good to help me understand the book better and to help me improve my functional chops. To further explore this fun little language, I'm going to start a long series here on this blog. Here's the plan:

• Its target audience will be people like myself: self-taught programmers whose primary education has been in the humanities.
• This won't be a tutorial on how to program: The audience should have a little programming experience, preferably in Perl, Python, or Ruby.
• But you won't need to have any experience in lisp, functional programming, or concurrent programming. I'll touch on those as we go along.
• The problems I'm going to tackle will be oriented to processing text documents and analyzing the language in them. The techniques I'll cover will be helpful to those interested in stylistics and other literary studies, or to those interested in corpus linguistics.
• Many of the things I'll describe—such as tokenization–will be very basic and far back from the cutting-edge. Since this is also an introduction to Clojure, I'll cover the basics also.
• But there will also be some odd gaps. For example, I'll explain regular expressions just enough to implement tokenization. Then I'll point you to one of the myriad-thousand excellent online tutorials if you want to learn more about them.
• Since one of the main points of Clojure is concurrency and parallel processing, I'll cover that also.
• The examples will build on each other, and in the end, we'll have a system for doing parallel processing of text documents. We'll build a variety of tasks to do some standard analyses, and we'll design it so creating more tasks and inserting them into the processing stream will be relatively easy.

Well, this is long enough for today. Tomorrow I'll start with a brief introduction to Clojure and a look at why you might want to learn it.

Editing Lisp

In which I decide to start hacking in lisp, and that rocks my editing world. The options for editing lisp in Vim are pretty limited. Well, let's be precise. Vim is a pretty good lisp editor, when you compare it with almost anything except emacs. There have been attempts to bring Vim up to emacs' level. There was a project to integrate ECL with Vim. It was part of a larger project, Slim-Vim to write a Vim interface to SLIME, the excellent emacs lisp development system. Unfortunately, Slim-Vim is dead. Both of its primary developers have abandoned it. According to their farewell emails, one now develops lisp using a commercial IDE, and the other uses emacs and SLIME. Generally, the advice for people like me is to suck it up and use emacs. You will be assimilated. Others do still use Vim for lisp, and I still prefer Vim for actually editing lisp. I've done some crazy, complicated things to get it to integrate with a running lisp system. The version of lisp I usually use, SBCL, doesn't include readline in the REPL ("Use SLIME," they say), and using it without command history, etc., is painful. So I would start emacs and SLIME, and I would start Vim. I would open the file in both emacs and Vim. Editing would occur in Vim, then I would refresh the buffer in emacs and feed my changes into SBCL. I know. Suck it up. Use emacs. Eventually, that will be how this story ends. First, I'm going to try Cusp, an Eclipse plug in for lisp development. Aside from being a lot prettier, I'm not sure what it has over emacs+SLIME. What do you use for lisp hacking? (If I get no answers, I'll assume everyone uses emacs.)

CarbonLisp Status

Just a short update today. I've started working on CarbonLisp again. I have a small lispish interpreter written in C#. It will be the first step in bootstrapping the lisp system. Of course, I'm not sure when I'll get to work on it next.

Targeting the DLR

I've been circling warily around CarbonLisp again. I think I've decided not to target the DLR initially. Instead, I will have a compiler that, like Common Lisp, will produce code that varies from very dynamic to static. Once that's done, I may add a DLR component. Of course, I have to get started first.

Common Lisp Tutorial

If you want to learn Common Lisp, an excellent option is Practical Common Lisp, which is also a Lisp tutorial. It's available both in print and on-line. (Note, I get nothing from this shameless plug. But I do honestly recommend it. I have my own copy, and I refer to it regularly.)

LiSP, Revisited

Remember that I ordered a copy of Lisp in Small Pieces from a sales that Amazon.ca was putting on? Like many people, I was informed that the price was due to a mistake, and they were canceling my order. Easy come, easy go. C'est la vie. Fill in your favorite cliché here. I did look at a copy of the book in a library. (I don't qualify for a library card there, so I didn't check the book out.) It looks like a great book. Maybe I'll have to get off my wallet and buy it anyway. While browsing through the book, I realized that I'm attempting to implement too much of Common Lisp initially. For example, the lisp reader needs access to the full-fledged lisp system in order to modify its own settings. It's a chicken-and-egg situation. You need a full lisp system to completely implement the reader, and you need a reader to completely implement the system. The answer, of course, is to implement a very restricted subset and bootstrap your way up. Duh. Anyway, now I think I have a plan to go forward with. I haven't done anything on it yet, though. I'll keep you posted when I do.

Lisp in Small Pieces

Amazon.ca was having an unexpected sale on Lisp in Small Pieces, one of the best books on implementing lisp. For about a day it was selling better than Harry Potter. Hmm. I'm implementing lisp, and I could always use a new book. My copy will arrive in two to four weeks. Until that time, CarbonLisp is going on vacation. In the meantime, I'll work on and blog about other projects.

Roadblocks and Dynamic Coding

What's happened since my last posting?

• Google's identified this blog as a potential spam-blog and locked me out. I've gotten it unlocked. I guess i need to post more often.
• I've gotten most of the framework code for the DLR finished. Nothing exciting, and I'm sure it's bug-ridden, but it's a start.
• I've started on the lisp reader, and I realized I don't know what I'm doing. Fools rush in where angels fear to tread, and all that. Part of the problem is that I'm really doing exploratory programming, but in a static language. I'm not a big static language fan, anyway. I'm considering doing a toy implementation of lisp in Python to get started and to figure out what I'm doing.

I know, I know. That last point is just crazy talk. Of course, this whole blog has turned into the ramblings of a madman, anyway, so why the hell not get some traction by doing some exploratory programming in Python, then moving back to C#.

Back from Vacation and Names

Most of last week I was on vacation. The rest of the week, I've spent a little time digging into ToyScript and filling out the framework I'll need for implementing lisp on the DLR. More about that later. I've temporarily decided to call this project SiliconLisp. This roughly follows the convention of naming dynamic languages for the CLR as Iron-. I've just changed the element it's made out of. Actually, I was originally going to call it CarbonLisp, but I didn't want to confuse this with the Mac windowing library Carbon. I may go with CarbonLisp anyway. If you have a suggestion for the name, please leave me a comment.

Lisp and .NET

The Truth about Lisp Why does that link matter? Because I've been thinking about implementing Common Lisp on the CLR and the DLR. When interpreted, it would primarily work with the DLR, but as declare statements are added and the Lisp code becomes more static, it would be able to leverage the CLR directly. At one point, I was thinking about implementing Scheme, but I've moved away from that. When I started learning Common Lisp, I thought that I preferred Scheme. But now I've done a project in Scheme (a simple todo-list manager, similar to todo.txt, and I've decided that I prefer Common Lisp after all. Scheme's still great. I have a nice clean feeling whenever I code in it, but I think I prefer CL's more complete language specification. It's still not really complete enough. (Hello? threads, networking, Hello?) Also, I'm beginning to grok continuations, but I don't have a clue how I'd implement them in .NET. So I'm just not doing that right now. Still, what craziness is this? Well, I've been doing a lot of .NET at work, and I'd like to be able to use a more powerful, sane (for some definition of sane) language. I'd also like to be able to produce assemblies that can be called from C# or whatever. Basically, I want a lisp that is a first-class citizen of the CLR. And I've been wanting to do more Lisp for my personal projects. I'll let you know how it goes. Or I'll put my head between my knees and breath deeply until this latest light-headedness passes.