Farewell, National Blog Posting Month

I wasn't officially part of National Blog Posting Month (NaBloPoMo), but I have been trying to post everyday this month. I skipped one Saturday, then I completely fell off the wagon just before Thanksgiving. I posted a few times after Thanksgiving, but it was never really the same. Still, it was a good experience, and I hope to keep blogging here more regularly, although probably not daily. I found that I would post stuff just to get my posting quota in. The postings weren't always as informative or helpful as I would like. I wasn't happy about that, and I probably won't be posting as often, but I should be happier with the quality of my posts. So now I say, Farewell, National Blog Posting Month! And thank goodness you're over.

Writing and Coding, Part 3: Artistic Expression

All right, I'll admit it. I've been putting this part of the series off. This is a huge topic, and the more I've thought about it, the less I feel I can do it justice in the time I have to prepare and the space I have to expound on it here. So instead of doing an in-depth exploration, I'm going to provide a very high-level overview. Put on your running shoes. Artistic Expression Programming seems to be a creative, artistic endeavor in two ways. First, you have phenomena like perl poetry, where code is explicitly, consciously used as poetry. This is related to the school of thought that looks at programming's primary purpose being communicating algorithms to a person. Executing the code on a computer is secondary, a means of making sure you are correctly expressing the algorithm. Aesthetics: De Gustibus Non Disputandum Personally, I tend to having simple, regular syntax, even at the expense of having simple, regular semantics (although I like that to be simple also). This bias is evident in a couple of ways: First, I prefer Python to Ruby. Ruby's semantics are probably more regular (everything's an object vs everything's an object, but it may be a classic object and some things are sealed), but the few times I've attempted to learn Ruby, the syntax just seems a little irregular. Not in a bad way, necessarily, but enough that it's squelched what little motivation I've had so far to learn what is probably a fine language. But that's beside the point. What I'm trying to illustrate is that people judge programming languages partially according to criteria that is strictly non-utilitarian. Having simple syntax doesn't make any difference in how capable Python is at building, say, web apps. (As I type this, I'm mentally constructing an argument that it does, but in general, I stand by this statement.) Of course, the academic in me is wanting to step back and define terms now. What does aesthetics mean? Maybe it's time to step back, pull out some literary theory books, and see where this leads us. Is there anything more dull than someone who is both a literature- and computing-geek?

The Kindle and Big Brother

Mark Pilgrim has an great post that draws correlations between the Kindle's DRM and Orwell's 1984. The point here really isn't the Kindle. Amazon's e-book reader is just the latest example of such policies. (It may be breaking new ground, so to speak, in that it is an inherently connected device, and its DRM reflects that.) Right now I'm reading (paper book) Rainbows End by Vernor Vinge. In that future world, privacy is virtually non-existent. DHS has monitoring devices in everything. Vinge explicitly says at one point that people had to adjust their quaint notions of privacy. (I don't have the book in front of me, so I don't know that he put it like that — quaint, and so on — but that was the impression that I got. Britain and other countries are moving quickly toward a surveillance society. I remember several years ago, I was helping to chaperon a study-abroad program in England, and we warned the students that they should be careful about such things. In a previous term, a student was caught trying to buy drugs when a video camera installed in an alley recorded him. I would imagine that the United States is moving in that direction. Certainly the current administration would like that. We already have cameras surveilling intersections for traffic violations. The sheer size of the US means that we probably won't be able to monitor everything. We'll have to find some scaled-down version of universal surveillance that only monitors high-risk areas. Until the computers can watch us for ourselves. Then we can monitor everything.

Programming Fun

I just ran across a great post on Ruby being designed to be fun. I don't program in Ruby myself, yet. I've attempted to learn it several times. I get started working through Programming Ruby, but I never get very far. I never have much motivation to pick it up. It seems to occupy a niche that is happily occupied by Python in my programming ecosystem. If another language is going to displace Python, it will probably be some form of lisp. In the end, what finally makes me set aside Ruby are aesthetic reasons I'll talk about more later (when I finish the Writing and Coding series I'm working on). Of course, I also looked at Python once or twice before it clicked for me, so maybe there is some Ruby in my future. Still, I think that it is great that Ruby is designed for fun. When I did finally "get" Python, I had just come out of bad experiences with two other programming languages: I had hit Perl's 1000-line limit; then I switched to Java. Java's great, but it's not fun. Python made programming fun again. That sounds like I had just gotten out of two bad relationships when I met her. And maybe that's what it was.

Playing Catch-Up

Things have settled down after the holiday, but I'm still trying to go through all the electronica that has accumulated in 120 hours of not touching a computer. In the meantime, entertain yourselves by browsing the Most Unusual Books of the World (via Geoffrey Rockwell).

Happy Thanksgiving

Yesterday I said that I would return the to the writing and coding series today. I lied. Later this week, is Thanksgiving Day in the US. Originally, I was going to struggle through all the relatives and food and attempt to post anyway. I've decided not to. While I'm gone, I'll leave you with this link: Kindling Openness and Impact. This touches on some of the issues I see with the Kindle. If you are in the US, happy Thanksgiving. And to everyone, I hope the rest of your week is great.

Paradigm Shift

Amazon's getting ready to release an e-book reader, Kindle. This sounds pretty cool. Too bad $400 is more than I want to sink into an reader. Robert Scoble points out that a lot of people aren't loving this device. Whether it succeeds or fails, however, I think this device is fascinating, for the same reason I find the the current writer's strike interesting. We have a traditional, risk adverse industry (publishing, music, TV, movies). A new, disruptive technology comes along (the Internet). Now, those industries have to figure out a new way to make money at what they do. Everything changes. This is an old story. The same thing happened at the invention of the printing press. Initially, the printers were the main ones making money. A publisher would buy a book from an author, and that was the end of the author's rights. The printer could publish the book. He could change the book. He could put his name on it. Somehow, we went from there to today's situation: authors are paid a more or less fair percentage of the sales of the book, they are credited, they have some say in what changes get made to the book before publishing. Today, we (publishers, studio executives, writers, and consumers) are trying to figure out how to make money at this. The old media probably aren't going away soon, but they are becoming less important. The new-fangled Internets are messing everything up. Will the solutions that we reach now be the final ones? (Final here means until the next disruptive technology comes along tomorrow.) Maybe not. That's what's interesting.

---

(Tomorrow, back to the writing and coding series.)

Writing and Coding, Part 2: Self-Discovery

One reason people create things, art generally, but I'm arguing here that creation in itself of anything is an act of self-discovery. People say that writing is an act of self-discovery. Journaling is writing for no purpose other than self-discovery. But even writing that is meant for public consumption also often helps the writer learn about herself. Of course, the act of self-discovery—like creativity—is usually messy, and writers rarely let it out in public. I remember that for one class I wrote a lot of personal essays, a form that I really enjoy (hmm, a blog?). I would pick a topic to write on. I'd get started, heading in one direction. About half way through, I'd discover something about myself or about the topic I was writing on. Because of that revelation, the essay would take a sudden turn and head off in another direction. That was good; I counted on it, hoped for it. But it made more work for me, because then I would have to fix the first half, make the whole essay go in the direction the last half did. Part of the essay would be thrown away, but in the end, I'd learned something, and the essay was much better. I've heard about others who draft novels by throwing away the first, the second, and even the third drafts. That astounds me. Knowing how much work goes into a novel, I have trouble imagining throwing a draft away. On the other hand, it would probably make a better final product. I just wonder if more planning up front could take the place of making such drastic edits. So when I write for self-discovery, I don't know where the work is headed, and I have to be willing to cut drastically, even throw out whole drafts. How does this apply to programming? Supposedly, Richard Feynman had written on his chalkboard, "What I cannot create I do not understand." Part of the programming I do is to help me understand problems. I'm not by nature a detailed thinker. I prefer to stay in the clouds, thinking about things in an abstract way. Programming requires me to be specific, however, and to think problems through thoroughly. In solving a problem in code, I learn something about the problem and what it takes to solve it. In learning to solve a problem this way, I often work my way to a solution gradually. I open up a REPL (lisp read-eval-print loop) or Python interactive interpreter. I open up a text editor. I go back and forth between the two, feeling my way to a solution. Working this way is fun, but it is wasteful. As I'm doing it, I may attempt several solutions that have to be thrown away. And often, the code that results is a mess and has to be thrown away anyway. But at that point, I understand the problem and what it takes to solve it, and I can quickly write something much better. That process sounds like what I did with the personal essays. But I am learning something about myself on this? After all, I'm talking about self discovery in this posting, not problem understanding. It can be. I should probably write another post focusing on this assertion, arguing just it, but the act of creating anything teaches us about ourselves, if we're listening. How do we interact with the environment we're changing? How is our approach different than other people's? Why is this problem interesting, not that one?

Writing and Coding, Part 1

When I named this blog, part of the reason I chose this name was because I enjoy both writing and coding, and I wanted to talk about both. I didn't want to start two blogs, however, and later abandon both. I would rather just abandon one. (There are already a few abandoned blogs with my name on them. No sense adding to that number without reason.) The other reason is that I've always thought that there are similarities between the two activities, as well as big differences, and I wanted to explore these. A lot of people have written on these before or presented papers. This is just my take. This is a complicated, many-faceted topic, so it will take me several days to discuss this. Here's the plan:

• Writing and Coding, Part 2: Self-Discovery
• Writing and Coding, Part 3: Artistic Expression
• Writing and Coding, Part 4: Test-Driven Writing
• Writing and Coding, Part 5: Ambiguity
• Writing and Coding, Part 6: Conclusion

Expanding Short Stories

The other day, I mentioned that I was working on a short story I was calling "Voodoo Christmas." As I was fleshing out the concept, I realized a number of things. First, there are a lot of characters. Second, it's not so much about Christmas as it is about the year after Christmas, so there's also a long expanse of time involved. I've even considered it spanning a year. Third, it will just take a while to set up the plot, develop the characters, get things rolling, and tell the tale. Allow me to introduce the novella/novel Voodoo Christmas.

Editing Lisp

In which I decide to start hacking in lisp, and that rocks my editing world. The options for editing lisp in Vim are pretty limited. Well, let's be precise. Vim is a pretty good lisp editor, when you compare it with almost anything except emacs. There have been attempts to bring Vim up to emacs' level. There was a project to integrate ECL with Vim. It was part of a larger project, Slim-Vim to write a Vim interface to SLIME, the excellent emacs lisp development system. Unfortunately, Slim-Vim is dead. Both of its primary developers have abandoned it. According to their farewell emails, one now develops lisp using a commercial IDE, and the other uses emacs and SLIME. Generally, the advice for people like me is to suck it up and use emacs. You will be assimilated. Others do still use Vim for lisp, and I still prefer Vim for actually editing lisp. I've done some crazy, complicated things to get it to integrate with a running lisp system. The version of lisp I usually use, SBCL, doesn't include readline in the REPL ("Use SLIME," they say), and using it without command history, etc., is painful. So I would start emacs and SLIME, and I would start Vim. I would open the file in both emacs and Vim. Editing would occur in Vim, then I would refresh the buffer in emacs and feed my changes into SBCL. I know. Suck it up. Use emacs. Eventually, that will be how this story ends. First, I'm going to try Cusp, an Eclipse plug in for lisp development. Aside from being a lot prettier, I'm not sure what it has over emacs+SLIME. What do you use for lisp hacking? (If I get no answers, I'll assume everyone uses emacs.)

A Theory of Text Editors

When I started working with LINUX, I used GNU emacs. At the time, I was using some low-end hardware, even for the time. One day, I realized that my morning routine was to boot up, start emacs, and make a cup of coffee while I waited for emacs to wake up in the morning. That was insane. There was a lot I liked about emacs, but I needed something lighter weight. So I switched to Vim, and I still use it for all my editing needs. Vim messes with your brain. Once you've internalized the keystrokes, you're ruined for other editors. In the past, I would sometimes think that I needed to enter the 20th century and try some other editor—Eclipse, jEdit, or something. I'd install whatever it was I planned on using and tweaked it out, but I only used it at most a day, and the entire time I'd be thinking, "I could do this so much more quickly in Vim." Finally, I'd switch back just to make myself shut up. Of course, to really test drive a text editor, you need to give it a few weeks, so I never gave any of these a fair shake. I even tried viper once. This was a mode in emacs that used vi keystrokes and commands. I only used it about thirty minutes. It was too confusing. I remember the emacs keystrokes enough that I couldn't never quite decide whether I should use them or vi. I was less productive with it than I was with emacs by itself. My theory is that there are two levels to text editing: High-level semantics and low level syntax. I'll be abusing both of these terms, so let me explain what I mean by them. If you're editing Java or C# or Python or Lisp, semantics involves the libraries you're using. What properties does a Button have? What arguments does socket.socket take? Which comes first in elt, the index or the sequence? Eventually, if you use the language and its libraries enough, these questions go away. The other level is syntax. This involves editing at the character level: entering text, copying, pasting, moving, search-and-replace. This never goes away. In my experience, most IDEs are great at the semantic level, but poor at the syntactic level. They will answer any questions you have about the libraries and the language, but if you have to a sequence of keystrokes 1000 times, it is painful. Usually possible, but painful. On the other hand, Vim is brain dead at semantic stuff. At the syntactic level, however, once the keystrokes have branded themselves onto your brain, no other editor comes close. It tells you nothing about the objects, methods, or functions that you're using, but if you have to repeat something 1000 times, bang, you're done. So what's the point? Well, this posting actually started out as a discussion of editing lisp. It seems to have gotten derailed though. Lisp will have to wait for tomorrow.

Stalled Novel

Well, my novel's spinning its wheels right now. More precisely, I'm spinning my wheels instead of working on my novel right now. I decided that I needed another wrinkle in the plot. I decided what it was, but I haven't been able to make myself work on it since then. Instead of working on that, I'm taking a break. I've had some ideas for short stories, and I'm working on those. I'm tentatively calling the first one "Voodoo Christmas." Should be fun. I'll let you know if anything comes of it.

Erlang Nostalgia

A few months ago, I wrote a system for controlling distributed processing using Erlang, but since then, I haven't really done much with Erlang. Finishing up that series of articles on Erlang reminded me of what fun Erlang is. Looking at problems concurrently is interesting and fun, and while I'm not always fond of the language's syntax, I still enjoy hacking in it. I'll have to do more Erlang in the near future. Something else for my to-do list.

CarbonLisp Status

Just a short update today. I've started working on CarbonLisp again. I have a small lispish interpreter written in C#. It will be the first step in bootstrapping the lisp system. Of course, I'm not sure when I'll get to work on it next.

Language Dreams

Continuing yesterday's quest to prove how strange I am, here is the story of another language-related dream. When I was in school, one of my areas was medieval literature. I took a lot of Middle English (ME) courses (Chaucer and others), as well as several Old English (OE) courses (*Beowulf*; no comment today on the upcoming movie). These courses, particularly the OE ones, are very time intensive. I became faster at translating OE the more I did it, but I never got enough practice to be very quick. I would spend hours a day translating a small passage for the next class. It consumed me. And at night, I couldn't escape. Invariably, once or twice during the semester I would dream about OE. It was always the same dream: A charcoal-gray background. Words in light gray with a slight halo. The words would speed by faster than I could read. Some of the words contained OE letters like thorns, eths, aesces. Other words were obviously ME. The two mashed together, interlocked with each other. And they kept coming. Faster than I could read, they scrolled by, they never stopped. It probably sounds like the dream had some emotional content, but it really didn't. There were just words without meaning or feeling. I don't know that these were bad dreams in themselves, but the fact that I had them both dismayed and amused me.

Grammar Police

The other night I dreamed I was explaining grammatical rules to someone. Actually, I was explaining how there are rules, rules, and rules. There are rules that someone in the past made up and—if I abide by them at all—I pay attention to them only because other people would think poorly of me if I break them, and language is nothing if not conventional. Generally, all the big, bad grammar rules your teacher used to yell at you about fall into this category. Then there are rules that are good, have basis, and I follow them sometimes. After all, rules are made to be broken. Finally, there are real rules that, when you broken them, you're no longer talking English. If you're a native speaker, you've probably internalized these rules and breaking one just wouldn't occur to you. When I woke up from my dream, I was explaining this to someone, and I pointed out that it didn't matter if you explained, for instance, why the rule about split infinitives is bogus. Those who like rules are going to insist that all these rules must be followed at all costs. Some people just can't handle the freedom, I guess. (Of course, any discussion of faux grammar rules is incomplete without mentioning the phrase attributed to Winston Churchill (falsely): "This is the kind of pedantic nonsense up with which I will not put!" Unfortunately, this refers to ending a sentence with a preposition, not split infinitives.) Against my better judgment, I ask: does anyone else here dream about grammar? The resounding silence reinforces how strange I am,

Concurrent Thinking, Part 3

When I decided to dust off and publish the old post I had written, I wasn't expecting it to turn into a week-log series. And it's not. This is the last.

---

If you'll remember, gentle reader, I've been foreshadowing trouble. Today's the payoff. You'll remember that the reason I didn't just slurp the entire file into memory at one time and build lists from it is that I need to be able to process files that were much larger than would fit in memory. If the process reading the file can read all of it before the transformation process can handle much of it, or before the output process can write the output, the file will be read into memory in its entirety, more or less, and this entire exercise has failed. To imagine how this could happen, suppose the input comes from a local file, but the output is being written across a network. The input process could read all the file, and the transformer process could handle the data, but the messages would still pile up in the output process's mailbox, filling memory and bringing the system to its knees. Or suppose the transformation being applied involves a time-intensive computation. Its mailbox would fill up memory when the input process sends it the entire file while the first few messages are being transformed. Again, the system dies, horribly, horribly. I won't tell you how I solved this when it happened to me. Looking back, it was a nasty, nasty kludge that I only used because I was under time pressure. What I should have done was use the functional solution someone mentioned in a comment to Concurrent Thinking, Part 1. This solution involves having the input and transformation functions return a tuple with two values: data and another function. The new function will return more data and yet another function. Eventually, when there's no more data, the function will just return a flag indicating that. I won't rewrite the entire example from yesterday in this style, but the input function would look like this:

input(Filename) ->
    {ok, F} = file:open(Filename, [read]),
    input_continuation(F).

input_continuation(F) ->
    fun() ->
        case file:read(F, 1024) of
            eof ->
                file:close(F),
                eof;
            Data ->
                {Data, input_continuation(F)}
        end
    end.

Here, input/1 opens the file and calls input_continuation/1. This returns a function, A. When you call A, you get the first block of data and a new function, B. B will return the second block of data and a function C. This happens until there is no more data, at which point the file is closed and eof is returned. This solution is very Erlang-ish, and it throttles nicely so it doesn't accidentally fill up memory. End of series.

Concurrent Thinking, Part 2

So, briefly, the problem involves piping large amounts of data through a series of transformations, usually just one. In other languages I would use some form of lazy lists to plug together the input reader, transformations, and output writer. In Erlang, I decided to use processes. One interesting thing about Erlang processes: If you squint at them just right, they act like classes do in other languages, just without inheritance. The processes communicate by passing messages, and those messages trigger the processes to perform predefined actions. This seems very like message-passing paradigm of object oriented programming. Basically, one process reads the input and sends it to the transformer process, which in turn sends it on to the output process. Here's the pipeline I implemented in Python above (it has to be saved in a file named "erltransform.erl" to work):

  -module(erltransform).

  -export([input/2, transform/1, output/1, main/2]).

  input(Filename, NextPid) ->
      {ok, F} = file:open(Filename, [read]),
      input(F, file:read(F, 1024), NextPid).

  input(F, {ok, Data}, Pid) ->
      Pid ! Data,
      input(F, file:read(F, 1024), Pid);
  input(F, eof, Pid) ->
      io:format("closing input.~n", []),
      ok = file:close(F),
      Pid !  eof.

  transform(OutPid) ->
      transform(OutPid, 0).

  transform(OutPid, 0) ->
      receive
          eof ->
              OutPid ! eof;
          _Data ->
              transform(OutPid, 1)
      end;
  transform(OutPid, 1) ->
      receive
          eof ->
              OutPid ! eof;
          Data ->
              OutPid ! Data,
              transform(OutPid, 0)
      end.

  output(Filename) ->
      {ok, F} = file:open(Filename, [write]),
      write_output(F).

  write_output(F) ->
      receive
          eof ->
              ok = file:close(F),
              io:format("closing output. done.~n", []),
              eof;
          Data ->
              ok = file:write(F, Data),
              write_output(F)
      end.

  main(InFilename, OutFilename) ->
      OutPid = spawn(erltransform, output, [OutFilename]),
      TransformPid = spawn(erltransform, transform, [OutPid]),
      spawn(erltransform, input, [InFilename, TransformPid]).

Hmm. This is a lot longer, and it doesn't do as much. Erlang's libraries are OK, but the standard distribution doesn't have Python's batteries included approach. Since Erlang doesn't come with a library to read and write CSV files, this just reads a block of 1024 bytes, and it drops every other block. Some explanation: At the beginning of the code above, the input/2 function opens the input file, reads the first block, and calls input/3, which sends the data to the transformer PID and loops. When it reaches the end of the file, it closes the file, prints a message, and exits. The transform/1 function calls transform/2 with a flag indicating whether it should keep or drop the next block of data that's sent to it. Based on the value of this, it either sends the data on or ignores it. When it receives eof, it sends that on and exits. The output/1 function opens the file and sends it on to write_output/1. (Since both of these functions have the same arity, I had to give the second function a different name.) Output just writes the data it receives out and closes the file when told to. Finally, main/2 spawns all the processes, linking them together, then it gets out of the way. In this case, the messages being passed are simple: the data read, dropped, or written. Yesterday, I hinted that this approach got me in trouble. How? That, gentle reader, is tomorrow's episode. Stay tuned.

Concurrent Thinking, Part 1

Note: I originally wrote this back in June, but never finished it. An now, presenting part one.... Concurrency is the key idea to come to terms with in Erlang and the key tool that it gives you. It's the modeling tool to use when solving a problem. I realized this when I was implementing a code transformation pipeline. The pipeline would go input to transformation to output. I wanted to be able to interchange any of those pieces, so I could read from multiple formats, transform the data in different ways, and write the data out in different formats. And it needed to be able to handle a lot of data. On my first attempt, I approached the problem the way I might with Python: Input, transformation, and output are iterators which are just chained together. For example, the input would read CSV, the transformation would drop every other item, and the output would write it back to CSV::

  from __future__ import with_statement

  import csv

  def input(filename):
      with open(filename, 'rb') as f:
          for row in csv.reader(f):
              yield row

  def transform(rows):
      rows = iter(rows)
      while True:
          rows.next()
          yield rows.next()

  def output(rows, filename):
      with open(filename, 'wb') as f:
          writer = csv.writer(f)
          writer.writerows(rows)

That's not very useful, but you get an idea of the problem and what I was trying to do. Using iterators/generators is nice here because I don't have to worry about too much input filling up memory. Also, I don't build a list I don't need or use. Unfortunately, Erlang doesn't have iterators. I thought I would use lists instead and work from there. I could have had the input function read everything into a list, the transformation function transforms it into another list, and the output function writes a list's contents to disk. It wouldn't have been pretty, but it would have worked. Part way through coding this, I realized that Erlang has a great solution to this: concurrency.

---

Stay tuned tomorrow, when the hero attempts to use powerful Erlang concurrency and ends up hoisted on his own petard.

Plot Twists

Today's Sunday, it's late, and I still have a lot to do today. So you get the reward. Here's a toy. I don't know how useful it is, but it is fun to play with. Plot Twist Generator

Novel Writing

I mentioned NaNoWriMo a couple of days ago. I'm sure that at the end of this, many people who actually managed to turn out 50,000 words of precious, absolutely brilliant, ground-breaking, Pulitzer-winning material just want to see it in print. Lots of people will do that for you—or pretend to—for a price. When you think about it, this is a situation ripe for someone to take advantage of. A writes a book. Convinced that it is the next Great American Novel, he or she sends it off to several publishers and agents. He doesn't hear anything for a while, but when he does, no one seems to share his opinion of his work. All say, "Sorry, not right for us." Finally, he finds an agent who guarantees results, but who wants a reading fee. That seems reasonable. Or he finds a publisher who charges him to publish the work, but points out that they will be in featured on Amazon and in a major national chain. Surely he'll recover the fee and be making more money than he ever dreamed of. I've written one novel so far, and I haven't gotten any published yet (I haven't tried, since it was crap). So my credentials on this are thin. But I do know this: writing is hard. All stages of the writing process are hard. Writing is hard, editing is hard, getting an agent is hard, getting a publisher is hard (which is why people hire agents to do it), marketing your book is hard (which is why publishers help). There are no shortcuts. Talent helps. Being willing to look hard at your work and to take criticism helps. But perseverance, stubbornness, and sweat are critical. The point? Be careful. You are probably fine, but many new writers aren't aware of what rubbish their work is, and many people are willing to feed them lies, take their money, and deliver nothing. Before you try to get published, do two things. First, ask yourself why you want to be published. Do you want to hold a printed book in your hand with your name on the spine? Fine, self-publishing or print on demand (POD) is probably fine. But if you want to be read by strangers or make money writing fiction, the traditional publishing industry—while it has its problems—is your best chance right now. And face it, most people write to be read, being published is just a means to that end. Second, do your homework. Visit Writers Beware. Hopefully, you'll be establishing a long-term business relationship with your agent. Make sure it's someone honest, someone good at their job, someone you can work with. Write well. Be safe.

Django

For work, I've been moving some of our processes to be more database-centric. Databases themselves aren't that useful or interesting for the average user, who doesn't want to get grubby with SQL. I've been putting a web front end on things, and I decided to use Django to do it. Before this, I had worked through the tutorial a couple of times, but I had never actually done anything with the web framework. Overall, I really like it. It could use some brushing up in parts, but then it's not at the 1.0 release yet. And some parts are incredible. The admin interface deserves everything that's been said about it. Here you get a complete interface on your database tables for almost no effort. Incredible. The main downside is my own insecurities. Am I doing this the best way? Am I structuring my code the best way. I've divided the project up into about half a dozen apps, based upon categories that make sense according to our processes, but which are ultimately arbitrary. I should probably get on (IRC) #django and ask some questions. But if history is any indication, I will continue to stumble along on my own.

National Blog Posting Month

No, this blog is not dead! I've been considering changing the focus somewhat, and I've been busy with work and life (as usual). But, this is National Blog Posting Month (NaBloPoMo). In honor of which, I've decided to join that and post here every day for a month. I could have joined the larger, original movement, National Novel Writing Month (NaNoWriMo). I am still working on my novel—I'm sure I'll talk about that more soon—but I'm not taking part in that. I'm not that crazy. Is anyone here taking part in either of these insanities?

What's in the Box?

That is the theme for my own personal media conglomerate. (I'm sure it says something that I don't have a lone TV station, radio station, or web site for my internal media. It's a conglomerate. At least, that's what the voices tell me.) Anyway, early in the book I'm working on, I wanted to up the ante for a scene, so I added a box. The box isn't opened. It's just a closed, locked box, about 5in by 6in by 1in, covered in dusty, musty smelling leather. Two characters steal it for someone else, without knowing what's in it. My plan was to have them steal the box to get the plot moving. Then they would turn it over to the person who wanted them to steal it, and the box would disappear. Unfortunately, the box refuses to disappear. It's a "gun." In playwriting, if you introduce a gun in act one, it has to fire in act three. The gun is just an example of the principle that everything you introduce at the beginning of a story is a promise. And by the end, you have to deliver on that promise. In my case, once I introduce the box so early in the story, it has to play some role before the end of the book, and someone has to open it. So I'm left with the box. I don't know what's in it, but it had damn well better be something good. Later, I'll tell you what my wife said was in the box. In the meantime, what do you think is in the box?

---

You might be wondering, after my previous post, why I'm talking about my writing. Two reasons. First, I track the visitors to this site, and I can say authoritatively that no one really comes here. (If you do visit this site, thanks, I appreciate it. But it's pretty much just you and me.) So writing here is more like journaling or talking to myself than it is blogging. Second, from what I've said here, do you really know that much more than you did?

Novel, Draft One, Day Two

I had forgotten about this: writing a first draft is always an exercise in learning to ignore myself. I want to know if I'm in the process of crashing and burning, but at this point, my gut does a spectacularly poor job of evaluating my writing. My first drafts always suck. I'm much better at revising my drafts than I am at creating them. In general, life would be so much easier if I could just get things right the first time.

Novel, Draft One

I finished plotting, character development, and planning for my novel. I'll start writing the first draft on Sunday. Insert small, quiet party here.

Targeting the DLR

I've been circling warily around CarbonLisp again. I think I've decided not to target the DLR initially. Instead, I will have a compiler that, like Common Lisp, will produce code that varies from very dynamic to static. Once that's done, I may add a DLR component. Of course, I have to get started first.

Speculative Fiction

I just read Margaret Atwood's piece on the uses of science fiction. It's very good, and I want to comment on it more some other time. Right now, however, what caught my eye is her comments on what science fiction is. As a genre, science fiction is interesting. There are different ways of defining it and different ways of subdividing it (alternate history, space opera, etc.). Some people want to include fantasy or science fantasy in the mix; some want to leave out anything that's not strictly science. Some want to include only science as we know it; some want to broaden it to include any stories that use a science-like explanation for their fantastical elements. What made me pause in Atwood's piece was her definition of science fiction and speculative fiction. Here's what she says:

I like to make a distinction between science fiction proper and speculative fiction. For me, the science fiction label belongs on books with things in them that we can't yet do, such as going through a wormhole in space to another universe; and speculative fiction means a work that employs the means already to hand, such as DNA identification and credit cards, and that takes place on Planet Earth. But the terms are fluid. Some use speculative fiction as an umbrella covering science fiction and all its hyphenated forms - science fiction fantasy, and so forth - and others choose the reverse.

I've never heard anyone divide the genre up that way. I've generally heard what she calls *speculative fiction* referred to as *hard science fiction* or *mundane science fiction*. I'm sure that many others use *speculative fiction* in this way, but I don't remember ever seeing it myself before. Of course, the naming is complicated by science fiction's inferiority complex. Is it literature? Is it a backwater of poorly written dreck? This is another topic for yet another day. I tend to use *speculative fiction* myself. What attracts me to the genre is best captured by the word *speculative*. Whether using science (naturalistic explanations) or fantasy (supernatural explanations), it gives me a lot of room to explore interesting territories and to tell fascinating stories. Which term do you prefer or use?

Heroes Season Premier

The season two of Heroes started last night. I thought this episode was all right. In general, I like the show's broad scope, but all those plot threads is also its weakness. This episode was an example of both its strengths and weaknesses: by itself, this episode seemed diffused, but it promised a lot for future episodes. In a sense, however, that's exactly what you want for the beginning. I'm looking forward to seeing where they take all the characters over the course of this season. Chuck seems really good too.

Talking about Writing

The first week of September, my wife and I went to the beach with my extended family. We had a great time. While I was there, I took my laptop and worked on the novel I'm writing. Generally, I would try to slip out to the porch with a cup of coffee where I would write, sip coffee, stare at the ocean, and pretend to work. It was wonderful. One afternoon, my brother-in-law asked me what I was working on. I stammered around a bit, but I didn't have a good answer. Actually, I could have told him exactly what I was writing, but I prefer not to talk about works in progress (WIP). My wife knows this, and she jumped in and saved me. Eventually, I just told him it was a fantasy novel and let it go at that. He doesn't read fantasy himself, I don't think, so he is probably now thinking Lord of the Rings. (It's nothing like LOTR.) I have read other writers talk about how they don't like discussing their WIP, so I'm not alone in this. But when he asked, I really, really felt like I was by myself in the spotlight. I didn't want to give the answer, but I couldn't think what else to say. Part of my "logic" on this is gut-level superstition: if I talk about it, that would ruin the work in some obscure way. However, part of my reluctance is more rational: I write to express something, and if I talk about it, I might not need to write it anymore. Or maybe I'm just crazy. What do you think? How should I have answered my brother in law?

Writing, not Coding

I realize that I've been talking a lot about the coding that I would like to do. But the name of this blog is Writing/Coding, and I haven't talked at all about writing. That is ironic, since I've been writing for pleasure more than I have been coding. I always wanted to write a novel, and to write one well. Last year, I sat down to write a science fiction/fantasy (SF/F) novel. Immediately, I was struck by writer's block, striving to write something original. I knew consciously that my writer's block made no sense: it would be my first novel and utter crap, no matter how original it was. Because of that, I charted a new course: I would write the most derivative, cliché-ridden work possible, intentionally. I wrote more than 125,000 words (a novel is typically 80,000-100,000 words), and as it turns out, originality was not really the problem. Original or not, it was still utter crap. While individual scenes built and maintained tension, I never was able to carry that tension over a longer story arc. In the end, I decided I would have to start again almost from the beginning, and because the concept was unoriginal, I just wasn't motivated to continue that story. So I let it drop. This year, I resolved to write another novel, again SF/F. It took me a long time to decide on and develop an idea I was happy with, but finally I started writing in July or so. I've been developing an idea since then. I've worked on the universe and the characters, and now I'm developing the plot. Six days a week, I get up early and work for an hour to an hour and a half. It's going all right so far: I like the idea I am developing, and I'm reasonably happy with the execution thus far. And that's the news on the writing front.

Common Lisp Tutorial

If you want to learn Common Lisp, an excellent option is Practical Common Lisp, which is also a Lisp tutorial. It's available both in print and on-line. (Note, I get nothing from this shameless plug. But I do honestly recommend it. I have my own copy, and I refer to it regularly.)

LiSP, Revisited

Remember that I ordered a copy of Lisp in Small Pieces from a sales that Amazon.ca was putting on? Like many people, I was informed that the price was due to a mistake, and they were canceling my order. Easy come, easy go. C'est la vie. Fill in your favorite cliché here. I did look at a copy of the book in a library. (I don't qualify for a library card there, so I didn't check the book out.) It looks like a great book. Maybe I'll have to get off my wallet and buy it anyway. While browsing through the book, I realized that I'm attempting to implement too much of Common Lisp initially. For example, the lisp reader needs access to the full-fledged lisp system in order to modify its own settings. It's a chicken-and-egg situation. You need a full lisp system to completely implement the reader, and you need a reader to completely implement the system. The answer, of course, is to implement a very restricted subset and bootstrap your way up. Duh. Anyway, now I think I have a plan to go forward with. I haven't done anything on it yet, though. I'll keep you posted when I do.

I'm Baaaack

After almost a month, I'm back. Work has been crushing, and I went to the beach for a week. That's really no excuse, though. More posts are on the way.

Lisp in Small Pieces

Amazon.ca was having an unexpected sale on Lisp in Small Pieces, one of the best books on implementing lisp. For about a day it was selling better than Harry Potter. Hmm. I'm implementing lisp, and I could always use a new book. My copy will arrive in two to four weeks. Until that time, CarbonLisp is going on vacation. In the meantime, I'll work on and blog about other projects.

Roadblocks and Dynamic Coding

What's happened since my last posting?

• Google's identified this blog as a potential spam-blog and locked me out. I've gotten it unlocked. I guess i need to post more often.
• I've gotten most of the framework code for the DLR finished. Nothing exciting, and I'm sure it's bug-ridden, but it's a start.
• I've started on the lisp reader, and I realized I don't know what I'm doing. Fools rush in where angels fear to tread, and all that. Part of the problem is that I'm really doing exploratory programming, but in a static language. I'm not a big static language fan, anyway. I'm considering doing a toy implementation of lisp in Python to get started and to figure out what I'm doing.

I know, I know. That last point is just crazy talk. Of course, this whole blog has turned into the ramblings of a madman, anyway, so why the hell not get some traction by doing some exploratory programming in Python, then moving back to C#.

Back from Vacation and Names

Most of last week I was on vacation. The rest of the week, I've spent a little time digging into ToyScript and filling out the framework I'll need for implementing lisp on the DLR. More about that later. I've temporarily decided to call this project SiliconLisp. This roughly follows the convention of naming dynamic languages for the CLR as Iron-. I've just changed the element it's made out of. Actually, I was originally going to call it CarbonLisp, but I didn't want to confuse this with the Mac windowing library Carbon. I may go with CarbonLisp anyway. If you have a suggestion for the name, please leave me a comment.

Looking into ToyScript

I'm checking out John Lam's ToyScript. It's an implementation of a small scripting language, and it was written as an introduction to working on the DLR.

Lisp and .NET

The Truth about Lisp Why does that link matter? Because I've been thinking about implementing Common Lisp on the CLR and the DLR. When interpreted, it would primarily work with the DLR, but as declare statements are added and the Lisp code becomes more static, it would be able to leverage the CLR directly. At one point, I was thinking about implementing Scheme, but I've moved away from that. When I started learning Common Lisp, I thought that I preferred Scheme. But now I've done a project in Scheme (a simple todo-list manager, similar to todo.txt, and I've decided that I prefer Common Lisp after all. Scheme's still great. I have a nice clean feeling whenever I code in it, but I think I prefer CL's more complete language specification. It's still not really complete enough. (Hello? threads, networking, Hello?) Also, I'm beginning to grok continuations, but I don't have a clue how I'd implement them in .NET. So I'm just not doing that right now. Still, what craziness is this? Well, I've been doing a lot of .NET at work, and I'd like to be able to use a more powerful, sane (for some definition of sane) language. I'd also like to be able to produce assemblies that can be called from C# or whatever. Basically, I want a lisp that is a first-class citizen of the CLR. And I've been wanting to do more Lisp for my personal projects. I'll let you know how it goes. Or I'll put my head between my knees and breath deeply until this latest light-headedness passes.

Update

I've had a post about Erlang and thinking concurrently in the works for a month, while life and work have gotten in the way. That article's still in the works, but I'd also like to resurrect this and post about some other things.

Erlang: The Pros

I had meant to post this much sooner, but there's been this thing called work. It gets in the way. Finally, here is the second half of my overview of Erlang. Concurrency Given it's background, it's no surprise that concurrency is easy and cheap in Erlang. Here's a contrived example:

12> PrintRandom = fun() ->              
        {A, B, C} = erlang:now(),           
        random:seed(A, B, C),
        io:format("~p~n", [random:uniform()])
        end.
#Fun

13> lists:foreach(fun(_) -> spawn(PrintRandom) end, lists:seq(1, 10)).
0.664105                                                             
0.664137
0.664170
0.664203
0.993887
0.993920
0.993953
0.993986
0.994019
0.994052
ok

First, the state for the random number generator is stored in each process, and it's initialized from a standard value. (Actually, maybe the random module should be on my Erlang: Cons list, although having the random module not produce random numbers probably makes debugging easier.) Line 13 spawns 10 processes, each of which prints a random number. Not very useful, but it should illustrate how easy it is to create processes. Also, you can easily spawn a lot processes. This code generates a random number and throws it away a million times in a million processes, and it executes in about 3.5 seconds on my machine:

-module(timespawn).

-export([make_random/0, spawn_random/1, ts/1]).

make_random() ->
    {A, B, C} = erlang:now(),
    random:seed(A, B, C),
    random:uniform().

spawn_random(0) ->
    ok;
spawn_random(X) ->
    spawn(fun make_random/0),
    spawn_random(X-1).

ts(X) ->
    timer:tc(?MODULE, spawn_random, [X]).

Distribution Another nice aspect Erlang is how easy it is to distribute processing. This is partially due to its open security model, which I listed as a negative. For example, in two separate console windows, I can start two different instances of Erlang. As long as I pass both the -sname argument (with different values) and the -setcookie argument (with the same value), they can talk to each other, even if they're on different computers in the same network. Functional I've worked with functional languages in the past, but I haven't really drunk the functional Kool Aid until now. I'm enjoying it this time, though, and want to start working more with one of the Lisps (probably Scheme), Haskell, or ML. When I do start on one (or more likely, on all) of these, I'm sure I'll talk about it here. OTP The OTP (Open Telecom Platform) is the standard Erlang library, and it contains modules that make writing fault tolerant, supervised, client-server, and distributed applications easy and fast. Conclusion I'm glad I've gotten to do a lot of Erlang recently. It's changed the way I think about concurrency and distribution, and it's raised my expectations of other systems that tackle the same problem. Even if I do most of my concurrent programming primarily in other languages, this will give me a good basis. And really, what more can I ask for from a programming language?

Erlang: The Cons

Today, I'm going to talk about the warts I've seen in Erlang so far. I'm sure I'll find other things I don't like, and I'm also sure that I'll make peace with some of the things I've listed here. Nevertheless, here is my list of the bad and the ugly so far. (This isn't a tutorial on Erlang, so I'm not going to explain the code snippets below. Sorry.) Strings The one thing that really made me go "Yuck" as I was learning Erlang is strings. Strings are just lists of integers, which are treated as strings for printing. For example, if you crank up the read-eval print loop (REPL) and type in a list of integers, it gets printed as a string if there are no control characters in the string:

Eshell V5.5.4  (abort with ^G)
1> [69, 114, 105, 99].
"Eric"

Likewise, if you type in a string, it's interpreted as a list of integers:

4> lists:foreach(fun(X) -> io:format("~p ", [X]) end, "Rochester").
82 111 99 104 101 115 116 101 114 ok

Initially, this just felt wrong to me. It still does in some respects, but I've also learned to appreciate how this makes string processing easy most of the time, particularly when it's coupled with Erlang's fantastic pattern matching features. For example, the function get_ints in the module below walks through a string and returns a list of all the integers in the string:

-module(get_ints).
-export([get_ints/1]).

get_ints(String) ->
    get_ints(String, [], []).

get_ints([], [], Ints) ->
    lists:reverse(Ints);
get_ints([], Current, Ints) ->
    CurrentInt = list_to_integer(lists:reverse(Current)),
    lists:reverse([CurrentInt|Ints]);
get_ints([Chr|Data], Current, Ints) when Chr >= $0, Chr =< $9 ->
    get_ints(Data, [Chr|Current], Ints);
get_ints([_Chr|Data], [], Ints) ->
    get_ints(Data, [], Ints);
get_ints([_Chr|Data], Current, Ints) ->
    CurrentInt = list_to_integer(lists:reverse(Current)),
    get_ints(Data, [], [CurrentInt|Ints]).

To use this, save it in a file named "get_ints.erl", compile it, and call the function:

8> c(get_ints).
{ok,get_ints}
9> get_ints:get_ints("This is the answer: 42").
"*"
10> get_ints:get_ints("This is the answer: 0, 42").
[0,42]
11> get_ints:get_ints("This is the answer: 0, 42, 23").
[0,42,23]

Notice that the first list of one integer ([42]) is interpreted as a string. I included the number zero (a control character) in the next test to force Erlang to print the results as a list of integers. Another small complaint with strings involves Unicode. I've done enough processing XML and processing phonetic data that good Unicode handling is important to me, no matter whether I'm using it much at the time or not. In one sense, Erlang handles Unicode just fine. A string containing a schwa character is just [601]. Unfortunately, this is the depth of its Unicode handling. It doesn't give you any information about the Unicode data points or a function to change a character from upper-case to lower- or vice versa. Security Another complaint is Erlang's security model. On the one hand, it has the virtue of being easy to set up, but if two nodes can connect and communicate, there's nothing one can't do on the other node. Having more fine-grained control over things would be nice. Speed Also, there's the issue of speed. Erlang is generally fast enough, and where it's not, you can easily set up a node written in C, Java, or C#. Still, being able to deal with everything in Erlang would be more convenient. REPL Finally, there are restrictions in working from the REPL that I could do without. To create a function like get_ints above, I more or less have to create a new module in a file and put the function there. There are ways to work around this, but they seem unnatural. I'd rather not use them. Nothing on this list is a deal-killer for me. I've been doing a lot of Erlang the past few weeks, and I've really enjoyed it. It's been productive and interesting. Still, I can always dream about a better world.

Erlang: An Overview

Recently I've been programming in Erlang. It's a new computer language for me, and I thought I would talk about it. This will take several posts. Today I'll just provide an overview of the language and environment. Erlang was developed by Ericsson telecommunications for developing soft real-time concurrent systems. I've never developed real-time systems before, soft or hard, so I won't talk about that much. I haven't done a lot with concurrency, either, except a touch of threading here and there. Nevertheless, the concurrency and distributed features of this language are what really attracted me to it. Originally, Ericsson implemented Erlang in PROLOG, and this still shows in some parts of Erlang, such as the way it uses pattern-matching. As an undergraduate, I went through a period where I programmed more or less only in PROLOG for about two years, so a lot of this feels like a stroll down memory lane. Other parts are very new. It's primarily a functional language. It has a lot of features designed for doing telecom programming, such as processing binary data. Erlang code is organized into modules, which are compiled and run from a read-eval-print-loop (REPL). A short "hello world" style module would look like this:

-module(hello). -exports([hello/1]). hello(Name) -> io:format("Hello, ~p.~n", [Name]).

The -module line identifies the module, which is also the name of the file containing the module. The -exports line identifies the functions from this module that are exposed to the outside world. The hello function is the rest of the module. io:format is the standard way to print to the console. Next, I want to look at some of the parts of Erlang that I'm less fond of.

Introduction

Welcome. I am interested in language and literature, technology and computers. This is my blog for exploring both, separately and together. A lot of interesting things are happening where these topics intersect, and I'm looking forward to tracking my thoughts, as well as pointing to what others are doing. Currently, I'm working for a company that does litigation support. This blog isn't about that, and this will be the last I mention that here. My interests in literature are fairly wide ranging. I did most of my graduate work in medieval literature, so you'll probably see a lot of references to Beowulf, Chaucer, and Sir Gawain and the Green Knight. Analyzing ancient literature with computers creates interesting problems beyond the standard issues that analyzing just any text creates. More on that later. When I'm working with computers, I do a fair amount of programming. Currently, I primarily use C#, Python, and Erlang. I keep looking for excuses to use Common Lisp or Scheme, but I haven't had the time to get around to doing more in those languages. Most of the topics I am currently planning to talk about are computer-related, although they might touch on humanities topics also. I'd like to write posts that are more directly and purely about literature, too, but in the meantime, be prepared for technology. Glad you're here. Stick around.