In my mental model of knowledge, I split skills between three major mastery levels:

• Those we can use professionally. I program professionally, and I practice daily to strive for mastery. I’m getting to the point where I can handle any task that is thrown at me.
• Those we can use. I can cook without killing myself of burning down my apartment complex, even though I can’t make many types of food and wouldn’t recognize a Mother Sauce if it was poured down my shirt.
• Those we couldn’t really use if our life depended on it. I’m not good at drawing. I’ve never spent time on it, and I have no motivation to learn. I can “draw” diagrams and get my point across, and this is enough for me.

All people treat the skills from the first category differently than the other two: we use them with confidence. Some people use them with arrogance. We know these fields well enough to absorb innovations and new ways of thinking. We can confidently discuss them with other people.

In my mental model, starting from scratch in our expert area causes people to reject a change. For instance: an Excel or Word guru would never willingly switch to a completely unrelated product. Relearning things line inserting line-breaks and adding pictures to a text document is completely orthogonal to anything they actually have to do on the job, much less something more complicated like Mail Merge. Instead of using it like it should be used, they would likely try using it like they used their old product, declare that it can’t ramrod the flimjam, and switch back to their old way of doing things, regardless of which is really better.

Interfaces and APIs

Altered interfaces are difficult for the end user. This isn’t just about GUIs or the placement of buttons in a menu; this extends to software APIs as well.

An altered menu causes all users to waste time looking for new ways to do something they could do 5 minutes ago. Changes to an API cause the programmers relearn a library they already knew — not to mention the laundry list of changes that need to happen to the new stuff!

APIs and Interfaces

The interface for a library or class develops massive unavoidable dependencies with use. I’ve called this “code inertia” in the past. Joel and Jeff’s recent discussion of dependencies between unit tests and code is a perfect example of code inertia: making nontrivial changes to an application can force a nontrivial number of changes to your automated testing environment.

This has not only a physical effect on the code, but also a mental effect for programmers! I’ve actually heard arguments from programmers based on the IDEA of possible future change: we shouldn’t use this library because it’s changed significantly in the past, and it might change again! Having to relearn the API combined with the frustration of updating broken code leaves a giant red flag in a programmer’s mind, even though any codebase that is being properly maintained will go through interface changes as new versions are released.

Excel

I was fortunate enough to take a Numerical Methods and Analysis class during my senior year of college.

If you’re not familiar, it’s the mathematics of computation: finding efficient approximations, determining rate of convergence, and knowing the margin of error at each stage.

This was my only college class where a calculator was ever useful. Most of our homework involved finding the values and error in iterative processes, so the professor suggested a “complex math package” for our homework: Excel.

Despite my initial skepticism, Excel is AWESOME with iterative numerical computation: type in a relative formula, highlight 50 rows, and click “fill down.” Voila! You can put a bunch of results in adjacent columns and compare convergence rates visually before you even prove that it converges at a certain rate. It’s quick, it’s interactive, and it supports easy experimentation.

Why is this relevant? My college upgraded to the Microsoft Office 2007 suite the previous summer, toolstrip and all. Every single class was punctuated by my professor loudly complaining that she HATED the new Excel because the interface was WORSE and she couldn’t find anything.

Even though she knew where everything was after 2 weeks of use, we got the same complaint every week. She had already made up her mind: it was worse.

KDE and Linus

A few years ago, Linus Torvalds publicly announced his Linux desktop preference on the GNOME message boards:

I personally just encourage people to switch to KDE.

This “users are idiots, and are confused by functionality” mentality of Gnome is a disease. If you think your users are idiots, only idiots will use it. I don’t use Gnome, because in striving to be simple, it has long since reached the point where it simply doesn’t do what I need it to do.

Please, just tell people to use KDE.

Linus

The different camps typed their representative emoticons — and D: — and then life continued as normal. However, Linus was eventually forced to switch when the next version of KDE changed too much:

Interviewer: Another open source project that underwent a big change was KDE with version 4.0. They released a lot of fundamental architectural changes with 4.0 and it received some negative reviews. As a KDE user how has this impacted you?

Linus: I used to be a KDE user. I thought KDE 4.0 was such a disaster I switched to GNOME. I hate the fact that my right button doesn’t do what I want it to do. But the whole “break everything” model is painful for users and they can choose to use something else.

I realise the reason for the 4.0 release, but I think they did it badly. They did so may[sic] changes it was a half-baked release. It may turn out to be the right decision in the end and I will re-try KDE, but I suspect I’m not the only person they lost.

I got the update through Fedora and there was a mismatch from KDE 3 to KDE 4.0. The desktop was not as functional and it was just a bad experience for me. I’ll revisit it when I reinstall the next machine which tends to be every six to eight months.

The GNOME people are talking about doing major surgery so it could also go the other way.

Linus realizes that it might be the right decision in the long run, but there’s no escaping the fact that even the big guns in the field can’t spend his time relearning the basics every day, even if it’s the right thing to do. He has a mental model of how the desktop is supposed to work, and when KDE broke it, he fixed the problem.

Adult Learning and Malcolm Knowles

To me, a lot of the principles of API and interface design fall under adult learning. To state the obvious, most of the indended clients for interfaces are adults. We’d like them to learn how to use our products with the minimal amount of fuss.

However, if your interface is exactly like every other interface, there would be no reason to use it. You’re trying to solve a problem for yourself, and you’re trying to solve a problem for your intended users. Your users will have to learn how to solve their problems with your interface, and it had better be easy.

Enter the research of Malcolm Knowles. He is one of the big-shots in the field of adult learning, having done a lot of the early theoretical work. This site has a good description of his life’s work with researching, for the interested.

In particular, Knowles attempted to find the theory of what caused adults to best learn. Believe it or not, there hadn’t been extensive research in the field before he started. The closest that he was able to come was his theory of informal learning, through group sessions or other kinds of informal meetings. He also developed his assumptions by which adults learn:

Malcolm Knowle’s Assumptions on Adult Learning

1. Self-concept: As a person matures his self concept moves from one of being a dependent personality toward one of being a self-directed human being.
2. Experience: As a person matures he accumulates a growing reservoir of experience that becomes an increasing resource for learning.
3. Readiness to learn. As a person matures his readiness to learn becomes oriented increasingly to the developmental tasks of his social roles.
4. Orientation to learning. As a person matures his time perspective changes from one of postponed application of knowledge to immediacy of application, and accordingly his orientation toward learning shifts from one of subject-centeredness to one of problem centredness.
5. Motivation to learn: As a person matures the motivation to learn is internal.

The salient bullet point is #2: experience. It’s directly listed as a basis for adult learning. I’m not saying that this is a direct proof of my mental model of knowledge, because quite honestly it’s just not a proper conclusion. However, designers of APIs and GUIs should take this as an indication that experience is something to build on, not disregard.

To use an aforementioned example, I really like the ribbon interface in Office 2007! I was proficient after only a few hours, and I enjoyed using fewer keystrokes to achieve my goals. I thought that it mapped pretty well to the old menu-based methods, and it was pretty obvious when it didn’t. If I could use only keyboard shortcuts to navigate, it would be the ideal control.

However, it’s probably a bad idea releasing it in the same year as Vista, which already scrambled some menus, changed some positions, altered some interfaces, renamed some popular folders, added UAC, etc. It all added up to too much of a change: when people sat down at it for the first time, nobody couldn’t use the darned thing.

Live and learn!

Footnote

I remember a minor fuss when Linux 2.6.0 came out because the device driver interface changed. Linus himself descended from the cloud to state that he makes the right design choices for Linux without regard to compatibility issues. It was going to tie in really nicely with my point, and my Google Fu has failed me. up. The device driver interface changed from 2.4 to 2.6, and that there was a of device driver complaints. At this point, I just want to know if I was making this up.

Overview of BattleCode

The competition’s focus tends to vary from year to year. Last year’s competition could be won either by capturing most of the towers on the map or by completely wiping out the players on the other team.

It’s even better this year: it’s primarily a resource-managing competition. Certain tiles on the game board contains a resource (flux) that you can gather. If you have more when the round ends — or cross a magic threshold during the match — then your team will be declared the winner.

It’s probably still possible to create a winning team that is entirely militaristic, but I have a feeling that the organizers are trying to reward teams that split-up. Most of the teams that split up into groups last year were defeated by monolithic swarms, but this mechanism of resource mining and control might better side with teams that split.

The good news is that it seems like there is a core group of functionality that remains the same from year to year, so code that is written for a previous BattleCode tournament is still usable the next year. I just found out about it, so I’m in the process of developing a player from scratch.

Great Design Decisions

A great BattleCode design decision involves the maximum number of units: it’s self-regulating. There is no ‘official’ limit to the number of units that can run at any one time, but all units must return to one of 6 central energon-producing units (Archons) from time-to-time or they die. The Archons produce Energon at a constant rate, preventing too many units from existing at once.

The genius of the design of the soft-cap is that it allows two extra strategies into the game: first, you can send some of your units on a suicide run guilt-free! If a unit decides that it’s FUBAR, it can send a message home saying “Just trying to kill some bugs, sir!” and its parent can feel free to spawn a new unit at its earliest convenience.

As a result, the Archons are as valuable as a King in chess: it’s not very powerful, but you can’t afford to lose it.

Java-Based Programming

The competition software uses the Java VM to control the matches. The user overrides a Runnable class named RobotPlayer. The run() function lets the user define the behavior of the AI agent. They provide a default function that solely moves the AI, but it probably won’t get you very far in the main competition.

The run() function is written as a loop, and robots don’t have a clear idea of a separation of turns: they are limited to 3000 Java bytecodes per “turn,” and the controller switches to the next robot automatically. A “yield()” function is provided that artificially ends a turn.

Very Algorithm-Friendly

There is no centralized user-written player to pass strategy and tactics to individual units: they have to decide that amongst themselves. You need to tell them how to move, how to fight, and they need to be able to manage their own health.

The two major categories of algorithms that need to be developed are dynamic path-finding algorithms (because a robot needs to know where it’s going!) and distributed organization. You need to be able to use your units to protect your Archons, they need to be effective fighters, and it would be a poor unit that gets stuck on the first non-passable square it encounters.

I have no doubt that some teams are eventually able to publish papers on some of the organization techniques that they use: quite a bit of work can be accomplished in 3000 bytecodes, and I bet that some of the organization and attack techniques are more sophisticated than many papers that have been actually published.

My Plans

My goal is to have a force that can fully spawn, move, and fight by the end of January, and my final strategy and tactics to be decided by the end of February. I will most likely merge with a team that has a few of my friends by the end of the month, but you’ll be able to find me by my username (jakevoytko).

Quick-Links for the TL;DR Crowd

About Lisp: Describes Lisp as an overview.
High Level Overview: Describes in general what is needed to program with Lisp.
All you need

• SBCL
• Emacs
• SLIME
• asdf-install

Code examples: A few rudimentary Lisp snippets.
Other Lisp Features: Features that I did not feel like explaining
Resources

About Lisp

Note: For a complete introduction, I recommend this e-book. If you only want to learn enough to get started, read on!

Interactive

Most languages take an input file and produce an executable or run a script. Lisp is interactive: all input is entered into the REPL, and the computation’s result is printed to the screen.

REPL: Read Evaluate Print Loop. A C++ coder can think of it like this:

void lisp()
{
    while(true){
        cout<<">"; // prompt
        cout<<evaluate(read())<<endl;
    }
}

If you’ve ever used Python’s command line prompt, you’ve used a REPL. A bash prompt is a REPL. So is Lisp.

Prefix

Most expressions in C-style languages are prefix, but exceptions are made for arithmetic, such as 1/(x+3). All expressions in Lisp are prefix expressions: the first argument of an S-expression (Sexp) is a function name, and the rest of the atoms of the S-expression are the arguments for the function. The following are examples of Lisp function calls:

(/ 1 (+ x 3))         ; 1/(x+3)
(format t "~A~%" x)    ; prints the value of 'x' and a newline.

All Expressions Return Values

All expressions in Lisp return values. For instance, the ‘if‘ statement in Lisp not only chooses which execution branch to follow, but returns the result of the executed expression. ‘nil‘ is the default return value of an expression, which also evaluates to false (‘t‘ is true).

In C-style languages, expressions don’t necessarily return values. The expression ‘1+3‘ will return ‘4‘, but ‘cout<<1+3‘ doesn’t return anything (They return a reference to an ostream, so this is a mistake) void functions don’t return anything; they can’t be used as a sub-expression for a larger statement of code.

Functions in Common Lisp can return multiple values. The ‘multiple-value-bind‘ macro is used to bind multiple return values to multiple variables.

High Level Overview: What You Need To Get Started

1) An implementation. All you need is an implementation that conforms to the standards. All of them have an edge case or two, but odds are it won’t affect you. If you absolutely need to switch later (for CLisp’s fast arbitrary precision arithmetic, for instance), nothing is stopping you; most libraries support all popular implementations.

2) An editor. I personally recommend Emacs with the SLIME extension. You can also use Vim and Limp if you either prefer Vim or you can’t stand Emacs. I left Vim before I started using Emacs, so I’ve never used Limp. Caveat emptor!

You don’t absolutely need to use an editor that interacts with Lisp. It’s POSSIBLE to run a Lisp program like a script, but it usually makes as much sense as opening a bag of chips with a hammer. You will want to use the REPL for most of your Lisp tasks, and this will only be bearable from within a good editor.

3) A Lisp tutorial or reference. I highly recommend “ANSI Common Lisp” by Paul Graham [see my review].

4) An installation system, like asdf-install. Installation systems automatically load Lisp projects into your current image, and are essential for working with a large project. asdf-install allows you to download libraries from cliki.net and load them into your Lisp image from within the interactive Lisp command prompt!

Details: What You Need To Get Started

SBCL

About

Steel Bank Common Lisp (SBCL) was forked from Carnagie-Mellon University Common Lisp (CMUCL) in 1999. It is named after Carnagie and Mellon’s respective industries, steel and banking.

Most Lisps can be interpreted or compiled, depending on the context. However, SBCL compiles all Lisp code it receives. Having used SBCL for 6 months or so, there’s not much of a pause even for large Lisp files. On the other hand, the compiler is very verbose, but this can be tweaked.

Why SBCL: Popular

Naturally, I can’t find good numbers on the popularity of SBCL. However, it is noteworthy that asdf-install and LIMP were initially designed to work with SBCL. This seems to be par for the course for all of the libraries I’ve used.

Why SBCL: Active Development

Development of SBCL is still roaring along, with 1,228 commits to their Sourceforge page at the time of writing, including several in the past few weeks. New releases come at the beginning of every month, and this month was no exception: version 1.0.23 was produced December 1, 2008.

Installing SBCL

Installing SBCL: Linux

SBCL can be installed like any other package:

sudo aptitude install sbcl
# Or your local variant.

Once that’s finished, it’s easy to test the installation:

jake@justalaptop:~/code/genesis$ sbcl
(+ 2 2)
*
4

Installing SBCL: Windows

There is an “experimental” Windows installation binary: [link].

I’ve ran it for brief periods and it seems to work fine, so I’m not sure what’s so experimental about it. Again, Caveat emptor!

After installation, run SBCL. You will get a command-line prompt, and type the following:

> (+ 2 2)
*
4

It works!

Emacs

You can use whatever editor that you want to write Lisp. Emacs has good advantages:

ELisp: Emacs is scripted in its own subset of Common Lisp. Called ‘elisp’, anybody who has been working with Lisp for a few months and can type “ELisp manual” into Google can customize Emacs to their taste.

Irony alert: I typed “elisp manula” into Google on my first try.

Automatic indentation: Yeah, I know, every editor in the history of the world does automatic indentation. However, Emacs’ authors primarily write Lisp code, so they paid close attention to how Emacs interacts with Lisp. I rarely see Emacs do the Wrong Thing. When it does, I’m usually writing awful code.

SLIME: The Superior Lisp Interaction Mode for Emacs. In order to effectively code in Lisp, you need an editor mode that will interact with a Lisp implementation. Typing the code directly into a command line is nice for small experiments, but is painful for anything more than 30 lines of code (for me at least.. your pain tolerance may vary).

SLIME

SLIME combines the editing power of Emacs with the interactive nature of Lisp.

If you’ve ever used Python’s command-line prompt, you’re aware that it’s painful to use for any amount of time. The editing capability is limited to that of your console, and when you exit Python, all of your code disappears!

SBCL (and all other Lisp implementations) is no different. If this were the only option for working with Lisp, nobody would. You must work with the REPL from within a real editor. You COULD write code in an external file and load it into SBCL from the command line, but that removes the interactive nature of Lisp.

Enter SLIME (Superir Lisp Interactive Mode for Emacs). SLIME spawns a new Lisp process and acts as the liaison between you and the process.

To start the whole process, just type “M-x slime“, and a new REPL buffer opens. Any Lisp command can be entered into this buffer, and you get all of the editing commands from Emacs. slime-mode begins in all of your open Lisp buffers, which gives you code completion and function argument hints for all compiled functions.

If slime-mode is enabled in a Lisp buffer, you can compile the file with “C-c C-k“, or compile a single function with “C-c C-c” (it looks to me like dependencies are propagated correctly). SLIME knows what capability your Lisp has, so it can take advantage of implementation-specific capabilities like debugging.

Installing SLIME

SLIME is installed the same way as any other Emacs plugin:

• Download and extract.
• Add the load path to your .emacs file
• Add any other code snippets required.

In this case, you’re going to be adding the following code to your .emacs file (open using “M-x M-f ~/.emacs“):

(add-to-list 'load-path "<em>/the/path/to/slime</em>")
(require 'slime)
(add-hook 'lisp-mode-hook (lambda () (slime-mode t)))
(add-hook 'inferior-lisp-mode-hook (lambda () (inferior-slime-mode t)))
(setq inferior-lisp-program "sbcl")

I haven’t tested it on Windows, so I’m not sure if there are any extra gotchas. If you find that you’re running into undefined function errors, one possible workaround is to install Cygwin and add C:\cygwin\bin and c:\cygwin\usr\bin to your %PATH% environment variable. Some people run away in horror at the idea, so if you don’t like it, you’re on your own.

asdf-install

One thing that you should research is asdf-install, a package manager for Lisp. If you find that you need a package that is found on cliki.net, you can download it with one step:

> (require 'asdf-install) ; Not the one step.
("ASDF-INSTALL")
> (asdf-install:install 'postmodern) ; The one step.
NIL

The above installs Postmodern, a simple Postgres interface for Common Lisp. Once you’ve done this, you only need to load the package into your Lisp image. This is also accomplished using asdf-install:

> (asdf:oos 'asdf:load-op :postmodern)
NIL

Using those 3 lines, you can now start to define database connections and operations on the database, without ever leaving Emacs!

Starting to Code

I highly recommend that you find a real reference or a real tutorial and start using that to write code. However, if you’re looking for things to enter into the REPL, I’ll give you some overviews of the basics of Lisp.

Arithmetic

> (+ 2 2)
4

This adds 2 and 2. Notice that Lisp arithmetic (and all Lisp functions) have prefix notation: the function name always comes first. You can nest values like the following:

> (+ (+ 1 1) 1)
3

Printing

> (format t "Hello, ~A~%" "Jacob")
"Hello, Jacob"
nil

‘format‘ is the function name. This is the Lisp-version of ‘printf‘,

The second argument is the destination stream. You can give this ‘t‘ or ‘nil‘ for true or false, or the name of a stream.

When ‘t‘ is given, the value is printed to Standard Output. When ‘nil‘ is given, the value is returned as a string.

The third argument is the formatting string. “~” is the escape character. “~A” means that it takes an argument that follows (like printf), and ~% is a newline.

There are other functions that you can use for printing, like ‘princ‘. I usually stick with ‘format‘, but there are different print semantics for different functions.

Defining functions

> (defun adding-function (arg1 arg2)
    "Adds arg1 and arg2."
    (+ arg1 arg2))
ADDING-FUNCTION
 
> (adding-function 1 2)
3
 
> (+ (adding-function 1 2) 1)
4

Here we see a function definition and two function calls. Let’s examine the function definition:

> (defun adding-function (arg1 arg2)

This can be considered the prototype line. ‘adding-function’ is the name of the function. It takes two mandatory parameters, ‘arg1′ and ‘arg2′, both with no default values.

"Adds arg1 and arg2."

This is the “documentation string.” A string may optionally be the first atom of a function, and it can be accessed from within Lisp using Lisp’s documentation reading abilities. Note that unlike wimpy strings in most languages, Lisp strings can contain newlines with no extra syntactical work.

(+ arg1 arg2))

This is the executed statement within the function body. A Lisp function returns the return value of the last statement. Since this is the only statement, it returns the addition of arg1 and arg2.

Defining variables

Defining a variable for the first time in SBCL:

(defvar my-var) ; my-var is 'nil'.
(defvar my-var 'a-value) ; my-var is 'a-value'.

Notice the quote in 'a-value. The single quotation mark indicates that the following expression is data, not code. Otherwise, it would try to look up the value of the variable a-value, which we may or may not have defined. Either way, it’s not what we want.

If a variable is already defined, you can use ‘setf‘:

> (setf my-var 3)
3

Conditionals

For statements that only execute when something is true, use ‘when‘. For statements that only execute when something is false, use ‘unless‘.

> (when (= 1 1)
    t)
t
> (unless (= 1 1)
    t)
nil

For statements of the form “if (true), do (x), otherwise do (y)“, use ‘if‘.

> (if expression
    x
    y)

If ‘expression‘ doesn’t evaluate to ‘nil‘, then ‘x‘ is executed and its return value is returned. Otherwise, ‘y‘ is executed and its return value is returned.

For statements involving a lot of if, else-if clauses in other languages, use cond:

(cond ((test1) (expression1))
      ((test2) (expression2))
      ; ...
      (t (default-expression)))

Iteration

The two easiest ways to iterate are the macros ‘dotimes‘ and ‘dolist‘.

> (dotimes (i 3)
    (format t "~A" i)
012
nil

‘dotimes‘ is exactly as it sounds: it executes a statement a number of times. You assign a variable (in our case, ‘i‘) and tell it the number of times it shall execute, and the variable takes on the values of all of the integers in the range [0, n).

> (dolist (i '(1 2 3))
    (format t "~A" i))
123
nil

‘dolist‘ iterates through a list much in the same way ‘dotimes‘ does.

Reduce

The idea of ‘reduce‘ is to iterate through a set, applying a function to each value that acts as an accumulator. For instance, we can do this with addition:

> (reduce #'+ '(1 2 3))
6

Lambdas

If you want to define your own functions for ‘reduce‘ without writing formal functions, you can do them as a ‘lambda‘. That’s just a fancy word for ‘function with no name’. It consists of the name, the argument list, and a function body to execute:

> (lambda (x) (+ x 2))

This defines an anonymous function that adds 2 to the input given. The only difference from ‘reduce‘ is that we need to provide two parameters: the accumulated value and the next value in the list.

You’re going to use this. A lot. Formally defining every function is not worth it. A lot of Lisp code ends up much cleaner with functional approaches.

To define our own addition function for ‘reduce‘:

> (reduce (lambda (x y) (+ x y)) '(1 2 3))
6

Other features of Common Lisp

• Macros: Creating code on-the-fly
• CLOS: Common Lisp Object System. Lisp’s answer to Object Oriented programming.
• Packages: This is how Lisp does namespacing.
• More iteration constructs than you know what to do with.
• ‘mapcar‘: Apply a function to each element of a list
• Hash tables
• Arbitrary precision integers and numbers
• cons: “Construct” function. This is the function that builds lists.car: Returns the first element of a list.
• cdr: Returns the tail of a list (everything but the car).
• Arbitrary binary manipulation
• ‘eval‘: Generate code from data at runtime.
• File streams, input streams, output streams, stream redirections, oh my!
• FFI: Foreign function interface. This lets you interface Lisp with C. You’ll probably need to do this if you want to add functionality to Common Lisp. It’s actually surprisingly easy to use.

Resources

Books

ANSI Common Lisp by Paul Graham. A great first Lisp book. It can be used as a textbook.

Paradigms of Artificial Intelligence Programming by Peter Norvig. The best programming book I have ever read. It does have quite a bit on AI, but it also has extensive sections on advanced Lisp programming.

Websites

SBCL: The Lisp implementation I currently use.

SLIME: A Lisp interaction plugin for Emacs.

Limp: A SBCL plugin for Vim.

cliki.net: A general Lisp resource. The home of all projects remotely installable using asdf-install.

Common Lisp Cookbook: An incomplete, yet still helpful, recipe book for Lisp code.

I’ve been programming a lot recently (instead of blogging about programming!). I caught the genetic algorithm bug along with the rest of the internet, so I’m in the middle of writing a Lisp library to make it easier to develop genetic algorithms: Genesis [Github link].

I’ve made some progress in the past few weeks!

The first version of my program was only a genetic programming library, and barely. It produced a rule set that could be evaluated as a function, but it wasn’t anything different than hill-climbing (a phrase I learned from snobby Reddit commenters). It was inefficient, it was unwieldy, but it produced great results when it ran long enough.

I’ve added a population! An arbitrary number of critters evolve in parallel at the moment. Previously, the individual would just mutate. I also added a basic gene-sharing algorithm: random merging.

It’s faster! In my first version, I used two ‘eval‘s per rule per round. Now I use 1, and I will cut it down to 2 per rule evaluation (instead of per round) soon.

It’s in a package! I finally decided to follow my own advice and namespace everything properly. The package layout still needs work, as I’ve been too busy coding to figure out the ASDF system for installing packages.

The package system in Lisp is a lot more flexible than most module systems I’ve ever used, so I’ve been reading a few other popular Lisp packages to see how they are organized before I jump off the deep end without my swimmies.

I started to add tests! Some of the functions are very fundamental in nature, so I’ve started to add solid tests for them.

I segregated the example! There are currently 2 different files: genesis.lisp and square-root-sample.lisp. square-root-sample shows the code necessary to produce a simple/stupid example: finding an algorithm to calculate square roots. As I’ve mentioned before, the results are very good:

All you need to do in order to run this example is load genesis.lisp and square-root-sample.lisp and call the following:

(square-root-sample <em>generations population-size</em>)

After it has finished (run a small number of generations to get a sense for the time it takes), you can call the following to get the best answer so far:

(funcall-best *CURRENT-POPULATION* #'sample-fitness-function 16)

Work Needed

A recent popular example of genetic programming is the production of the Mona Lisa using random polygons. My program should almost be able to handle this. I don’t allow for a “modify rule” function at the moment, only a “new rule” function. It would probably work, but convergence would be even slower than the example given.

I’m also not allowing for one of the most powerful methods of genetic algorithm development: binary serialization. Imagine: if you can represent your whole algorithm as a binary string, then crossover, reproduction, and mutation are all made trivial. I am instead using this idea for lists, which is convenient in Lisp, but I feel it lacks some of the punch.

On The Horizon

Of course, this is still in the toy phase. I still have a pretty substantial to-do list, but here are some of the high-notes:

• Finish writing tests for core functions.
• Multiple populations, including cross-breeding between populations.
• Add multithreading.
• Give the user much greater control over (genetic-algorithm) by allowing extra key arguments.
• Self-awareness: Gather statistics on which alterations produce better improvements, and make beneficial changes more likely to occur.

What are Genetic Algorithms?

Genetic algorithms are one of the best things that computer science has produced. Rather than figuring out a good algorithm yourself, you let your random number generator find one while you sleep! Well, after an up-front design cost to generate proper random functions, and evaluating them, and determining how you’ll evolve them, and how long they should run…

I’ve long been interested in genetic algorithms, and now that I’ve learned my first language with an eval — Lisp — I’m starting to make a library. I spent the past few days prototyping and testing my code,
and I got some encouraging initial results: I got a great almost-linear approximation to my simple test problem (finding square roots) using only addition, subtraction, multiplication, and division (see below).

I know, square roots. What a scorcher. But real people are making real solutions to real problems using genetic algorithms! NASA engineers produced an antenna. They determined acceptable design parameters for the antenna, figured out how to evaluate new designs, and let randomness determine the rest.

The result is described here.

My Genetic Algorithm Plans

I want to produce a Lisp package over the next few weeks (since I should have a boatload of freetime around Christmas) that makes it easy to define new genetic algorithms. I’m going to do it in a few stages, and blog about intermediate results if and when they are interesting.

This will also give internet denizens plenty of opportunity to criticize my awful Lisp code! I am a nublet when it comes to Lisp, and I have no illusions about the quality of the code I’m writing.

This week: prototyping. I picked an easy problem to work out the basics of genetic algorithms. My problem? Finding square roots between 0 and 99, inclusive, with just one rule set.

Disclaimer: This is potentially the stupidest use of genetic programming known to man. Attempting to regularly solve numeric problems with genetic algorithms can lead to blindness, irrational exuberance towards Republican politics, and death.

To give you a sense of the data it ingests, I present the main interface:

(defun genetic-algorithm (generations starting-rules rule-fun
                          fitness-fun)
  "Runs 'generations' number of generations. The rules are initially
  set to to 'starting-rules', the rule generating function is
  'rule-fun', and the evaluation function is 'fitness-fun'"
  (prep-rules starting-rules)
  (dotimes (gen-num generations)
    (run-generation rule-fun fitness-fun)))

Here is the invocation I used for my square root problem:

(genetic-algorithm 32000 '((/ num 2) (+ num .2))
                   #'sample-rule-fun #'sample-fitness-fun)

The typical work for defining genetic algorithms is split into a few stages:

1) Problem definition: What do I even want to solve?
2) Evaluation function: How far am I from right?
3) Picking components: How can I build my algorithms randomly?

Problem Definition

Find the square root of an integer, n, in the range [0, 99].

Evaluation Function

Sum of the squared error of all of the integers in the problem statement.

Interestingly, we can predict the quality and form of the results: since we are minimizing the sum of the squared errors, the result should mimic a least-squared approach. Since I’m only using basic arithmetic, it probably won’t be a very good one (linear at best..).

Prototype Implementation

Warning: There be dragons in the parens ahead.

The rules are stored as a list of lists:

(defvar *RULES* nil)

We also store the possible operators:

(defvar *sample-nodes* #(leaf + leaf - leaf / leaf * leaf))

Woah, what’s with all of the “leaf”s? Well, a balanced binary tree has about as many nodes as leafs (I’m generalizing…), so I started at leaf/operator parity and moved up until I stopped getting stack overflows when generating new functions, which happened at one .

How do the leafs get translated into values? First, they are picked from an array of random values:

(defvar *sample-values*
    #(-1 1 num num randomnum randomfrac))
 
(defun sample-generate-leaf (list)
  "Picks a random element from 'list' and translates it into the proper atom
  or list."
  (let ((elt (random-array-element list)))
    (case elt
         (randomnum (random 1024))
         (randomfrac
          (let ((val (ignore-errors (/ (random 1024) (random 1024)))))
            (when (null val)
              (setf val (/ 1 2)))
            val))
         (t elt))))

These are pretty self descriptive (num is the input number, and it is included twice because I got better results ). ‘randomnum‘ is a random integer in [0, 1023], and ‘randomfrac‘ is a fraction whose
coefficients are in [0, 1023].

So how is all of this put together?

(defun sample-rule-fun ()
  "Example random rule generator. Generates a random arithmetical
  expression."
  (let ((elt (random-array-element *sample-nodes*)))
    (case elt
      (leaf (sample-generate-leaf *sample-values*))
      (t
       (append (list elt) (at-least-once #'sample-rule-fun))))))
 
(defun sample-generate-leaf (arr)
  "Picks a random element from 'arr' and translates it into the proper atom
  or list."
  (let ((elt (random-array-element arr)))
    (case elt
    (randomnum (random 1024))
    (randomfrac
     (let ((val (ignore-errors (/ (random 1024) (random 1024)))))
       (when (null val)
         (setf val (/ 1 2))) ;; avoid divide by zero
       val))
    (t elt))))

This is also fairly straightforward: if you pick an operator, generate a new list, if you pick a leaf, generate a new value.

Results

Square Root Function Test

I ran it for about 18 hours (overnight and after work), and when I came back, the results were nothing short of amazing:

I included a range of values outside of the initial zone because the results are better than I could have hoped for.

I saved the data, and then was trying to save the rules (there were 18 in all), and then I lost them from my first Emacs freeze. C’est la vie. However, my initial guess that the data would fit as a least-squares appears to be a better guess than I ever could have hoped. I want to add arbitrary polynomials, and I hope to see a higher-order best-squares fit of the data plot, with much less error.

Implementation Time

The first implementation was done in a 4-hour chunk of one evening (during which I played X-Moto for at least an hour…), which I find impressive. I probably would have spent at least 3 times that on the prototype in C++. This problem is made extremely because of several of Lisp’s features:

eval: My method is based on runtime list generation, and eval-ing it as a lambda.

Insanely simple Lisp syntax: The same syntax that defines lists also defines their programs? This reduces random function generation to a random list generation problem, which is almost trivial.

REPL: Lisp is a REPL (read-eval-print loop) language, meaning that it has an interactive prompt. Combining this with Emacs’ multiple buffer capabilities, and it’s super easy to make changes and reload.

TODO

It’s never finished!

Rules

Allow more than one rule set, define the maximum number of rules per set, define rules as arrays for better lookup, etc.

Better use of eval

I eval the rules every time I need them, as opposed to the once I should generate them. I will probably see a significant speedup when I do this.

Define a package

Namespacing everything has obvious benefits.

Cross-breed rules

Allow true genetic properties by copying subsets of rules into other rules.

Better rule trimming

My current program produces some stupid rules: such as f(x)=x. In fact, my “amazing results” had 4 rules that evaluated a number to itself out of a total if 18 rules.

Allowing the user to define rule inputs.

Putting the package in an easy-to-use location

Stay tuned for more!

What is “Code Inertia”?

I’ve been interested in “code inertia” for a few years now. It should sound familiar to anyone with a basic understanding of physics: code with a lot of inertia is very resistant to change. You’d either have to be crazy, uninformed, or very well paid to try to shake the way the system is organized.

I met my first gigantic and arcane system at my college internship. It was a Java-based web platform with 8 years of cruft built on top. There was no documentation for even creating a development environment, and it had a complex and lengthy build system. It was a mountain of pain ready to strike. There was no way in hell that the system could be considered “beginner friendly,” and there was no clear path for gaining knowledge of the system.

It was so complex that only a handful of individuals had the Keys to the Kingdom. Everybody else used the tools produced by the Chosen Ones and went on their merry way. I tried to become a Holder of Knowledge, but the system proved too great for me, the college intern.

Why do I mention it? Because the whole system had a gigantic amount of inertia! There was so much code built on top of the mountain, that nothing could be changed. You couldn’t change the build system, because then you’d have to alter a mountain of configurations for hundreds of sites. Despite the amount of code, very little of it was actually reusable, and there was no central repository of knowledge for code reuse.

The most helpful changes became the ones that were the most impossible to make. It was a big, suffocating force, and the company eventually abandoned it for a shiny new toy called Sharepoint. The old way of doing things simply wasn’t sustainable. Whether they learned their lesson or not I’ll never know, because I have since left the company and landed a real programming job.

Because of the monstrosity I mentioned, I appreciate my current job’s gigantic, fairly documented, and accessible code base quite a bit. It’s not close to perfect, but it does a lot of real world tasks very well, and beginners can use a good chunk of it pretty quickly.

Windows.h

If you want THE BEST example of inertia, just look at “windows.h“. It might have the greatest code inertia of any file not defined in a language standard. It is the basis for every Win32 application. Could it be modified? Potentially, but only by appending to the standard. The file itself could never be modified without wasting an impressive number of programmer hours.

This is unfortunate, because it has classic design errors. Windows programming still isn’t very “easy” with it, and you have to watch #include order and know what macros it defines. Yuck. However, none of these things can really be fixed, because even innocent-seeming changes would probably break builds for arcane reasons. Obviously, it should be replaced.

The .NET framework is the first Microsoft offering that has the potential to deliver a knockout blow to the old way of Win32 programming, since MFC wasn’t up to the task. Some people reject .NET for the runtime size, and others reject it because it’s Microsoft-specific, but it’s a lot easier to work with than Win32 for your run-of-the-mill windowed application.

Reasons for Inertia

Code can pick up inertia for different reasons:

So many files depend on this code that changes would be prohibitively expensive.

This is (in my personal experience) the most common kind of code inertia, as it’s hard to avoid even for well-designed systems. You design a few nice classes, build useful utilities around them, and use the library in 162 programs.

You could easily change the low-level code underneath the utility abstractions. Unfortunately, the top-level interface has picked up quite a bit of inertia. If you want your whole system to be consistent, you’ll need to make quite a few changes. Otherwise, you can do sneaky things that are good enough for most situations, like deprecate the interface while maintaining a backwards-compatibility layer.

The code itself is so large that changes are prohibitively expensive.

This is related to the previous point, but sans dependences. That finance application you were writing for that coal business? Well, they suddenly decided that they wanted to enter the asphalt business, and want the application to reflect that.

The code is so complex and ill-written that changing it is likely to break everything.

This is likely to have been written “by somebody else.” Everybody tacks their changes on to the system instead of modifying how it works, and hope it falls apart when the next guy uses it.

That guy who knew how everything works quit.

“Fricksticks, Frank’s gone. Now nobody knows how the the whole Foo suite works. Frick frick frick.”

Predictive Power of Inertia

Let’s say that our company has two million lines of code that works wonders with widgets, and our CEO suddenly sees a golden opportunity in the gadget market.

We have a big problem: our code wasn’t designed with gadgets in mind! You might find some core libraries are abstracted enough to work with gadgets. The rest of it explicitly assumes widgets.

What’s going to happen? As nice as refactoring the entire code base would be– and I think I threw up a little inside thinking about it– it’s obvious that gadget code is going to be added on top of the widget code base, instead of working with it. That’s a prediction that anybody can make.

Adding the basic gadget functionality you need NOW will be much less effort than changing your code to handle widgets, gadgets, gizmos, and pandas. It’s the right way to go from a business perspective (less time, less expense, same results), but it’s unsatisfying from a technical perspective, and probably misses some nice opportunities for generalization.

Some people will point out that there are lost opportunities for the future. You may want to also be able to work with thingamajigs and whatchamacallits in the future. If you tack on gadget code, you’re going to be tacking on code for everything else.

It’s important to realize that there’s never a free lunch in any of this regard. Even if all of your library code has perfect abstractions, you’ll still need to build up some extra gadget-specific code that reads and writes them, interfaces with your existing utilities, etc. It’s all just a matter of ease of modification.

Law of Large Code Bases

All code bases cross a size threshold where addition is less costly than modification.

When you need new functionality in your code, you have two options:

1. Modify the existing code to solve your problem.
2. Creating new code to solve your problem.

Sometimes, Option 1 is unacceptable, and it’s unfortunate. In a perfect world, we would write code once, and it would work for all of our situations. In a less perfect world, we should always be able to modify our existing code to best be generalized. Unfortunately, software is a business, and rewriting/heavy modification isn’t always in the cards.

Option two is a short term solution, but an interesting corollary becomes immediately obvious:

When a code base has crossed the threshold mentioned in the Law of Large Code Bases, adding additional adds additional inertia to the code.

When code crosses this boundary, you can consider it a marked man until it is changed for the general case.

Can We Plan Around Code Inertia?

The planning phase of the project is obviously the best chance to take code inertia into account. Unfortunately, there’s not usually a whole lot that you can do. You’re not always under control of the scope of your project, and you don’t know what problems you’re going to need to solve in 2 years. That’s fine.

Trying to find flexible solutions is a great way to start. Just take a look at Steve Yegge’s post on Property Lists as a programming paradigm. You can express a tremendous number of solutions in terms of
property lists, and they’re very flexible if you use them right.

Another successful strategy is the One Thing Well philosophy. Unix systems come with a broad variety of tools that “do one thing, but do it well.” Chaining the tools together is easy, and any system programming problem can easily be solved in terms of these tools.

In writing this post, I noticed that deprecation/change seems to be the most successful way of supplanting old methodologies. Python 3000 is a great example of this. So is the module system of the Linux 2.6 kernel. It turns out that it’s really easy to change code that’s underneath a layer of abstraction, but you can’t change code that relies on the abstraction.

“Programs must be written for people to read, and only incidentally for machines to execute.”

~Abelson and Sussman, SICP

If 8 years of programming has taught me anything, it’s that programmers need to give themselves every advantage imaginable. I’m not disparaging other programmers.  I say this as someone who has been humbled on the altar of bugs many times.

The fact that my first language (C++) is one of the most complex languages probably didn’t help. C++ isn’t my favorite language, but I always find myself coming home to it, whether for school assignments, or more recently for money. Plus, it ranks higher than Java in my book. Yuck!

Most programming languages have fundamental shortcomings that prevent them from being very human-friendly. Like what? Like nested parentheses and braces and brackets. Fortunately, along with programming languages evolved code editors that do great things with indentation, coloring, and matching brace highlighting. The actual problem isn’t fixed (a colored, indented C++ program is still the exact same thing as a non-colored, non-indented C++ program), but the tools exist to make it a non-issue.

There is a wrong way to do it, and a right way to do it, but none of it is enforced by the compiler.

Assignment Within if(){

When I was 14 and 15 and hadn’t been using the language for very long, I made a whole litany of embarrassing mistakes. For instance, I would spend an hour trying to figure out why the following wasn’t doing what I expected:

 for(int i=0; i<10; ++i);{
     do_something(i);
 }

Fortunately, I stopped falling for it after a few times. However, I couldn’t seem to shake the following error:

 if(my_ptr = NULL){ // Wrong, but no compile error!
     do_something();
 }

To the rookie, this looks correct at a quick glance. Even worse, it compiles without warning. In fact, this was one of the evil no-nos that Java outlawed altogether, even though it has legitimate uses.

No matter how many times I swore I’d look closer, 2 weeks later I’d find this same bug after 30 minutes of frustrated debugging.

Eventually, I eliminated a large subset of this error by writing the equality test differently:

 if(NULL = my_ptr){ // compile error!
     do_something();
 }

This caused a comparatively small boost in my productivity. However, this helped, and it required no effort. Every little bit counts. Interestingly, this kicked me out of my groove: I started making this error a whole lot less frequently because I was thinking more about my equalities. Unless projects have strict style guides, I still code it this way.

For being a simple style change, it comes with a few benefits:

1. Anybody can understand this code.
2. When you screw up, it doesn’t compile.

//TODO: Lists

Some code generators, like Visual Studio 2003, fill generated functions with comments:

 void importantFoo()
 {
    // TODO: Implement this!
 }

For the authors of the code generator, this suits its purpose: informing the programmer that they have some work to do while generating compilable code.

Believe it or not, some code generators improve upon this. For instance, I’ve seen some code generators taking an exceptional approach:

 void importantFoo() {
     throw std::runtime_error("TODO: Implement function importantFoo()!");
 }

This comes with benefits: unless the programmer is a complete ninny, this code will be executed, and the nascent program will be prematurely snuffed from this Earth.

If this doesn’t cause the programmer to get off his lazy rear end and code, then what will?

The // TODO! statement is certainly not unique to code generators: it seems to be pretty popular amongst programmers, too. I am no exception them, too.

For a while, I took the comment approach to the TODO:

 void my_foo() {
     // TODO: Implement me! I am a yummy pizza!
 }

I’ve always felt a little dirty writing these. Realistically, your brain’s stack can only hold 5-9 items, so I can see an argument for temporarily using them. However, all you’re doing is passing the buck to an older, wiser future version of yourself in a brave new world with flying cars and butler monkeys.

It’s obvious that if you had flying cars and butler monkeys, the LAST thing that you would want to do would be to implement a // TODO statement left by a barbaric past self.

It’s my opinion that if you HAVE to write one, you should resolve the issue ASAP. For instance, before you  compile or interpret or “(read)” or whatever your language uses.

But recently, I found an even better TODO method for compiled languages:

 void my_foo() {
     TODO: Implement me! I am a yummy pizza!
 }

In C++, this generates about a trillion compiler errors (more or less). Compiler errors, for whatever reason, put me on a mission to fix a problem, so this is is an easy and helpful motivator.

I call these “Quit your bitchin’” TODO lists: Quit your bitchin’ and implement me! You have to do it sooner or later!

It is important to note that you should NEVER do this in Python. Python is expressive enough that this syntax will likely cause the International Space Station to fall to earth while buttering bread on the wrong side. </bad_joke>

Conclusion

Modern programming languages are designed  with human readability in mind, but there isn’t a “perfect language” in this regard. Best practices develop around programming languages, and is imperative that newbies find out what these are ASAP, because they are only hurting themselves if they do things their own way.

In addition to using best practices, sometimes you can make things easier for yourself with little effort.

If you’re like me, you want to know what you can do before you start. For my previous 2 programming jobs, I asked my interviewer what I could do to better prepare myself for my first day. They both laughed and said, “Just show up!” True story.

I ended up working on personal projects instead of preparing for the job. It didn’t really hurt or help in the long run, but I felt that the time could have been better spent.

If you really only have a few weeks before you start your job, don’t keep preparing for your interview. Don’t practice arcane language trivia questions and don’t try to solve programming puzzles (unless you do it as a hobby!). Avoid trivia in this stage. If your interviewer was competent, s/he already thinks that you can do the work. Diving back into Meyers, the Gang of Four, or Knuth might give you a trick or two, but in the long run it’ll all get lost in the noise.

Don’t get me wrong! Studying is never a bad move. I graduated 3 months ago and I still spend a lot of my time studying. It’s not the best short-term solution to getting you started in a programming job.

One of the things you should do (but not the most important) is to read domain and company-specific material. Academic journals, conference proceedings, blogs, company websites, and Econ books all fall into this category.

If you’re still in school, take a Microeconomics course. Trust me, it’s worth it. Look at the press releases on your company’s website. They’ll give you a hint at the focus of your job specifics.

Why should you waste your time with this crap? Odds are that your company has a few goals:

1. To contribute to its particular field of expertise.
2. To do better than its competitors, for some value of better.
3. To turn a profit.

It is certainly possible to make good programming/GUI interface/business decisions without knowing anything about your field, your competitors, or your business.

Is it easier with the insider knowledge? Probably. If you didn’t spend time thinking about your field in college, you should at least flip through an Econ textbook or a few trade journals.

This shouldn’t command the bulk of your time. At the end of the day, you won’t master your new field in the few weeks before your job starts. I don’t mean to undercut the importance of this step! You will learn your business sooner or later, and starting early will give you a big help on your first day.

What will REALLY help you prepare?

Use, read, and modify code written by other programmers. A lot.

Why? You’re going to spend most of your career as a programmer doing two things:

1) Reading and correcting what others have written.
2) Building on top of what others have designed.

Likewise, other people will spend time correcting what you have written, and building on top of your designs.

Unless you have donated a significant amount of time to Open Source projects, s/he has minimal experience dealing with unfamiliar code. Maybe a few libraries here and there, but not much else.

Many of my friends and I were very active programmers in our spare time, but we spent our time designing our own code, not using the code of others. I started seeing the light towards the end of my senior year, but the full implications never struck me until I entered the work world as a professional programmer.

Most of my school assignments were based around creating new code. Sometimes I didn’t even have to design it! Use the professor’s framework, fill in the blanks, get an A, and move on to something more interesting.

I landed a Summer of Code project, but the project was based around creating a helper for some Boost libraries (for being a messy solution to an ill-defined problem under time constraint while working another full-time job, it ended up producing decent results). I learned how to use a few of the useful Boost libraries, learned I never wanted to use others ever again, and got a window into the OSS world. Unfortunately for me, my experience reading and modifying the code of others was minimal.

Some of my harder assignments were still reasonable projects to complete from scratch. I took a cryptography class, and my final project was to implement the quadratic sieve. I used GNU’s MPZ library, and built the linear algebra/number theoretic functions from scratch. I just as easily could have used a library like LAPACK++. In fact, it’s very likely that such a library would have been much faster than my personal implementation, and the project would have been finished sooner.

If it is so important, why weren’t you asked about using the code of others in your interview? It could be any number of reasons, really. Maybe the company’s code base is so massive that EVERYONE needs time to digest the structure. Maybe they haven’t hired a junior developer recently. Maybe the company wants to completely mold the habits of new developers, before they learn naughty practices.

Will you be OK without doing this? Yes. I started my job with only minimal experience with unfamiliar code, and I haven’t starved to death on the streets. Yet. However, I wish I had spent more time figuring out how to use different libraries.

A week or two ago, he threw me a curveball:

“How’s it going? Do you understand it all yet?”

“No. I’m making progress, though. I’m probably not going to have this down for a while. It’s a completely different way of thinking from the programming I normally do.”

“How is it different?”

I froze. He’s “Mr.-Fix-It,” and understands the rock quarry/asphault business inside and out. He’s very analytical, and it’s almost impossible to stump him with my favorite “Devil’s advocate” questions. However, aside from what I’ve told him, he just doesn’t know much about programming.

“Metaprogramming” and “first-class functions” wouldn’t have meant anything. “It lets you write programs faster” is a cop-out; he’s given me detailed explanations of his business. I wanted him to understand.

I was able to stammer out the fact that the language encourages breaking problems into smaller versions of the same problem (recursion), but that other programming languages also supported it.

I eventually told him that I hadn’t even thought about explaining the difference to a non-programmer.

“Well, if you’re ever able to come up with it, let me know.”

In college, I was a Computer Science tutor. I could explain most CS concepts that students saw in early C++ and Java courses to the layman on the street. However, I had nothing for explaining the difference between Lisp and C++ to him, which was a little scary. I spent some time dreaming up examples.

Functions

All models are wrong, but some are useful.

~George Box

Abstract math notions are boring to most people, so I try to relate programming concepts to real-world items.

A fast-food restaurant is a great example of a function: An order and some money is input, and you get cheeseburgers, fries, a soda, and your change. Whether a person likes math or not, they know what a function is: you put something in, you get something out.

The idea is easily extensible to Object Oriented programming: all Burger Kings have orders, and you can place a specific order at an instance of a Burger King. A Burger King “is-a” fast food restaraunt.

I will use this metaphor to explain why Lisp is different from the programming I usually do (C++/Java/C#/Python).

Lambdas

Any restaraunt probably has a few different ways to order food: over the phone, at a drive-through window, at a counter, through a waiter, etc. However, there might be unofficial ways to order food: Bill Gates might write a letter requesting a buffet with a few hundreds slipped in as proof. Someone might tell the owner on his way to work that he’d like him to come back with a pizza.

The owner can easily define new ways to order food through a restaraunt.

Lisp, through “lambdas”, lets a programmer define new functions as easily as a restaraunt owner can devise new ways to order. The inputs and outputs are the same, but making new functions can have very low overhead compared to other programming languages.

Brevity

“Wiz Wit”.

This is my usual order at Pat’s in Philly. With just two words, I get a cheesesteak with Cheez-Wiz and fried onions in less than 5 seconds. The system is designed to maximize the number of cheesesteaks output per hour: orders are short, and you usually get your cheesesteak before you’re done paying [and sometimes, before you're done speaking!]

This lesson can be applied to writing: It’s easier to convey a short thought. Unless you are Charles Dickens or Neal Stephenson, economy of language should be a goal.

Lisp’s syntax is very simple and short, so you can usually convey a thought in Lisp much quicker than you can convey a thought in most other programming languages.

Macros

An order can come in several different forms: it can be written, it can be gestured, it can be be spoken, and it can be implicit.

A lot of programming languages let you generalize the details of what an order can receive. Most orders are the same, so you can handle the contents the same even though the orders are different. In C++, you could even use “templates” to write code that generates code that handles this idea.

Lisp takes this idea all the way to its logical conclusion through “macros.” You can easily write functions that define orders. The difference is that you can easily do this at any scope!

If you notice that all fast food restaraunts are the same except for color scheme, specials, and slogans, it is easy to write functions that define new fast food chains. If you notice that your functions for defining fast food chains are similar to your functions for defining hardware chains, you can write functions that generate these producers.

Lisp lets you easily do this manufacturing on a level that is unimaginable with any other language. It may be possible, but nothing beats Lisp’s ease-of-use.

Self-Modifying Code

The metaphor breaks down here, and I won’t force it. However, the idea is simple when it is explained.

Most programming languages are designed to modify memory. They all store their information in memory, retrieve it, modify it, and write it to memory again.

Some programming languages try to hide the fact that they’re based on a memory-modification model, but they are extremely similar to languages that don’t hide it.

Lisp, however, is written to modify lists. Lisp is also written using lists, so a Lisp program can be programmed to modify itself.

Conclusion

This is just the tip of the iceberg! There are a lot more ideas that lay beneath the surface here, and you can bet that I’ll think about how to explain them to an educated adult. However, this should get someone with passing familiarity to programming to understand what makes Lisp stand out: ease-of-use and brevity.

History of SHA-0

In 1993, SHA-0 descended from heaven. The NSA published SHA-0 in FIPS-180, which contained the algorithm with no explanation of its security. Concerns were thrown aside as the algorithm became widely adopted. The stated collision resistance [the resistance most pertinent to the attack listed below] was the theoretical maximum of 2^80.

In 1995, the NiST released an improvement on the algorithm in FIPS 180-1 . The new algorithm (called SHA-1) had just an extra shift operation in the message expansion. Other than this detail, the new algorithm was identical to SHA-0. According to the NSA, this corrected a weakness in the message expansion.

In 1998, researchers Chabard and Joux released an elementary attack on SHA-0, described below. It exploited a fundamental weakness in the simple round structure of SHA-1, combined with an exceptionally simple message expansion. Their attack employs a search algorithm that requires only examining 2^61 messages instead of 2^80 messages to find a collision.

This was only a theoretical break in SHA-0: the collision resistance was still improbably high (though breakable through a distributed computing project), and the only collisions produced were gibberish.

More than one weakness in SHA-0 was found in the meantime! Biham and Chen discovered another route to finding collisions and near-collisions, and Chinese researchers Yang, Yu, and Yin have effectively pwnt [to use a technical term] SHA-0. In their most recent attack, they found a search space that requires examining only 2^39 different messages. This is possible on a beefy desktop computer. They have only released the attack vector and a high-level overview of how they arrived at the vector. They have not released the details for how they developed their attack in an English language journal.

The Biham and Chen attack is actually cooler than the Chibaud-Joux attack, but much more complicated. If this essay is well-received, I will write a post on that.

Description of SHA-0

SHA-0 is a block hash with an input of 512 bits. This means that any input message is broken into 512-bit blocks. The output from processing the first block is the input for processing the second block, and so on and so forth. The output of the final processing step is your hash value.

Writing the description is far less clear than just reading the pseudocode, so I urge you to read the pseudocode listed on Wikipedia or a reference implementation I wrote in Python [note: Python is not my primary language, and I wrote it so that a Math major with no coding experience could understand it, so I make no guarantees about it.]

The overall structure of SHA is as follows:

1) Message padding: Because SHA-0 operates on blocks, we need to make sure that we can evenly break the message into 512-bit blocks. The length of the message must also be appended on the end of the message. If the message + the length is not 0 (mod 512), then in between the message and the appended length, the string “100…..000″ is added to make the message the right length.

2) Message Expansion: The message is then divided into 512-bit blocks. Each 512-bit block can be broken down into 16 32-bit words. However, just plugging these 16 words into the round functions (described below) would give attackers too much control over the internal state of the round function, the 16 words are expanded into 80 words using the recurrence W[i] = W[i-3] ^ W[i-8] ^ W[i-14] ^ W[i-16] for 17<i<80. Note that the first 16 words of the expansion are the 16 words from the block.

3) 80 rounds : The input of the hash function, in addition to the message, is also a 5 word magic constant. These 5 words are transformed using the results of the message expansion, and then added to the original magic word to produce the hash.

After that, the resulting 5 registers are added with the initial state of the 5 registers. For the first block, this means the result is added to the “magic number”.

Two messages side-by-side

If we take two messages, m1 and m2, and let m1=m2, obviously SHA-0(m1) = SHA-0(m2). Cryptographic hash functions MUST be deterministic by definition.

The ultimate goal is to find two messages that hash to the same value: Hash(some_message) = Hash(other_message). This, by itself, is overwhelming. So Chabaud and Joux decided to cheat: They would find collisions for successively stronger versions of SHA-0 until they found a collision attack against the full SHA-0.

Local Collisions

Chabaud and Joux needed better tools to attack SHA-0, so they decided to find necessary conditions for finding collisions for just a few rounds of SHA-0. They decided to see what happens if they flip a single bit in a round, and ignore the message expansion. So they are allowed to put whatever they want into each round of the hash function by the rules of this simplified attack.

Let’s go back to identical messages m1 and m2. Let’s alter m2 a little: we will flip a single bit. If we make a change to a bit in some round r , the register A will be the only register affected in round r . In round r+1 , the registers A and B will be affected. So on and so forth, so it will take until round r+4 for the bit difference in m2 to propagate all the way to the last register, E .

Chabaud and Joux decided that they wanted to find the minimum amount of rounds necessary to createa  local collision from r . It turns out that if you have full control over the input to the hash function, you can create a local collision at round r+5 : The hash at round r+5 of m1 and m2 is the same even though m1 and m2 are different functions. This is called a 6-round local collision (Rounds: r, r+1, r+2, r+3, r+4, r+5 ).

How?

Let’s look at round r+1 , the round after the first bit-change occurred. The changes that happen to the A register are defined as:

A = (A <<< 5) + f
(B, C, D) + E + round_constant + input[r+1
]

where f is one of the 4 round functions defined in SHA-0. We have full control over the term “input[r+1]“.

and the changes that happen to the B register are:

B = A

We can’t correct the difference in B that propagates from A (because we have no additive term in this equation), but we CAN correct the difference that occurs in A. After all, we know what bit we changed, so getting A to match up in m1 and m2 is easy: we just have to do the math.

Chabaud and Joux define specific masks for r+1, r+2, r+3, r+4, and r+5 , and voila! A local collision.

Why does this work?

This works because of a special fact about the A register: it is the only register that is directly modified by the message, AND it is the only register whose value is produced by the values of other registers. Since the other registers aren’t sharing bit differences, we only have to fight the changes in the A register in order to succeed.

Problems!

If it were really this easy to find a local collision, SHA-0 would have been completely unusable. Fortunately for its strength, there are a few complicating factors:

Message expansion : The message expansion produces an additional constraint: this produces the need to chain together a bunch of local collisions inside of the 80 rounds in order to come up with one full collision. The constraints of the message expansion need to be satisfied by your chosen starting points for all of the local collisions (as your bit differences that you introduce will be scattered by the message expansion in the exact same way).

Round-specific functions : To make matters worse, there are round-specific functions that change every 20 rounds. Each of the functions are functions of the registers B , C , and D , and can therefore be viewed as functions of past values of A . Two of the functions are non-linear : they can’t be modeled by an XOR, and they have their own specific constraints for when a local collision can be produced. The round functions prevent the local collision from being automatic: depending on the round we start the collision, we need to search for which bits we need to flip. The Chabaud-Joux attack on SHA-0 has a search space of 2^61 for bits that need to be flipped in order to find the correct hash.

The weak message expansion

However, there is still an extra foothold that Chabaud and Joux were able to use: the message expansion of SHA-0 is just an XOR. When they were satisfying the constraints of the bits they were flipping, they were able to keep flipping the same bit in each message word instead of needing to come up with complicated arrangements of bits that needed to be flipped. The extra rotation function added to SHA-1 caused this particular attack to be impractical: the search space for bits to be flipped rose dramatically, and new approaches had to be developed.