peter.michaux.ca

Scheme from Scratch - Bootstrap - Conclusion

2010-01-27T18:58:00

The main goal of this Scheme from Scratch series, as the title implies, was to record the creation of a bootstrap Scheme interpreter in a low-level language like C. This bootstrap interpreter could to be used to self-compile a Scheme compiler written in Scheme. It took about three weeks and the interpter’s code is less than a couple thousand lines lines of code. It is not efficient code but it is straightforward and easy to understand. It doesn’t have macros or continuations but it is enough to write a compiler. It does have closures so it has a better feature set then some languages do.

Going Meta-Circular

The read, eval, and write primitives in Scheme make it very easy to write a little Scheme REPL in Scheme. It almost feels like cheating.

$ ./scheme 
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (define env (environment))
ok
> (define (repl)
    (write-char #\])
    (write-char #\space)
    (write (eval (read) env))
    (write-char #\newline)
    (repl))
ok
> (repl)
] (cons 1 2)
(1 . 2)
] (define (double x) (* 2 x))
ok
] (double 21)
42

The bootstrap interpreter was modeled after the first meta-circular interpreter in SICP. The ability to implement a programming language in our new interpreter implementation is a great test. You can download the SICP meta-circular evaluator (cache) and run it yourself. I first wrote a little helper file called run-meta-circular.scm to load and start everything that is required.

; Add standard definitions to complete our 
; Scheme implementation: cadr, length, etc.
(load "stdlib.scm")

; Add extra definitions needed by
; the meta-circular implementation.
(define true #t)
(define false #f)
; Cheat a bit but actually results
; in a better REPL
(define display write)
(define newline (lambda () (write-char #\newline)))

; Load the meta-circular implementation
; exactly as it appears in SICP.
(load "ch4-mceval.scm")

; Run the meta-circular interpreter.
; These lines appear but are commented out 
; in the meta-circular implementation.
(define the-global-environment (setup-environment))
(driver-loop)

With that file it is easy to fire up the meta-circular interpreter exactly as it is written in SICP.

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (load "run-meta-circular.scm")


";;; M-Eval input:"
(define a (cons 1 2))

";;; M-Eval value:"
ok

";;; M-Eval input:"
(car a)

";;; M-Eval value:"
1

";;; M-Eval input:"
((lambda (x) x) 21)

";;; M-Eval value:"
21

Unfortunately the SICP code doesn’t define many primitives in the meta-circular interrupter so the above example is not too rich. If you expand the interpreter you can just fill your boots with meta-circular goodness.

It was really great to see in the GitHub feeds that other people were running the meta-circular interpreter days ago and I hadn’t even mentioned it yet.

- - - -

The reason for documenting this whole adventure was I couldn’t find a book that I could just buy and read implementing Scheme in C. Maybe one exists and I didn’t search hard enough. Maybe one should still be written. Maybe these articles has for will fill the need for at least a few readers. Regardless it has been fun documenting it and watching other enjoy implementing their own interpreters along the way. Enjoy your interpreter!

Previous article: Garbage Collection

Scheme from Scratch - Bootstrap v0.22 - Garbage Collection

2010-01-26T10:15:00

Garbage collection is an essential part of a production Scheme system. Bootstrap Scheme is not a production Scheme system. It is intended to be just enough of a Scheme interpreter to execute a Scheme compiler once. If your computer has enough memory that it won’t run out during the execution of a compiler than you don’t need to implement a garbage collector in Bootstrap Scheme.

I think that the simplest garbage collection system you could add to the interpreter is a mark-and-sweep system. Mark-and-sweep is easier to implement than many collectors, isn’t foiled by circular references, and doesn’t require hardware-specific knowledge. The garbage collection pauses in a mark-and-sweep system can be noticeably long but that doesn’t matter for a bootstrap interpreter.

The C function alloc_object in the interpreter currently calls malloc for every object allocated and doesn’t record anything about what memory has been allocated. The fundamental goal is to replace alloc_object so a new version of it manipulates a heap of preallocated objects that can be recycled.

This page describes one way to implement a mark-and-sweep algorithm. This is basically the mark-and-sweep algorithm I’ve seen in use in some interpreters.

When the interpreter begins, the C init function in the model layer is executed. This is the right place to set up a heap of Scheme objects. You can statically allocate or dynamically malloc enough space for thousands of Scheme objects (i.e. struct object). Each object needs a new field, char mark, that must be set to 0 during initialization. Each object also needs a struct object *next field to be used for tracking which memory is free or active. At the beginning, all memory is free. Link all the heap objects together into a linked list with the next field. The last object in the list can point to C’s NULL. Assign the head of this linked list to a C global variable free_list. There is also a global active_list that is set to NULL now and that will hold all objects in use by the program.

When alloc_object is called, it looks to see if the free_list has an available object. If the list does have an available object then alloc_object moves that object from the free_list to the active_list and returns that object. If the free_list is empty then alloc_object calls a gc function to run the mark-and-sweep algorithm. When gc returns, alloc_object can recheck for available objects on the free_list and return one if there is one or print an error message and exit the program.

The gc function is reasonably simple with its two phases: mark and sweep.

In the mark phase, the garbage collector starts at all the “root” objects (described below) and walks all reachable memory (e.g. pairs can point to other objects) and marks each object. Each reachable object will have its mark field set to 1. When all reachable objects are marked, then move on to the sweep phase.

In the sweep phase, the garbage collector starts at the top of the active_list and examines each object in that list. Objects that have mark set to 0 are moved to the free_list. (Symbols and strings also need to have their data.symbol.value field, for example, freed.) Objects in the active_list that have mark set to 1 have their mark set back to 0 in preparation for the next mark phase.

All of that is pretty easy and nicely contained to a very small area of the C source code. Where everything becomes messy is keeping track of the roots for the mark-and-sweep algorithm. The root objects are the ones where the mark phase starts. These are objects that may not have any other objects pointing to them but should still not be garbage collected. the_global_environment is an example of a known root object. It is easy to have the sweep phase start at all the known roots.

What isn’t easy is managing the stack roots. Have a look at the following C function, for example,

object *make_if(object *predicate, object *consequent,
                object *alternative) {
    return cons(if_symbol,
                cons(predicate,
                     cons(consequent,
                          cons(alternative,
                               the_empty_list))));
}

There are a total of four calls to cons. Each call to cons will call alloc_object. Each call to alloc_object can potentially trigger a round of garbage collection. The objects returned by the three inner cons calls above are not rooted. That is, there is no known garbage collection root, that when followed, will lead to those returned objects. We need a way to protect the returned objects so that an outer call to cons doesn’t move the previous returned value back to the free_list.

We need to maintain a global stack_root list of these otherwise unprotected objects. The mark phase can then use these stack_root objects as additional roots. In order to do this, the make_if function needs to be changed to push and pop from the stack_root list.

object *make_if(object *predicate, object *consequent,
                object *alternative) {
    object *result;
    
    push_stack_root(&result);
    result = cons(alternative, the_empty_list);
    result = cons(consequent, result);
    result = cons(predicate, result);
    result = cons(if_symbol, result);
    pop_stack_root();
    return result;
}

Ugg. Not pretty, is it? What is neat, and keeps the implementation from being even uglier, is that it is a pointer to the result pointer that is pushed onto the stack_root list. That way, anything pointed to by result is protected. You’ll notice that the result isn’t protected when it is returned to make_if’s caller. It is up to that caller to protect the returned object.

If you forget to protect a stack variable you can have dangling pointer. If you forget to pop a stack root you will have a memory leak. The garbage collection code is complex and invades almost all functions. Code with care!

I think you can see why I didn’t cloud the implementation up to now with garbage collection worries. Focusing on the model, reader and evaluation layers was plenty as they were being introduced. I think worrying about garbage collection along the way would have made things intimidating and less fun.

Other types of garbage collectors, with hardware-specific knowledge, allow you to avoid making this mess. The Bohem conservative garbage collector is one such library you could link to your interpreter. If you have a look in the source code of that library you will probably be more frightened by it than you are by the approach I’ve presented.

One more article tomorrow to wrap up things.

Previous article: Standard Library
Next article: Conclusion

Scheme from Scratch - Bootstrap v0.21 - Standard Library

2010-01-25T10:35:00

With I/O in place, especially the load form, we can continue to expand our Scheme implementation by programming new compound procedures in Scheme. The standard library defines forms like cadr, map, not, etc. Feel free to add whatever you think you would like to have available.

A sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (load "stdlib.scm")
stdlib-loaded
> (define (double x) (* 2 x))
ok
> (map double '(0 1 2 3)) 
(0 2 4 6)

There is no C programming for this version. That feels like a move in the right direction given a bootstrap interpreter is to eventually allow programming in the implemented language.

Three weeks to the day and even though there isn’t garbage collection, the functionality of the language is complete. As long as programs are not too big, run too long, or process too much data, the experience of using your bootstrap interpreter is what it set out to be. Sit back and enjoy watching it run. :-)

There is a v0.21 branch on github for this version.

Previous article: I/O
Next article: Garbage Collection

Scheme from Scratch - Bootstrap v0.20 - I/O

2010-01-24T15:00:00

The last major missing piece of the puzzle for a bootstrap interpreter is the ability to work with files. Scheme has several input and output primitive procedures and I’m implementing the ones I think will be useful. It is a slightly larger amount to implement than some days have been but hopefully quite straightforward for anyone who has come this far.

input:

load
read
read-char
peek-char
input-port?
open-input-file
close-input-file
eof-object?

output:

write
write-char
output-port?
open-output-file
close-output-file
error

A sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (define out (open-output-port "asdf.txt"))
ok
> (write-car #\c out)
ok
> (close-output-port out)
ok
> (load "program.scm")
program-loaded
> (error "bad move")
"bad move"
exiting

Some of these functions required refactoring in other areas of the interpreter. I refactored the C read function to handle the end of a source file properly. I refactored the C write function to take a stream parameter so that the Scheme write could write to any port. I’ll backport these changes to previous versions eventually.

I’m not doing any of the the current-input-port, call-with-input-port, and with-input-from-file business. If a port is not specified as an optional parameter to read, write-char, etc then C’s stdin and stdout are used.

I added the error output form which writes all of its arguments and then exits. This form is not required by R5RS, for example, but is useful.

There is still a little bit to do but we are oh so close.

There is a v0.20 branch on github for this version.

Previous article: Eval
Next article: Standard Library

Scheme from Scratch - Bootstrap v0.19 - Eval

2010-01-23T10:36:00

The eval form is a bit like apply. They are both procedures with an unusual tail call requirements.

The second argument to eval is an environment, so exposing eval also requires exposing some environment-related procedures. All together I implemented the following procedures for this version:

interaction-environment: Returns the top-level environment of the current REPL session.
null-environment: Returns an empty environment with only syntactic keywords like if, define, etc.
environment: Returns a new environment like the environment present when the REPL starts.
eval: Evaluates an expression in a given environment.

Scheme from Scratch - Bootstrap v0.18 - Apply

2010-01-22T09:57:00

The apply form is an interesting beast in Scheme. It is a procedure so can be passed around and assigned to variables. It has a special tail call requirement that most other primitive procedure has and that needs to handled carefully.

Sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (apply + '(1 2 3))
6
> (apply + 1 2 '(3))
6
> (apply + 1 2 3 '())
6

The first example above is the important one to have. I implemented apply to allow the last two examples for the fun of the implementation.

In my implementation, I did set up apply as a primitive procedure but it is one where the C function for the primitive procedure should never be called. It is a regular primitive procedure because that means apply can be passed around in the Scheme programs. I implemented a small trick in eval to handle apply and have the tail call requirement satisfied. I don’t want to enter the world of trampolines for my primitive procedures.

I have a list of what remains. I looks like there are only about six more days of implementation for Bootstrap Scheme and then a wrap-up article.

There is a v0.18 branch on github for this version.

Previous article: And and Or
Next article: Eval

Scheme from Scratch - Bootstrap v0.17 - And and Or

2010-01-21T09:45:00

One more round of library syntax: the and and or forms. These could be split into two different versions but they are so similar I think biting them off in one chunk is justified. If you do and first then or will be easy.

Sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (and 1 2 3)
3
> (and)
#t
> (or #f 2 #t)    
2
> (or)
#f

An important aspect of and and or is they are short-circuiting. Here is an example demonstrating the importance of and short-circuiting when side-effects are involved:

> (define a 1)
ok
> (and #f (set! a 2))
#f
> a
1
> (and #t (set! a 2))
ok
> a
2

If you are not already consulting the Scheme spec when implementing features in your interpreter, now might be a good time to give that a try. I have R5RS printed on real paper which is handy.

I’m not 100% sure but I think it might now be possible to run all the examples in The Little Schemer in your interpreter.

I’ve enjoyed the watch feature on GitHub to follow other’s progress. It is motivating to see implementations moving forward and really great to see people have actually implemented features I haven’t discussed yet: apply, load, etc.

There is a v0.17 branch on github for this version.

Previous article: Let
Next article: Apply

Scheme from Scratch - Bootstrap v0.16 - Let

2010-01-20T08:08:00

We could have lived our lives without adding cond. It is a convenience form and there are a few others that are handy to have around: let is one of them.

Sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (let ((x (+ 1 1))
        (y (- 5 2)))
    (+ x y))
5

The let form is also “library syntax” and so the implementation is another abstract syntax tree manipulation. This time it is a manipulation that converts the let form to a procedure application with a lambda form.

At this point you may be thinking these abstract syntax tree manipulations could become tedious to implement. Macros exist for this reason and so that the programmer can invent his own new library syntax. We only need a few bits of library syntax for a bootstrap interpreter and it is easier to implement directly in C then to implement macros in C and then the library syntax in Scheme.

There is a v0.16 branch on github for this version.

Previous article: Cond
Previous article: And and Or

Scheme from Scratch - Bootstrap v0.15 - Cond

2010-01-19T08:55:00

The only remaining part of the first meta-circular interpreter of SICP still to port to our interpreter is the cond form.

Sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (cond (#f          1)
        ((eq? 'a 'a) 2)
        (else        3))
2

The cond form is “library syntax” and so the implementation of cond is an abstract syntax tree manipulation to convert the form to a series of nested if forms. The conversion algorithm is spelled out for us in SICP.

SICP notes and discusses how inefficient the implementation is. Every time a single cond form is evaluated it is (re)converted to nested if forms. This is inefficient and the second meta-circular interpreter addresses this issue; however, for the purpose of a bootstrap intereter, simple is more important than efficient.

We’ve reached an important milestone along our path. The port from SICP is complete. Any extra functionality we need for our interpreter to be a useful bootstrap interpreter we will need to figure out how to implement on our own.

There is a v0.15 branch on github for this version.

Previous article: Begin
Next article: Let

Scheme from Scratch - Bootstrap v0.14 - Begin

2010-01-18T10:13:00

The last few days have been some meaty implementation steps. Today is quite light with the implementation of the begin form.

Sample REPL session:

$ ./scheme
Welcome to Bootstrap Scheme. Use ctrl-c to exit.
> (begin 1 2 3)
3

The begin form doesn’t introduce a new lexical scope like a lambda does but begin does have a tail position that must be handled correctly.

As noted by readers in the comments below, a compound procedure has an implicit begin form for the body of the procedure. The procedure application portion of eval can be refactored to use this fact. I’ve made this change to the code. Perhaps introducing begin before lambda would have been a better pedagogical progression but I was getting impatient to get to lambda.

There is a v0.14 branch on github for this version.

Previous article: Lambda the Ultimate
Next article: Cond