The lexer is complete with tests. It fully encapsulates the logic
of splitting an input document into a stream of tokens. It can be
instantiated from an Rc<str>, meaning no lifetimes need be managed
references to the original document (like a stringview) can be
passed around carelessly. The Lexer implements the iterator
method which should help elegantly design repls / compilers, etc.
The S-Expression data type represents the parsed AST. The actual
parsing logic is yet to be added. It is intended that the AST be
the last step before compiling to bytecode. The data representation
here is cons cells of datum. Formatting is implemented.
Signed-off-by: Ava Affine <ava@sunnypup.io>
