Ava Affine
6554a0639a
The lexer is complete with tests. It fully encapsulates the logic of splitting an input document into a stream of tokens. It can be instantiated from an Rc<str>, meaning no lifetimes need be managed references to the original document (like a stringview) can be passed around carelessly. The Lexer implements the iterator method which should help elegantly design repls / compilers, etc. The S-Expression data type represents the parsed AST. The actual parsing logic is yet to be added. It is intended that the AST be the last step before compiling to bytecode. The data representation here is cons cells of datum. Formatting is implemented. Signed-off-by: Ava Affine <ava@sunnypup.io>