flesh/Readme.org

#+Title: Relish: Rusty Expressive LIsp SHell
#+Author: Ava Hahn

Note: this document is best read using a dedicated ORG mode editor

* Purpose statement
The purpose of Relish is to create a highly portable, easy to integrate language that can be used in many environments.

* Goals
- Iterate on the ideas and designs that were tested with [[https://gitlab.com/whom/shs][SHS]]
- Create a usable POSIX shell
- Create usable applications/scripts
- To have quality code coverage
- No unsafe code nessesary

  Note: as of release 0.3.0, the Posix module requires unsafe code in order to interface with libc.
  Users are still able to compile and use Relish without any of the unsafe code involved by removing
  the POSIX module. See the Compilation section for more details.

** Stretch Goals
- Create an interpreter that can be booted on one or more SOCs
- Create an interpreter that can be embedded in another application
- Create UI bindings

* Contact
[[https://matrix.to/#/#relish:matrix.sunnypup.io][Matrix chat: #relish:matrix.sunnypup.io]]

If you like Relish and want to support me in working on it consider donating: [[https://ko-fi.com/avaaffine][Ava's Ko-Fi]].

* Documentation
** Writing in Relish
Users who are new to Relish, or who are seeking documentation on how to effectively write in Relish should reference [[file:Writing.org][the main language documentation]]. This will go over the following:
- How Relish operates under the hood
- Relish's syntax
- Relish control flow constructs
- Stdlib functions
- Builting documentation
- Common patterns in using Relish
** Using Relish as your shell
As of version 0.3.0 Relish implements all the features of an interactive shell.
See further documentation in [[file:Shell.org][the shell documentation]]. This material goes over the following:
- How to start and view jobs in Relish
- Special forms for modifying the file descriptors of new jobs
- Piping and other special shell control flow
- Special snippets used to provide a first class shell experience
* Configuration
By default Relish will read from ~/.relishrc for configuration, but the default shell will also accept a filename from the RELISH_CFG_FILE environment variable.
See [[file:snippets/basic_minimal_configuration.rls][the minimal shell configuration example]] for an example of a basic configuration file.
Other snippets, including mood-prompt and avas-laptop-prompt demonstrate more complex configurations.

** The configuration file
The configuration file is a script containing arbitrary Relish code.
On start, any shell which leverages the configuration code in the config module ([[file:src/run.rs][run.rs]]) will create a clean seperate context, including default configuration values, within which the standard library will be initialized.
The configuration file is evaluated and run as a standalone script and may include arbitrary executable code.
Afterwards, configuration values found in the variable map will be used to configure the standard library function mappings that the shell will use.
Errors during configuration are non-terminal. In such a case any defaults which have not been overwritten will remain present.

**** Important points to note
- When the configuration file is run, it will be run with default configuration values.
- The user/script interpreter will be run with the standard library configured to use the previously defined configuration variables.
- The standard library will then be re-processed and re-added to the symbol table with new configuration.
- Variables and functions defined during configuration will carry over to the user/script interpreter, allowing the user to load any number of custom functions and variables.

** Configuration Values
- CFG_RELISH_POSIX (default true): when true, enables POSIX style job control and shell features.
- CFG_RELISH_ENV (default true): when true, interpreter's variable table and environment variable table are kept in sync.
- CFG_RELISH_L_PROMPT (default 'λ'): a function that is called with no arguments to output the left hand of the prompt
- CFG_RELISH_R_PROMPT (default ''): a function that is called with no arguments to output the right hand of the prompt
- CFG_RELISH_PROMPT_DELIMITER (default '>'): a function that is called with no arguments to output the delimiter separating prompt from user input

** Prompt design
For an example of prompt design see [[file:snippets/mood-prompt.rls][the mood prompt]]
For a more complex example see [[file:snippets/avas-laptop-prompt.rls][Ava's laptop prompt]]

** Further configuration
Further configuration can be done by loading scripts that contain more functions and data to evaluate.
Variables and functions defined in an external script loaded by your interpreter will persist in the symbol table.

#+BEGIN_SRC lisp
  (call "my-extra-library-functions.rls")
#+END_SRC

* Compilation
Compiling Relish is as simple as kicking off a build with Cargo.
#+BEGIN_EXAMPLE sh
  cargo build
#+END_EXAMPLE

This will produce a binary at [[file:target/debug/relish][target/debug/relish]] which includes all of the features relish has to offer. This provides a REPL with a full interactive shell that also can manage variables in the Unix environment. It is possible to compile a smaller REPL that does not interact with environment variables and does not offer any shell features. Simply pass the ~--no-default-features~ flag to cargo:

#+BEGIN_EXAMPLE sh
  cargo build --no-default-features
#+END_EXAMPLE

In order to run Relish it is recommended to run the resulting binary at [[file:target/debug/relish][target/debug/relish]].

** Testing
Relish has upwards of 120 unit tests. Full functionality of the core AST with lex, parse, and eval routines as well as the symbol table and relevant code can be tested with Cargo. Unit tests are also included for the standard library of data manipulations and special cases around quote/eval and lambda use:
#+BEGIN_EXAMPLE sh
  cargo test
#+END_EXAMPLE

Userlib tests can be triggered by loading the userlib as well as its test suite directly from a ~cargo run~ invocation:
#+BEGIN_EXAMPLE sh
  cargo run snippets/userlib.rls snippets/userlib-tests.rls
#+END_EXAMPLE

* The codebase
** The [[file:tests][tests directory]]
Start here if you are new.
*** [[file:tests/test_eval.rs][Eval tests]]
These are particularly easy to read and write tests.
They primarily cover execution paths in the evaluation process.
*** [[file:tests/test_func.rs][Func tests]]
These tests extend the eval tests to cover the co-recursive nature between eval and func calls.
*** [[file:tests/test_lex.rs][Lex tests]]
These tests verify the handling of syntax.
*** Lib tests: (tests/test_lib*)
These tests are unique per stdlib module and work to prove the functionality of builtin functions in the language.
** [[file:src][Source directory]]
This directory contains all of the user facing code in relish.
Just a few entries of note:
*** [[file:src/segment.rs][Segment module]]
This file lays out the data structures that the interpreter operates on.
Representation of code trees, traversals, and type annotations all live here.
It provides the core representation of data used in Relish, and could provide supplementary refrence material for users seeking a deeper understanding of how their code is stored in memory.
*** [[file:src/lib.rs][lib.rs]]
This defines a library that can be included to provide an interpreter interface within any Rust project.
The components defined here can certainly be used to support language development for other LISP (or non LISP) langauges. An external project may use or not use any number of these components.
*** [[file:src/sym.rs][Symbol module]]
This file contains all code related to symbol expansion and function calling.
The types defined in this file include SymTable, Args, Symbol, and more.
Code to call Lambda functions also exists in here.
*** [[file:src/run.rs][Run module]]
This file contains functions which load and run the configuration file script.
For more information see the configuraiton section above in this Readme.
*** [[file:src/stl.rs][Standard library module]]
This defines the ~static_stdlib~ function and the ~dynamic_stdlib~ function.
The ~static_stdlib~ function loads all symbols in the standard library which do not need further configuration into the symbol table.
The ~dynamic_stdlib~ function loads all symbols in the standard library which *do* need configuration into the symbol table.
The ~dynamic_stdlib~ function uses variables saved in the symbol table to configure the functions and variables it loads.
This module also contains definitions for the default configuration values.
Any new addition to the stdlib must make its way here to be included in the main shell (and any other shell using the included stdlib functions).
You may choose to override these functions if you would like to include your own special functions in your own special interpreter, or if you would like to pare down the stdlib to a lighter subet of what it is.
You can view the code for standard library functions in [[file:src/stl/][the standard library directory]].
*** [[file:src/bin/][binary directory]]
This contains any executable target of this project. Notably [[file:src/bin/relish.rs][the main shell]].
* Current Status / TODO list
Note: this section will not show the status of each item unless you are viewing it with a proper orgmode viewer.
Note: this section only tracks the state of incomplete TODO items. Having everything on here would be cluttered.

** DONE Alpha Release
(See tag: v0.2.0)
** DONE Beta tasks
(See tag: v0.3.0)
** TODO v1.0 tasks
- islist type query
- Can pass args to relish scripts (via interpreter)
- Can pass args to relish scripts (via command line)
- File operations
  - read-to-string
  - write-to-file
  - file exists?
  - (path functions)
  - (add this all to the readme)
- finish basic goals in the [[file:snippets/interactive-devel.rls][interactive development library]]
- Release CI
- Rename to Flesh
- Make an icon if you feel like it
- Post release to relevant channels
** TODO v1.1 tasks
- finish stretch goals in the [[file:snippets/interactive-devel.rls][interactive development library]]
- Stl boolean assert builtin
- History length configurable (env var?)
- Lex function
- Read function (Input + Lex)
- execute configurable function on cd
- Post to relevant channels
- Implement Compose for lambdas
  (add to readme)
- color control library
  - probably more escapes in the lexer
  - a snippet with a bunch of color constants
- Search delim configurable
** TODO v1.2 release tasks
- Emacs syntax highlighting and/or LSP implementation
- Bindings for the simplest possible UI library?
- Scheme compatibility layer?
- Get on that bug tracker
- Network library
  - HTTP Client
  - TCP Stream client
  - UDP Client
  - TCP Listener
  - HTTP Listener
  - UDP Listener
* Special thanks
Special thanks to [[https://nul.srht.site/]['Underscore Nul']] for consulting with me in the early stages of this project. Meeting my goal of only using safe rust (with the exception of the posix module) would have been a much bigger challenge if not for having someone to experiment on design ideas with.