flesh/Readme.org

#+Title: Flesh: Flexible Shell
#+Author: Ava Affine

Note: this document is best read using a dedicated ORG mode editor

* Purpose statement
The purpose of Flesh is to create a highly portable, easy to integrate language that can be used in many environments.

* Goals
- Iterate on the ideas and designs that were tested with [[https://gitlab.com/whom/shs][SHS]]
- Create a usable POSIX shell
- Create usable applications/scripts
- To have quality code coverage
- No unsafe code nessesary

  Note: as of release 0.3.0, the Posix module requires unsafe code in order to interface with libc.
  Users are still able to compile and use Flesh without any of the unsafe code involved by removing
  the POSIX module. See the Compilation section for more details.

** Stretch Goals
- Create an interpreter that can be booted on one or more SOCs
- Create an interpreter that can be embedded in another application
- Create UI bindings

* Contact
[[https://matrix.to/#/#relish:matrix.sunnypup.io][Matrix chat: #relish:matrix.sunnypup.io]]

If you like Flesh and want to support me in working on it consider donating: [[https://ko-fi.com/avaaffine][Ava's Ko-Fi]].

* Documentation
** Writing in Flesh
Users who are new to Flesh, or who are seeking documentation on how to effectively write in Flesh should reference [[file:Writing.org][the main language documentation]]. This will go over the following:
- How Flesh operates under the hood
- Flesh syntax
- Flesh control flow constructs
- Stdlib functions
- Builting documentation
- Common patterns in using Flesh

** Using Flesh as your shell
As of version 0.3.0 Flesh implements all the features of an interactive shell.
See further documentation in [[file:Shell.org][the shell documentation]]. This material goes over the following:
- How to start and view jobs in Flesh
- Special forms for modifying the file descriptors of new jobs
- Piping and other special shell control flow
- Special snippets used to provide a first class shell experience
* Configuration
By default Flesh will read from ~/.fleshrc for configuration, but the default shell will also accept a filename from the FLESH_CFG_FILE environment variable.
See [[file:snippets/basic_minimal_configuration.f][the minimal shell configuration example]] for an example of a basic configuration file.
Other snippets, including mood-prompt and avas-laptop-prompt demonstrate more complex configurations.

** The configuration file
The configuration file is a script containing arbitrary Flesh code.
On start, any shell which leverages the configuration code in the config module ([[file:src/run.rs][run.rs]]) will create a clean seperate context, including default configuration values, within which the standard library will be initialized.
The configuration file is evaluated and run as a standalone script and may include arbitrary executable code.
Afterwards, configuration values found in the variable map will be used to configure the standard library function mappings that the shell will use.
Errors during configuration are non-terminal. In such a case any defaults which have not been overwritten will remain present.

**** Important points to note
- When the configuration file is run, it will be run with default configuration values.
- The user/script interpreter will be run with the standard library configured to use the previously defined configuration variables.
- The standard library will then be re-processed and re-added to the symbol table with new configuration.
- Variables and functions defined during configuration will carry over to the user/script interpreter, allowing the user to load any number of custom functions and variables.

** Configuration Values
- CFG_FLESH_POSIX (default true): when true, enables POSIX style job control and shell features.
- CFG_FLESH_ENV (default true): when true, interpreter's variable table and environment variable table are kept in sync.
- CFG_FLESH_L_PROMPT (default 'λ'): a function that is called with no arguments to output the left hand of the prompt
- CFG_FLESH_R_PROMPT (default ''): a function that is called with no arguments to output the right hand of the prompt
- CFG_FLESH_PROMPT_DELIMITER (default '>'): a function that is called with no arguments to output the delimiter separating prompt from user input
- CFG_FLESH_CD_CB (default None): optional function / lambda that is called with no arguments on every change of CD. See [[file:Shell.org][The Shell Documentation]] for more information.

** Prompt design
For an example of prompt design see [[file:snippets/mood-prompt.f][the mood prompt]]
For a more complex example see [[file:snippets/avas-laptop-prompt.f][Ava's laptop prompt]]

** Further configuration
Further configuration can be done by loading scripts that contain more functions and data to evaluate.
Variables and functions defined in an external script loaded by your interpreter will persist in the symbol table.

#+BEGIN_SRC lisp
  (call "my-extra-library-functions.f")
#+END_SRC

* Syntax Highlighting
Syntax Highlighting is implemented for Emacs under [[file:snippets/flesh-mode.el][the Flesh Mode module (snippets/flesh-mode.el)]].
For installation, consider adding the following to your ~\~/.emacs~:
#+BEGIN_SRC el
  (load-file "/PATH/TO/flesh/snippets/flesh-mode.el")
#+END_SRC

* Compilation
Compiling Flesh is as simple as kicking off a build with Cargo.
#+BEGIN_EXAMPLE sh
  cargo build -p flesh-shell
#+END_EXAMPLE

This will produce a binary at [[file:target/debug/flesh][target/debug/flesh]] which includes all of the features flesh has to offer. This provides a REPL with a full interactive shell that also can manage variables in the Unix environment. It is possible to compile a smaller REPL that does not interact with environment variables and does not offer any shell features. Simply pass the ~--no-default-features~ flag to cargo:

#+BEGIN_EXAMPLE sh
  cargo build -p flesh-shell --no-default-features
#+END_EXAMPLE

In order to run Flesh it is recommended to run the resulting binary at [[file:target/debug/flesh][target/debug/flesh]].

** Testing
Flesh has upwards of 120 unit tests. Full functionality of the core AST with lex, parse, and eval routines as well as the symbol table and relevant code can be tested with Cargo. Unit tests are also included for the standard library of data manipulations and special cases around quote/eval and lambda use:
#+BEGIN_EXAMPLE sh
  cargo test
#+END_EXAMPLE

Userlib tests can be triggered by loading the userlib as well as its test suite directly from a ~cargo run~ invocation:
#+BEGIN_EXAMPLE sh
  cargo run snippets/userlib.f snippets/userlib-tests.f
#+END_EXAMPLE

* Current Status / TODO list
Note: this section will not show the status of each item unless you are viewing it with a proper orgmode viewer.
Note: this section only tracks the state of incomplete TODO items. Having everything on here would be cluttered.

** DONE Alpha Release
(See tag: v0.2.0)
** DONE Beta tasks
(See tag: v0.3.0)
** TODO v1.0 tasks
- Pipe also operates on stderr
- fix let var -> env var bug
- lexer/parser shorthand sugar for quote and eval
- quote/eval based macro procedures
  - constexpr equivalent
  - const var equivalent
- finish the [[file:snippets/interactive-devel.f][interactive development library]]
- core code is nostd compatible?
- cut down all extra clones of code tree branches
** TODO v1.1 tasks
- memoize builtin
- def marks user defined functions / lambdas as pure/impure
- optimize routine implemented
  - precomputes pure code branches
  - memoizes pure functions
- def should call optimize routine on any function or lambda definition
- interactive development library calls optimize routine
- Can pass args to flesh scripts (via command line)
- Can pass args to flesh scripts (via interpreter)
- all autocomplete is done via configurable userfunction, default documented with 1:1 functionality
- better emacs language mode
- Post to relevant channels
** TODO v1.2 release tasks
- History length configurable (env var?)
- Search delim configurable
- Implement Compose for lambdas
  (add to readme)
- color control library
  - probably more escapes in the lexer
  - a snippet with a bunch of color constants
- Bindings for the simplest possible UI library?
- Scheme compatibility layer?
- Get on that bug tracker
- Network library
  - HTTP Client
  - TCP Stream client
  - UDP Client
  - TCP Listener
  - HTTP Listener
  - UDP Listener
* Special thanks
Special thanks to [[https://nul.srht.site/]['Underscore Nul']] for consulting with me in the early stages of this project. Meeting my goal of only using safe rust (with the exception of the posix module) would have been a much bigger challenge if not for having someone to experiment on design ideas with.