Flesh: Flexible Shell

• Purpose statement
• Goals
  • Stretch Goals
• Contact
• Documentation
  • Writing in Flesh
  • Using Flesh as your shell
• Configuration
  • The configuration file
    • Important points to note
  • Configuration Values
  • Prompt design
  • Further configuration
• Syntax Highlighting
• Compilation
  • Testing
• The codebase
  • The tests directory
    • Eval tests
    • Func tests
    • Lex tests
    • Lib tests: (tests/test_lib*)
  • Source directory
    • Segment module
    • lib.rs
    • Symbol module
    • Run module
    • Standard library module
    • binary directory
• Current Status / TODO list
  • Alpha Release
  • Beta tasks
  • v1.0 tasks
  • v1.1 tasks
  • v1.2 release tasks
• Special thanks

Note: this document is best read using a dedicated ORG mode editor

Purpose statement

The purpose of Flesh is to create a highly portable, easy to integrate language that can be used in many environments.

Goals

• Iterate on the ideas and designs that were tested with SHS
• Create a usable POSIX shell
• Create usable applications/scripts
• To have quality code coverage
• No unsafe code nessesary Note: as of release 0.3.0, the Posix module requires unsafe code in order to interface with libc. Users are still able to compile and use Flesh without any of the unsafe code involved by removing the POSIX module. See the Compilation section for more details.

Stretch Goals

• Create an interpreter that can be booted on one or more SOCs
• Create an interpreter that can be embedded in another application
• Create UI bindings

Contact

Matrix chat: #relish:matrix.sunnypup.io

If you like Flesh and want to support me in working on it consider donating: Ava's Ko-Fi.

Documentation

Writing in Flesh

Users who are new to Flesh, or who are seeking documentation on how to effectively write in Flesh should reference the main language documentation. This will go over the following:

• How Flesh operates under the hood
• Flesh syntax
• Flesh control flow constructs
• Stdlib functions
• Builting documentation
• Common patterns in using Flesh

Using Flesh as your shell

As of version 0.3.0 Flesh implements all the features of an interactive shell. See further documentation in the shell documentation. This material goes over the following:

• How to start and view jobs in Flesh
• Special forms for modifying the file descriptors of new jobs
• Piping and other special shell control flow
• Special snippets used to provide a first class shell experience

Configuration

By default Flesh will read from ~/.fleshrc for configuration, but the default shell will also accept a filename from the FLESH_CFG_FILE environment variable. See the minimal shell configuration example for an example of a basic configuration file. Other snippets, including mood-prompt and avas-laptop-prompt demonstrate more complex configurations.

The configuration file

The configuration file is a script containing arbitrary Flesh code. On start, any shell which leverages the configuration code in the config module (run.rs) will create a clean seperate context, including default configuration values, within which the standard library will be initialized. The configuration file is evaluated and run as a standalone script and may include arbitrary executable code. Afterwards, configuration values found in the variable map will be used to configure the standard library function mappings that the shell will use. Errors during configuration are non-terminal. In such a case any defaults which have not been overwritten will remain present.

Important points to note

• When the configuration file is run, it will be run with default configuration values.
• The user/script interpreter will be run with the standard library configured to use the previously defined configuration variables.
• The standard library will then be re-processed and re-added to the symbol table with new configuration.
• Variables and functions defined during configuration will carry over to the user/script interpreter, allowing the user to load any number of custom functions and variables.

Configuration Values

• CFG_FLESH_POSIX (default true): when true, enables POSIX style job control and shell features.
• CFG_FLESH_ENV (default true): when true, interpreter's variable table and environment variable table are kept in sync.
• CFG_FLESH_L_PROMPT (default 'λ'): a function that is called with no arguments to output the left hand of the prompt
• CFG_FLESH_R_PROMPT (default ''): a function that is called with no arguments to output the right hand of the prompt
• CFG_FLESH_PROMPT_DELIMITER (default '>'): a function that is called with no arguments to output the delimiter separating prompt from user input
• CFG_FLESH_CD_CB (default None): optional function / lambda that is called with no arguments on every change of CD. See The Shell Documentation for more information.

Prompt design

For an example of prompt design see the mood prompt For a more complex example see Ava's laptop prompt

Further configuration

Further configuration can be done by loading scripts that contain more functions and data to evaluate. Variables and functions defined in an external script loaded by your interpreter will persist in the symbol table.

  (call "my-extra-library-functions.f")

Syntax Highlighting

Syntax Highlighting is implemented for Emacs under the Flesh Mode module (snippets/flesh-mode.el). For installation, consider adding the following to your \~/.emacs:

  (load-file "/PATH/TO/flesh/snippets/flesh-mode.el")

Compilation

Compiling Flesh is as simple as kicking off a build with Cargo.

  cargo build

This will produce a binary at target/debug/flesh which includes all of the features flesh has to offer. This provides a REPL with a full interactive shell that also can manage variables in the Unix environment. It is possible to compile a smaller REPL that does not interact with environment variables and does not offer any shell features. Simply pass the --no-default-features flag to cargo:

  cargo build --no-default-features

In order to run Flesh it is recommended to run the resulting binary at target/debug/flesh.

Testing

Flesh has upwards of 120 unit tests. Full functionality of the core AST with lex, parse, and eval routines as well as the symbol table and relevant code can be tested with Cargo. Unit tests are also included for the standard library of data manipulations and special cases around quote/eval and lambda use:

  cargo test

Userlib tests can be triggered by loading the userlib as well as its test suite directly from a cargo run invocation:

  cargo run snippets/userlib.f snippets/userlib-tests.f

The codebase

The tests directory

Start here if you are new.

Eval tests

These are particularly easy to read and write tests. They primarily cover execution paths in the evaluation process.

Func tests

These tests extend the eval tests to cover the co-recursive nature between eval and func calls.

Lex tests

These tests verify the handling of syntax.

Lib tests: (tests/test_lib*)

These tests are unique per stdlib module and work to prove the functionality of builtin functions in the language.

Source directory

This directory contains all of the user facing code in flesh. Just a few entries of note:

Segment module

This file lays out the data structures that the interpreter operates on. Representation of code trees, traversals, and type annotations all live here. It provides the core representation of data used in Flesh, and could provide supplementary refrence material for users seeking a deeper understanding of how their code is stored in memory.

lib.rs

This defines a library that can be included to provide an interpreter interface within any Rust project. The components defined here can certainly be used to support language development for other LISP (or non LISP) langauges. An external project may use or not use any number of these components.

Symbol module

This file contains all code related to symbol expansion and function calling. The types defined in this file include SymTable, Args, Symbol, and more. Code to call Lambda functions also exists in here.

Run module

This file contains functions which load and run the configuration file script. For more information see the configuraiton section above in this Readme.

Standard library module

This defines the static_stdlib function and the dynamic_stdlib function. The static_stdlib function loads all symbols in the standard library which do not need further configuration into the symbol table. The dynamic_stdlib function loads all symbols in the standard library which do need configuration into the symbol table. The dynamic_stdlib function uses variables saved in the symbol table to configure the functions and variables it loads. This module also contains definitions for the default configuration values. Any new addition to the stdlib must make its way here to be included in the main shell (and any other shell using the included stdlib functions). You may choose to override these functions if you would like to include your own special functions in your own special interpreter, or if you would like to pare down the stdlib to a lighter subet of what it is. You can view the code for standard library functions in the standard library directory.

binary directory

This contains any executable target of this project. Notably the main shell.

Current Status / TODO list

Note: this section will not show the status of each item unless you are viewing it with a proper orgmode viewer. Note: this section only tracks the state of incomplete TODO items. Having everything on here would be cluttered.

DONE Alpha Release

(See tag: v0.2.0)

DONE Beta tasks

(See tag: v0.3.0)

TODO v1.0 tasks

• Can pass args to flesh scripts (via command line)
• Can pass args to flesh scripts (via interpreter)
• declare macros

TODO v1.1 tasks

• all autocomplete is done via configurable userfunction, default documented with 1:1 functionality
• Pipe also operates on stderr
• better emacs language mode
• finish the interactive development library
• History length configurable (env var?)
• Post to relevant channels
• Implement Compose for lambdas (add to readme)

• color control library

  • probably more escapes in the lexer
  • a snippet with a bunch of color constants
• Search delim configurable

TODO v1.2 release tasks

• Bindings for the simplest possible UI library?
• Scheme compatibility layer?
• Get on that bug tracker

• Network library

  • HTTP Client
  • TCP Stream client
  • UDP Client
  • TCP Listener
  • HTTP Listener
  • UDP Listener

Special thanks

Special thanks to 'Underscore Nul' for consulting with me in the early stages of this project. Meeting my goal of only using safe rust (with the exception of the posix module) would have been a much bigger challenge if not for having someone to experiment on design ideas with.