extend build instructions

more readme edits

See merge request whom/shs!11

Readme.md

@@ -1,88 +1,166 @@
-# shs
+# SHS
 Syntactically Homogeneous Shell
 
 ## Overview
-This shell was created to have extremely simple syntax. S-Expressions were chosen to represent statements and the scope of language features were constrained to what could be considered practical for daily shell use. This program is meant to be practical for administrators and daily power users. It can be used for both system administration (as one might use bash or zsh) as well as for the creation of simple programs.
+This shell was created to have extremely simple syntax. S-Expressions were chosen to represent statements and the scope of the language features were constrained to what could be considered practical for daily shell use. This program is meant to be practical for administrators and daily power users. It can be used for both system administration (as one might use bash or zsh) as well as for the creation of simple programs as one might use Python, Racket, or Visual Basic.
 
 ## Basic Syntax
-When in doubt the `print_ast` utility can be used to examine the output of the Lex+Parse process. Here you can spot any bugs regarding syntax.
+When in doubt the `print_ast` utility can be used to examine the output of the Lexing and Parsing process. Here you can spot any bugs regarding syntax.
+
+`$ print_ast (example invocation)`
 
 ### Lists
-Any sequence of items within a set of parenthesis is a list
-`(1 "two" three 4)`
+Any sequence of items within a set of parenthesis is a list. 
 
-Lists can be infinitely nested
-`("one" (2 3 4 (5)))`
+For Example: `(1 "two" three 4)`
+
+Lists can be infinitely nested: `("one" (2 3 4 (5)))`
 
 ### Data types
-We use the following data types
-* Number: 1, 2.0, etc
-* String: "this is a string" (string delimiters: ' " \`)
-* Bool: T or F
-* Symbol: a string with no delimiters
-* List: a sequence of elements within parenthesis
+Elements in a list can be one of the following data types:
+* **Number**: A number can be written in one of the following formats:
+    - integers: 1, 2, 3, etc...
+    - rational numbers: 1.22, 2.24, etc...
+* **String**: Strings must use one of the following delimiters:
+    - "this string is a proper string"
+    - 'this is also a proper string'
+    - \` this is another proper string\`
+* **Boolean**: Boolean values can take the following forms:
+    - true: "T"
+    - false: "F"
+* **Symbol**:
+    - A symbol looks like a string without delimiters.
+    - A symbol denotes a variable or a function.
+* **List**: Any combination of multiple items seperated by a space. (See above.)
 
 ### Function calls
-Any list beginning in a symbol will be considered a function call.
-From within the `shs_repl` utility, unknown symbols will be assumed to be system binaries.
+Any list beginning in a symbol will be considered a function call. A user may typically expect to be able to start a list with a variable. However, given the current architecture this pattern must be maintained in order to enable shell functionality. Thus, from within the `shs` utility unknown symbols at the start of a list will be assumed to be system binaries. 
 
-`(append () 1 2 3)`
-`(vim Readme.md)`
-`(if (eq "example" (fread 'test_file')) (print "test worked) (rm -rf /))`
+Function call: `(append () 1 2 3)`
+
+System binary call: `(vim Readme.md)`
+
+```
+; complex call 
+(if (eq "example" (fread 'test_file'))
+    (print "test worked")
+    (rm -rf /))
+```
+
+There may be alternative REPLs, officially or from community members, that do not exhibit this behavior.
 
 ### Variable declaration
-There are a few ways to export variables
-* export: `(export NAME (value))`
-* let: `(let ((var1 val1) (var2 val2)) (form_to_be_evaluated))`
+Export is used to define variables.
 
-Currently, let has yet to be implemented
+`(export NAME (value))`
+
+Example export:
+```
+(export vim 'nvim')
+```
+
+Users may also leverage the `export` function to create aliases in the SHS shell.
+
+```
+(export vim nvim)
+(vim /path/to/file.c)
+```
+
+The above example will call exec with the arguments "nvim" and "/path/to/file.c".
 
 ### Function declaration
 Use the `func` function from the stdlib:
-`(func name (var1, var2, var3) (form_to_be_evaluated))` 
-In this case, `(form_to_be_evaluated)` will not be evaluated until the function is called.
+
+Example call: `(func name (var1, var2, var3) (form_to_be_evaluated))` 
+
+In the above example, a function is defined with the following attributes:
+- The function is named "name".
+- The function takes three arguments: arg1, arg2, and arg3.
+- When the function is called, "(form_to_be_evaluated)" is evaluated.
+
+SHS will only lex (form_to_be_evaluated). It will not parse or execute it until the function is invoked.
+
+In the below example a function named addthree is defined. The function takes three arguments, and returns the sum of them.
+
+```
+(func addthree (a b c) (+ a b c))
+(addthree 2 3 4)
+Output: 9
+```
 
 ### Control flow
-See `stdlib/control_flow.go`. We have if, while, and progn forms:
-`(if (cond) (then) (else))`
-`(when (cond) (form1)....... (formN))`
-`(progn (form1)..... (formN))`
-If and While should be self explanatory. For those new to LISP, the rough idea of progn is to evaluate a sequence of N forms and return the result of the final one.
-We also have functioning implementations of map and reduce in the stdlib (incomplete)
+SHS currently uses the following control flow forms for daily use. Although, it is not hard to create your own in the SHS source code.
+#### if statements
+The if form takes 3 arguments. It evaluates the first argument, the condition, and if it evaluates to true (T) it evaluates the second argument. If the first argument evaluates to false (F) the `if` routine then evaluates the third argument. The argument that is not used will not be parsed or evaluated. The whole statement will, however, be lexed.
+
+```
+(if (cond) (then) (else))
+(if (eq 1 1.0) 
+    (print "numbers are equal") 
+    (print "numbers are not equal"))
+```
+
+#### while loops
+The while loop takes N arguments. The first argument, the conditional, must evaluate to a boolean value (T or F). If the first argument is evaluated to T, all subsequent arguments are evaluated. If the first argument evaluates to F, while returns. Finally, the conditional is re-evaluated and the loop begins again.
+
+```
+(while (cond) (form1)....... (formN))
+
+(export cond T)
+(while cond 
+    (print "this will only be printed once")
+    (export cond F))
+
+Output:
+this will only be printed once
+```
 
 ## Comments
 The standard delimiter for comments is ;
-any characters after a semicolon will be ignored until end of line
+Any characters after a semicolon will be ignored until end of line. Semicolons found in strings will not be considered comment delimiters.
+
+```
+; this comment explains the value in myvar
+(export myvar "testvar")
+```
 
 ## How to build
 ### Compiling/Installation
-- For now simply run `go install cmd/...` for each utility you wish to use. If you have GOPATH and GOBIN set it should be usable from PATH
+1. [follow Google's instructions to install Go](https://golang.org/doc/install)
+2. run `$ mkdir -p ~/go`
+3. add the following commands to your shell configuration, and run them in your shell
+    - `export GOPATH $HOME/go`
+    - `export PATH $PATH:$GOPATH/bin`
+4. run `$ go get -u gitlab.com/whom/shs`
+5. change directory to `$GOPATH/src/gitlab.com/whom/shs`
+6. run `$ go install ./cmd/shs.go`
+7. you can now call `shs` via the shell
 
 ### Adding SHS to your application
-* Make sure to set ast.SyncTablesWithOSEnviron, ast.ExecWhenFuncUndef. All of which control integrations with the underlying system. 
+Here are some important tips for integrating an SHS REPL into another codebase.
+
+* Make sure to set ast.SyncTablesWithOSEnviron and ast.ExecWhenFuncUndef, both of which control integrations with the underlying system. 
     - If you do not want the user to be able to set environment variables set ast.SyncTablesWithOSEnviron to false.
     - If you do not want the user to be able to call binaries from the host system, set ast.ExecWhenFuncUndef to false.
-- Get text you are interested in parsing
-- Create a new VarTable and FuncTable (see ast/var_table.go and ast/func_table.go)
-- Call `Lex(text)` on the `text` you want to evaluate to recieve a tree of parsed lexemes.
-- Call `tree.Eval(FuncTable, VarTable, false)` where tree is the returned data from Lex, and the final boolean argument is whether or not to convert unknown symbols to strings. (this is a helpful option if you are writing functions such as those in stdlib/call.go, or any funciton in which you may want to be able to edit and transform the final ast based on your own varaiable table)
-- Make sure the GPLv3 is adhered to
-- *OVERRIDE THE STDLIB GenFuncTable FUNCTION.* You very likely do NOT want an available function to call system binaries in your embedded shell. Make sure the stdlib Call function is not included.
+- Create a new VarTable and FuncTable (see ast/var_table.go and ast/func_table.go).
+- Make sure to adhere to the terms and conditions stated in the GPLv3.
+- *OVERRIDE THE STDLIB GenFuncTable FUNCTION.* You very likely do NOT want an available function to call system binaries in your embedded shell. Make sure the stdlib Call function is not included in your GenFuncTable inplementation.
 
 ## Configuration
-* variables exported in the repl, if of types string or number, will result in a corresponding variable added to the Environment.
-* one can write arbitrary shs script into `.shsrc` including function and variable declarations
-* of note are the following variables
+* Variables exported in the REPL, if of type string or number, will result in a corresponding environment variable.
+* One can write arbitrary SHS script into `.shsrc` including function and variable declarations
+* Particularly useful are the following variables:
     - `SH_LOGGING` Sets the log level (from 0 to 3)
     - `SHS_STATIC_PROMPT` Sets the prompt
     - `SH_HIST_FILE` Sets the history file
     - `SH_DEBUG_MODE` Adds additional debug output for the lexer (high clutter)
-* additionally, the repl will evaluate any function you define as `_SH_PROMPT` before the shell prompt
+* Additionally, the REPL will evaluate any function you define as `_SH_PROMPT` before the shell prompt.
     - if defined, the function will be evaluated before printing the prompt
     - the function will be given 0 arguments
     - if the function does not return a string, its output will be discarded
     - afterwards, the repl will print the values in `SHS_STATIC_PROMPT`
-Here is an example of a shs configuration file:
+
+Here is an example of a SHS configuration file:
 ```lisp
 (export GOPATH (concat HOME     "/go"))
 (export GOBIN  (concat GOPATH  "/bin"))
@@ -95,12 +173,14 @@ Here is an example of a shs configuration file:
 ```
 
 ## The Docs
+
+#### Godoc
+What follows are links to documentation for the code and interfaces used by SHS. This documentation is automatically generated by godoc, and in times of transition may not be complete.
 - [Documentation on language interpreter](https://godoc.org/gitlab.com/whom/shs/ast)
 - [Documentation on logging module](https://godoc.org/gitlab.com/whom/shs/log)
 - [Documentation on configuration module](https://godoc.org/gitlab.com/whom/shs/config)
 - [Documentation on utility functions](https://godoc.org/gitlab.com/whom/shs/util)
 - [Documentation on standard library](https://godoc.org/gitlab.com/whom/shs/stdlib)
-- [Guide to standard library](https://gitlab.com/whom/shs/-/blob/master/stdlib/Readme.md)
 
 ## Contributing
 - Any contribution to this software is welcome as long as it adheres to the conduct guidelines specified in the `Contributing.md` file in this repository.

ast/eval.go

@@ -17,7 +17,9 @@
 
 package ast
 
-import "gitlab.com/whom/shs/log"
+import (
+    "gitlab.com/whom/shs/log"
+)
 
 /* determines whether or not to execute a system binary
  * when a function cannot be found in the functable

ast/func_table.go

@@ -18,7 +18,6 @@
 package ast
 
 import (
-    "fmt"
     "gitlab.com/whom/shs/log"
 )
 
@@ -38,8 +37,18 @@ type Function struct {
     // number of times user has called this function
     TimesCalled int
 
-    // number of args required
-    Args        int
+    // list of types (LIST, NUMBER, STRING, etc) representing args
+    Args        []Token_t
+    NumArgs     int // -1 means infinite
+
+    // lazy arg checking (use NumArgs instead of args)
+    ArgLazy     bool
+
+    // dont fail on undefined symbol (passed to eval when parsing args)
+    SymLazy   bool
+
+    // dont eval args at all, leave that to the function
+    EvalLazy    bool
 }
 
 /* holds a mapping of key to function
@@ -52,24 +61,58 @@ type FuncTable *map[string]*Function
  * makes sure correct arguments are passed in
  */
 func (f Function) ParseFunction(args *Token) bool {
-    // handle infinite args
-    if f.Args < 0 {
+    inc := 0
+    for iter := args; iter != nil; iter = iter.Next {
+        inc += 1
+        if inc > len(f.Args) {
+            log.Log(log.ERR,
+                "too many arguments",
+                "ftable")
+            return false
+        }
+
+        if iter.Tag != f.Args[inc - 1] {
+            log.Log(log.ERR,
+                "argument is " + GetTagAsStr(iter.Tag) +
+                " should be " + GetTagAsStr(f.Args[inc - 1]),
+                "ftable")
+            return false
+        }
+    }
+
+    if inc < len(f.Args) {
+        log.Log(log.ERR,
+            "not enough args given",
+            "ftable")
+        return false
+    }
+
+    return true
+}
+
+/* same as ParseFunction but only evaluates the number of args
+ */
+func (f Function) LazyParseFunction(args *Token) bool {
+    if f.NumArgs < 0 {
         return true
     }
 
-    i := f.Args
+    total := 0
     for iter := args; iter != nil; iter = iter.Next {
-        i -= 1
+        total += 1
     }
 
-    if i != 0 {
+    if total < f.NumArgs {
         log.Log(log.ERR,
-            "Incorrect number of arguments",
-            "eval")
-        log.Log(log.DEBUG,
-            fmt.Sprintf("Function %s expects %d arguments. You've provided %d arguments.",
-                f.Name, f.Args, f.Args - i),
-            "eval")
+            "expected more arguments, try calling info on function",
+            "ftable")
+        return false
+    }
+
+    if total > f.NumArgs {
+        log.Log(log.ERR,
+            "too many args. try calling info on function",
+            "ftable")
         return false
     }
 
@@ -80,7 +123,19 @@ func (f Function) ParseFunction(args *Token) bool {
  * calls ParseFunction and increments TimesCalled
  */
 func (f Function) CallFunction(args *Token, vt VarTable, ft FuncTable) *Token {
-    if !f.ParseFunction(args) {
+    n_args := args
+    if !f.EvalLazy {
+        n_args = args.Eval(ft, vt, f.SymLazy)
+    }
+
+    passes := false
+    if f.ArgLazy {
+        passes = f.LazyParseFunction(n_args)
+    } else {
+        passes = f.ParseFunction(n_args)
+    }
+
+    if !passes {
         log.Log(log.ERR,
                 "Couldnt call " + f.Name,
                 "eval")
@@ -88,7 +143,7 @@ func (f Function) CallFunction(args *Token, vt VarTable, ft FuncTable) *Token {
     }
 
     f.TimesCalled += 1
-    return f.Function(args, vt, ft)
+    return f.Function(n_args, vt, ft)
 }
 
 /* searches for function mapped to argument in FuncTable

ast/lex.go

@@ -162,14 +162,22 @@ func lex(input string) *Token {
 
         // comment case
         case ';':
+            i = matchLineEnd(i)
             start_pos = i + 1
-            i = matchLineEnd(start_pos)
+
+        // this isnt to handle string escaping
+        // its only to make sure that escaped spaces stay in
+        // the same token.
+        case '\\':
+            if i != len(input) - 1 && input[i+1] == ' '{
+                // eat the backslash
+                input = input[:i] + input[i+1:]
+            }
         }
 
         if needs_alloc {
             needs_alloc = false
             if (i < 0) {
-                // TODO: Maybe not overload this.
                 start_pos = i
                 goto error
             }
@@ -186,8 +194,6 @@ func lex(input string) *Token {
     return ret
 
 error:
-    // TODO: Hook into error module
-    // TODO: Finalize and GC alloced tokens
     if start_pos == -1 {
         log.Log(log.ERR,
                 "Unmatched string delimiter in input. discarding.",
@@ -209,6 +215,11 @@ error:
 func StrIsNumber(arg string) bool {
     dotCount := 0
 
+    // negative nums
+    if len(arg) > 1 && arg[0] == '-' {
+        arg = arg[1:]
+    }
+
     for _, char := range arg {
         if !unicode.IsDigit(char) {
             if char == '.' && dotCount == 0 {

cmd/shs.go

@@ -95,7 +95,7 @@ func main() {
     }
 
     dyn_prompt := ast.GetFunction("_SH_PROMPT", funcs)
-    if dyn_prompt == nil || dyn_prompt.Args != 0 {
+    if dyn_prompt == nil || dyn_prompt.NumArgs != 0 {
         dyn_prompt = nil
     }
 
@@ -149,6 +149,11 @@ func main() {
         fmt.Printf(prePrompt)
         text, err := line.Prompt(prompt)
         if err != nil && err != liner.ErrPromptAborted{
+            // must be a better way to do this check
+            if err.Error() == "EOF" {
+                return
+            }
+
             log.Log(log.ERR, "couldnt read user input: " + err.Error(), "repl")
             continue
         }

stdlib/arith.go

@@ -36,14 +36,6 @@ import (
  * Example: (number "3.4")
  */
 func NumCast(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
-    in = in.Eval(f, a, false)
-    if in.Tag != ast.STRING {
-        log.Log(log.ERR,
-            "only a string can successfully be cast to a number",
-            "number_cast")
-        return nil
-    }
-
     if !ast.StrIsNumber(in.Value()) {
         log.Log(log.ERR,
             "string failed number cast",
@@ -64,9 +56,6 @@ func NumCast(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
  */
 func Add(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
     var res float64
-
-    in = in.Eval(f, a, false)
-
     for i := in; i != nil; i = i.Next {
         if i.Tag != ast.NUMBER {
             log.Log(log.ERR, "Non-number given to ADD", "add")
@@ -112,9 +101,6 @@ func Add(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
 func Sub(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
     var res float64
     var sub float64
-
-    in = in.Eval(f, a, false)
-
     for i := in; i != nil; i = i.Next {
         if i.Tag != ast.NUMBER {
             log.Log(log.ERR, "Non-number given to SUB", "sub")
@@ -166,9 +152,6 @@ func Sub(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
  */
 func Mult(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
     res := 1.0
-
-    in = in.Eval(f, a, false)
-
     for i := in; i != nil; i = i.Next {
         if i.Tag != ast.NUMBER {
             log.Log(log.ERR, "Non-number given to MULT", "mult")
@@ -214,9 +197,6 @@ func Mult(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
  */
 func Div(in *ast.Token, a ast.VarTable, f ast.FuncTable) *ast.Token {
     var res float64
-
-    in = in.Eval(f, a, false)
-
     for i := in; i != nil; i = i.Next {
         inner := 0.0
 

stdlib/bool.go

@@ -30,14 +30,6 @@ import (
  * Example: (bool "F")
  */
 func BoolCast(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-    if in.Tag == ast.LIST || in.Tag == ast.NUMBER {
-        log.Log(log.ERR,
-            "only strings successfully cast to bool",
-            "bool cast")
-        return nil
-    }
-
     body := in.Value()
     if body != ast.TRUE && body != ast.FALSE {
         log.Log(log.ERR,
@@ -52,13 +44,6 @@ func BoolCast(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
 }
 
 func Not(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-
-    if in.Tag != ast.BOOL {
-        log.Log(log.ERR, "non-bool argument to 'not'", "not")
-        return nil
-    }
-
     out := ast.TRUE
     if in.Value() == ast.TRUE {
         out = ast.FALSE
@@ -69,10 +54,9 @@ func Not(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
     return t
 }
 
+// Lazy args this
 func Eq(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
     out := ast.TRUE
-
-    in = in.Eval(ft, vt, false)
     second := in.Next
 
     if in.Tag != second.Tag {
@@ -155,13 +139,6 @@ func Lt(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
     out := ast.TRUE
     second := in.Next
 
-    in = in.Eval(ft, vt, false)
-
-    if in.Tag != ast.NUMBER || second.Tag != ast.NUMBER {
-        log.Log(log.ERR, "non-number argument to numeric boolean operator", ">/<=")
-        return nil
-    }
-
     l, _ := strconv.ParseInt(in.Value(), 10, 64)
     r, _ := strconv.ParseInt(second.Value(), 10, 64)
 
@@ -178,13 +155,6 @@ func Gt(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
     out := ast.TRUE
     second := in.Next
 
-    in = in.Eval(ft, vt, false)
-
-    if in.Tag != ast.NUMBER || second.Tag != ast.NUMBER {
-        log.Log(log.ERR, "non-number argument to numeric boolean operator", ">/<=")
-        return nil
-    }
-
     l, _ := strconv.ParseInt(in.Value(), 10, 64)
     r, _ := strconv.ParseInt(second.Value(), 10, 64)
 

stdlib/control_flow.go

@@ -78,8 +78,8 @@ func ShsIf(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * has rather progn like behavior in that it returns the result of the last form to be evaluated
  *
  * Example:
- *     (export cond F)
- *     (while cond (export cond T) (print "will only be printed once") (+ 1 2))
+ *     (export cond T)
+ *     (while cond (export cond F) (print "will only be printed once") (+ 1 2))
  *     loop will iter one time, print "will only be printed once" and return 3
  */
 func ShsWhile(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {

stdlib/filesys.go

@@ -54,20 +54,6 @@ func AbsPath(arg string) string {
  *     (cd (concat HOME "/go"))
  */
 func Cd(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
-
-    if in == nil {
-        log.Log(log.ERR,
-            "arguments to cd evaluated to nil!",
-            "cd")
-        return nil
-    }
-
-    if in.Tag == ast.LIST {
-        log.Log(log.ERR, "Couldnt change dir to a list", "cd")
-        return nil
-    }
-
     err := os.Chdir(in.Value())
     if err != nil {
         log.Log(log.ERR, err.Error(), "cd")
@@ -84,14 +70,6 @@ func Cd(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  *    (fexists test)
  */
 func Fexists(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-    if in == nil || in.Tag != ast.STRING {
-        log.Log(log.ERR, 
-            "argument to fexists must be a string",
-            "fexists")
-        return nil
-    }
-
     filename := in.Value()
     out := ast.TRUE
 
@@ -115,7 +93,6 @@ func Fexists(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  *    (fread (concat HOME ".shsrc"))
  */
 func Fread(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
     exists := Fexists(in, vt, ft) // some waste, extra use of Eval
     if exists == nil || exists.Tag != ast.BOOL || exists.Value() == ast.FALSE {
         log.Log(log.ERR,
@@ -146,26 +123,12 @@ func Fread(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  *     (fwrite "test" "one two three")
  */
 func Fwrite(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-    if in == nil || in.Tag == ast.SYMBOL || in.Tag == ast.LIST {
-        log.Log(log.ERR,
-            "first argument must be a filename",
-            "fwrite")
-        return nil
-    }
-
     text := in.Next
-    if text == nil || text.Tag == ast.SYMBOL || text.Tag == ast.LIST {
-        log.Log(log.ERR,
-            "second argument must be stringable",
-            "fwrite")
-        return nil
-    }
-
     err := ioutil.WriteFile(
         AbsPath(in.Value()),
         []byte(text.Value()),
         0644)
+
     if err != nil {
         log.Log(log.ERR,
             "error writing file: " + err.Error(),
@@ -183,22 +146,7 @@ func Fwrite(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  *     (fwrite "test" "one two three")
  */
 func Fappend(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-    if in == nil || in.Tag == ast.SYMBOL || in.Tag == ast.LIST {
-        log.Log(log.ERR,
-            "first argument must be a filename",
-            "fappend")
-        return nil
-    }
-
     text := in.Next
-    if text == nil || text.Tag == ast.SYMBOL || text.Tag == ast.LIST {
-        log.Log(log.ERR,
-            "second argument must be stringable",
-            "fappend")
-        return nil
-    }
-
     exists := Fexists(in, vt, ft)
     if exists.Value() == ast.FALSE {
         log.Log(log.ERR,
@@ -212,14 +160,15 @@ func Fappend(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
         AbsPath(in.Value()),
         os.O_APPEND|os.O_CREATE|os.O_WRONLY,
         0644)
+
     if err != nil {
         log.Log(log.ERR,
             "couldnt open file for append: " + err.Error(),
             "fappend")
         return nil
     }
-    defer f.Close()
 
+    defer f.Close()
     if _, err := f.WriteString(text.Value()); err != nil {
         log.Log(log.ERR,
             "error appending to file: " + err.Error(),

stdlib/funcs.go

@@ -113,7 +113,8 @@ func DeclFunc(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Tok
         Function:    inner,
         Name: name.Value(),
         TimesCalled:     0,
-        Args:      numArgs,
+        NumArgs:   numArgs,
+        ArgLazy:     true,
     }
     return nil
 }

stdlib/list.go

@@ -18,6 +18,8 @@
 package stdlib
 
 import (
+    "fmt"
+    "strconv"
     "gitlab.com/whom/shs/ast"
     "gitlab.com/whom/shs/log"
 )
@@ -28,12 +30,7 @@ import (
  * in event a not-list is passed in, returns the arg.
  */
 func Expand(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
-    if input.Tag != ast.LIST {
-        log.Log(log.INFO, "expand called on not a list", "expand")
-        return input
-    }
-
-    return input.Eval(funcs, vars, false).Expand()
+    return input.Expand()
 }
 
 /* L_APPEND (append from repl)
@@ -68,3 +65,187 @@ func L_append(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Tok
 
     return src
 }
+
+/* Len
+ * Returns length of list or string as a number
+ * Returns nil if not a list or string
+ *
+ * Example: () -> 0
+ * Example: (1 2 3) -> 3
+ */
+func Len(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
+    if input.Tag != ast.LIST && input.Tag != ast.STRING {
+        log.Log(log.ERR,
+            "must provide list or strinig",
+            "head")
+        return nil
+    }
+
+    length := 0
+    if input.Tag == ast.STRING {
+        length = len(input.Value())
+
+    } else {
+        for iter := input.Expand(); iter != nil; iter = iter.Next {
+            length += 1
+        }
+    }
+
+    ret := &ast.Token{Tag: ast.NUMBER}
+    ret.Set(fmt.Sprintf("%d", length))
+    return ret
+}
+
+/* Takes two arguments
+ * An index and a list
+ * Returns element indexed by index arg
+ * can return nil if index is out of bounds
+ *
+ * Example (index 2 (1 2 3)) -> 2
+ */
+func Index(in *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
+    idx, err := strconv.ParseInt(in.Value(), 10, 64)
+    if err != nil || idx < 0 {
+        log.Log(log.ERR,
+            "index must be a positive integer: " + err.Error(),
+            "index")
+        return nil
+    }
+
+    series := in.Next.Expand()
+    iter := &series
+    i := int64(0)
+    for i <= idx {
+        if *iter == nil {
+            log.Log(log.ERR,
+                "index out of bounds",
+                "index")
+            return nil
+        }
+
+        if i < idx {
+            iter = &((*iter).Next)
+        }
+
+        i += 1
+    }
+
+    ret := (*iter).Copy()
+    ret.Next = nil
+    return ret
+}
+
+/* Head
+ * Returns first element in the list
+ * Returns nil if input is not a list or if list is empty
+ *
+ * Example: (head (2 3 4)) -> 2
+ */
+func Head(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
+    li := input.Expand().Copy()
+    if li != nil {
+        li.Next = nil
+    }
+
+    return li
+}
+
+/* Tail
+ * Returns last element in a list
+ * Returns nil if not a list
+ *
+ * Example: (tail (2 3 4)) -> 4
+ */
+func Tail(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
+    iter := input.Expand()
+    for iter != nil && iter.Next != nil {
+        iter = iter.Next
+    }
+
+    return iter.Copy()
+}
+
+/* Slice
+ * Takes 3 args and returns a list
+ * Arg 1: starting index of sublist
+ * Arg 2: end index of sublist
+ * Arg 3: source list
+ * returns sublist, or nil if non list applied, or nil if start or end arent INTEGERS
+ * first index in a list is 0
+ *
+ * Example: (slice 1 2 (1 2 3)) -> (2 3)
+ * Example: (slice 0 0 (1 2 3)) -> (1)
+ */
+func Slice(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token {
+    start := input
+    end := input.Next
+    source := end.Next
+
+    st, err := strconv.ParseInt(start.Value(), 10, 64)
+    en, errr := strconv.ParseInt(end.Value(), 10, 64)
+
+    if err != nil {
+        log.Log(log.ERR,
+            "couldnt parse integer from start value: " + err.Error(),
+            "slice")
+        return nil
+    }
+
+    if errr != nil {
+        log.Log(log.ERR,
+            "couldnt parse integer from end value: " + errr.Error(),
+            "slice")
+        return nil
+    }
+
+    if st < 0 || en < 0 {
+        log.Log(log.ERR,
+            "both indices must be positive",
+            "slice")
+        return nil
+    }
+
+    if st > en {
+        log.Log(log.ERR,
+            "end index must be greater than start index",
+            "slice")
+        return nil
+    }
+
+    en = en - st
+    var inner *ast.Token
+    buildIter := &inner
+    sourceIter := source.Expand()
+
+    for st > 0 {
+        if sourceIter == nil {
+            log.Log(log.ERR,
+                "start index out of bounds",
+                "slice")
+            return nil
+        }
+
+        sourceIter = sourceIter.Next
+        st -= 1
+    }
+
+    for en >= 0 {
+        if sourceIter == nil {
+            log.Log(log.ERR,
+                "end index out of bounds",
+                "slice")
+            return nil
+        }
+
+        *buildIter = sourceIter.Copy()
+        (*buildIter).Next = nil
+
+        buildIter = &((*buildIter).Next)
+        sourceIter = sourceIter.Next
+        en -= 1
+    }
+
+    ret := &ast.Token{Tag: ast.LIST}
+    ret.Direct(inner)
+    return ret
+}

stdlib/shell.go

@@ -225,8 +225,8 @@ func LaunchProcess(
  * Example (l vim file.txt)
  */
 func Call(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
     if in == nil {
+        log.Log(log.ERR, "no arguments given", "call")
         return nil
     }
 
@@ -261,11 +261,6 @@ func Call(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (bg vim file.txt)
  */
 func Bgcall(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
-    if in == nil {
-        return nil
-    }
-
     if in.Tag == ast.LIST {
         log.Log(log.ERR, "couldnt exec, target bin is a list", "call")
         return nil
@@ -304,14 +299,6 @@ func Fg(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
         return nil
     }
 
-    in = in.Eval(ft, vt, false)
-    if in.Tag != ast.NUMBER && in.Tag != ast.STRING {
-        log.Log(log.ERR,
-            "must supply a number or string to fg",
-            "fg")
-        return nil
-    }
-
     pid, err := strconv.ParseFloat(in.Value(), 64)
     if err != nil {
         log.Log(log.ERR,
@@ -374,12 +361,6 @@ func Jobs(in *ast.Token, vt ast.VarTable, fg ast.FuncTable) *ast.Token {
  * Example: ($ echo hello world)
  */
 func ReadCmd(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
-
-    if in == nil {
-        return nil
-    }
-
     if in.Tag == ast.LIST {
         log.Log(log.ERR, "couldnt exec, target bin is a list", "call")
         return nil
@@ -434,13 +415,6 @@ func GetExit(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * (this function not added to functable in stdlib.go)
  */
 func Kill(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
-
-    if in.Tag == ast.LIST {
-        log.Log(log.ERR, "non-number argument to kill function", "kill")
-        return nil
-    }
-
     pid, err := strconv.ParseInt(in.Value(), 10, 64)
     if err != nil {
         log.Log(log.ERR, "error parsing arg to kill: " + err.Error(), "kill")

stdlib/stdlib.go

@@ -20,7 +20,6 @@ package stdlib
 import (
     "os"
     "fmt"
-    "gitlab.com/whom/shs/log"
     "gitlab.com/whom/shs/ast"
     "gitlab.com/whom/shs/util"
 )
@@ -32,227 +31,334 @@ func GenFuncTable() ast.FuncTable {
     var stdlib ast.FuncTable
     stdlib = &map[string]*ast.Function{
         "if": &ast.Function{
-            Function:  ShsIf,
+            Function:   ShsIf,
             Name:       "if",
-            TimesCalled:   0,
-            Args:          3,
+            NumArgs:    3,
+            EvalLazy:   true,
+            ArgLazy:    true,
         },
 
         "while": &ast.Function{
-            Function:  ShsWhile,
+            Function:   ShsWhile,
             Name:       "while",
-            TimesCalled:      0,
-            Args:            -1,
+            NumArgs:    -1,
+            EvalLazy:   true,
+            ArgLazy:    true,
         },
 
         "progn": &ast.Function{
             Function:   ShsProgn,
-            Name:    "shs_progn",
-            TimesCalled:       0,
-            Args:             -1,
+            Name:       "shs_progn",
+            NumArgs:    -1,
+            EvalLazy:   true,
+            ArgLazy:    true,
         },
 
         "func": &ast.Function{
-            Function:  DeclFunc,
-            Name:   "decl_func",
-            TimesCalled:      0,
-            Args:             3,
+            Function:   DeclFunc,
+            Name:       "decl_func",
+            Args:   []ast.Token_t{
+                ast.SYMBOL,
+                ast.LIST,
+                ast.LIST,
+            },
+            EvalLazy:   true,
+        },
+
+        "len": &ast.Function{
+            Function:   Len,
+            Name:       "len",
+            NumArgs:    1,
+            ArgLazy:    true,
+        },
+
+        "index": &ast.Function{
+            Function:   Index,
+            Name:       "index",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.LIST,
+            },
+        },
+
+        "head": &ast.Function{
+            Function:   Head,
+            Name:       "head",
+            Args:   []ast.Token_t{
+                ast.LIST,
+            },
+        },
+
+        "tail": &ast.Function{
+            Function:   Tail,
+            Name:       "tail",
+            Args:   []ast.Token_t{
+                ast.LIST,
+            },
+        },
+
+        "slice": &ast.Function{
+            Function:   Slice,
+            Name:       "slice",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+                ast.LIST,
+            },
         },
 
         "export": &ast.Function{
-            Function: Export,
-            Name:   "export",
-            TimesCalled:   0,
-            Args:          2,
+            Function:   Export,
+            Name:       "export",
+            EvalLazy:   true,
+            ArgLazy:    true,
+            NumArgs:    2,
         },
 
         "input": &ast.Function{
-            Function:  Input,
-            Name:    "input",
-            TimesCalled:   0,
-            Args:          1,
+            Function:   Input,
+            Name:       "input",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "load": &ast.Function{
-            Function:    Load,
-            Name:      "load",
-            TimesCalled:    0,
-            Args:           1,
+            Function:   Load,
+            Name:       "load",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "bool": &ast.Function{
-            Function:  BoolCast,
-            Name:        "bool",
-            TimesCalled:      0,
-            Args:             1,
+            Function:   BoolCast,
+            Name:       "bool",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "string": &ast.Function{
-            Function:  StrCast,
-            Name:     "string",
-            TimesCalled:     0,
-            Args:            1,
+            Function:   StrCast,
+            Name:       "string",
+            NumArgs:    1,
+            ArgLazy:    true,
         },
 
         "number": &ast.Function{
-            Function:  NumCast,
-            Name:     "number",
-            TimesCalled:     0,
-            Args:            1,
+            Function:   NumCast,
+            Name:       "number",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
-       "...": &ast.Function{
-            Function: Expand,
-            Name:      "...",
-            TimesCalled:   0,
-            Args:          1,
+        "...": &ast.Function{
+            Function:   Expand,
+            Name:       "...",
+            Args:   []ast.Token_t{
+                ast.LIST,
+            },
         },
 
         "append": &ast.Function{
-            Function: L_append,
-            Name:     "append",
-            TimesCalled:     0,
-            Args:           -1,
+            Function:   L_append,
+            Name:       "append",
+            NumArgs:    -1,
+            ArgLazy:    true,
+       },
+
+        "join": &ast.Function{
+            Function:   Join,
+            Name:       "join",
+            Args:   []ast.Token_t{
+                ast.STRING,
+                ast.LIST,
+            },
+        },
+
+        "split": &ast.Function{
+            Function:   Split,
+            Name:       "split",
+            Args:   []ast.Token_t{
+                ast.STRING,
+                ast.STRING,
+            },
+        },
+
+        "replace": &ast.Function{
+            Function:   Replace,
+            Name:       "replace",
+            Args:   []ast.Token_t{
+                ast.STRING,
+                ast.STRING,
+                ast.STRING,
+            },
+        },
+
+        "substr": &ast.Function{
+            Function:   Substr,
+            Name:       "substr",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+                ast.STRING,
+            },
         },
 
         "exit": &ast.Function{
-            Function:  ExitShell,
-            Name:         "exit",
-            TimesCalled:       0,
-            Args:              0,
+            Function:   ExitShell,
+            Name:       "exit",
+            Args:   []ast.Token_t{},
         },
 
         "eq": &ast.Function{
-            Function:         Eq,
-            Name:           "==",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Eq,
+            Name:       "==",
+            NumArgs:       2,
+            ArgLazy:    true,
         },
 
         "ne": &ast.Function{
-            Function:         Ne,
-            Name:           "!=",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Ne,
+            Name:       "!=",
+            NumArgs:       2,
+            ArgLazy:    true,
         },
 
         "<": &ast.Function{
-            Function:         Lt,
-            Name:            "<",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Lt,
+            Name:       "<",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+            },
         },
 
         ">": &ast.Function{
-            Function:         Gt,
-            Name:            ">",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Gt,
+            Name:       ">",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+            },
         },
 
         "<=": &ast.Function{
-            Function:        Lte,
-            Name:           "<=",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Lte,
+            Name:       "<=",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+            },
         },
 
         ">=": &ast.Function{
-            Function:        Gte,
-            Name:           ">=",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Gte,
+            Name:       ">=",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+                ast.NUMBER,
+            },
         },
 
         "!": &ast.Function{
-            Function:        Not,
-            Name:            "!",
-            TimesCalled:       0,
-            Args:              1,
+            Function:   Not,
+            Name:       "!",
+            Args:   []ast.Token_t{
+                ast.BOOL,
+            },
         },
 
         "+": &ast.Function{
-            Function:      Add,
-            Name:        "add",
-            TimesCalled:     0,
-            Args:           -1,
+            Function:   Add,
+            Name:       "add",
+            NumArgs:    -1,
+            ArgLazy:    true,
         },
 
         "-": &ast.Function{
-            Function:      Sub,
-            Name:        "sub",
-            TimesCalled:     0,
-            Args:           -1,
+            Function:   Sub,
+            Name:       "sub",
+            NumArgs:    -1,
+            ArgLazy:    true,
         },
 
         "*": &ast.Function{
-            Function:     Mult,
+            Function:   Mult,
             Name:       "mult",
-            TimesCalled:     0,
-            Args:           -1,
+            NumArgs:    -1,
+            ArgLazy:    true,
         },
 
         "/": &ast.Function{
-            Function:      Div,
-            Name:        "div",
-            TimesCalled:     0,
-            Args:           -1,
+            Function:   Div,
+            Name:       "div",
+            NumArgs:    -1,
+            ArgLazy:    true,
         },
 
         "cd": &ast.Function{
-            Function:       Cd,
-            Name:  "changedir",
-            TimesCalled:     0,
-            Args:            1,
+            Function:   Cd,
+            Name:       "changedir",
+            SymLazy:    true,
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "concat": &ast.Function{
             Function:   Concat,
-            Name:"concatenate",
-            TimesCalled:     0,
-            Args:           -1,
+            Name:       "concatenate",
+            NumArgs:    -1,
+            ArgLazy:    true,
         },
 
         "print": &ast.Function{
-            Function: PrintStr,
-            Name:      "print",
-            TimesCalled:     0,
-            Args:            1,
+            Function:   PrintStr,
+            Name:       "print",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "l": &ast.Function{
-            Function:     Call,
+            Function:   Call,
             Name:       "call",
-            TimesCalled:     0,
-            Args:           -1,
+            NumArgs:    -1,
+            ArgLazy:    true,
+            SymLazy:    true,
         },
 
         "bg": &ast.Function{
-            Function:        Bgcall,
-            Name: "background call",
-            TimesCalled:          0,
-            Args:                -1,
+            Function:   Bgcall,
+            Name:       "background call",
+            NumArgs:    -1,
+            ArgLazy:    true,
+            SymLazy:    true,
         },
 
         "fg": &ast.Function{
-            Function:       Fg,
-            Name: "foreground",
-            TimesCalled:     0,
-            Args:            1,
+            Function:   Fg,
+            Name:       "foreground",
+            Args:   []ast.Token_t{
+                ast.NUMBER,
+            },
         },
 
         "$": &ast.Function{
-            Function:  ReadCmd,
-            Name:   "read cmd",
-            TimesCalled:     0,
-            Args:           -1,
+            Function:   ReadCmd,
+            Name:       "read cmd",
+            NumArgs:   -1,
+            ArgLazy:    true,
+            SymLazy:    true,
         },
 
         "?": &ast.Function{
-            Function:    GetExit,
-            Name:"get exit code",
-            TimesCalled:       0,
-            Args:              0,
+            Function:   GetExit,
+            Name:       "get exit code",
+            Args:   []ast.Token_t{},
         },
 
         /*
@@ -260,51 +366,56 @@ func GenFuncTable() ast.FuncTable {
         "kill": &ast.Function{
             Function:       kill,
             Name:     "kill job",
-            TimesCalled:       0,
             Args:              1,
         },
         */
 
         "jobs": &ast.Function{
-            Function:       Jobs,
-            Name:    "list jobs",
-            TimesCalled:       0,
-            Args:              0,
+            Function:   Jobs,
+            Name:       "list jobs",
+            Args:   []ast.Token_t{},
         },
 
         "info": &ast.Function{
-            Function:     ShInfo,
-            Name:   "Shell Info",
-            TimesCalled:       0,
-            Args:              1,
+            Function:   ShInfo,
+            Name:       "Shell Info",
+            Args:   []ast.Token_t{
+                ast.SYMBOL,
+            },
         },
 
         "fexists": &ast.Function{
-            Function:    Fexists,
-            Name:  "file exists",
-            TimesCalled:       0,
-            Args:              1,
+            Function:   Fexists,
+            Name:       "file exists",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "fread": &ast.Function{
-            Function:      Fread,
-            Name:    "read file",
-            TimesCalled:       0,
-            Args:              1,
+            Function:   Fread,
+            Name:       "read file",
+            Args:   []ast.Token_t{
+                ast.STRING,
+            },
         },
 
         "fwrite": &ast.Function{
-            Function:     Fwrite,
-            Name:   "write file",
-            TimesCalled:       0,
-            Args:              2,
+            Function:   Fwrite,
+            Name:       "write file",
+            Args:   []ast.Token_t{
+                ast.STRING,
+                ast.STRING,
+            },
         },
 
         "fappend": &ast.Function{
-            Function:     Fappend,
-            Name:"append to file",
-            TimesCalled:        0,
-            Args:               2,
+            Function:   Fappend,
+            Name:       "append to file",
+            Args:   []ast.Token_t{
+                ast.STRING,
+                ast.STRING,
+            },
         },
     }
 
@@ -327,32 +438,32 @@ func ExitShell(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (info append)
  */
 func ShInfo(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    switch in.Tag {
-    case ast.BOOL:
-        fmt.Printf("BOOL LITERAL\nValue: %s\n", in.Value())
-    case ast.STRING:
-        fmt.Printf("STRING LITERAL \nValue: %s\n", in.Value())
-    case ast.NUMBER:
-        fmt.Printf("NUMBER LITERAL \nValue: %s\n", in.Value())
-    case ast.LIST:
-        fmt.Printf("LIST \nString Value: %s, AST:\n", in.String())
-        ast.PrintSExprsIndividually(in)
-    case ast.SYMBOL:
-        repr := ast.GetVar(in.Value(), vt)
-        if repr != nil {
-            fmt.Printf("VARIABLE\nTYPE: %s\nVALUE: %s\n", ast.GetTagAsStr(repr.Tag), repr.Value())
-            break
-        }
-
-        funct := ast.GetFunction(in.Value(), ft)
-        if funct != nil {
-            fmt.Printf("FUNCTION\nNAME: %s\nTIMES CALLED: %s\nNUM ARGS: %d\n", funct.Name, funct.TimesCalled, funct.Args)
-            break
-        }
-
-        fmt.Printf("UNKNOWN SYMBOL\n")
+    repr := ast.GetVar(in.Value(), vt)
+    if repr != nil {
+        fmt.Printf("VARIABLE\nTYPE: %s\nVALUE: %s\n", ast.GetTagAsStr(repr.Tag), repr.Value())
+        return nil
     }
 
+    funct := ast.GetFunction(in.Value(), ft)
+    if funct != nil {
+        fmt.Printf("FUNCTION\nNAME: %s\nTIMES CALLED: %s\n\n", funct.Name, funct.TimesCalled)
+        if funct.ArgLazy {
+            fmt.Printf("function does not evaluate args by type\nARG COUNT: %s\n", funct.NumArgs)
+        } else {
+            fmt.Printf("function evaluates args by type\nARGS: ")
+            for _, i := range(funct.Args) {
+                fmt.Printf(ast.GetTagAsStr(i))
+            }
+        }
+
+        fmt.Printf("SYMLAZY: %t\n", funct.SymLazy)
+        fmt.Printf("(can undefined symbols be passed in)\n")
+        fmt.Printf("EVALLAZY: %t\n", funct.EvalLazy)
+        fmt.Printf("(are all forms in evaluated before function call)\n")
+        return nil
+    }
+
+    fmt.Printf("UNKNOWN SYMBOL\n")
     return nil
 }
 
@@ -364,14 +475,6 @@ func ShInfo(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (print (input))
  */
 func Input(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-    if in.Tag != ast.STRING && in.Tag != ast.NUMBER {
-        log.Log(log.ERR,
-            "argument to input must be a string or number",
-            "input")
-        return nil
-    }
-
     prompt := in.Value()
     var output string
 
@@ -389,14 +492,6 @@ func Input(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (load "myscript.shs")
  */
 func Load(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, true)
-    if in.Tag != ast.STRING {
-        log.Log(log.ERR,
-            "argument to load must be a string",
-            "load")
-        return nil
-    }
-
     bp := in.Value()
     bp = AbsPath(bp)
     util.LoadScript(bp, vt, ft)

stdlib/string.go

@@ -19,6 +19,7 @@ package stdlib
 
 import (
     "fmt"
+    "strings"
     "strconv"
     "gitlab.com/whom/shs/ast"
     "gitlab.com/whom/shs/log"
@@ -29,8 +30,6 @@ import (
  * Example: (concat "hello" " " "world")
  */
 func Concat(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    in = in.Eval(ft, vt, false)
-
     var res string
     for i := in; i != nil; i = i.Next {
         if i.Tag == ast.LIST {
@@ -53,12 +52,148 @@ func Concat(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (string 1) -> 1.0
  */
 func StrCast(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    body := in.Eval(ft, vt, false).String()
+    body := in.String()
     res := &ast.Token{ Tag: ast.STRING }
     res.Set(body)
     return res
 }
 
+/* Takes 2 arguments, a string and a delimiter
+ * returns a list of substrings found delimited by the delimiter from the parent string
+ * Filters out all empty segments between delimiters
+ *
+ * Example: (split "/path/to/file" "/") -> ("path" "to" "file")
+ */
+func Split(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
+    body := in.Value()
+    delim := in.Next.Value()
+
+    var res *ast.Token
+    builder := &res
+    strtoks := strings.Split(body, delim)
+    for _, i := range strtoks {
+        if i == "" {
+            continue
+        }
+
+        *builder = &ast.Token{Tag: ast.STRING}
+        (*builder).Set(i)
+        builder = &(*builder).Next
+    }
+
+    return res
+}
+
+/* Takes three args
+ * 1. source string
+ * 2. token to be replaced
+ * 3. token to swap in place
+ * All three args are strings
+ * Returns final string
+ */
+func Replace(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
+    body := in.Value()
+    sub := in.Next.Value()
+    tok := in.Next.Next.Value()
+
+    body = strings.ReplaceAll(body, sub, tok)
+
+    res := &ast.Token{Tag: ast.STRING}
+    res.Set(body)
+    return res
+}
+
+/* Takes two args, a delimiter and a list of strings
+ * Returns the list of strings concatenated together with the delimiter in between each element
+ * On error returns nil
+ *
+ * Example: (join ", " ("apple" "ananas" "pear")) -> "apple, ananas, pear"
+ */
+func Join(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
+    delim := in
+    strs := in.Next
+    de := delim.Value()
+    res := ""
+
+    for i := strs.Expand(); i != nil; i = i.Next {
+        if i.Tag != ast.STRING {
+            log.Log(log.ERR,
+                "all items to be joined must be strings",
+                "join")
+            return nil
+        }
+
+        res += i.Value()
+        if i.Next != nil {
+            res += de
+        }
+    }
+
+    ret := &ast.Token{Tag: ast.STRING}
+    ret.Set(res)
+    return ret
+}
+
+/* takes three arguments:
+ *   1. start index
+ *   2. end index
+ *   3. source
+ * Returns a substring from source delimited by args 1 and 2.
+ * First two args must be integers (4 or 4.0 but not 4.3)
+ *
+ * Example: (substr 1 5 "Linus Torvalds") -> "inus "
+ */
+func Substr(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
+    start := in
+    end := start.Next
+    str := end.Next
+
+    ed_idx := 0
+    st_idx, err := strconv.Atoi(start.Value())
+    ed_idx, err = strconv.Atoi(end.Value())
+    if err != nil {
+        log.Log(log.ERR,
+            "error parsing args: " + err.Error(),
+            "substr")
+        return nil
+    }
+
+    strlen := len(str.Value())
+    if st_idx < 0 {
+        st_idx += strlen
+    }
+
+    if ed_idx < 0 {
+        ed_idx += strlen
+    }
+
+    if st_idx < 0 || st_idx >= strlen {
+        log.Log(log.ERR,
+            "first index out of bounds",
+            "substr")
+        return nil
+    }
+
+    if ed_idx < 0 || ed_idx >= strlen {
+        log.Log(log.ERR,
+            "last index out of bounds",
+            "substr")
+        return nil
+    }
+
+    if st_idx > ed_idx {
+        log.Log(log.ERR,
+            "start must be less than end",
+            "substr")
+        return nil
+    }
+
+    res := str.Value()[st_idx:ed_idx]
+    ret := &ast.Token{Tag: ast.STRING}
+    ret.Set(res)
+    return ret
+}
+
 /* Takes one arg, returns nil
  * Prints a string to stdout
  * Unquotes string so user can add escaped chars like \n, \t, etc
@@ -66,7 +201,7 @@ func StrCast(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
  * Example: (print "Line: \n, Tab: \t")
  */
 func PrintStr(in *ast.Token, vt ast.VarTable, ft ast.FuncTable) *ast.Token {
-    body := in.Eval(ft, vt, false).String()
+    body := in.String()
     if body[0] != body[len(body)-1] && body[0] != '"' {
         body = "`" + body + "`"
     }

stdlib/vars.go

@@ -34,6 +34,12 @@ func Export(input *ast.Token, vars ast.VarTable, funcs ast.FuncTable) *ast.Token
     name := input
 
     form := name.Next.Eval(funcs, vars, false)
+
+    // error in eval process
+    if form == nil {
+        return nil
+    }
+
     if name.Tag != ast.SYMBOL {
         log.Log(log.ERR,
             "first arg should be a symbol",

util/shell_complete.go

@@ -80,7 +80,8 @@ func ShellCompleter(line string, vt ast.VarTable, ft ast.FuncTable) []string {
     completions := []string{}
     for _, i := range compSource {
         if strings.HasPrefix(i, tail) {
-            completions = append(completions, head + i)
+            str := strings.ReplaceAll(i, " ", "\\ ")
+            completions = append(completions, head + str)
         }
     }