# Parser for the scripting language of ion shell

This provides a parse for scripting language of [ion shell]

## Inner working of the parsing 

### Phase Tokenisation

Scripts are tokenized before the actual parsing.
It determines all the special keywords, builtin commands, operators, assignment operators 
and words between the noise. Noises are white spaces, new lines or tabs.
The tokeniser returns those noises as tokens too. 
These noises tokens are useful for formatting files. 
It also important for the syntactic of the script language of ion shell.
Arguments within calls of commands and values in square brackets are separated by white spaces for example. 
Every token also has the location where it starts, line and column count.
This property is needed by the LSP server to communicate with the client properly.  
See [this file][token_file] for the rules of the tokenisation.

### Phase Parsing

The parsing is concerned about the syntax of a script.
Parsing solves the following problems:

- Validate the correct order of tokens.

The formal grammar is documented under [this file][grammar_parser_file]

**Note** 

The parser detects if a script file sources another one. However it does not load the files 
automatically. It is up to the user to get the sourced file. However the parser 
as an iterator returns a marker item to tell the user 
if and which file is sourced.

## Changelog

See the [changelogs](./CHANGELOG.md)

## Contributing

See the following [guide lines][contributing] 

## License 

This project, all workspace members and the visual code extension are licensed under [MIT License][LICENSE]

[License]: LICENSE
[contributing]: ../CONTRIBUTING.md 
[token_file]: rules_for_tokens.pest
[grammar_parser_file]: grammar_for_syntax.txt
[insta]: https://docs.rs/insta/latest/insta/
[insta_cli_tool]: https://crates.io/crates/cargo-insta