**Table of Contents** - [Code Product Library](#code-product-library) - [The Product Syntax](#the-product-syntax) - [Product Scopes](#product-scopes) - [Linear Scopes](#linear-scopes) - [Definitions](#definitions) - [References](#references) - [Escaping the $ character](#escaping-the--character) - [Examples](#examples) - [Preprocessor like Macro Replacement](#preprocessor-like-macro-replacement) - [EBNF Syntax](#ebnf-syntax) # Code Product Library This library implements an engine for expanding one `TokenTree` into another `TokenTree` by generating code productions. # The Product Syntax The product syntax contains three elements described in detail below: 1. Product scopes for product definitions using curly braces: ```text ${ ... } ``` 2. Linear scopes that expand all defintions sequentially instead generating the product: ```text $[ ... ] ``` 3. The definitions using parentheses: ```text $( ... ) ``` 4. References, either indexed or named: ```text $0 $foo ``` Since we use Rust `TokenTrees`, the only requirement is that everything tokenizes. This means literals must be properly formatted, and all opening brackets must be closed. All syntactic elements relevant to the product syntax start with the pound sign `$`. ## Product Scopes Allow local product definitions in `$` followed by braces. These scopes are evaluated inside-out and expand the product of all definitions; named references that are not local will be forwarded to the parent scope. ```text ${ $(name: (one)(two)) $(number: (1)(2)) $name:$number ; } ``` expands to: ```text one:1 ; one:2 ; two:1 ; two:2 ; ``` ## Linear Scopes Allow local definitions in `$` followed by brackets. All definitions in a linear scope must have the same number of elements. These scopes are evaluated inside-out and expand the elements of all definitions *together* instead their product; named references that are not local will be forwarded to the parent scope. ```text $[ $(name: (one)(two)) $(number: (1)(2)) $name:$number ; ] ``` expands to: ```text one:1 ; two:2 ; ``` ## Definitions A list of nested parentheses. The inner parentheses contain the produced Rust tokens. A product definition can be named or unnamed, hidden or visible. This leads to the three main forms (unnamed-hidden is not supported). For named definitions with only one item, the parentheses can be omitted. Names can be used to reference the product at multiple places. This works for names from outer scopes as well. Unnamed definitions can only be referenced by their numeric position in the product list; these will be local and not reference outer scopes. Hidden definitions will not be expanded at the place of their definition but can be referenced later. This is useful for defining a production first and using it multiple times. Visible definitions will be expanded at the place of their definition and can be referenced later as well. - `$((a)(b))` - unnamed, visible: Defining an unnamed product will implicitly leave a numeric reference to it in place. Thus, it will be expanded at the place of its definition. This is especially useful for defining a production that is used only once. - `$($name:(a)(b))` - named, visible: Defining a named production with a `$` in front of its name will leave a reference to it in place. Thus, it will be expanded at the place of its definition. - `$(name:(a)(b))` - named, hidden: Defining a named production without the `$` in front of its name will leave no reference to it. Thus, it will not be expanded at the place of its definition. This can be used when one wants to define a set of productions at the start of a scope which later become referenced. - `$($name:item)` and `$(name:item)` - special form for single items: Single named non-parenthesized items can be written without the surrounding parentheses. Note: When the item needs to be parenthesized, one needs to put it in double parentheses. The number of the product definitions starts with one; named definitions are included in the numbering. **Note 1:** Productions can only contain Rust tokens, not product definitions or references. This is intentionally chosen for simplicity for now. **Note 2:** Code expansion is the product of all defined products. Hidden products that are not referenced will still create the same code multiple times. This also applies to empty definitions: ```text $(unused: (1)()(3)) $(used: (a)(b)) foo($used) ``` will expand to: ```text foo(a) foo(a) foo(a) foo(b) foo(b) foo(b) ``` ## References References are used to reference an earlier defined production. They are written as `$` followed by a number or the name of a named product definition. When named, this can refer to a name from an outer scope. Numbered references are only valid in the scope of the product definition they are defined; indexing starts at zero. Unresolved references will lead to a compile error. Everything has to be defined before being used. ## Escaping the $ character The `$` character is used to start product syntax elements. To use a literal `$` character, just double it. # Examples The following three examples will all expand to: ```text impl Trait for MyType {} impl Trait for MyType {} impl Trait for MyType {} ``` 1. Using unnamed product definitions: ```text impl Trait<$((Foo)(Bar)(Baz))> for MyType<$1> {} ``` 2. Using a named, hidden product definition: ```text $(T: (Foo)(Bar)(Baz)) impl Trait<$T> for MyType<$T> {} ``` 3. Using a named, visible product definition: ```text impl Trait<$($T: (Foo)(Bar)(Baz))> for MyType<$T> {} ``` # Preprocessor like Macro Replacement When product definitions contain only one item, then code product expansion acts like a simple preprocessor, substituting defined products. Parentheses become optional then. For example: ```text $(T: Foo) $(U: Bar) impl Trait<$T> for MyType<$U> {} ``` will expand to: ```text impl Trait for MyType {} ``` # EBNF Syntax Here is an informal code product syntax in EBNF for reference. The actual implementation differs slightly depending on what `ScopeMode` is chosen. ```ebnf start = { product_entity | rust } ; rust = ?any_valid_rust_token? ; product_entity = "$" ( scope | definition | reference ) ; scope = product_scope | linear_scope ; product_scope = "{" { product_entity | rust } "}" ; linear_scope = "[" { product_entity | rust } "]" ; definition = "(" [ "$" name ":" ] { product } ")" ; product = "(" { rust } ")" ; name = ?identifier? ; reference = "#" ( ?number? | ?identifier? ) ; ```