Basic lexical analyzer for parsing and compiling.