lexer_builder

A lexer generator for dart.

Features

  • Match tokens using RegExp syntax.
  • Rules dependent on lexer state.
  • Generates lexer code automatically.

Caveats

  • Generated lexers using regex are likely slower than handwritten ones.

Getting started

Include lexer_builder and build_runner as dev_dependencies and lexer_builder_runtime in your pubspec.yml.

Usage

Annotate a class with Lexer() and methods in it with Rule() to define a lexer. See the example for detailed instructions.

TODO

  • Eventually generate custom code for the rules instead of using RegExp internally.
  • Support async lexing with that by accurately measuring partly-matched subgroups, dispatching a rule if the next character doesn't make any match longer.

Additional information

For more information about lexers in general, see the flex lexer generator and its documentation.

Libraries

builder
The lexer generators. You shouldn't need to include this, running dart run build_runner build calls the builder.