Lexer Validation
There are additional restrictions that are enforced at compile-time apart from the ones covered previously
Duplicated Terminals
Having duplicated terminals is likely a mistake, because those terminals are interchangeable in the syntax tree. Likewise, you cannot have 2 terminals with no literals.
use teleparse::prelude::*; #[derive_lexicon] pub enum MyToken { #[teleparse(terminal(Zero = "0", Another = "0"))] Integer, } fn main() {}
error: Duplicate literal pattern `0` for variant `Integer`.
--> tests/ui/lex_no_dupe_literal.rs:4:48
|
4 | #[teleparse(terminal(Zero = "0", Another = "0"))]
| ^^^
Regex Features
Teleparse uses the logos
crate
for the lexer, which combines all the rules into a single
state machine for performance. Logos also imposes additional
restrictions on regex features that requires backtracking.
Please refer to their documentation for more information.