Parser Grammar Engine

The Parser Grammar Engine (PGE, originally the Parrot Grammar Engine) is a compiler and runtime system for Raku rules for the Parrot virtual machine. PGE uses these rules to convert a parsing expression grammar into Parrot bytecode. It is therefore compiling rules into a program, unlike most virtual machines and runtimes, which store regular expressions in a secondary internal format that is then interpreted at runtime by a regular expression engine. The rules format used by PGE can express any regular expression and most formal grammars, and as such it forms the first link in the compiler chain for all of Parrot's front-end languages.

When executed, the bytecode generated by PGE will parse text as described in the input rules, generating a parse tree. The parse tree can be manipulated directly, or fed into the next stage of the Parrot compiler toolchain to generate an abstract syntax tree (AST) from which code can be generated; if the grammar describes a programming language.

History

Originally named P6GE and written in C, PGE was translated to native Parrot and renamed not long after its initial release in November 2004. Its author is Patrick R. Michaud. PGE was written to reduce the amount of work needed to implement a compiler on Parrot. It was also written to allow Perl 6 to easily self-host, though current Pugs development no longer uses PGE as its main rules back-end in favor of a native engine named PCR.

Internals

PGE combines three styles of parsing:

• Raku rules
• an operator-precedence parser
• custom parse subroutines

The primary form is Raku rules, so a PGE rule might look like this for an addition-only grammar:

 rule term   { <number> | \( <expr> \) }
 rule number { \d+ }
 rule expr   { <term> ( '+' <term> )* }

The operator precedence parser allows an operator table to be built and used directly in a Perl 6 rule style parser like so:

 rule expr is optable { ... }
 rule term   { <number> | \( <expr> \) }
 rule number { \d+ }
 proto term: is precedence('=')
             is parsed(&term) {...}
 proto infix:+ is looser('term:') {...}

This accomplishes the same goal of defining a simple, addition-only grammar, but does so using a combination of a Raku style regex/rules for term and number and a shift-reduce optable for everything else.

Code generation

Though PGE outputs code which will parse the grammar described by a rule, and can be used at runtime to handle simple grammars and regular expressions found in code, its main purpose is to parse high-level programming languages.

The Parrot compiler toolchain is broken into several parts, of which PGE is the first. PGE converts source code to parse trees. The tree grammar engine (TGE) then converts these into a Parrot abstract syntax trees (PAST). A second TGE pass then converts a PAST into Parrot opcode syntax trees (POST) which can be directly transformed into executable bytecode.

External links

• "Parsers, Perl 6 Rules, and the Parrot Grammar Engine". 2006-06-28.