Options
All
  • Public
  • Public/Protected
  • All
Menu

Splits an input string into lexical tokens, i.e. smaller strings that are easily identifiable by tokens.tokenType().

Lexing starts always in a "stream" context. Incomplete input may be buffered until a complete token can be emitted.

In addition to slices of the original input, the following control characters may also be emitted:

  • \x02 (Start of Text): A document starts with the next token
  • \x18 (Cancel): Unexpected end of flow-mode (indicates an error)
  • \x1f (Unit Separator): Next token is a scalar value
  • \u{FEFF} (Byte order mark): Emitted separately outside documents

Hierarchy

  • Lexer

Index

Constructors

constructor

Properties

Private atEnd

atEnd: any

Flag indicating whether the end of the current buffer marks the end of all input

Private atLineEnd

atLineEnd: any

Private blockScalarIndent

blockScalarIndent: any

Explicit indent set in block scalar header, as an offset from the current minimum indent, so e.g. set to 1 from a header |2+. Set to -1 if not explicitly set.

Private blockScalarKeep

blockScalarKeep: any

Block scalars that include a + (keep) chomping indicator in their header include trailing empty lines, which are otherwise excluded from the scalar's contents.

Private buffer

buffer: any

Current input

Private charAt

charAt: any

Private continueScalar

continueScalar: any

Private flowKey

flowKey: any

Flag noting whether the map value indicator : can immediately follow this node within a flow context.

Private flowLevel

flowLevel: any

Count of surrounding flow collection levels.

Private getLine

getLine: any

Private hasChars

hasChars: any

Private indentNext

indentNext: any

Minimum level of indentation required for next lines to be parsed as a part of the current scalar value.

Private indentValue

indentValue: any

Indentation level of the current line.

Private lineEndPos

lineEndPos: any

Position of the next \n character.

Private next

next: any

Stores the state of the lexer if reaching the end of incpomplete input

Private parseBlockScalar

parseBlockScalar: any

Private parseBlockScalarHeader

parseBlockScalarHeader: any

Private parseBlockStart

parseBlockStart: any

Private parseDocument

parseDocument: any

Private parseFlowCollection

parseFlowCollection: any

Private parseLineStart

parseLineStart: any

Private parseNext

parseNext: any

Private parsePlainScalar

parsePlainScalar: any

Private parseQuotedScalar

parseQuotedScalar: any

Private parseStream

parseStream: any

Private peek

peek: any

Private pos

pos: any

A pointer to buffer; the current position of the lexer.

Private pushCount

pushCount: any

Private pushIndicators

pushIndicators: any

Private pushNewline

pushNewline: any

Private pushSpaces

pushSpaces: any

Private pushTag

pushTag: any

Private pushToIndex

pushToIndex: any

Private pushUntil

pushUntil: any

Private setNext

setNext: any

Methods

lex

  • lex(source: string, incomplete?: boolean): Generator<string, void, unknown>
  • Generate YAML tokens from the source string. If incomplete, a part of the last line may be left as a buffer for the next call.

    Parameters

    • source: string
    • Optional incomplete: boolean

    Returns Generator<string, void, unknown>

    A generator of lexical tokens

Generated using TypeDoc