lexer

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer is the engine to process input and emit Tokens

func New(input string) *Lexer

func (l *Lexer) HasNext() bool

func (l *Lexer) NextToken() token.Token

type StateFn ¶

type StateFn func(*Lexer) StateFn

StateFn represents a function which is capable of lexing parts of the input. It returns another StateFn to proceed with.

Typically a state function would get called from LexStartFn and should return LexStartFn to go back to the decision loop. It also could return another non start state function if the partial input to parse is abiguous.