spudplate::Lexer Class Reference

Tokenises a spudlang source string into a stream of Token values. More...

#include <lexer.h>

Public Member Functions
	Lexer (std::string source)
	Constructs a Lexer over the given source string.
Token	nextToken ()
	Returns the next Token from the source, advancing the position.

Detailed Description

Tokenises a spudlang source string into a stream of Token values.

Call nextToken() repeatedly to consume tokens one at a time. The lexer returns an EOF_TOKEN once the entire input has been consumed, and an ERROR token for any unrecognised character.