jasy.script.tokenize package

Submodules

jasy.script.tokenize.Lang module

JavaScript 1.7 keywords

jasy.script.tokenize.Tokenizer module

exception jasy.script.tokenize.Tokenizer.ParseError(message, fileId, line)

Bases: Exception

class jasy.script.tokenize.Tokenizer.Token

Bases: object

assignOp

end

line

start

type

value

class jasy.script.tokenize.Tokenizer.Tokenizer(source, fileId='', line=1)

Bases: object

done()

get(scanOperand=False)

It consumes input only if there is no lookahead. Dispatches to the appropriate lexing function depending on the input.

getComments()

input_

lexDot(ch)

lexExponent()

lexIdent(ch)

lexNumber(ch)

lexOp(ch)

lexRegExp(ch)

lexString(ch)

lexZeroNumber(ch)

match(tokenType, scanOperand=False)

mustMatch(tokenType)

peek(scanOperand=False)

peekOnSameLine(scanOperand=False)

skip()

Eats comments and whitespace.

token

unget()

Match depends on unget returning undefined.

Module contents