lexical analyzer

views updated

lexical analyzer (scanner) The part of a compiler that breaks up the input into meaningful units, e.g. names, constants, reserved words, operators. The lexical analyzer will also remove redundant characters, e.g. spaces, and may deal with character-set mappings, e.g. replacing upper-case letters by the equivalent lower-case letters. The units recognized by the lexical analyzer are called tokens, and are output in some conveniently coded form for subsequent processing by the compiler.

About this article

lexical analyzer

Updated About encyclopedia.com content Print Article Share Article