Lexical Analyzer
1. Process the input character that constitute a high level program in to valid set of token
2.It skips the comment and white space while creating these tokens.
3.if any erroneous input is provided by the user in the program
Lexical analyser correlate that errors with source file and line number
Terminology used in Lexical Analysis.
1. Token: A set of input string which are related through a similar pattern
Any word that starts with an alphabet and can contain any number or alphabet in between is called an identifier.
Identifier is called a token: raghu , raghu123
2. Lexme: The actual input string which represent the token.
Identifier-->token
raghu,raghu123-->Lexme
3. Pattern: Rule which a Lexical analyser follow to create a token
Consider this expression
sum=3+2;
The main task of lexical Analyzer is to read a stream of characters as an input and produce a sequence of tokens such as names,keywords,punctuation marks etc. It discards the white space and comments between the tokens and also keep track of line numbers. It read from left-to-right and grouped into tokens.Tokens are sequences of characters with a collective meaning.The lexical analyzer takes a source program as input, and produces a stream of tokens as output.
previous topic next topic
1. Process the input character that constitute a high level program in to valid set of token
2.It skips the comment and white space while creating these tokens.
3.if any erroneous input is provided by the user in the program
Lexical analyser correlate that errors with source file and line number
Terminology used in Lexical Analysis.
1. Token: A set of input string which are related through a similar pattern
Any word that starts with an alphabet and can contain any number or alphabet in between is called an identifier.
Identifier is called a token: raghu , raghu123
2. Lexme: The actual input string which represent the token.
Identifier-->token
raghu,raghu123-->Lexme
3. Pattern: Rule which a Lexical analyser follow to create a token
Consider this expression
sum=3+2;
The main task of lexical Analyzer is to read a stream of characters as an input and produce a sequence of tokens such as names,keywords,punctuation marks etc. It discards the white space and comments between the tokens and also keep track of line numbers. It read from left-to-right and grouped into tokens.Tokens are sequences of characters with a collective meaning.The lexical analyzer takes a source program as input, and produces a stream of tokens as output.
previous topic next topic