Design and implement a scanner for a programming language whose lexical specifications are given below. The scanner identifies and outputs tokens $($valid words and punctuation$)$ in the source program. Its output is a token that can thereafter be used by the syntax analyzer to verify that the program is syntactically correct.

When invoked, the scanner should extract the next token from the source program and should be able to output a token even if the input does not form a correct program. The syntax of the language will be specified later in Part $2.$

Atomic lexical elements of the language:

id ::$=$ letter alphanum$*$

alphanum ::$=$ letter $|$ digit $|\_$

integer ::$=$ nonzero digit$*|0$

float ::$=$ integer fraction $[$e$[+|]$ integer$]$

fraction ::$=.$digit$*$ nonzero $|\mathrm{.}0$

letter ::$=$ a$..$z $|$A$..$Z

digit ::$=0.\mathrm{.}9$

nonzero ::$=1.\mathrm{.}9$

Operators, punctuation and reserved words:

$==+($ if

$<>-)$ then

$<*\{$ else

$>/\}$ for

$<==[$ class

$>=$ && $]$ integer

; $!/*$ float

$,||*/$ read

$.//$ write

: return

:: main

Notes:

$$ You are responsible for providing appropriate test cases that test for a wide variety of valid and invalid cases.

$$ It is up to you to analyze this lexical definition and figure out if there are ambiguities or other errors. Any changes to this definition, if any, must be justified and should not result in diminishing the expressive power of the language. Also, you have to design the lexical analyzer in a flexible way, so that it can easily be adapted if changes are needed during the designing of syntax analyzer.

$$ The tokens $//$ and $/*$ followed $*/$ by denote comments in the source code

Deliverables:

Document

$$ Lexical specifications: Identify the lexical specification you used in the design of the scanner, as well as any changes that you may have applied to original lexical specifications.

$$ Finite state automaton: Finite state machine describing the operation of your scanner.

$$ Error reporting and recovery: Identify all the possible lexical errors that the scanner can encounter. Identify and explain the error recovery technique that you implement.

$$ Design: Give a brief overview of the overall structure of your solution, as well as a brief description of the role of each component of your implementation.

$$ Implementation Tools: Identify all the tools$/$libraries$/$techniques that you have used in your analysis or implementation and justify why you have used these particular ones as opposed to others.

Implementation

$1.$ Scanner:

$1\mathrm{.}1.$ Develop a scanner that recognizes the above$-$mentioned tokens.

$1\mathrm{.}2.$ It should be a function that returns a data structure containing the information about the next token identified in the source program file.

$1\mathrm{.}3.$ The data structure should contain information such as $(1)$ the token type $(2)$ its value $($or lexeme$)$ when applicable and $(3)$ its location in the source code. Fully describe this structure in your documentation.

$2.$ Driver:

$2\mathrm{.}1.$ Include a driver that repeatedly calls the lexical analysis function and prints the token type of each token until the end of the source program is reached.

$2\mathrm{.}2.$ The scanner should optionally print the token stream to a file in the AtoCC format.

$2\mathrm{.}3.$ Another file $($for verification purposes$)$ should contain representative error messages each time an error is encountered in the input program.

$2\mathrm{.}4.$ The scanner should not stop after encountering an error. Error messages should be clear and should identify the location of the error in the source code.

$3.$ Test cases:

$3\mathrm{.}1.$ Include many test cases that test a wide variety of valid and invalid cases.

Selected examples:

$0123[$Invalid number:$0123]$ OR $[$integer:$0][$integer:$123]$

$01\mathrm{.}23[$Invalid number:$01\mathrm{.}23]$ OR $[$integer:$0][$float:$1\mathrm{.}23]$

$12\mathrm{.}340[$Invalid number:$12\mathrm{.}340]$ OR $[$float:$12\mathrm{.}34][$integer:$0]$

$12345\mathrm{.}6789$e$-123[$float:$12345\mathrm{.}6789$e$-123]$

$12345[$integer:$12345]$

abc $[$id:abc$]$

abc$\_1[$id:abc$\_1]$

$\_$abc$1[$Invalid identifier:$\_$abc$1]$

$1$abc $[$Invalid identifier:$1$abc$]$ OR $[$integer:$1][$id:abc$]$