Question: I need full java code for this desciption. In this project, we will begin our lexer. Our lexer will start by reading the strings of

I need full java code for this desciption. In this project, we will begin our lexer. Our lexer will start by reading the strings of the BASIC file that the user wants to run. It will break the BASIC code up into words or tokens and build a collection of these tokens. We can consider the lexer complete when it can take any BASIC file and output a list of the tokens being generated.We will not be using the Scanner class that you may be familiar with for reading from a file; we are, instead, using Files.readAllBytes. This is a much simpler way of dealing with files.Example of readAllLines:Path myPath = Paths.get(someFile.basic);String content = new String(Files.readAllBytes (myPath);A second concept you may not be familiar with is enum. Enum, short for enumeration. This is a Java language construct that lets us create a variable that may be any one of a list of things. We will use this to define the types of tokens a tokenType.enum colorType { RED, GREEN, BLUE }colorType myFavorite = colorType.BLUE;System.out.println(myFavorite); // prints BLUEThe general way to think about a lexer is the same way that you learned to read. Remember following the line of letters with your finger? We will do the same thing we will create a class (I called mine CodeHandler). This class will manage the incoming stream of letters, allowing the lexer to peek ahead in the stream, get a character (move the finger one forward), and tell us if we are at the end of the document.With the CodeHandler written, we will move on to the lexer itself. For now, we will only deal with a few types of tokens words, numbers and new line. Of course, just reading through the document doesnt do anything we will make tokens objects that hold the type (using an enum) and a string of the value. For example:WORD(hello)DetailsThis assignment must have four different source code files. Basic.Java Basic.java must contain main. Your main must ensure that there is one and only one argument (args). If there are none or more than 1, it must print an appropriate error message and exit. That one argument will be considered as a filename. You will pass the filename to an instance of Lexer (which we will build below). You will then call lex() on your lexer. It will return a linked list of Token. Print each token out. This is just debugging output the format isnt super important so long as it makes sense to a programmer. CodeHandlerThe code handler class should have a private string to hold the document (BASIC file) and a private integer index (the finger position). It should have methods:char Peek(i)-looks i characters ahead and returns that character; doesnt move the indexString PeekString(i) returns a string of the next i characters but doesnt move the indexchar GetChar() returns the next character and moves the indexvoid Swallow(i) moves the index ahead i positionsboolean IsDone() returns true if we are at the end of the documentString Remainder() returns the rest of the document as a stringSome of these are used in next assignment. Note that there is no accessor for the string array. You must use the methods.The constructor will take a filename. Basic will call Lexer which will call CodeHandler. Use ReadAllBytes to read from the file and populate the private internal string.TokenCreate the Token class. It needs an enum of TokenType (values: WORD, NUMBER, ENDOFLINE) and a string to hold the value of the token (for example, hello and goodbye are both WORD, but with different values. The token will also hold the line number and character position of the start of the token. Create two constructors one for TokenType, line number and position and one that also has a value; some tokens dont have a value because it is doesnt matter (new line). Make sure to add a ToString method. Your exact format isnt critical; I output the token type and the value in parentheses if it is set.LexerThe final file must be called Lexer.java. The Lexer class must contain a lex method that accepts a single string (the filename) and returns a linked list of Tokens. Your lexer must keep track of the line number and character position within the line. This will help you generate good error messages.Lex is the method that will break the data from CodeHandler into a linked list of tokens. While there is still data in CodeHandler, we want to peek at the next character to get an idea what to do with it. If the character is a space or tab, we will just move past it (increment position). If the character is a linefeed (
), we will create a new EndOfLine token with no value and add it to token list. We should also increment the line number and set line position to 0.If the character is a carriage return (\r), we will ignore it.If the character is a letter, we need to call ProcessWord (see below) and add the result to our list of tokens.If the character is a digit, we need to call ProcessDigit (see below) and add the result to our list of tokens.Throw an exception if you encounter a character you dont recognize.ProcessWord is a method that returns a Token. It accepts letters, digits and underscores (_) and make a String of them. When it encounters a character that is NOT one of those, it stops and makes a WORD token, setting the value to be the String it has accumulated. Make sure that you are using peek so that whatever character does NOT belong to the WORD stays in the StringHandler class. Also remember to increment the position value. Words can also end with $ and %.ProcessNumber is similar to ProcessWord, but accepts 0-9 and one .. Create Junit tests for your lexer and for CodeHandler.Test with multi-line strings. Test with words then numbers and numbers and then words. Some examples of valid input and the result output are:InputOutputan empty lineEndOfLine5 helloNUMBER (5) WORD(hello) EndOfLine5.238.53NUMBER(5.23) NUMBER(8.5) NUMBER(3) EndOfLine8499999NUMBER (8) NUMBER (4) NUMBER (99999) EndOfLine7431NUMBER(7) NUMBER(4) NUMBER(3) NUMBER(1) EndOfLine2 number 3NUMBER(2) WORD (number) NUMBER (3) EndOfLineHINTSDo not wait until the assignment is nearly due to begin. Start early so that you can ask questions.You may find it useful to make some private helper methods in your lexer. This is encouraged.Read the rubric carefully.Be careful about good OOP very little in this assignment should be static. Members should always be private.

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!