The set of legal tokens of Core are: Reserved words (11 reserved words): program, begin, end,...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
The set of legal tokens of Core are: Reserved words (11 reserved words): program, begin, end, int, if, then, else, while, loop, read, write Special symbols (19 special symbols): ; , = ! [ ] && || ( ) + - * != == < > <= >= Integers (unsigned) Identifiers: start with uppercase letter, followed by zero or more uppercase letters and zero or more digits. Note that something like "ABCend" is illegal as an id because of the lowercase letters; and it is not two tokens because of lack of whitespace. But ABC123 and A1B2C3 are both legal. White space requirement: White space, i.e., one or more blanks or tab characters or carriage returns, is required between any pair of tokens unless one or both of them is/are special symbols. If one or both of them is/are special symbols. white space is optional. You should not treat white space as a regular token. We will number these tokens 1 through 11 for the reserved words, 12 through 30 for the special symbols, 31 for integer, and 32 for identifier. Note that 31 just tells us that the token in question is an integer, not the actual value of the integer. Similarly, 32 just tells us that the particular token is an identifier, not the name of the identifier. We will also use two additional numbers, 33 and 34, to indicate not actual tokens but situations that the Scanner has to deal with. We will use 33 to indicate that the Scanner is at the end-of-file so there are no more tokens. And 34 will indicate that the Scanner came across a string of characters in the input stream that is not a legal token, in other words, an error token. The Scanner's interface should consist of the following (public) methods: Constructor (): The Scanner's constructor should receive, as parameter, the name of the input file that contains the string of characters to be tokenized. The constructor should start by appropriately instantiating a buffered reader (if you are using Java) corresponding to that file in the usual manner. The constructor should then call a private method tokenizeLine(). This method will read a line from the input file and convert the sequence of characters in that line into a sequence of Core tokens and save them in a private data structure, possibly an array. It should then set a cursor index that points to the first token in that structure. It would be useful to include, in the same structure, the string of characters making up each token since you will need that for the case of integer tokens and identifier tokens. If a line that tokenizeLine() reads consists entirely of white space characters, it should skip that line and read the next line skipping as many lines as necessary to get to a non-empty line. If the line contains a string of characters that is not a valid token, tokenize Line() should still tokenize up to that point with the next number in its private data structure being 34 to indicate illegal token. tokenizeLine() also has to take account of greedy tokenizing. int getToken(): This returns a number between 1 and 32 if the current token is a proper Core token; 33 if the Scanner is at the end of the file and there are no more tokens; 34 if the Scanner comes across a string that is not a legal token. Note that getToken() does not move the "cursor" forward. So repeated calls to getToken() will return the same value. To move past the current token, the Parser has to call skipToken(). void skipToken(): This method moves the "token cursor" to the next token (but does not return anything). If there are no more tokens in the current line, skipToken() calls tokenizeLine() which will read the next line from the input file and convert the sequence of characters in that line into a sequence of Core tokens and save them in the Scanner's data structure and set the cursor index to point to the first token and return. ● If, when skipToken() is called, the current token is 33, i.e., we are already at the end-of-file, the cursor is not moved since there are no more lines or tokens to read. Similarly, if the current token is 34, i.e., we have an illegal token, the cursor is not moved since we do not go past an illlegal token. Your main() function should receive the name of the input file as a command line argument. It should then call the constructor of Scanner, passing that name as an argument.main() should then repeatedly call getToken(), print the returned token number, and call skipToken() until, after some number of iterations, getToken() returns 33 to indicate end-of-file or 34 to indicate invalid token. In either case, it should print an appropriate message and terminate. main() should print each token number on a separate line. Suppose the input file contains the following Core program: program int X; begin X=25; write X; end Your main() function should produce the following sequence of numbers: 14 32 12 2 32...33 corresponding to the tokens, "program", "int", "X", ";", "begin", "X", ..., EOF, but with each number.| The set of legal tokens of Core are: Reserved words (11 reserved words): program, begin, end, int, if, then, else, while, loop, read, write Special symbols (19 special symbols): ; , = ! [ ] && || ( ) + - * != == < > <= >= Integers (unsigned) Identifiers: start with uppercase letter, followed by zero or more uppercase letters and zero or more digits. Note that something like "ABCend" is illegal as an id because of the lowercase letters; and it is not two tokens because of lack of whitespace. But ABC123 and A1B2C3 are both legal. White space requirement: White space, i.e., one or more blanks or tab characters or carriage returns, is required between any pair of tokens unless one or both of them is/are special symbols. If one or both of them is/are special symbols. white space is optional. You should not treat white space as a regular token. We will number these tokens 1 through 11 for the reserved words, 12 through 30 for the special symbols, 31 for integer, and 32 for identifier. Note that 31 just tells us that the token in question is an integer, not the actual value of the integer. Similarly, 32 just tells us that the particular token is an identifier, not the name of the identifier. We will also use two additional numbers, 33 and 34, to indicate not actual tokens but situations that the Scanner has to deal with. We will use 33 to indicate that the Scanner is at the end-of-file so there are no more tokens. And 34 will indicate that the Scanner came across a string of characters in the input stream that is not a legal token, in other words, an error token. The Scanner's interface should consist of the following (public) methods: Constructor (): The Scanner's constructor should receive, as parameter, the name of the input file that contains the string of characters to be tokenized. The constructor should start by appropriately instantiating a buffered reader (if you are using Java) corresponding to that file in the usual manner. The constructor should then call a private method tokenizeLine(). This method will read a line from the input file and convert the sequence of characters in that line into a sequence of Core tokens and save them in a private data structure, possibly an array. It should then set a cursor index that points to the first token in that structure. It would be useful to include, in the same structure, the string of characters making up each token since you will need that for the case of integer tokens and identifier tokens. If a line that tokenizeLine() reads consists entirely of white space characters, it should skip that line and read the next line skipping as many lines as necessary to get to a non-empty line. If the line contains a string of characters that is not a valid token, tokenize Line() should still tokenize up to that point with the next number in its private data structure being 34 to indicate illegal token. tokenizeLine() also has to take account of greedy tokenizing. int getToken(): This returns a number between 1 and 32 if the current token is a proper Core token; 33 if the Scanner is at the end of the file and there are no more tokens; 34 if the Scanner comes across a string that is not a legal token. Note that getToken() does not move the "cursor" forward. So repeated calls to getToken() will return the same value. To move past the current token, the Parser has to call skipToken(). void skipToken(): This method moves the "token cursor" to the next token (but does not return anything). If there are no more tokens in the current line, skipToken() calls tokenizeLine() which will read the next line from the input file and convert the sequence of characters in that line into a sequence of Core tokens and save them in the Scanner's data structure and set the cursor index to point to the first token and return. ● If, when skipToken() is called, the current token is 33, i.e., we are already at the end-of-file, the cursor is not moved since there are no more lines or tokens to read. Similarly, if the current token is 34, i.e., we have an illegal token, the cursor is not moved since we do not go past an illlegal token. Your main() function should receive the name of the input file as a command line argument. It should then call the constructor of Scanner, passing that name as an argument.main() should then repeatedly call getToken(), print the returned token number, and call skipToken() until, after some number of iterations, getToken() returns 33 to indicate end-of-file or 34 to indicate invalid token. In either case, it should print an appropriate message and terminate. main() should print each token number on a separate line. Suppose the input file contains the following Core program: program int X; begin X=25; write X; end Your main() function should produce the following sequence of numbers: 14 32 12 2 32...33 corresponding to the tokens, "program", "int", "X", ";", "begin", "X", ..., EOF, but with each number.|
Expert Answer:
Answer rating: 100% (QA)
The legal tokens of Core can be classified into the following categories Reserved wordsThese are words that have a special meaning to the Core compile... View the full answer
Related Book For
Building Java Programs A Back To Basics Approach
ISBN: 9780135471944
5th Edition
Authors: Stuart Reges, Marty Stepp
Posted Date:
Students also viewed these programming questions
-
1) If given the choice, which situation would you choose for a team of persons to lead, and why? a) All team members are star performers, who consistently perform above expectations b) Less than 50%...
-
Identify some impacts on the financial services industry caused by changes in general economic conditions. (Identify the economic and political climate relating to the financial services industry).
-
The Crazy Eddie fraud may appear smaller and gentler than the massive billion-dollar frauds exposed in recent times, such as Bernie Madoffs Ponzi scheme, frauds in the subprime mortgage market, the...
-
As shown in Fig. 4.60, a horizontal beam is hinged to the wall at point A. The length of the beam is 1 = 2 m and it weighs W = 150 N. Point C is the center of gravity of the beam and it is...
-
Calculate the final pressure of a sample of water vapour that expands reversibly and adiabatically from 87.3 Torr and 500 dm3 to a final volume of 3.0 dm3. Take Y = 1.3
-
Give two reasons why KH(IO3)2 is preferred over benzoic acid as a primary standard for a 0.010 M NaOH solution.
-
Which of the following statements about neural networks is false? a. It can be very easy to overfit a neural network due to number of weights to be estimated and the number of ways these weights are...
-
The vice president of operations if Avigi IQ Company is evaluating the performance of two division organized as investment centers. Invested assets and condensed income statement data for the past...
-
A company manufactures a single product. Budget and standard cost details for next year include: Selling price per unit RM100.00 Variable production cost per unit RM44.50 Fixed production costs...
-
What are the two elements that an employee must show to make a successful Title VII claim? Do you think the motivating factor standard is appropriate for discrimination claims based on national...
-
Given the following information, what is the initial net cash outflow? Purchase Price of the new machine Rs . 8 0 0 0 0 Shipping and Installation Charges Rs . 2 0 0 0 0 Sale Price of Old Machine Rs ....
-
Suppose that you flip 50 "honest" coins. (a) How many microstates are there? Give your answer as a factor of the order of unity times an integral power of 10. (b) How many microstates are there...
-
Second Rate Ballroom, Inc, a national convention planner has 1,500 common shares and 2,000 bonds outstanding. Its securities have been performing very well in the marketplace. It has a common stock...
-
You want to buy a $33,000 car. The company is offering a 5% interest rate for 5 years. What will your monthly payments be? Feel free to use the Online Basic Financial Calculator $
-
Alison is a risk-averse investor. He has $10,000 and he will invest in Share A or Share B. Payoffs and probabilities are shown below: Share A High future profitability Low future profitability (0.70)...
-
Calculate the cell potential and the equilibrium constant for the following reaction at 298 K: Sn2+(aq) + Cu(s) >> Sn(s) + Cu+(aq) Hint: Carry at least 5 significant figures during intermediate...
-
Identify the motor being used in the ceiling fan and explain the method of its control.
-
What are conversion costs? What are prime costs?
-
Consider the following array of int elements: int[] numbers = {7, 2, 8, 4, 1, 11, 9, 5, 3, 10}; a. Show the state of the elements after five passes of the outermost loop of selection sort have...
-
Javas type int has a limit on how large an integer it can store. This limit can be circumvented by representing an integer as an array of digits. Write an interactive program that adds two integers...
-
Write a program called FarewellGoodBye that prints the following lyrics. Use static methods to show structure and eliminate redundancy in your solution. Farewell, goodbye, au revoir, good night! It's...
-
The responses most likely to be associated with use of a force-coercion change strategy are best described as __________. (a) internalized commitment (b) temporary compliance (c) passive cooptation...
-
The assessment center approach to employee selection relies heavily on ____________ to evaluate a candidates job skills. (a) intelligence tests (b) simulations and experiential exercises (c) 360...
-
Which of the following questions can an interviewer legally ask a job candidate during a telephone interview? (a) Are you pregnant or planning to soon start a family? (b) What skills do you have that...
Study smarter with the SolutionInn App