Question: With the lexical analysis code below how can I add these symbols? = (token type must be EQUALS), =, (NOTEQUALS), ( LPAREN, ) RPAREN, anything
With the lexical analysis code below how can I add these symbols? = (token type must be EQUALS), <, <=, >, >=, <> (NOTEQUALS), ( LPAREN, ) RPAREN, anything except a close quote (STRING)
The lexer class should also accept words.
Using a HashMap of known words and their token types. Unknown tokens are given the token type IDENTIFIER
import java.util.ArrayList; import java.util.regex.Matcher; import java.util.regex.Pattern; public class Lexer { public static ArrayList lex(String input) { // The tokens to return ArrayList tokens = new ArrayList(); // Lexer logic begins here StringBuffer tokenPatternsBuffer = new StringBuffer(); for (Token.TokenType tokenType : Token.TokenType.values()) tokenPatternsBuffer.append(String.format("|(?<%s>%s)", tokenType.name(), tokenType.pattern)); Pattern tokenPatterns = Pattern.compile(new String(tokenPatternsBuffer.substring(1))); // Begin matching tokens Matcher matcher = tokenPatterns.matcher(input); while (matcher.find()) { if (matcher.group(Token.TokenType.NUMBER.name()) != null) { tokens.add(new Token(Token.TokenType.NUMBER, matcher.group(Token.TokenType.NUMBER.name()))); continue; } else if (matcher.group(Token.TokenType.BINARYOP.name()) != null) { tokens.add(new Token(Token.TokenType.BINARYOP, matcher.group(Token.TokenType.BINARYOP.name()))); continue; } else if (matcher.group(Token.TokenType.WHITESPACE.name()) != null) continue; } return tokens; } } Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
