Create a tokenizer in Ruby. Your tokenizer will take two command line arguments: The first will be a grammar specification and the second will be a file to tokenize. Tokenize the file and print the tokens (symbol, lexeme, line) to the screen. If the file cannot be tokenized, print an error message identifying the line with the error.

sample files

File 1:

NUM -> \d+ ADDOP -> [-+] MULOP -> [*/] LP -> \( RP -> \) EQ -> = ID -> [A-Z]\w* comment -> \{[^}]*\}

S -> ID EQ expr expr -> expr ADDOP term | term term -> term MULOP factor | factor factor -> ID | NUM | LP expr RP

File 2:

4+2 { this is a comment } + 6

File 3:

1 +

2