Question: Python programming; pleasee help!! We want to build a tokenizer for simple expressions such as xpr = res = 3 + x_sum*11. Such expressions comprise
Python programming; pleasee help!!
We want to build a tokenizer for simple expressions such as xpr = "res = 3 + x_sum*11". Such expressions comprise only three tokens, as follows: (1) Integer literals: one or more digits e.g., 3 11; (2) Identifiers: strings starting with a letter or an underscore and followed by more letters, digits, or underscores e.g., res x_sum; (3) Operators: = + * . Leading or trailing whitespace characters should be skipped.
(b) The problem with the above is that re.findall returns a list of all lexemes but not their respective tokens (as defined by the grammar). Modify your regex pattern so that each matched lexeme is in its own (regex) group. The list returned should now be as follows:
[('', 'res', ''), ('', '', '='), ('3', '', ''), ('', '', '+'), ('', 'x_sum', ''), ('', '', '*'), ('11', '', '')]
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
