Question: Python programming; pleasee help!! We want to build a tokenizer for simple expressions such as xpr = res = 3 + x_sum*11. Such expressions comprise

Python programming; pleasee help!!

We want to build a tokenizer for simple expressions such as xpr = "res = 3 + x_sum*11". Such expressions comprise only three tokens, as follows: (1) Integer literals: one or more digits e.g., 3 11; (2) Identifiers: strings starting with a letter or an underscore and followed by more letters, digits, or underscores e.g., res x_sum; (3) Operators: = + * . Leading or trailing whitespace characters should be skipped.

(b) The problem with the above is that re.findall returns a list of all lexemes but not their respective tokens (as defined by the grammar). Modify your regex pattern so that each matched lexeme is in its own (regex) group. The list returned should now be as follows:

[('', 'res', ''), ('', '', '='), ('3', '', ''), ('', '', '+'), ('', 'x_sum', ''), ('', '', '*'), ('11', '', '')]

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!