I am writing a python program to try to remove all special sentences between and all non-alphanumeric characters except space from a folder of txt 'UTF-16' encoded book files. however i am encountering problems, Below is my code.

import osfor path, dirs, files in os.walk("/Users/16258/Desktop/dataset/"): for f in files: filename = os.path.join(path, f) myBooks = open(filename, 'r', encoding='UTF-16')f = open('cleaned.txt', 'w', encoding='UTF-16')for files in myBooks: for line in files: start = line.startswith(') end = line.endswith('>') if not start: if not end: f.write(line)f.close()

the problem of this code is that the output only removes the "" characters but not the sentences with it. For example input is AAA, the output of my code is I love to study AAA instead of AAA. Also the code only reads 1 file from the folder somehow. Graphs are attached.

cleaned.txt - Notepad File Edit Format View Help Shakespeare -- TWELFTH-NIGHT; OR WHAT YOU WILL from Online Library of Liberty (http://oll.libertyfund.org) Unicode .txt version by Mike Scott (http://www.lexically.net) from "The Complete Works of William Shakespeare" ed. with a glossary by W.J. Craig M.A. (London: Oxford University Press, 1916) STAGE DIR Scene.-A City in Illyria; and the Sea-coast near it. /STAGE DIR ACT 1 SCENE 1 A Room in the Duke's Palace. Enter Duke, Curio, Lords; Musicians attending. DUKE 0% If music be the food of love, play on; Give me excess of it, that, surfeiting, The appetite may sicken, and so die. That strain again! it had a dying fall: O! it came o'er my ear like the sweet sound That breathes upon a bank of violets, Stealing and giving odour. Enough! no more: Ln 1, Col 1 100% Windows (CRLF) UTF-16 LETwelfth-Night; or What You Will.txt - Notepad 0 X File Edit Format View Help (/STAGE DIR> If music be the food of love, play on; Give me excess of it, that, surfeiting, The appetite may sicken, and so die. That strain again! it had a dying fall: O! it came o'er my ear like the sweet sound That breathes upon a bank of violets, Stealing and giving odour. Enough! no more: Ln 1, Col 1 100% Windows (CRLF) UTF-16 LE