Java Text Parser

Overview:

Given a input text file you need to parse it and answer a set of frequency related questions.

Technical Requirement of Solution:

You are required to do this from scratch. This means, your solution cannot use any library methods in Java except the ones listed below.

- String.split() and other String operations can be used wherever required.

- You can use any Regular Expression related facilities (java.util.regex) to match target words and phrases.

- You are also allowed to use different variants of array and list based built-in data structures such as Array, List, ArrayList, Vector.

- Standard file IO facilities for reading/writing, such as BufferedReader.

*Library functions that leverage hash and maps are Not Allowed .*

Create as many files, intermediate array-based data structures as you wish, and allocate as much heap memory from JVM as you need. BUT you are allowed to read the input file EXACTLY ONCE to answer ALL the questions. You however can use any internal array based representation of the whole file to do multiple rounds of processing if needed after having read it EXACTLY ONCE.

For the following questions, list all matching output if there are ties

1. List the most frequent word(s) in the whole file and its frequency.

2. List the 3rd most frequent word(s) in the whole file and its frequency.

3. List the word(s) with the highest frequency in a sentence across all sentences in the whole file, also print its frequency and the corresponding sentence.

4. List sentence(s) with the maximum no. of occurrences of the word "the" in the entire file and also list the corresponding frequency.

5. List sentence(s) with the maximum no. of occurrences of the word "of" in the entire file and also list the corresponding frequency.

6. List sentence(s) with the maximum no. of occurrences of the word "was" in the entire file and also list the corresponding frequency.

7. List sentence(s) with the maximum no. of occurrences of the phrase "but the" in the entire file and also list the corresponding frequency.

8. List sentence(s) with the maximum no. of occurrences of the phrase "it was" in the entire file and also list the corresponding frequency.

9. List sentence(s) with the maximum no. of occurrences of the phrase "in my" in the entire file and also list the corresponding frequency.

Implementation Detail:

Inputs

The program has two arguments:

- The first argument: path to the input text file.

- The second argument: name prefix for the output files

For example:

$ java HW1 "./input.txt" "output" 

input file: A text document. Assume each newline ( ) defines a paragraph. Each period (.) defines end of an sentence. Or if a sentence is the last in a paragraph and doesnt have an explicit period (.), its end marker is the same as a newline. Each space within a sentence (character 32) define the word delimiter. The assignment is case insensitive so you must transform and work in lower case.

Outputs: As grading is automated, your output must conform to the following specifications. For each of the 9 questions you must create one single output file. So your program should produce 9 output files each time you run it. If a question has multiple output (multiple sentences/words/) you should print each sentence in a new line. Do not print them on the same line! The order of the sentences/words/phrases is not important. However, the order of the output file name must be matching the order of the questions. For example, given prefix output, the output file of the first question should be output1.txt and for the second question it is output2.txt. The output format depends on the question type and it must be:

For question 1 and 2: word:frequency e.g. the:76

For question 3: word:frequency:sentence e.g. the:8:when you combine the ideas of whistles at night the presence...

For question 4-8: word:frequency:sentence e.g. was:2:then it was withdrawn as suddenly as it appeared... was:2:the 4 a week was a lure which must draw him and...

For question 9: word:frequency:sentence e.g. in my:0:excellent in my:0:you are not averse to this trip watson

(In cases where the frequency is 0, list all sentences)

Other details

For Word related questions, Words are defined as whole word separated by a space. So there does not count towards the frequence of the. Whereas for phrases you are required to consider substrings as well, for example within my will also count towards the frequence of in my

When printing the sentences, dont print out the period (.) at the end

When there is a tie, print all words/sentences with the maximum frequency

Sample Input Text:

Excellent. You are not averse to this trip Watson By no means. Then we shall both come. What are you going to do yourself I have one or two things which I would wish to do now that I am in town. But I shall return by the twelve oclock train so as to be there in time for your coming. And you may expect us early in the afternoon. I have myself some small business matters to attend to. Will you not wait and breakfast No I must go. My heart is lightened already since I have confided my trouble to you. I shall look forward to seeing you again this afternoon. She dropped her thick black veil over her face and glided from the room. And what do you think of it all Watson asked Sherlock Holmes leaning back in his chair. It seems to me to be a most dark and sinister business. Dark enough and sinister enough. Yet if the lady is correct in saying that the flooring and walls are sound and that the door window and chimney are impassable then her sister must have been undoubtedly alone when she met her mysterious end. What becomes then of these nocturnal whistles and what of the very peculiar words of the dying woman I cannot think. When you combine the ideas of whistles at night the presence of a band of gipsies who are on intimate terms with this old doctor the fact that we have every reason to believe that the doctor has an interest in preventing his stepdaughters marriage the dying allusion to a band and finally the fact that Miss Helen Stoner heard a metallic clang which might have been caused by one of those metal bars that secured the shutters falling back into its place I think that there is good ground to think that the mystery may be cleared along those lines. But what then did the gipsies do I cannot imagine. I see many objections to any such theory. And so do I. It is precisely for that reason that we are going to Stoke Moran this day. I want to see whether the objections are fatal or if they may be explained away. But what in the name of the devil The ejaculation had been drawn from my companion by the fact that our door had been suddenly dashed open and that a huge man had framed himself in the aperture. His costume was a peculiar mixture of the professional and of the agricultural having a black top hat a long frock coat and a pair of high gaiters with a hunting crop swinging in his hand. So tall was he that his hat actually brushed the cross bar of the doorway and his breadth seemed to span it across from side to side. A large face seared with a thousand wrinkles burned yellow with the sun and marked with every evil passion was turned from one to the other of us while his deep set bile shot eyes and his high thin fleshless nose gave him somewhat the resemblance to a fierce old bird of prey. Which of you is Holmes asked this apparition. My name sir but you have the advantage of me said my companion quietly. I am Dr. Grimesby Roylott of Stoke Moran. Indeed Doctor said Holmes blandly. Pray take a seat. I will do nothing of the kind. My stepdaughter has been here. I have traced her. What has she been saying to you It is a little cold for the time of the year said Holmes. What has she been saying to you screamed the old man furiously. But I have heard that the crocuses promise well continued my companion imperturbably. Ha You put me off do you said our new visitor taking a step forward and shaking his hunting crop. I know you you scoundrel I have heard of you before. You are Holmes the meddler. My friend smiled. Holmes the busybody His smile broadened. Holmes the Scotland Yard Jack in office Holmes chuckled heartily. Your conversation is most entertaining said he. When you go out close the door for there is a decided draught. I will go when I have said my say. Dont you dare to meddle with my affairs. I know that Miss Stoner has been here. I traced her I am a dangerous man to fall foul of See here. He stepped swiftly forward seized the poker and bent it into a curve with his huge brown hands. See that you keep yourself out of my grip he snarled and hurling the twisted poker into the fireplace he strode out of the room. He seems a very amiable person said Holmes laughing. I am not quite so bulky but if he had remained I might have shown him that my grip was not much more feeble than his own. As he spoke he picked up the steel poker and with a sudden effort straightened it out again. Fancy his having the insolence to confound me with the official detective force This incident gives zest to our investigation however and I only trust that our little friend will not suffer from her imprudence in allowing this brute to trace her. And now Watson we shall order breakfast and afterwards I shall walk down to Doctors Commons where I hope to get some data which may help us in this matter. It was nearly one oclock when Sherlock Holmes returned from his excursion. He held in his hand a sheet of blue paper scrawled over with notes and figures. I have seen the will of the deceased wife said he. To determine its exact meaning I have been obliged to work out the present prices of the investments with which it is concerned. The total income which at the time of the wifes death was little short of 1100 is now through the fall in agricultural prices not more than 750. Each daughter can claim an income of 250 in case of marriage. It is evident therefore that if both girls had married this beauty would have had a mere pittance while even one of them would cripple him to a very serious extent. My mornings work has not been wasted since it has proved that he has the very strongest motives for standing in the way of anything of the sort. And now Watson this is too serious for dawdling especially as the old man is aware that we are interesting ourselves in his affairs so if you are ready we shall call a cab and drive to Waterloo. I should be very much obliged if you would slip your revolver into your pocket. An Eleys No. 2 is an excellent argument with gentlemen who can twist steel pokers into knots. That and a tooth brush are I think all that we need. At Waterloo we were fortunate in catching a train for Leatherhead where we hired a trap at the station inn and drove for four or five miles through the lovely Surrey lanes. It was a perfect day with a bright sun and a few fleecy clouds in the heavens. The trees and wayside hedges were just throwing out their first green shoots and the air was full of the pleasant smell of the moist earth. To me at least there was a strange contrast between the sweet promise of the spring and this sinister quest upon which we were engaged. My companion sat in the front of the trap his arms folded his hat pulled down over his eyes and his chin sunk upon his breast buried in the deepest thought. Suddenly however he started tapped me on the shoulder and pointed over the meadows. Look there said he. A heavily timbered park stretched up in a gentle slope thickening into a grove at the highest point. From amid the branches there jutted out the grey gables and high roof tree of a very old mansion. Stoke Moran said he.