Course: Information AND organization retrieval

3.15. Plot the document slope curves for a sample of web pages. The sample should include at least one page containing a news article. Test the accuracy of the simple optimization algorithm for detecting the main content block.

Write your own program or use the code from http://www.aidanf.net/software/bte-bodytext-extraction. Describe the cases where the algorithm fails.

Would an algorithm that searched explicitly for low-slope areas of the document slope curve be successful in these cases?

language: java

comments: required for each line of code