Java Multithreading Problem

The WebLoader program loads a list url strings from a file. It presents the urls in the left column of a table. When one of the Fetch buttons is clicked, it forks off one or more threads to download the HTML for each url. A Stop button can kill off the downloading threads if desired. A progress bar and some other status fields show the progress of the downloads -- the number of worker threads that have completed their run, the number of threads currently running, and (when done running) the elapsed time.

There are two main classes that make up the WebLoader program

WebFrame

The WebFrame should contain the GUI, keep pointers to the main elements, and manage the overall program flow.

WebWorker

WebWorker is a subclass of Thread that downloads the content for one url. The "Fetch" buttons ultimately fork off a few WebWorkers.

Fetching URLs

When one of the Fetch buttons is clicked, the GUI should change to a "running" state -- fetch buttons disabled, stop button enabled, status strings reset, maximum on the progress bar set to the number of urls.

The swing thread cannot wait around to launch all the workers, so create a special "launcher" thread to create and start all the workers. We will include the launcher thread in the count of "running" threads. Having a separate launcher thread helps keep the GUI snappy we leave the GUI thread to service the GUI. Notice that GUI reacts quickly to mouse button clicks, etc. even as the launcher and all the worker threads are working and blocking.

The launcher should do the following

Run a loop to create and start WebWorker objects, one for each url.

Rather than starting all the workers at once, we will use a limit on the number of workers running at one time. This is a common technique -- starting a thousand threads at once can bog the system down. It's better to limit the number of concurrent workers to some reasonable number. When the Single Thread Fetch button is clicked, limit at one worker. If the Concurrent Fetch button is clicked, limit at the int value in the text field at the time of the click.

Use a semaphore to enforce the limit -- the launcher should not create/start more workers than the limit, and each worker at the very end of its run can signal the launcher that it can go ahead and create/start another worker.

At the very start of its run(), each worker should increment the running- threads count. At the very end of its run, each worker should decrement the running-threads count. The launcher thread should also do this to account for itself, so the running-threads count includes the launcher plus all the currently running workers.

You can detect that the whole run is done when the running thread count gets back down to zero. When that finally happens, compute the elapsed time and switch the GUI back to the "ready" state -- Fetch buttons enabled, Stop button disabled, and progress bar value set to 0.

When the launcher/limit system is working, the running-threads status string should hover right at or below the limit value plus one (for the launcher) at first, and gradually go down to zero. Make a new semaphore for each Fetch run -- that way you do not depend on the semaphore state left behind by the previous run. Interruption may mess up the internal state of the semaphore.

WebWorker Strategy

The WebWorker constructor should take a String url, the int row number in the table that it should update, and a pointer back to the WebFrame to send it messages.

The WebWorker should have a download() method, called from its run(), that tries to download the content of the url. The download() may or may not succeed for any number of reasons, it will probably take between a fraction of a second and a couple seconds to run

Given a url-string, Java has classes that make parsing the URL and retrieving its content pretty easy. The standard code to get an input stream from a URL and download its content is shown below (see the URLConnection API docs, and this code is in the starter file). The process may fail in many different ways at runtime, in which case the flow of control jumps down to the catch clauses and continues from there. If the download succeeds, control gets to the "Success" line.

//Starting Files

//WebWorker.java

import java.io.*;

import java.net.*;

import java.text.SimpleDateFormat;

import java.util.Date;

import javax.swing.*;

public class WebWorker extends Thread {

/*

This is the core web/download i/o code...*/

String urlString;

public WebWorker(String url) {

urlString = url;

}

public void run() {

System.out.println("Fetching...." + urlString);

InputStream input = null;

StringBuilder contents = null;

try {

URL url = new URL(urlString);

URLConnection connection = url.openConnection();

// Set connect() to throw an IOException

// if connection does not succeed in this many msecs.

connection.setConnectTimeout(5000);

connection.connect();

input = connection.getInputStream();

BufferedReader reader = new BufferedReader(new InputStreamReader(input));

char[] array = new char[1000];

int len;

contents = new StringBuilder(1000);

while ((len = reader.read(array, 0, array.length)) > 0) {

System.out.println("Fetching...." + urlString + len);

contents.append(array, 0, len);

Thread.sleep(100);

}

System.out.print(contents.toString());

}

// Otherwise control jumps to a catch...

catch(MalformedURLException ignored) {

System.out.println("Exception: " + ignored.toString());

}

catch(InterruptedException exception) {

// YOUR CODE HERE

// deal with interruption

System.out.println("Exception: " + exception.toString());

}

catch(IOException ignored) {

System.out.println("Exception: " + ignored.toString());

}

// "finally" clause, to close the input stream

// in any case

finally {

try{

if (input != null) input.close();

}

catch(IOException ignored) {}

}

}

}

//WebFrame.java

import java.awt.*; import javax.swing.*; import java.awt.event.*; import javax.swing.table.DefaultTableModel;

public class WebFrame extends JFrame {

public static void main(String[] args) { WebFrame frame = new WebFrame(); DefaultTableModel model = new DefaultTableModel(new String[] { "url", "status"}, 0); JTable table = new JTable(model); table.setAutoResizeMode(JTable.AUTO_RESIZE_ALL_COLUMNS); model.addRow(new String[] { "http://www.sjsu.edu/cs/", ""});

JScrollPane scrollpane = new JScrollPane(table); scrollpane.setPreferredSize(new Dimension(600,300)); frame.add(scrollpane); frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); frame.pack(); frame.setVisible(true);

} }