Lab $41$: Using ToolRunner and Passing Parameters

Files and Directories Used in this Exercise

Eclipse project: toolrunner

Java files:

AverageReducer.java $($Reducer from AverageWordLength$)$

LetterMapper.java $($Mapper from AverageWordLength$)$

AvgWordLength.java $($driver from AverageWordLength$)$

Exercise directory: ~$/$workspace$/$toolrunner

In this Exercise, you will implement a driver using ToolRunner.

Follow the steps below to start with the Average Word Length program you wrote in an earlier lab, and modify the driver to use ToolRunner.$$ Then modify the Mapper to reference a Boolean parameter called caseSensitive; if true, the mapper should treat upper and lower case letters as different; if false or unset, all letters should be converted to lower case.

Modify the Average Word Length Driver to use Toolrunner

$1.$ Copy the Reducer, Mapper and driver code you completed in the $$Writing Java MapReduce Programs$$ lab earlier, in the averagewordlength project.

Copying Source Files

You can use Eclipse to copy a Java source file from one project or package to another by right$-$clicking on the file and selecting Copy, then right$-$clicking the new package and selecting Paste. If the packages have different names $($e$.$g$.$ if you copy from averagewordlength.solution to toolrunner.stubs$),$ Eclipse will automatically change the package directive at the top of the file. If you copy the file using a file browser or the shell, you will have to do that manually.

$2.$ Modify the AvgWordLength driver to use ToolRunner. Refer to the slides for details.

a$.$ Implement the run method

b$.$ Modify main to call run

$3.$ Jar your solution and test it before continuing; it should continue to function exactly as it did before. Refer to the Writing a Java MapReduce Program lab for how to assemble and test if you need a reminder.

Modify the Mapper to use a configuration parameter

$4.$ Modify the LetterMapper class to

a$.$ Override the setup method to get the value of a configuration parameter called caseSensitive, and use it to set a member variable indicating whether to do case sensitive or case insensitive processing.

b$.$ In the map method, choose whether to do case sensitive processing $($leave the letters as$-$is$),$ or insensitive processing $($convert all letters to lower$-$case$)$ based on that variable.

Pass a parameter programmatically

$5.$ Modify the driver$$s run method to set a Boolean configuration parameter called caseSensitive. $($Hint: Use the Configuration.setBoolean method.$)$

$6.$ Test your code twice, once passing false and once passing true. When set to true, your final output should have both upper and lower case letters; when false, it should have only lower case letters. Hint: Remember to rebuild your Jar file to test changes to your code. Pass a parameter as a runtime parameter

Submit:$$

$1.$a$.$ Code when you set caseSensitive $=$ true

$$ b$.$ Ouput when caseSensitive $=$ true$$

$($name the file part$-4-00000\_$programatically$\_$true$)$

$$

$2.$a$.$ Code when you set caseSensitive $=$ false

$$ b$.$ Ouput when caseSensitive $=$ false

$($name the file part$-4-00000\_$programatically$\_$false$)$

$$

$7.$ Comment out the code that sets the parameter programmatically. $($Eclipse hint: Select the code to comment and then select Source $>$ Toggle Comment$).$ Test again, this time passing the parameter value using $$D on the Hadoop command line, e$.$g$.$:

$ hadoop jar toolrunner.jar stubs.AvgWordLength $\backslash$

$-$DcaseSensitive$=$true shakespeare toolrunnerout

$$ Hint: you may need to use the following in your driver class:

$$ Job job $=$ new Job$($getConf$())$;

Submit:$$

$3.$a$.$ Code when you set caseSensitive $=$ false

$$ b$.$ Both the outputs:$$

$$ i$.$ Ouput when caseSensitive $=$ true$$

$($name the file part$-4-00000\_$toolrunner$\_$true$)$

$$ ii$.$ Ouput when caseSensitive $=$ false

$($name the file part$-4-00000\_$toolrunner$\_$false$)$

$$

$8.$ Test passing both true and false to confirm the parameter works correctly.$$

Submission:

Please submit $1,2$ and $3$ from above.$$ When submitting the code, remember the following:

Create a jar file of the final version of the project and submit your jar file after you have tested. Any jar files which do not execute $($i$.$e$.$ have compilation errors$)$ will automatically be given a $0.$

Also when you create your jar file, make sure you have the following checked:

$$Export Java source files and resources$$ and

$$Export generated class files and resources"

I need help with figuring out the proper code needed for all steps of the project. Thanks!