Question: Suppose you have a 7-node cluster with replication factor of 3. Describe what MapReduce has to do after it determines that a node has crashed

Suppose you have a 7-node cluster with replication factor of 3. Describe what MapReduce has to do after it determines that a node has crashed while a job is being processed. For simplicity, assume that the failed node is not replaced and your cluster is reduced to 6 nodes. Specifically:

i)What does HDFS (the storage layer) have to do in response to node failure in this case?

ii)What does MapReduce engine have to do to respond to the node failure? Assume thatthere was a job in progress because otherwise MapReduce does not need to do anything.b)Where does the Mapper store output key-value pairs before they are sent to Reducers?

iii)Can Reducers begin processing before Mapper phase is complete? Why or why not?

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!