[Solved] MapReduce to Spark


This is a very broad question, but the short of it is:

  1. Create an RDD of the input data.
  2. Call map with your mapper code. Output key-value pairs.
  3. Call reduceByKey with your reducer code.
  4. Write the resulting RDD to disk.

Spark is more flexible than MapReduce: there is a great variety of methods that you could use between steps 1 and 4 to transform the data.

0

solved MapReduce to Spark