[Solved] spark- scala:How to read data from .dat file transform it and finally store in HDFS

by


Please find the solution

val rdd = sc.textFile("/path/Test.dat")
val rddmap = rdd.map(i => i.split(" ")).map(i => (i(1),i(2))).sortByKey().map(i => i._1 + "%$" + i._2)
rddmap.repartition(1).saveAsTextFile("/path/TestOut1.dat")

output

Jasper%$Pinto

Jhon%$Ward

Shally%$Stun

1

solved spark- scala:How to read data from .dat file transform it and finally store in HDFS