To toDF()
, you must enable implicit conversions:
import spark.implicits._
In spark-shell
, it is enabled by default and that’s why the code works there. :imports
command can be used to see what imports are already present in your shell:
scala> :imports
1) import org.apache.spark.SparkContext._ (70 terms, 1 are implicit)
2) import spark.implicits._ (1 types, 67 terms, 37 are implicit)
3) import spark.sql (1 terms)
4) import org.apache.spark.sql.functions._ (385 terms)
3
solved toDF is not working in spark scala ide , but works perfectly in spark-shell [duplicate]