I think that you should build Hive Datawarehouse using Hive or MongoDB Datawarehouse using MongoDB. I didn’t understand how you are going to mix them, but I will try to answer the question anyway. Usually, you configure for a BI tool a JDBC driver for DB of your choice (e.g. Hive) and the BI tool … Read more
You can foldLeft over the input string, with a state of List[(Char, Int)]. Note that if you use Map[Char, Int], all occurrences of each character would be added up, weather they’re beside each other or not. s.foldLeft(List.empty[(Char, Int)]) { case (Nil, newChar) => (newChar, 1) :: Nil case ([email protected](headChar, headCount) :: tail, newChar) => if … Read more
Your use of for does not behave as you expect. You’re using this for-comprehension: for (data <- category(i)) { if (data.startsWith(x)) true else false } This expression “desugars” into (i.e. is shorthand for): category(i).foreach(data => { if (data.startsWith(x)) true else false }) foreach returns Unit, and therefore the type of this expression (and the type … Read more
This is a simple approach that converts the txt output data into a data structure (that can easily be written into a csv file). The basic idea is using data structures along with the amount of headers / columns in order to parse entry sets from the one liner txt output. Have a look at … Read more
You could achieve this as follows: import java.nio.file.Paths val path = “/home/gmc/exists.csv” val fileName = Paths.get(path).getFileName // Convert the path string to a Path object and get the “base name” from that path. val extension = fileName.toString.split(“\\.”).last // Split the “base name” on a . and take the last element – which is the extension. … Read more
Here is an informal explaination: The map() method, when applied to a collection of one type of thing (e.g. a collections of lines in a file) and provided a function (e.g. extract the second and third item from the given string) will return a collection of the results of applying that function to each item … Read more
The error is pretty self explanatory, so it is probably something simple that you are missing. Can you modify your script and run it as shown below. Please modify the “fileName” value to where you think the file is. import java.nio.file.{Paths, Files} import sys.process._ /************ Modify this line with your data’s file name **************/ val … Read more
I think this might solve your problem t.countByValue().filter(tupleOfCount=>Try(tupleOfCount._1.toInt).toOption.isEmpty).print() Use of isInstanceOf should be the last resort as @sergey said , so this code must solve the issue or else the pattern matching would be a good option too. solved Scala Filter Only Digits
you cannot use dataframes in RDD code (you use dataframes in days.foreach), th dataframes are null here as it only lives on the driver, but not on the executors 1 solved NullPointerException on DataFrame where
The most iconic way ot do it is to unwrap values with scala is to use pattern matching to unwrap the value. entities match { case Some(queryEntities: QueryEntities) => queryEntities.entities.foreach { case e => println(e.columnFamily) println(e.fromDate.getOrElse(“defaultFromDateHere”) println(e.toDate.getOrElse(“defaultToDateHere”)) } case None => println(“No value”) } 9 solved How can I access a method which return Option … Read more