Floating point calculation is reasonably complex subject. This has to do with the binary representation of a floating point number, that doesn’t guarantee every possible number (obviously), to have an exact representation, which can lead to errors in operations, and yes, these errors can propagate. Here is a link on the subject, although it isn’t … Read more
You could use replaceAll to do the job without regex : val name: String = “cust_id” val newName: String = name.replaceAll(“_”,” “) println(newName) solved Scala: How to replace all consecutive underscore with a single space?
Not super-short: List(1,2,3,4).grouped(2).collect{ case List(k,v) => k -> v }.toMap 2 solved How to convert scala List in to Map
Your import is wrong. It should be import com.typesafe.scalalogging.LazyLogging and not import com.typesafe.scalalogging.slf4j.LazyLogging 4 solved Scala logging error, com.typesafe.scalalogging.LazyLogging.$init$ method not found
Try this. scala> val df = Seq((4, 30),(2,50),(3,10),(5,30),(1,50),(6,25)).toDF(“id”,”price”) df: org.apache.spark.sql.DataFrame = [id: int, price: int] scala> df.show +—+—–+ | id|price| +—+—–+ | 4| 30| | 2| 50| | 3| 10| | 5| 30| | 1| 50| | 6| 25| +—+—–+ scala> df.sort(desc(“price”), asc(“id”)).show +—+—–+ | id|price| +—+—–+ | 1| 50| | 2| 50| | 4| … Read more
Valid Scala syntax is ab.toCharArray.map(x => if (“aeiou”.indexOf(x) >= 0) x else ‘ ‘) On contrary ab.chars().map(x -> “aeiou”.indexOf(x) >= 0 ? x : ‘ ‘); is Java syntax. 1 solved scala how to mimic a ternary operator in anonymous functions
Scala string split method uses regular expression, { is a special character in regular expression which is used for quantifying matched patterns. If you want to treat it as literal, you need to escape the character with , \\{: val s = “””word, {“..Json Structure…”}””” // s: String = word, {“..Json Structure…”} s.split(“, \\{“) // … Read more
You probably want something like this: trait BinTree[+A] case object Leaf extends BinTree[Nothing] case class Branch[+A](node: A, left: BinTree[A], right: BinTree[A]) extends BinTree[A] def buildTree[A](list: List[A]): BinTree[A] = list match { case Nil => Leaf case x :: xs => val (left, right) = xs.splitAt(xs.length/2) Branch(x, buildTree(left), buildTree(right)) } But you really need to get … Read more
You can try turning resultList into a function: import scalikejdbc.SQL def resultList(sqlQuery: String): List[List[String]] = SQL(sqlQuery) .map( rs => List( rs.string(“transaction_id”) ) ) .list() .apply() 16 solved How can i execute the sql query with parameter?
Please find the solution val rdd = sc.textFile(“/path/Test.dat”) val rddmap = rdd.map(i => i.split(” “)).map(i => (i(1),i(2))).sortByKey().map(i => i._1 + “%$” + i._2) rddmap.repartition(1).saveAsTextFile(“/path/TestOut1.dat”) output Jasper%$Pinto Jhon%$Ward Shally%$Stun 1 solved spark- scala:How to read data from .dat file transform it and finally store in HDFS
Try this val input = Seq((1,0),(2,0),(2,0),(1,1),(1,1),(2,1)) input.groupBy(_._2).collect{ case e => e._2.map(_._1) } //res3: scala.collection.immutable.Iterable[Seq[Int]] = List(List(1, 1, 2), List(1, 2, 2)) 0 solved How to group a Seq[Array[(Int, Int)]]) by the second value of array
If this is what you’re aiming for: List(List(r1), List(r2), List(r3 chain, r4), List(r5 chain, r6 chain, r7)) then here is a possibility: val rules = List(“r1”, “r2”, “r3 chain”, “r4”, “r5 chain”, “r6 chain”, “r7”) val (groups, last) = rules.foldLeft(List[List[String]](), List[String]()) { case ((groups, curGroup), rule) if rule.contains(“chain”) => (groups, rule :: curGroup) case ((groups, … Read more
val grouped = df.groupBy($”id”).count val res = df.join(grouped,Seq(“id”)) .withColumnRenamed(“count”,”repeatedcount”) Group By will give count of each id’s. Join that with original dataframe to get count against each id. solved How to add columns using Scala
For inner join you could use slick applicative-join with filter clause. For example: val query = for { (address, userAddressMapping) <- Address join UserAddressMapping on (_.id === _.addressId) if userAddressMapping.userId === 1 } yield (address.id, address.name) dbConfig.run(query.result) 12 solved Ineer Join query with where in scala slick
I would suggest you to use sqlContext api and use the schema you have defined val df = sqlContext.read .format(“com.databricks.spark.csv”) .option(“delimiter”, “\\t”) .schema(schema) .load(“path to your text file”) the schema is val schema = StructType(Seq( StructField(“ID”, IntegerType, true), StructField(“col1”, DoubleType, true), StructField(“col2”, IntegerType, true), StructField(“col3”, DoubleType, true), StructField(“col4”, DoubleType, true), StructField(“col5”, DoubleType, true), StructField(“col6”, DoubleType, … Read more