Quick note (to myself and you) that if you do not import
import org.apache.spark.SparkContext._
import org.apache.spark.SparkContext._
You will not get the functions which work on PairRDD's in Apache Spark while using Scala.
val domainsUserIds = records1.map(x => (x.getUserId.toString, x.getDomain.toString))
domainsUserIds.reduceByKey - Will not be available unless you explicitly do:
import org.apache.spark.SparkContext._
domainsUserIds.reduceByKey (now available)
No comments:
Post a Comment