value toDF is not a member of org.apache.spark.rdd.RDD[People]
2016-11-28 16:07
751 查看
编译如下代码时,出现value toDF is not a member of org.apache.spark.rdd.RDD[People] 错误
val rdd : RDD[People]= sparkSession.sparkContext.textFile(hdfsFile,2).map(line => line.split(",")).map(arr => People(arr(0),arr(1).trim.toInt))
rdd.toDF
参考http://stackoverflow.com/questions/33704831/value-todf-is-not-a-member-of-org-apache-spark-rdd-rdd,针对此错误有人提出需要做到以下两点
1. import sqlContext.implicits._ 语句需要放在获取sqlContext对象的语句之后
2. case class People(name : String, age : Int) 的定义需要放在方法的作用域之外(即java的成员变量位置)
实际上只需要做到第二点即可解决错误
val rdd : RDD[People]= sparkSession.sparkContext.textFile(hdfsFile,2).map(line => line.split(",")).map(arr => People(arr(0),arr(1).trim.toInt))
rdd.toDF
参考http://stackoverflow.com/questions/33704831/value-todf-is-not-a-member-of-org-apache-spark-rdd-rdd,针对此错误有人提出需要做到以下两点
1. import sqlContext.implicits._ 语句需要放在获取sqlContext对象的语句之后
2. case class People(name : String, age : Int) 的定义需要放在方法的作用域之外(即java的成员变量位置)
实际上只需要做到第二点即可解决错误
相关文章推荐
- value toDF is not a member of org.apache.spark.rdd.RDD[People]
- 解决value toDF is not a member of org.apache.spark.rdd.RDD[People]
- scala学习-Description Resource Path Location Type value toDF is not a member of org.apache.spark.rdd.R
- value toDF is not a member of org.apache.spark.rdd.RDD
- 拼写错误:value countBykey is not a member of org.apache.spark.rdd.RDD[(String, Int)]
- sbt编译spark程序提示value toDF is not a member of Seq()
- error: object kafka is not a member of package org.apache.spark.streaming
- Spark wordcount 编译错误 -- reduceByKey is not a member of RDD
- object hbase is not a member of package org.apache.hadoop
- org.apache.struts2.json.JSONWriter can not access a member of class
- org.apache.struts2.json.JSONWriter can not access a member of class
- Class org.apache.struts2.json.JSONWriter can not access a member of class org.springframework.aop.in
- Class org.apache.struts2.json.JSONWriter can not access a member of class org.springframework.aop.TruePointcut with modifiers "public"
- Class org.apache.struts2.json.JSONWriter can not access a member of *(转)
- Class org.apache.struts2.json.JSONWriter can not access a member of class
- org.apache.struts2.json.JSONWriter can not access a member of class
- Class org.apache.struts2.json.JSONWriter can not access a member of class oracle.jdbc.driver.Physica
- org.apache.struts2.json.JSONWriter can not access a member of class org.apache.commons.dbcp.PoolingD
- 遇到问题---org.apache.struts2.json.JSONWriter can not access a member of class
- Class org.apache.struts2.json.JSONWriter can not access a member of class oracle.jdbc.driver.Physica