解决spark程序报错:Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]
2017-05-28 18:23
453 查看
报错信息:
根据最后的Caused by信息和stack trace信息进行搜索,确定是broacast阶段超时,解决方法:
http://stackoverflow.com/questions/41123846/scala-spark-dataframes-join-java-util-concurrent-timeoutexception-futures-t
09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:49) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.aggregate.TungstenAggregate.doExecute(TungstenAggregate.scala:80) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.Exchange.prepareShuffleDependency(Exchange.scala:164) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:254) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:248) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - ... 64 more 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds] 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at scala.concurrent.Await$.result(package.scala:107) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.joins.BroadcastHashJoin.doExecute(BroadcastHashJoin.scala:107) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.Project.doExecute(basicOperators.scala:46) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:86) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:80) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48) 09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - ... 73 more
根据最后的Caused by信息和stack trace信息进行搜索,确定是broacast阶段超时,解决方法:
http://stackoverflow.com/questions/41123846/scala-spark-dataframes-join-java-util-concurrent-timeoutexception-futures-t
相关文章推荐
- Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
- Flume启动时报错Caused by: java.lang.InterruptedException: Timed out before HDFS call was made. Your hdfs.callTimeout might be set too low or HDFS calls are taking too long.解决办法(图文详解)
- tomcat内存溢出的解决方法(java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:)
- hbase Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.ThreadFactoryBu
- 项目中引入了大量的第三方架包所导致的tomcat内存溢出的解决方法(java.util.concurrent.ExecutionException: java.lang.OutOfMemoryErro
- tomcat内存溢出的解决方法(java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:)
- spark redis 中碰到java.net.SocketTimeoutException: Read timed out问题的解决
- tomcat servlet java.net.SocketTimeoutException: Read timed out 解决办法
- java.util.concurrent.TimeoutException: android.view.ThreadedRenderer.finalize() timed out after 10 s
- java.net.SocketTimeoutException: Read timed out 的解决办法 android平台 腾讯微博开发
- java.util.concurrent.TimeoutException: android.view.ThreadedRenderer.finalize() timed out after 10 s
- tomcat内存溢出的解决方法(java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:)
- java.net.SocketTimeoutException: Read timed out 的解决办法
- java.net.SocketTimeoutException: Read timed out 异常解决办法
- 执行Hive时出现org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.lang.NumberFormatException: For input string: "1s"错误的解决办法(图文详解)
- java.net.SocketTimeoutException: Read timed out 错误解决
- tomcat内存溢出的解决方法(java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:)
- java.net.SocketTimeoutException: Read timed out 错误解决
- java.net.SocketTimeoutException: Read timed out解决办法
- spring cloud java.util.concurrent.TimeoutException的三种解决方式