oozie server系统时钟偏差导致sqoop报错
2016-06-03 11:00
309 查看
使用oozie通过sqoop导数报错如下
通过查找源代码发现如下问题
分析得知是由于oozie job启动时间大于系统当前时间,去查cloudera oozie server发现两台oozie server中一台存在时钟偏差,由于是测试环境,直接将oozie ha禁用,只保留没有时钟偏差的server,重跑job问题解决。
当然后续必须得解决时钟偏差问题了,报错以作记录备忘。
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, begin > end in range (begin, end): (1464673410428, 1464673380508) java.lang.IllegalArgumentException: begin > end in range (begin, end): (1464673410428, 1464673380508) at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetApplicationsRequestPBImpl.setStartRange(GetApplicationsRequestPBImpl.java:338) at org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.getChildYarnJobs(LauncherMainHadoopUtils.java:57) at org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.killChildYarnJobs(LauncherMainHadoopUtils.java:88) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:167) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
通过查找源代码发现如下问题
long startTime = 0L; try { startTime = Long.parseLong((System.getProperty("oozie.job.launch.time"))); } catch(NumberFormatException nfe) { throw new RuntimeException("Could not find Oozie job launch time", nfe); } String tag = actionConf.get("mapreduce.job.tags"); if (tag == null) { throw new RuntimeException("Could not find Yarn tags property (mapreduce.job.tags)"); } GetApplicationsRequest gar = GetApplicationsRequest.newInstance(); gar.setScope(ApplicationsRequestScope.OWN); gar.setStartRange(startTime, System.currentTimeMillis());
public void setStartRange(long begin, long end) throws IllegalArgumentException { if(begin > end) { throw new IllegalArgumentException("begin > end in range (begin, end): (" + begin + ", " + end + ")"); } else { this.start = new LongRange(begin, end); } }
分析得知是由于oozie job启动时间大于系统当前时间,去查cloudera oozie server发现两台oozie server中一台存在时钟偏差,由于是测试环境,直接将oozie ha禁用,只保留没有时钟偏差的server,重跑job问题解决。
当然后续必须得解决时钟偏差问题了,报错以作记录备忘。
相关文章推荐
- Java中间件 - Tomcat 7 配置和部署
- nginx 403 forbidden 二种原因
- 电信数据挖掘中ID过少问题解决总结
- 解决Eclipse+JDK 32 bit+hadoop 32bit 遇到org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lan
- Tomcat7.0安装配置详细(图文)
- echarts toolbox 自定义工具
- linux登陆mysql数据库
- linux命令ps aux|grep xxx详解
- Makefile的基本规则
- HA高可用集群/LB负载均衡 添加一张虚拟网卡
- "Could not open '/sys/class/power_supply/genymotion_fake_path/present'
- linux系统c线程池的实现
- shell 脚本之for
- Linux中的ctags和Makefile使用
- centos7 安装nodejs
- Linux rpm 命令参数使用详解[介绍和应用]
- 深入浅出学Hive——Hive原理
- 深入浅出学Hive——Hive体系结构
- lxc(linux container) 命令学习
- 网站兼容性小方法