SQOOP之——安装部署笔记
2016-06-06 11:06
363 查看
转载请注明出处:http://blog.csdn.net/l1028386804/article/details/51594008
sqoop是一个把关系型数据库数据抽向hadoop的工具。同时,也支持将hive、pig等查询的结果导入关系型数据库中存储。由于,本人部署的hadoop版本是2.2.0,所以sqoop的版本是:sqoop-1.99.3-bin-hadoop200
1、下载sqoop
sqoop是一个把关系型数据库数据抽向hadoop的工具。同时,也支持将hive、pig等查询的结果导入关系型数据库中存储。由于,本人部署的hadoop版本是2.2.0,所以sqoop的版本是:sqoop-1.99.3-bin-hadoop200
1、下载sqoop
wget https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gz[/code]2、解压sqoop安装文件tar -zxvf sqoop-1.99.3-bin-hadoop200.tar.gz3、配置sqoop环境变量export SQOOP_HOME=/home/liuyazhuang/sqoop/sqoop export PATH = $SQOOP_HOME/bin:$PATH修改sqoop——server关于hadoop共享jar的引用vi server/conf/catalina.properties找到common.loader行,把/usr/lib/hadoop/lib/*.jar改成你的hadoop jar 包目录
例如:/home/hadoop/hadoop/share/hadoop/yarn/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/yarn/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/*.jarvi server/conf/sqoop.properties找到:mapreduce.configuration.directory行,修改值为你的hadoop配置文件目录
如:/home/liuyazhuang/hadoop/hadoop/etc/hadoop
并且替换@LOGDIR@ 和@BASEDIR@ :
0,$ s/@LOGDIR@/logs/g
0,$ s/@BASEDIR@/base/g
然后找到你的数据库jdbc驱动复制到sqoop/lib目录下,如果不存在则创建.vi /etc/profile增加以下内容:export SQOOP_HOME=/home/hadoop/sqoop export PATH=$PATH:$SQOOP_HOME/bin export CATALINA_BASE=$SQOOP_HOME/server export LOGDIR=$SQOOP_HOME/logs/启动 [root@liuyazhuang sqoop]# ./bin/sqoop.sh server start Sqoop home directory: /home/liuyazhuang/sqoop/sqoop Setting SQOOP_HTTP_PORT: 12000 Setting SQOOP_ADMIN_PORT: 12001 Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001 Using CATALINA_BASE: /home/liuyazhuang/sqoop/sqoop/server Using CATALINA_HOME: /home/liuyazhuang/sqoop/sqoop/server Using CATALINA_TMPDIR: /home/liuyazhuang/sqoop/sqoop/server/temp Using JRE_HOME: /usr/local/jdk1.7.0 Using CLASSPATH: /home/liuyazhuang/sqoop/sqoop/server/bin/bootstrap.jar (sqoop服务端是一个跑在tomcat上的服务程序) 关闭 sqoop server :./bin/sqoop.sh server stop 使用sqoop客户端: [root@liuyazhuang sqoop]# bin/sqoop.sh client Sqoop home directory: /home/liuyazhuang/sqoop/sqoop Sqoop Shell: Type 'help' or '\h' for help. sqoop:000> show version --all#显示版本:show version --all显示连接器:show connector --all创建连接:create connection --cid 1 client version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013 server version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013 Protocol version: [1] sqoop:000>
相关文章推荐
- linux 误删home目录
- Linux查看连接数,并发数
- JAVA抓取网站数据-----JSOUP
- nginx之location配置详解
- 浅谈自动化运维设计思想
- linux压缩和解压缩命令大全
- 利用JSCH实现远程操作Linux系统和hadoop集群
- centos yum
- 大括号在linux系统中使用技巧
- 网站性能优化:压缩图片
- openssl实现私有CA
- linux 清除缓存
- ubuntu下linux内核源码阅读工具和调试方法总结
- 【OpenCV】in-place操作
- Makefile介绍
- 强悍的 Linux —— linux 中 bin 和 sbin 目录的主要区别
- Apache AB 参数传递
- PopupWindow消失动画无效,脑洞大开的解决方案
- Linux 软件包安装
- Apache 服务器 自带的压力测试工具ab 的使用