hadoop的Hive操作hdfs文件第一天

hadoop的Hive操作hdfs文件第一天

hive的具体练习以下4个目标
1. 第一普通的hdfs文件能导入到hive中以供我们查询。

create table dept(deptID int,deptName string,address string);

load data local inpath '/home/dyq/Documents/dept' overwrite into table dept;

select * from dept;

hive> select * from dept;
OK
NULL NULL NULL
NULL NULL NULL
NULL NULL NULL
NULL NULL NULL
Time taken: 0.316 seconds, Fetched: 4 row(s)

从上面看导入成功但是查询的都是null那是因为没有加分隔.test表默认的有terminated by '\t'
lines terminated by '\n' 分隔符所以尽管有报错数据也是插入的。
正确的创建语法为
create table dept1(deptID int,deptName string,address string) row format delimited

fields terminated by ',' lines terminated by '\n' stored as textfile
导入数据的时候语句不变
load data local inpath '/home/dyq/Documents/dept' overwrite into table dept1;

hive> select * from dept1;
OK
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
Time taken: 0.153 seconds, Fetched: 4 row(s)
看看成功了

二、查询结果导出来。
从hive中把表中的数据导出来保存成文本类型。
先检索索要的结果
’
hive> select deptID,deptName from dept1;
OK
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
Time taken: 4.735 seconds, Fetched: 4 row(s)

将查询结果输出至本地目录

insert overwrite local directory '/home/dyq/Documents/outputdept' select a.deptID,a.deptName from dept1 a;

hive> insert overwrite local directory '/home/dyq/Documents/outputdept' select a.deptID,a.deptName from dept1 a;

WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
Query ID = dyq_20160828100454_07848914-19c7-47a7-9dd6-7fe50a1f9a82
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1472347049416_0003, Tracking URL = http://ubuntu:8088/proxy/application_1472347049416_0003/ Kill Command = /opt/hadoop-2.6.2/bin/hadoop job -kill job_1472347049416_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2016-08-28 10:05:14,004 Stage-1 map = 0%, reduce = 0%
2016-08-28 10:05:26,132 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.23 sec
MapReduce Total cumulative CPU time: 2 seconds 230 msec
Ended Job = job_1472347049416_0003
Moving data to local directory /home/dyq/Documents/outputdept
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.23 sec HDFS Read: 3564 HDFS Write: 49 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 230 msec
OK
Time taken: 34.259 seconds

三、查看到处结果
在/home/dyq/Documents下出现了outdept目录里面有000000_0文件打开

10ACCOUNTING
20RESEARCH
30SALES
40OPERATIONS
呵呵成功了额