Cloudera认证_开发者考试大纲

cca175开发者认证有10至12条题目，主要是基于cdh5的操作题。

要想通过此考试，需要掌握以下基本技能：

1.获取数据的能力 

这需要我们掌握sqoop的etl命令，flume的数据采集方式，以及如何使用hdfs的命令行加载数据。

2.数据规划，传输，存储的能力

使用spark读取hdfs上的数据，并进行一些基本的处理操作，将结果再返回到hdfs上，使用spark，我们不仅需要会用scala语言，也需要会python语言。

读取文件数据，对数据进行计算等等。

3.数据分析的能力

使用ddl语言对数据进行建表操作。建表的内容包括了建内部表，外部表，分区表，指定存储格式，指定分隔符，基于schema文件建表等。

这些都是非常基础的内容。多练即可。

CCA Spark and Hadoop Developer Exam (CCA175)

Number of Questions: 10–12 performance-based
(hands-on) tasks on CDH5 cluster. See below for full cluster
configuration

Time Limit: 120 minutes

Passing Score: 70%

Language: English, Japanese
(forthcoming)

Required Skills

Data Ingest

The skills to transfer data between external systems and your
cluster. This includes the following:

· Import
data from a MySQL database into HDFS using Sqoop

· Export
data to a MySQL database from HDFS using Sqoop

· Change
the delimiter and file format of data during import using
Sqoop

· Ingest
real-time and near-real time (NRT) streaming data into HDFS using
Flume

· Load
data into and out of HDFS using the Hadoop File System (FS)
commands

Transform, Stage, Store

Convert a set of data values in a given format stored in HDFS into
new data values and/or a new data format and write them into HDFS.
This includes writing Spark applications in both Scala and
Python:

· Load
data from HDFS and store results back to HDFS using
Spark

· Join
disparate datasets together using Spark

· Calculate
aggregate statistics (e.g., average or sum) using Spark

· Filter
data into a smaller dataset using Spark

· Write
a query that produces ranked or sorted data using Spark

Data Analysis

Use Data Definition Language (DDL) to create tables in the Hive
metastore for use by Hive and Impala.

· Read
and/or create a table in the Hive metastore in a given
schema

· Extract
an Avro schema from a set of datafiles using avro-tools

· Create
a table in the Hive metastore using the Avro file format and an
external schema file

· Improve
query performance by creating partitioned tables in the Hive
metastore

· Evolve
an Avro schema by changing JSON files