cdh4.2.1安装部署各目录用户组及权限详细说明

1、dfs.name.dir or dfs.namenode.name.dir 

sudo mkdir -p /data/1/dfs/nn /nfsmount/dfs/nn

sudo chown -R hdfs:hdfs /data/1/dfs/nn /nfsmount/dfs/nn

sudo chmod 700 /data/1/dfs/nn /nfsmount/dfs/nn

2、dfs.data.dir or dfs.datanode.data.dir

sudo mkdir -p /data/1/dfs/dn /data/2/dfs/dn /data/3/dfs/dn /data/4/dfs/dn

sudo chown -R hdfs:hdfs  /data/1/dfs/dn /data/2/dfs/dn /data/3/dfs/dn /data/4/dfs/dn

sudo chmod 700 /data/1/dfs/dn /data/2/dfs/dn /data/3/dfs/dn /data/4/dfs/dn

3、 mapred.local.dir

sudo mkdir -p /data/1/mapred/local /data/2/mapred/local /data/3/mapred/local /data/4/mapred/local

sudo chown -R mapred:hadoop /data/1/mapred/local /data/2/mapred/local /data/3/mapred/local /data/4/mapred/local

sudo chmod 751 /data/1/mapred/local /data/2/mapred/local /data/3/mapred/local /data/4/mapred/local

4、 the HDFS and MapReduce log directories

sudo chown -R hdfs:hdfs  $HDFS_LOG_DIR

sudo chmod 771 $HDFS_LOG_DIR

sudo chown -R mapred:mapred  $MAPRED_LOG_DIR

sudo chmod 771 $MAPRED_LOG_DIR

sudo chown -R mapred:anygroup  $MAPRED_LOG_DIR/userlogs

$MAPRED_LOG_DIR/userlogs目录的权限在进程启动的时候自动设置

5、hdfs /(root directory)

sudo chown  hdfs:hadoop /

sudo -u hdfs hadoop fs -chmod 755 /

6、 HDFS /tmp

sudo -u hdfs hadoop fs -mkdir /tmp

sudo -u hdfs hadoop fs -chmod -R 777 /tmp

7、mapreduce.jobtracker.staging.root.dir

sudo -u hdfs hadoop fs -mkdir -p /mapred/staging

sudo -u hdfs hadoop fs -chmod 777 /mapred/staging

sudo -u hdfs hadoop fs -chown -R mapred /mapred

8、mapred.system.dir

sudo -u hdfs hadoop fs -mkdir /mapred/sys

sudo -u hdfs hadoop fs -chown mapred:hadoop /mapred/sys

sudo -u hdfs hadoop fs -chmod -R 700 /mapred/sys

9、Verify the HDFS File Structure

sudo -u hdfs hadoop fs -ls -R /

drwxrwxrwt   - hdfs supergroup          0 2012-04-19 15:14 /tmp

drwxr-xr-x   - hdfs     supergroup          0 2012-04-19 15:16 /mapred

drwxr-xr-x   - mapred   supergroup          0 2012-04-19 15:29 /mapred/sys

drwxrwxrwt   - mapred   supergroup          0 2012-04-19 15:33 /mapred/staging

10、Create a Home Directory for each MapReduce User

Create a home directory for each MapReduce user. It is best to do this on the
NameNode; for example:

$ sudo -u hdfs hadoop fs -mkdir  /user/<user>

$ sudo -u hdfs hadoop fs -chown <user> /user/<user>

where <user> is the Linux username of each user.

Alternatively, you can log in as each Linux user (or write a script to do so) and create the home directory as

follows:
sudo -u hdfs hadoop fs -mkdir /user/$USER

sudo -u hdfs hadoop fs -chown $USER /user/$USER

sudo -u hdfs hadoop fs -chmod 711 /user/$USER