Hadoop MapReduce 学习
2010-04-27 22:12
316 查看
MapReduce
1. Map: (K, V) -> (K', V')
2. 合并相同K'值的项
(K', V') -> (K', V' *)
3. Reduce
(K', V'*) -> 新的(K,V)
Hadoop实现自动的 Fail-over, Redistribute M/R tasks
Hadoop为Master/Slave结构
1 Master ( Job Tracker) / M Slave (Task Tracker)
HDFS
文件分成固定块大小,“Write Once", 只能有一个写
HDFS也是Master/Salve结构
1 NameNode / N DataNode
其他相关资源:
Distributed Systems课程 http://www.cs.brandeis.edu/~cs147a/ 对HBase的评测: www.cs.duke.edu/~kcd/hadoop/
Cloud9: A library for Hadoop http://www.umiacs.umd.edu/~jimmylin/cloud9/umd-hadoop-dist/cloud9-docs/index.html Cloud Computing课程: http://www.umiacs.umd.edu/~jimmylin/cloud-computing/index.html UCSD的Network Services课程: http://www-cse.ucsd.edu/classes/fa07/cse124/assignments.html Virginia的入门课 http://www.cs.virginia.edu/~cbs6n/hadoop/
Hadoop的监控 http://www.x-trace.net/wiki/doku.php
Scaling up Hadoop (超强,改善Hadoop) http://www.cs.washington.edu/homes/ak/clusterworkshop/slides/YahooHadoopDISC08.pdf (此人开了咨询公司 http://www.spinnakerlabs.com/ )
Washington的课程 http://www.cs.washington.edu/education/courses/cse490h/07sp/index.html
Distributed Systems课程 http://www.cs.williams.edu/~jeannie/cs339/index.html http://pages.cs.wisc.edu/~dusseau/Classes/CS739/index.html
Parrallel Processing http://www.cs.colostate.edu/~cs575dl/
1. Map: (K, V) -> (K', V')
2. 合并相同K'值的项
(K', V') -> (K', V' *)
3. Reduce
(K', V'*) -> 新的(K,V)
Hadoop实现自动的 Fail-over, Redistribute M/R tasks
Hadoop为Master/Slave结构
1 Master ( Job Tracker) / M Slave (Task Tracker)
HDFS
文件分成固定块大小,“Write Once", 只能有一个写
HDFS也是Master/Salve结构
1 NameNode / N DataNode
其他相关资源:
Distributed Systems课程 http://www.cs.brandeis.edu/~cs147a/ 对HBase的评测: www.cs.duke.edu/~kcd/hadoop/
Cloud9: A library for Hadoop http://www.umiacs.umd.edu/~jimmylin/cloud9/umd-hadoop-dist/cloud9-docs/index.html Cloud Computing课程: http://www.umiacs.umd.edu/~jimmylin/cloud-computing/index.html UCSD的Network Services课程: http://www-cse.ucsd.edu/classes/fa07/cse124/assignments.html Virginia的入门课 http://www.cs.virginia.edu/~cbs6n/hadoop/
Hadoop的监控 http://www.x-trace.net/wiki/doku.php
Scaling up Hadoop (超强,改善Hadoop) http://www.cs.washington.edu/homes/ak/clusterworkshop/slides/YahooHadoopDISC08.pdf (此人开了咨询公司 http://www.spinnakerlabs.com/ )
Washington的课程 http://www.cs.washington.edu/education/courses/cse490h/07sp/index.html
Distributed Systems课程 http://www.cs.williams.edu/~jeannie/cs339/index.html http://pages.cs.wisc.edu/~dusseau/Classes/CS739/index.html
Parrallel Processing http://www.cs.colostate.edu/~cs575dl/
相关文章推荐
- hadoop 学习笔记:mapreduce框架详解
- Hadoop-统计红楼梦里出现名字的次数(MapReduce学习)
- hadoop 学习笔记 二 ----MapReduce
- Hadoop 学习笔记 (十) MapReduce实现排序 全局变量
- hadoop学习笔记2.windows上远程调试MapReduce
- Hadoop学习之MapReduce(三)
- 【Python学习系列四】Python程序通过hadoop-streaming提交到Hadoop集群执行MapReduce
- Hadoop学习笔记(五)MapReduce
- Hadoop学习之旅三:MapReduce
- Hadoop学习全程记录——在Eclipse中运行第一个MapReduce程序
- Hadoop学习笔记(2) 关于MapReduce
- Hadoop学习笔记(二):MapReduce的进度和状态
- Hadoop学习全程记录——在Eclipse中运行第一个MapReduce程序
- 【Big Data - Hadoop - MapReduce】hadoop 学习笔记:MapReduce框架详解
- Hadoop学习笔记-MapReduce工作原理
- Hadoop 学习笔记3 Develping MapReduce
- 云计算学习笔记005---Hadoop HDFS和MapReduce 架构浅析
- Hadoop学习之MapReduce框架详解
- Hadoop学习之路(二十二)MapReduce的输入和输出
- 【Hadoop学习笔记】——MapReduce