zhwj184
- 浏览: 740681 次
- 性别:
- 来自: 杭州
-
文章分类
最新评论
-
lgh1992314：
a offset: 26b offset: 24c offse ...
java jvm字节占用空间分析 -
ls0609：
语音实现在线听书http://blog.csdn.net/ls ...
Android 语音输入API使用 -
wangli61289：
http://viralpatel-net-tutorials ...
Android 语音输入API使用 -
zxjlwt：
学习了素人派http://surenpi.com
velocity宏加载顺序 -
tt5753：
谢啦........
Lucene的IndexWriter初始化时的LockObtainFailedException的解决方法
1.先去hadoop官网下载hadoop的源码 http://svn.apache.org/repos/asf/hadoop/common/trunk
2.下载maven3，当前hadoop的最新版必须使用maven3编译
3.到hadoop下载源码目录执行mvn clean install；mvn eclipse:eclipse；
4.将源码导入eclipse；
5.在eclipse设置执行的WordCount.java的jvm启动参数，最少需要两个，输入目录和输出目录
6.然后就可以设置断点进行调试了，我们在处理mapreduce的主干流程上设置断点
org.apache.hadoop.mapred.LocalJobRunner这个类的run方法上
我们看到在我们设置的输入输出目录，然后使用默认的hadoop单机配置下，mapTask有16个，reduceTask有1个
我们先看看我们的输入目录，刚好是16个文件，说明每个输入文件默认启动一个mapTask
而reduce怎么是一个，怎么处理16个mapTask的输出呢
在org.apache.hadoop.mapred.ReduceTask这个reduce处理中run方法中会对所有的map输出做一个merge，然后作为reduceTask的输入
if (!isLocal) {
Class combinerClass = conf.getCombinerClass();
CombineOutputCollector combineCollector =
(null != combinerClass) ?
new CombineOutputCollector(reduceCombineOutputCounter, reporter, conf) : null;
Shuffle shuffle =
new Shuffle(getTaskID(), job, FileSystem.getLocal(job), umbilical,
super.lDirAlloc, reporter, codec,
combinerClass, combineCollector,
spilledRecordsCounter, reduceCombineInputCounter,
shuffledMapsCounter,
reduceShuffleBytes, failedShuffleCounter,
mergedMapOutputsCounter,
taskStatus, copyPhase, sortPhase, this,
mapOutputFile);
rIter = shuffle.run();
} else {
// local job runner doesn't have a copy phase
copyPhase.complete();
final FileSystem rfs = FileSystem.getLocal(job).getRaw();
rIter = Merger.merge(job, rfs, job.getMapOutputKeyClass(),
job.getMapOutputValueClass(), codec,
getMapFiles(rfs, true),
!conf.getKeepFailedTaskFiles(),
job.getInt(JobContext.IO_SORT_FACTOR, 100),
new Path(getTaskID().toString()),
job.getOutputKeyComparator(),
reporter, spilledRecordsCounter, null, null);
}
由于用到的是单机模式，所以没有用到Shuffle的过程。
整个日志输出如下：
2012-11-05 12:46:37,006 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - session.id is deprecated. Instead, use dfs.metrics.session-id 2012-11-05 12:46:37,050 INFO [main] jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId= 2012-11-05 12:46:37,195 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2012-11-05 12:46:37,263 WARN [main] mapreduce.JobSubmitter (JobSubmitter.java:copyAndConfigureFiles(247)) - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 2012-11-05 12:46:37,315 INFO [main] input.FileInputFormat (FileInputFormat.java:listStatus(245)) - Total input paths to process : 16 2012-11-05 12:46:37,784 INFO [main] mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(368)) - number of splits:16 2012-11-05 12:46:37,832 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class 2012-11-05 12:46:37,833 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 2012-11-05 12:46:37,833 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.name is deprecated. Instead, use mapreduce.job.name 2012-11-05 12:46:37,833 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 2012-11-05 12:46:37,833 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 2012-11-05 12:46:37,833 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2012-11-05 12:46:37,834 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2012-11-05 12:46:37,834 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2012-11-05 12:46:37,834 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2012-11-05 12:46:37,835 WARN [main] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 2012-11-05 12:46:38,048 INFO [main] mapreduce.JobSubmitter (JobSubmitter.java:printTokens(438)) - Submitting tokens for job: job_local_0001 2012-11-05 12:46:38,597 INFO [main] mapreduce.Job (Job.java:submit(1222)) - The url to track the job: http://localhost:8080/ 2012-11-05 12:46:38,663 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1267)) - Running job: job_local_0001 2012-11-05 12:46:38,666 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(320)) - OutputCommitter set in config null 2012-11-05 12:46:38,677 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(338)) - OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 2012-11-05 12:46:39,666 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1288)) - Job job_local_0001 running in uber mode : false 2012-11-05 12:46:39,668 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1295)) - map 0% reduce 0% 2012-11-05 12:52:04,731 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000000_0 2012-11-05 12:52:04,894 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@16c4642 2012-11-05 12:52:04,903 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/capacity-scheduler.xml:0+7457 2012-11-05 12:52:05,006 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:05,007 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:05,007 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:05,007 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:05,007 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:05,062 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:05,063 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:05,063 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:05,063 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 10166; bufvoid = 104857600 2012-11-05 12:52:05,063 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26211052(104844208); length = 3345/6553600 2012-11-05 12:52:05,151 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:05,155 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000000_0 is done. And is in the process of committing 2012-11-05 12:52:05,166 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:05,166 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000000_0' done. 2012-11-05 12:52:05,166 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000000_0 2012-11-05 12:52:05,852 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1295)) - map 100% reduce 0% 2012-11-05 12:52:06,389 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000001_0 2012-11-05 12:52:06,389 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:run(386)) - Waiting for map tasks 2012-11-05 12:52:06,394 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@1a3c2bf 2012-11-05 12:52:06,398 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/hadoop-policy.xml:0+4644 2012-11-05 12:52:06,466 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:06,466 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:06,466 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:06,467 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:06,467 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:06,483 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:06,483 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:06,484 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:06,484 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 6454; bufvoid = 104857600 2012-11-05 12:52:06,484 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26212228(104848912); length = 2169/6553600 2012-11-05 12:52:06,508 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:06,515 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000001_0 is done. And is in the process of committing 2012-11-05 12:52:06,520 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:06,520 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000001_0' done. 2012-11-05 12:52:06,520 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000001_0 2012-11-05 12:52:06,521 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000002_0 2012-11-05 12:52:06,528 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@adc92c 2012-11-05 12:52:06,529 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/log4j.properties:0+4441 2012-11-05 12:52:06,615 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:06,616 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:06,616 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:06,616 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:06,616 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:06,724 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:06,725 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:06,725 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:06,725 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 5492; bufvoid = 104857600 2012-11-05 12:52:06,725 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26213316(104853264); length = 1081/6553600 2012-11-05 12:52:06,737 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:06,741 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000002_0 is done. And is in the process of committing 2012-11-05 12:52:06,743 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:06,744 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000002_0' done. 2012-11-05 12:52:06,744 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000002_0 2012-11-05 12:52:06,744 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000003_0 2012-11-05 12:52:06,747 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@1c6f86d 2012-11-05 12:52:06,749 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/hadoop-env.sh:0+2237 2012-11-05 12:52:07,288 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,288 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,288 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,289 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,289 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,322 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,322 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,322 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,322 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 3255; bufvoid = 104857600 2012-11-05 12:52:07,322 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26213356(104853424); length = 1041/6553600 2012-11-05 12:52:07,330 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,333 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000003_0 is done. And is in the process of committing 2012-11-05 12:52:07,334 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,335 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000003_0' done. 2012-11-05 12:52:07,335 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000003_0 2012-11-05 12:52:07,335 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000004_0 2012-11-05 12:52:07,339 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@19d34ca 2012-11-05 12:52:07,401 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/mapred-queue-acls.xml:0+2033 2012-11-05 12:52:07,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,484 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,484 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,484 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,485 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 3020; bufvoid = 104857600 2012-11-05 12:52:07,485 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26213292(104853168); length = 1105/6553600 2012-11-05 12:52:07,494 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,496 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000004_0 is done. And is in the process of committing 2012-11-05 12:52:07,498 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,498 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000004_0' done. 2012-11-05 12:52:07,498 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000004_0 2012-11-05 12:52:07,498 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000005_0 2012-11-05 12:52:07,500 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@1fe3515 2012-11-05 12:52:07,502 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/hadoop-metrics2.properties:0+1488 2012-11-05 12:52:07,560 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,560 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,560 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,560 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,560 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,566 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,566 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,566 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,566 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 1710; bufvoid = 104857600 2012-11-05 12:52:07,567 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214160(104856640); length = 237/6553600 2012-11-05 12:52:07,574 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,577 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000005_0 is done. And is in the process of committing 2012-11-05 12:52:07,582 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,582 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000005_0' done. 2012-11-05 12:52:07,583 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000005_0 2012-11-05 12:52:07,583 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000006_0 2012-11-05 12:52:07,585 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@4bc5a5 2012-11-05 12:52:07,587 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/ssl-client.xml.example:0+1243 2012-11-05 12:52:07,697 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,697 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,697 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,697 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,698 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,704 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,704 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,704 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,704 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 1530; bufvoid = 104857600 2012-11-05 12:52:07,704 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214044(104856176); length = 353/6553600 2012-11-05 12:52:07,709 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,712 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000006_0 is done. And is in the process of committing 2012-11-05 12:52:07,713 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,714 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000006_0' done. 2012-11-05 12:52:07,714 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000006_0 2012-11-05 12:52:07,714 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000007_0 2012-11-05 12:52:07,716 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@15f7c4b 2012-11-05 12:52:07,718 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/ssl-server.xml.example:0+1195 2012-11-05 12:52:07,783 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,783 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,783 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,783 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,783 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,788 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,789 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,789 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,789 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 1470; bufvoid = 104857600 2012-11-05 12:52:07,789 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214060(104856240); length = 337/6553600 2012-11-05 12:52:07,793 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,795 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000007_0 is done. And is in the process of committing 2012-11-05 12:52:07,796 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,797 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000007_0' done. 2012-11-05 12:52:07,797 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000007_0 2012-11-05 12:52:07,797 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000008_0 2012-11-05 12:52:07,798 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@17eb194 2012-11-05 12:52:07,799 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/configuration.xsl:0+535 2012-11-05 12:52:07,856 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,856 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,856 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,856 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,856 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:07,861 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:07,861 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:07,861 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:07,861 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 666; bufvoid = 104857600 2012-11-05 12:52:07,861 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214260(104857040); length = 137/6553600 2012-11-05 12:52:07,866 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:07,869 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000008_0 is done. And is in the process of committing 2012-11-05 12:52:07,870 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:07,871 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000008_0' done. 2012-11-05 12:52:07,871 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000008_0 2012-11-05 12:52:07,871 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000009_0 2012-11-05 12:52:07,873 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@14924fb 2012-11-05 12:52:07,874 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/taskcontroller.cfg:0+382 2012-11-05 12:52:07,993 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:07,993 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:07,993 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:07,993 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:07,994 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,001 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,001 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,001 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,001 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 546; bufvoid = 104857600 2012-11-05 12:52:08,001 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214236(104856944); length = 161/6553600 2012-11-05 12:52:08,007 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,010 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000009_0 is done. And is in the process of committing 2012-11-05 12:52:08,011 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,011 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000009_0' done. 2012-11-05 12:52:08,012 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000009_0 2012-11-05 12:52:08,012 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000010_0 2012-11-05 12:52:08,014 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@1e54167 2012-11-05 12:52:08,016 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/fair-scheduler.xml:0+327 2012-11-05 12:52:08,075 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,076 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,076 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,076 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,076 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,085 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,085 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,085 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,085 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 460; bufvoid = 104857600 2012-11-05 12:52:08,085 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214256(104857024); length = 141/6553600 2012-11-05 12:52:08,088 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,091 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000010_0 is done. And is in the process of committing 2012-11-05 12:52:08,092 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,092 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000010_0' done. 2012-11-05 12:52:08,092 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000010_0 2012-11-05 12:52:08,092 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000011_0 2012-11-05 12:52:08,094 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@ccde81 2012-11-05 12:52:08,095 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/core-site.xml:0+178 2012-11-05 12:52:08,154 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,154 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,154 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,154 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,154 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,157 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,157 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,157 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,157 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 239; bufvoid = 104857600 2012-11-05 12:52:08,157 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214336(104857344); length = 61/6553600 2012-11-05 12:52:08,162 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,168 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000011_0 is done. And is in the process of committing 2012-11-05 12:52:08,170 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,170 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000011_0' done. 2012-11-05 12:52:08,170 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000011_0 2012-11-05 12:52:08,170 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000012_0 2012-11-05 12:52:08,172 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@fbee67 2012-11-05 12:52:08,173 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/mapred-site.xml:0+178 2012-11-05 12:52:08,302 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,302 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,302 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,302 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,302 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,391 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,391 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,391 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,391 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 239; bufvoid = 104857600 2012-11-05 12:52:08,392 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214336(104857344); length = 61/6553600 2012-11-05 12:52:08,398 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,402 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000012_0 is done. And is in the process of committing 2012-11-05 12:52:08,404 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,405 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000012_0' done. 2012-11-05 12:52:08,405 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000012_0 2012-11-05 12:52:08,405 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000013_0 2012-11-05 12:52:08,407 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@ac8211 2012-11-05 12:52:08,409 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/hdfs-site.xml:0+178 2012-11-05 12:52:08,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,468 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,487 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,488 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,488 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,488 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 239; bufvoid = 104857600 2012-11-05 12:52:08,488 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214336(104857344); length = 61/6553600 2012-11-05 12:52:08,491 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,493 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000013_0 is done. And is in the process of committing 2012-11-05 12:52:08,495 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,496 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000013_0' done. 2012-11-05 12:52:08,496 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000013_0 2012-11-05 12:52:08,496 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000014_0 2012-11-05 12:52:08,498 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@5a4a14 2012-11-05 12:52:08,500 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/masters:0+10 2012-11-05 12:52:08,558 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,559 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,559 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,559 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,559 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,562 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,562 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,562 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,562 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 14; bufvoid = 104857600 2012-11-05 12:52:08,563 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600 2012-11-05 12:52:08,565 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,568 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000014_0 is done. And is in the process of committing 2012-11-05 12:52:08,570 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,570 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000014_0' done. 2012-11-05 12:52:08,570 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000014_0 2012-11-05 12:52:08,571 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(213)) - Starting task: attempt_local_0001_m_000015_0 2012-11-05 12:52:08,572 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@527e31 2012-11-05 12:52:08,574 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(699)) - Processing split: file:/home/weijianzhongwj/software/hadoop-1.1.0/conf/slaves:0+10 2012-11-05 12:52:08,682 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:setEquator(1133)) - (EQUATOR) 0 kvi 26214396(104857584) 2012-11-05 12:52:08,682 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(929)) - mapreduce.task.io.sort.mb: 100 2012-11-05 12:52:08,682 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(930)) - soft limit at 83886080 2012-11-05 12:52:08,682 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(931)) - bufstart = 0; bufvoid = 104857600 2012-11-05 12:52:08,682 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:<init>(932)) - kvstart = 26214396; length = 6553600 2012-11-05 12:52:08,688 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:52:08,688 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1395)) - Starting flush of map output 2012-11-05 12:52:08,688 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1414)) - Spilling map output 2012-11-05 12:52:08,688 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1415)) - bufstart = 0; bufend = 14; bufvoid = 104857600 2012-11-05 12:52:08,688 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1417)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600 2012-11-05 12:52:08,693 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1603)) - Finished spill 0 2012-11-05 12:52:08,695 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_m_000015_0 is done. And is in the process of committing 2012-11-05 12:52:08,698 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - map 2012-11-05 12:52:08,699 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_m_000015_0' done. 2012-11-05 12:52:08,699 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:run(238)) - Finishing task: attempt_local_0001_m_000015_0 2012-11-05 12:52:08,699 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:run(394)) - Map task executor complete. 2012-11-05 12:52:58,415 INFO [Thread-10] mapred.Task (Task.java:initialize(566)) - Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@e85102 2012-11-05 12:58:16,363 INFO [Thread-10] mapred.Merger (Merger.java:merge(549)) - Merging 16 sorted segments 2012-11-05 12:59:41,406 INFO [Thread-10] mapred.Merger (Merger.java:merge(653)) - Merging 7 intermediate segments out of a total of 16 2012-11-05 12:59:44,605 INFO [Thread-10] mapred.Merger (Merger.java:merge(648)) - Down to the last merge-pass, with 10 segments left of total size: 22378 bytes 2012-11-05 12:59:44,966 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - 2012-11-05 12:59:46,437 INFO [communication thread] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - reduce > reduce 2012-11-05 12:59:46,486 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1295)) - map 100% reduce 66% 2012-11-05 12:59:47,163 WARN [Thread-10] conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords 2012-11-05 12:59:47,198 INFO [Thread-10] mapred.Task (Task.java:done(980)) - Task:attempt_local_0001_r_000000_0 is done. And is in the process of committing 2012-11-05 12:59:47,198 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - reduce > reduce 2012-11-05 12:59:47,198 INFO [Thread-10] mapred.Task (Task.java:commit(1141)) - Task attempt_local_0001_r_000000_0 is allowed to commit now 2012-11-05 12:59:47,199 INFO [Thread-10] output.FileOutputCommitter (FileOutputCommitter.java:commitTask(432)) - Saved output of task 'attempt_local_0001_r_000000_0' to file:/home/weijianzhongwj/software/hadoop-1.1.0/out/_temporary/0/task_local_0001_r_000000 2012-11-05 12:59:47,199 INFO [Thread-10] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(501)) - reduce > reduce 2012-11-05 12:59:47,200 INFO [Thread-10] mapred.Task (Task.java:sendDone(1100)) - Task 'attempt_local_0001_r_000000_0' done. 2012-11-05 12:59:47,486 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1295)) - map 100% reduce 100% 2012-11-05 12:59:47,487 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1306)) - Job job_local_0001 completed successfully 2012-11-05 12:59:47,602 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1313)) - Counters: 27 File System Counters FILE: Number of bytes read=667992 FILE: Number of bytes written=2711113 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 Map-Reduce Framework Map input records=749 Map output records=2585 Map output bytes=35514 Map output materialized bytes=22535 Input split bytes=2154 Combine input records=2585 Combine output records=1089 Reduce input groups=789 Reduce shuffle bytes=0 Reduce input records=1089 Reduce output records=789 Spilled Records=2293 Shuffled Maps =0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=603 CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes)=4293918720 File Input Format Counters Bytes Read=26536 File Output Format Counters Bytes Written=15413
有很多的统计信息
我们再看下最终的输出目录：
0
顶
顶
0
踩
踩
分享到：
发表评论
-
对字符串进行验证之前先进行规范化
2013-09-17 23:18 13888对字符串进行验证之前先进行规范化 应用系统中经常对字 ... -
在Apache Tomcat 7设置redis作为session store
2013-08-21 17:00 1955原文：http://www.javaarch.ne ... -
使用telnet连接到基于spring的应用上执行容器中的bean的任意方法
2013-08-08 09:17 1433使用telnet连接到基于spring的应用上执行容器中 ... -
jdk7和8的一些新特性介绍
2013-07-06 16:07 10071更多ppt内容请查看：htt ... -
java对于接口和抽象类的代理实现，不需要有具体实现类
2013-06-12 09:50 2907原文链接：http://www.javaarch.net/j ... -
Excel2007格式分析和XML解析
2013-06-07 09:56 10612在物料清单采购中，用到excel上传文件解析功能，不 ... -
Java EE 7中对WebSocket 1.0的支持
2013-06-05 09:27 3799原文链接：http://www.javaarch.n ... -
java QRCode生成示例
2013-06-05 09:26 1468原文链接：http://www.javaarch.n ... -
Java Web使用swfobject调用flex图表
2013-05-28 19:05 1079Java Web使用swfobject调用 ... -
spring使用PropertyPlaceholderConfigurer扩展来满足不同环境的参数配置
2013-05-21 15:57 3289spring使用PropertyPlaceholderCon ... -
java国际化
2013-05-20 20:57 4438java国际化 本文来自：http://www.j ... -
RSS feeds with Java
2013-05-20 20:52 1184RSS feeds with Java 原文来自：htt ... -
使用ibatis将数据库从oracle迁移到mysql的几个修改点
2013-04-29 10:40 1635我们项目在公司的大战略下需要从oracle ... -
线上机器jvm dump分析脚本
2013-04-19 10:48 2863#!/bin/sh DUMP_PIDS=`p ... -
eclipse远程部署，静态文件实时同步插件
2013-04-06 20:18 5418eclipse 远程文件实时同步，eclipse远程 ... -
java价格处理的一个问题
2013-03-26 21:21 1794我们经常会处理一些价格，比如从运营上传的文件中将某 ... -
java 服务降级开关设计思路
2013-03-23 16:35 3720java 服务屏蔽开关系统，可以手工降级服务，关闭服 ... -
poi解析excel内存溢出
2013-03-20 22:21 6346真是悲剧啊，一个破内部使用系统20多个人使用的后 ... -
简单web安全框架
2013-03-16 11:56 1499web安全框架,主要用servlet filter方 ... -
基于servlet的简单的页面缓存框架
2013-03-11 19:27 1177基于servlet的页面级缓存框架的基本用法： 代码参考： ...
相关推荐
一套eclipse中的hadoop开发环境搭建教程，附带了eclipse中的hadoop的两个demo，一个是wordcount、一个是sort，大家一起学习进步！
文档详细的描述了Hadoop在Linux上的安装过程，并且附带了Wordcount程序示例
hadoop wordcount 打包部署
eclipse调试hadoop图示，阐述调试过程，有图有真相
hadoop wordcount2.0 包含省略标点，忽略大小写等内容
win7上eclipse调试hadoop必备
基于Windows eclipse maven Hadoop 的WordCount源码。下载后需要自己编译jar包，详情请看解压后的README.md
ubuntu上面运行hadoop自带的wordcount实例
在Windwos环境下使用Eclipse调试hadoop云计算环境[亲测成功]
使用hadoop实现WordCount详细实验报告，配有环境变量配置截图以及实验运行及结果详细过程描述与截图
eclipse配置hadoop，并且如何在eclipse中进行mapreduce的开发
Eclipse中远程调试Hadoop必备资料:hadoop-eclipse-plugin-1.1.1和hadoop-core-1.0.2-modified；已经在eclipse-jee-juno-SR1-win32-x86_64和hadoop1.1.1 下测试过。
1. 资源包括了eclipse Oxygen版本的hadoop插件，包括winutils 2. 文档详细介绍了windows 下的hadoop版本的配置 3. 实例程序可以跑通，有详细截图
Hadoop在eclipse上面安装插件，以及对Hadoop jar包和hadoop-common jar包的安装，环境变量的配置
用于windows下eclipse连接hadoop2.2.0的插件以及eclipse版本，亲测可用
Linux和Win下配置eclipse开发hadoop
window 下eclipse远程访问伪分布式hadoop 运行wordcount
hadoop wordcount 实验，成功运行 maven项目（全部源代码）
myeclipse或eclipse配置hadoop2.6的插件 包括hadoop-eclipse-plugin-2.6.0.jar winutils.exe hadoop.dll三个文件
eclipse-hadoop3x-master如题eclipse-hadoop3x-master，eclipse-hadoop3x-master，eclipse-hadoop3x-master