See the following commands and tutorials to learn how to test IO speed, debug and profile, do log file performance analysis, and more.

How-tos

Test IO speed

Use the following command to test IO speed.

hadoop jar /usr/lib/hadoop/hadoop-test-0.20.2+737.jar TestDFSIO -write -nrFiles 1 -fileSize 2000

Tutorials

Debugging and profiling with IsolationRunner

See: https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#IsolationRunner

Vaidya log file performance analysis

See: https://hadoop.apache.org/docs/r1.2.1/vaidya.html

Profiling

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Profiling

Debug scripts

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Debugging

Features

The distributed cache

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#DistributedCache

The HDFS rebalancing feature

See: http://hadoop.apache.org/docs/r1.2.1/hdfs_user_guide.html#Rebalancer