Package | Description |
---|---|
org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
org.apache.hadoop.mapred | |
org.apache.hadoop.mapreduce.task.reduce | |
org.apache.hadoop.util |
Common utilities.
|
Modifier and Type | Method and Description |
---|---|
Progress |
SequenceFile.Sorter.RawKeyValueIterator.getProgress()
Gets the Progress object; this has a float (0.0 - 1.0)
indicating the bytes processed by the iterator so far
|
Modifier and Type | Method and Description |
---|---|
Progress |
ShuffleConsumerPlugin.Context.getCopyPhase() |
Progress |
ShuffleConsumerPlugin.Context.getMergePhase() |
Progress |
RawKeyValueIterator.getProgress()
Gets the Progress object; this has a float (0.0 - 1.0)
indicating the bytes processed by the iterator so far
|
Progress |
Task.getProgress() |
Progress |
MapTask.MapOutputBuffer.MRResultIterator.getProgress() |
Progress |
MapTask.getSortPhase() |
Modifier and Type | Method and Description |
---|---|
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
int inMemSegments,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Counters.Counter mergedMapOutputsCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
int inMemSegments,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
Constructor and Description |
---|
ShuffleConsumerPlugin.Context(TaskAttemptID reduceId,
JobConf jobConf,
FileSystem localFS,
TaskUmbilicalProtocol umbilical,
LocalDirAllocator localDirAllocator,
Reporter reporter,
CompressionCodec codec,
Class<? extends Reducer> combinerClass,
Task.CombineOutputCollector<K,V> combineCollector,
Counters.Counter spilledRecordsCounter,
Counters.Counter reduceCombineInputCounter,
Counters.Counter shuffledMapsCounter,
Counters.Counter reduceShuffleBytes,
Counters.Counter failedShuffleCounter,
Counters.Counter mergedMapOutputsCounter,
TaskStatus status,
Progress copyPhase,
Progress mergePhase,
Task reduceTask,
MapOutputFile mapOutputFile) |
Constructor and Description |
---|
MergeManagerImpl(TaskAttemptID reduceId,
JobConf jobConf,
FileSystem localFS,
LocalDirAllocator localDirAllocator,
Reporter reporter,
CompressionCodec codec,
Class<? extends Reducer> combinerClass,
Task.CombineOutputCollector<K,V> combineCollector,
Counters.Counter spilledRecordsCounter,
Counters.Counter reduceCombineInputCounter,
Counters.Counter mergedMapOutputsCounter,
ExceptionReporter exceptionReporter,
Progress mergePhase,
MapOutputFile mapOutputFile) |
ShuffleSchedulerImpl(JobConf job,
TaskStatus status,
TaskAttemptID reduceId,
ExceptionReporter reporter,
Progress progress,
Counters.Counter shuffledMapsCounter,
Counters.Counter reduceShuffleBytes,
Counters.Counter failedShuffleCounter) |
Modifier and Type | Method and Description |
---|---|
Progress |
Progress.addPhase()
Adds a node to the tree.
|
Progress |
Progress.addPhase(float weightage)
Adds a node with a specified progress weightage to the tree.
|
Progress |
Progress.addPhase(String status)
Adds a named node to the tree.
|
Progress |
Progress.addPhase(String status,
float weightage)
Adds a named node with a specified progress weightage to the tree.
|
Progress |
Progress.phase()
Returns the current sub-node executing.
|
Copyright © 2013 Apache Software Foundation. All rights reserved.