Package | Description |
---|---|
org.apache.hadoop.examples |
Hadoop example code.
|
org.apache.hadoop.examples.pi |
This package consists of a map/reduce application,
distbbp,
which computes exact binary digits of the mathematical constant π.
|
org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
org.apache.hadoop.mapred | |
org.apache.hadoop.mapred.gridmix | |
org.apache.hadoop.mapred.lib | |
org.apache.hadoop.mapreduce | |
org.apache.hadoop.mapreduce.lib.db | |
org.apache.hadoop.mapreduce.lib.input | |
org.apache.hadoop.mapreduce.lib.join | |
org.apache.hadoop.mapreduce.lib.map | |
org.apache.hadoop.mapreduce.lib.output | |
org.apache.hadoop.mapreduce.lib.reduce | |
org.apache.hadoop.mapreduce.task | |
org.apache.hadoop.mapreduce.v2.app.commit | |
org.apache.hadoop.tools.mapred | |
org.apache.hadoop.tools.mapred.lib |
Modifier and Type | Method and Description |
---|---|
RecordReader<LongWritable,IntWritable> |
BaileyBorweinPlouffe.BbpInputFormat.createRecordReader(InputSplit generic,
TaskAttemptContext context)
Create a record reader for a given split.
|
RecordReader<MultiFileWordCount.WordOffset,Text> |
MultiFileWordCount.MyInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
void |
MultiFileWordCount.CombineFileLineRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
Constructor and Description |
---|
MultiFileWordCount.CombineFileLineRecordReader(CombineFileSplit split,
TaskAttemptContext context,
Integer index) |
Modifier and Type | Method and Description |
---|---|
RecordReader<NullWritable,SummationWritable> |
DistSum.Machine.AbstractInputFormat.createRecordReader(InputSplit generic,
TaskAttemptContext context)
Specify how to read the records
|
Modifier and Type | Method and Description |
---|---|
RecordReader<Text,Text> |
TeraInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
OutputCommitter |
TeraOutputFormat.getOutputCommitter(TaskAttemptContext context) |
RecordWriter<Text,Text> |
TeraOutputFormat.getRecordWriter(TaskAttemptContext job) |
Modifier and Type | Interface and Description |
---|---|
interface |
TaskAttemptContext |
Modifier and Type | Method and Description |
---|---|
void |
OutputCommitter.abortTask(TaskAttemptContext taskContext)
This method implements the new interface by calling the old method.
|
void |
OutputCommitter.commitTask(TaskAttemptContext taskContext)
This method implements the new interface by calling the old method.
|
boolean |
OutputCommitter.needsTaskCommit(TaskAttemptContext taskContext)
This method implements the new interface by calling the old method.
|
void |
OutputCommitter.recoverTask(TaskAttemptContext taskContext)
This method implements the new interface by calling the old method.
|
void |
OutputCommitter.setupTask(TaskAttemptContext taskContext)
This method implements the new interface by calling the old method.
|
Modifier and Type | Method and Description |
---|---|
RecordReader<LongWritable,LongWritable> |
SleepJob.SleepInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
Modifier and Type | Method and Description |
---|---|
RecordReader<K,V> |
CombineFileInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
Modifier and Type | Interface and Description |
---|---|
interface |
MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper . |
interface |
ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer . |
interface |
TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
Modifier and Type | Class and Description |
---|---|
class |
Mapper.Context
The
Context passed on to the Mapper implementations. |
class |
Reducer.Context
The
Context passed on to the Reducer implementations. |
Modifier and Type | Method and Description |
---|---|
abstract void |
OutputCommitter.abortTask(TaskAttemptContext taskContext)
Discard the task output.
|
abstract void |
RecordWriter.close(TaskAttemptContext context)
Close this
RecordWriter to future operations. |
abstract void |
OutputCommitter.commitTask(TaskAttemptContext taskContext)
To promote the task's temporary output to final output location.
|
abstract RecordReader<K,V> |
InputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split.
|
abstract OutputCommitter |
OutputFormat.getOutputCommitter(TaskAttemptContext context)
Get the output committer for this output format.
|
abstract RecordWriter<K,V> |
OutputFormat.getRecordWriter(TaskAttemptContext context)
Get the
RecordWriter for the given task. |
abstract void |
RecordReader.initialize(InputSplit split,
TaskAttemptContext context)
Called once at initialization.
|
abstract boolean |
OutputCommitter.needsTaskCommit(TaskAttemptContext taskContext)
Check whether task needs a commit.
|
void |
OutputCommitter.recoverTask(TaskAttemptContext taskContext)
Recover the task output.
|
abstract void |
OutputCommitter.setupTask(TaskAttemptContext taskContext)
Sets up output for the task.
|
Modifier and Type | Method and Description |
---|---|
void |
DBOutputFormat.DBRecordWriter.close(TaskAttemptContext context)
Close this
RecordWriter to future operations. |
RecordReader<LongWritable,T> |
DBInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split.
|
OutputCommitter |
DBOutputFormat.getOutputCommitter(TaskAttemptContext context) |
RecordWriter<K,V> |
DBOutputFormat.getRecordWriter(TaskAttemptContext context)
Get the
RecordWriter for the given task. |
void |
DBRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
Modifier and Type | Field and Description |
---|---|
protected TaskAttemptContext |
CombineFileRecordReader.context |
Modifier and Type | Method and Description |
---|---|
abstract RecordReader<K,V> |
CombineFileInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context)
This is not implemented yet.
|
RecordReader<LongWritable,Text> |
TextInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<K,V> |
SequenceFileInputFilter.createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for the given split
|
RecordReader<LongWritable,Text> |
CombineTextInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<K,V> |
CombineSequenceFileInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<BytesWritable,BytesWritable> |
SequenceFileAsBinaryInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<K,V> |
DelegatingInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<LongWritable,Text> |
NLineInputFormat.createRecordReader(InputSplit genericSplit,
TaskAttemptContext context) |
RecordReader<K,V> |
SequenceFileInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<Text,Text> |
SequenceFileAsTextInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<Text,Text> |
KeyValueTextInputFormat.createRecordReader(InputSplit genericSplit,
TaskAttemptContext context) |
void |
SequenceFileRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
KeyValueLineRecordReader.initialize(InputSplit genericSplit,
TaskAttemptContext context) |
void |
DelegatingRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
CombineFileRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
SequenceFileAsBinaryInputFormat.SequenceFileAsBinaryRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
SequenceFileAsTextRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
LineRecordReader.initialize(InputSplit genericSplit,
TaskAttemptContext context) |
void |
CombineFileRecordReaderWrapper.initialize(InputSplit split,
TaskAttemptContext context) |
Constructor and Description |
---|
CombineFileRecordReader(CombineFileSplit split,
TaskAttemptContext context,
Class<? extends RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders
for each chunk in the CombineFileSplit.
|
CombineFileRecordReaderWrapper(FileInputFormat<K,V> inputFormat,
CombineFileSplit split,
TaskAttemptContext context,
Integer idx) |
DelegatingRecordReader(InputSplit split,
TaskAttemptContext context)
Constructs the DelegatingRecordReader.
|
Modifier and Type | Method and Description |
---|---|
abstract ComposableRecordReader<K,V> |
ComposableInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<K,TupleWritable> |
CompositeInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext taskContext)
Construct a CompositeRecordReader for the children of this InputFormat
as defined in the init expression.
|
void |
CompositeRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
WrappedRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
void |
MultiFilterRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
Modifier and Type | Class and Description |
---|---|
class |
WrappedMapper.Context |
Modifier and Type | Method and Description |
---|---|
void |
FileOutputCommitter.abortTask(TaskAttemptContext context)
Delete the work directory
|
void |
FileOutputCommitter.abortTask(TaskAttemptContext context,
Path taskAttemptPath) |
void |
PartialFileOutputCommitter.cleanUpPartialOutputForTask(TaskAttemptContext context) |
void |
PartialOutputCommitter.cleanUpPartialOutputForTask(TaskAttemptContext context)
Remove all previously committed outputs from prior executions of this task.
|
void |
TextOutputFormat.LineRecordWriter.close(TaskAttemptContext context) |
void |
FilterOutputFormat.FilterRecordWriter.close(TaskAttemptContext context) |
void |
FileOutputCommitter.commitTask(TaskAttemptContext context)
Move the files from the work directory to the job output directory
|
void |
FileOutputCommitter.commitTask(TaskAttemptContext context,
Path taskAttemptPath) |
Path |
PartialFileOutputCommitter.getCommittedTaskPath(int appAttemptId,
TaskAttemptContext context) |
protected Path |
FileOutputCommitter.getCommittedTaskPath(int appAttemptId,
TaskAttemptContext context)
Compute the path where the output of a committed task is stored until the
entire job is committed for a specific application attempt.
|
Path |
FileOutputCommitter.getCommittedTaskPath(TaskAttemptContext context)
Compute the path where the output of a committed task is stored until
the entire job is committed.
|
static Path |
FileOutputCommitter.getCommittedTaskPath(TaskAttemptContext context,
Path out) |
Path |
FileOutputFormat.getDefaultWorkFile(TaskAttemptContext context,
String extension)
Get the default path and filename for the output format.
|
OutputCommitter |
NullOutputFormat.getOutputCommitter(TaskAttemptContext context) |
OutputCommitter |
FilterOutputFormat.getOutputCommitter(TaskAttemptContext context) |
OutputCommitter |
FileOutputFormat.getOutputCommitter(TaskAttemptContext context) |
OutputCommitter |
LazyOutputFormat.getOutputCommitter(TaskAttemptContext context) |
RecordWriter<K,V> |
TextOutputFormat.getRecordWriter(TaskAttemptContext job) |
RecordWriter<K,V> |
SequenceFileOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<WritableComparable<?>,Writable> |
MapFileOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<K,V> |
NullOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<K,V> |
FilterOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<BytesWritable,BytesWritable> |
SequenceFileAsBinaryOutputFormat.getRecordWriter(TaskAttemptContext context) |
abstract RecordWriter<K,V> |
FileOutputFormat.getRecordWriter(TaskAttemptContext job) |
RecordWriter<K,V> |
LazyOutputFormat.getRecordWriter(TaskAttemptContext context) |
protected SequenceFile.Writer |
SequenceFileOutputFormat.getSequenceWriter(TaskAttemptContext context,
Class<?> keyClass,
Class<?> valueClass) |
Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context)
Compute the path where the output of a task attempt is stored until
that task is committed.
|
static Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context,
Path out)
Compute the path where the output of a task attempt is stored until
that task is committed.
|
static String |
FileOutputFormat.getUniqueFile(TaskAttemptContext context,
String name,
String extension)
Generate a unique filename, based on the task id, name, and extension
|
boolean |
FileOutputCommitter.needsTaskCommit(TaskAttemptContext context)
Did this task write any files in the work directory?
|
boolean |
FileOutputCommitter.needsTaskCommit(TaskAttemptContext context,
Path taskAttemptPath) |
void |
FileOutputCommitter.recoverTask(TaskAttemptContext context) |
void |
FileOutputCommitter.setupTask(TaskAttemptContext context)
No task setup required.
|
Constructor and Description |
---|
FileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
PartialFileOutputCommitter(Path outputPath,
TaskAttemptContext context) |
Modifier and Type | Class and Description |
---|---|
class |
WrappedReducer.Context |
Modifier and Type | Class and Description |
---|---|
class |
MapContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper . |
class |
ReduceContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer . |
class |
TaskAttemptContextImpl
The context for task attempts.
|
class |
TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
Modifier and Type | Method and Description |
---|---|
TaskAttemptContext |
CommitterTaskAbortEvent.getAttemptContext() |
Constructor and Description |
---|
CommitterTaskAbortEvent(TaskAttemptId attemptID,
TaskAttemptContext attemptContext) |
Modifier and Type | Method and Description |
---|---|
RecordReader<Text,FileStatus> |
UniformSizeInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context)
Implementation of InputFormat::createRecordReader().
|
OutputCommitter |
CopyOutputFormat.getOutputCommitter(TaskAttemptContext context) |
Constructor and Description |
---|
CopyCommitter(Path outputPath,
TaskAttemptContext context)
Create a output committer
|
Modifier and Type | Method and Description |
---|---|
RecordReader<K,V> |
DynamicInputFormat.createRecordReader(InputSplit inputSplit,
TaskAttemptContext taskAttemptContext)
Implementation of Inputformat::createRecordReader().
|
void |
DynamicRecordReader.initialize(InputSplit inputSplit,
TaskAttemptContext taskAttemptContext)
Implementation for RecordReader::initialize().
|
Copyright © 2013 Apache Software Foundation. All rights reserved.