Package | Description |
---|---|
org.apache.hadoop.examples |
Hadoop example code.
|
org.apache.hadoop.examples.pi |
This package consists of a map/reduce application,
distbbp,
which computes exact binary digits of the mathematical constant π.
|
org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
org.apache.hadoop.mapred.gridmix | |
org.apache.hadoop.mapreduce | |
org.apache.hadoop.mapreduce.lib.aggregate | |
org.apache.hadoop.mapreduce.lib.chain | |
org.apache.hadoop.mapreduce.lib.db | |
org.apache.hadoop.mapreduce.lib.input | |
org.apache.hadoop.mapreduce.lib.jobcontrol | |
org.apache.hadoop.mapreduce.lib.map | |
org.apache.hadoop.mapreduce.lib.output | |
org.apache.hadoop.mapreduce.lib.partition | |
org.apache.hadoop.mapreduce.tools | |
org.apache.hadoop.tools |
Command-line tools for MapReduce.
|
org.apache.hadoop.tools.mapred |
Modifier and Type | Method and Description |
---|---|
Job |
Sort.getResult()
Get the last job that was run using this instance.
|
Modifier and Type | Method and Description |
---|---|
void |
DistSum.MapSide.init(Job job)
Initialize a job
|
void |
DistSum.ReduceSide.init(Job job)
Initialize a job
|
void |
DistSum.MixMachine.init(Job job)
Initialize a job
|
Modifier and Type | Method and Description |
---|---|
static void |
TeraSort.setOutputReplication(Job job,
int value) |
static void |
TeraSort.setUseSimplePartitioner(Job job,
boolean value) |
Modifier and Type | Method and Description |
---|---|
Job |
SleepJob.call() |
Modifier and Type | Method and Description |
---|---|
static org.apache.hadoop.mapred.gridmix.Statistics.JobStats |
Statistics.generateJobStats(Job job,
JobStory jobdesc)
Generates a job stats.
|
Modifier and Type | Method and Description |
---|---|
Job[] |
Cluster.getAllJobs()
Deprecated.
Use
Cluster.getAllJobStatuses() instead. |
static Job |
Job.getInstance()
|
static Job |
Job.getInstance(Cluster ignored)
Deprecated.
Use
getInstance() |
static Job |
Job.getInstance(Cluster ignored,
Configuration conf)
Deprecated.
|
static Job |
Job.getInstance(Cluster cluster,
JobStatus status,
Configuration conf)
|
static Job |
Job.getInstance(Configuration conf)
|
static Job |
Job.getInstance(Configuration conf,
String jobName)
|
static Job |
Job.getInstance(JobStatus status,
Configuration conf)
|
Job |
Cluster.getJob(JobID jobId)
Get job corresponding to jobid.
|
Modifier and Type | Method and Description |
---|---|
static Job |
ValueAggregatorJob.createValueAggregatorJob(Configuration conf,
String[] args)
Create an Aggregate based map/reduce job.
|
static Job |
ValueAggregatorJob.createValueAggregatorJob(String[] args,
Class<? extends ValueAggregatorDescriptor>[] descriptors) |
Modifier and Type | Method and Description |
---|---|
protected static void |
Chain.addMapper(boolean isMap,
Job job,
Class<? extends Mapper> klass,
Class<?> inputKeyClass,
Class<?> inputValueClass,
Class<?> outputKeyClass,
Class<?> outputValueClass,
Configuration mapperConf)
Adds a Mapper class to the chain job.
|
static void |
ChainMapper.addMapper(Job job,
Class<? extends Mapper> klass,
Class<?> inputKeyClass,
Class<?> inputValueClass,
Class<?> outputKeyClass,
Class<?> outputValueClass,
Configuration mapperConf)
Adds a
Mapper class to the chain mapper. |
static void |
ChainReducer.addMapper(Job job,
Class<? extends Mapper> klass,
Class<?> inputKeyClass,
Class<?> inputValueClass,
Class<?> outputKeyClass,
Class<?> outputValueClass,
Configuration mapperConf)
Adds a
Mapper class to the chain reducer. |
protected static void |
Chain.setReducer(Job job,
Class<? extends Reducer> klass,
Class<?> inputKeyClass,
Class<?> inputValueClass,
Class<?> outputKeyClass,
Class<?> outputValueClass,
Configuration reducerConf)
Sets the Reducer class to the chain job.
|
static void |
ChainReducer.setReducer(Job job,
Class<? extends Reducer> klass,
Class<?> inputKeyClass,
Class<?> inputValueClass,
Class<?> outputKeyClass,
Class<?> outputValueClass,
Configuration reducerConf)
Sets the
Reducer class to the chain job. |
Modifier and Type | Method and Description |
---|---|
static void |
DataDrivenDBInputFormat.setInput(Job job,
Class<? extends DBWritable> inputClass,
String inputQuery,
String inputBoundingQuery)
setInput() takes a custom query and a separate "bounding query" to use
instead of the custom "count query" used by DBInputFormat.
|
static void |
DBInputFormat.setInput(Job job,
Class<? extends DBWritable> inputClass,
String inputQuery,
String inputCountQuery)
Initializes the map-part of the job with the appropriate input settings.
|
static void |
DataDrivenDBInputFormat.setInput(Job job,
Class<? extends DBWritable> inputClass,
String tableName,
String conditions,
String splitBy,
String... fieldNames)
Note that the "orderBy" column is called the "splitBy" in this version.
|
static void |
DBInputFormat.setInput(Job job,
Class<? extends DBWritable> inputClass,
String tableName,
String conditions,
String orderBy,
String... fieldNames)
Initializes the map-part of the job with the appropriate input settings.
|
static void |
DBOutputFormat.setOutput(Job job,
String tableName,
int fieldCount)
Initializes the reduce-part of the job
with the appropriate output settings
|
static void |
DBOutputFormat.setOutput(Job job,
String tableName,
String... fieldNames)
Initializes the reduce-part of the job with
the appropriate output settings
|
Modifier and Type | Method and Description |
---|---|
static void |
FileInputFormat.addInputPath(Job job,
Path path)
Add a
Path to the list of inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
static void |
FileInputFormat.addInputPaths(Job job,
String commaSeparatedPaths)
Add the given comma separated paths to the list of inputs for
the map-reduce job.
|
static void |
SequenceFileInputFilter.setFilterClass(Job job,
Class<?> filterClass)
set the filter class
|
static void |
FileInputFormat.setInputDirRecursive(Job job,
boolean inputDirRecursive) |
static void |
FileInputFormat.setInputPathFilter(Job job,
Class<? extends PathFilter> filter)
Set a PathFilter to be applied to the input paths for the map-reduce job.
|
static void |
FileInputFormat.setInputPaths(Job job,
Path... inputPaths)
Set the array of
Path s as the list of inputs
for the map-reduce job. |
static void |
FileInputFormat.setInputPaths(Job job,
String commaSeparatedPaths)
Sets the given comma separated paths as the list of inputs
for the map-reduce job.
|
static void |
FileInputFormat.setMaxInputSplitSize(Job job,
long size)
Set the maximum split size
|
static void |
FileInputFormat.setMinInputSplitSize(Job job,
long size)
Set the minimum input split size
|
static void |
NLineInputFormat.setNumLinesPerSplit(Job job,
int numLines)
Set the number of lines per split
|
Modifier and Type | Method and Description |
---|---|
Job |
ControlledJob.getJob() |
Modifier and Type | Method and Description |
---|---|
void |
ControlledJob.setJob(Job job)
Set the mapreduce job
|
Constructor and Description |
---|
ControlledJob(Job job,
List<ControlledJob> dependingJobs)
Construct a job.
|
Modifier and Type | Method and Description |
---|---|
static <K1,V1,K2,V2> |
MultithreadedMapper.setMapperClass(Job job,
Class<? extends Mapper<K1,V1,K2,V2>> cls)
Set the application's mapper class.
|
static void |
MultithreadedMapper.setNumberOfThreads(Job job,
int threads)
Set the number of threads in the pool for running maps.
|
Modifier and Type | Method and Description |
---|---|
static void |
MultipleOutputs.addNamedOutput(Job job,
String namedOutput,
Class<? extends OutputFormat> outputFormatClass,
Class<?> keyClass,
Class<?> valueClass)
Adds a named output for the job.
|
static void |
FileOutputFormat.setCompressOutput(Job job,
boolean compress)
Set whether the output of the job is compressed.
|
static void |
MultipleOutputs.setCountersEnabled(Job job,
boolean enabled)
Enables or disables counters for the named outputs.
|
static void |
SequenceFileOutputFormat.setOutputCompressionType(Job job,
SequenceFile.CompressionType style)
Set the
SequenceFile.CompressionType for the output SequenceFile . |
static void |
FileOutputFormat.setOutputCompressorClass(Job job,
Class<? extends CompressionCodec> codecClass)
Set the
CompressionCodec to be used to compress job outputs. |
static void |
LazyOutputFormat.setOutputFormatClass(Job job,
Class<? extends OutputFormat> theClass)
Set the underlying output format for LazyOutputFormat.
|
static void |
FileOutputFormat.setOutputPath(Job job,
Path outputDir)
Set the
Path of the output directory for the map-reduce job. |
static void |
SequenceFileAsBinaryOutputFormat.setSequenceFileOutputKeyClass(Job job,
Class<?> theClass)
Set the key class for the
SequenceFile |
static void |
SequenceFileAsBinaryOutputFormat.setSequenceFileOutputValueClass(Job job,
Class<?> theClass)
Set the value class for the
SequenceFile |
Modifier and Type | Method and Description |
---|---|
K[] |
InputSampler.Sampler.getSample(InputFormat<K,V> inf,
Job job)
For a given job, collect and return a subset of the keys from the
input data.
|
K[] |
InputSampler.SplitSampler.getSample(InputFormat<K,V> inf,
Job job)
From each split sampled, take the first numSamples / numSplits records.
|
K[] |
InputSampler.RandomSampler.getSample(InputFormat<K,V> inf,
Job job)
Randomize the split order, then take the specified number of keys from
each split sampled, where each key is selected with the specified
probability and possibly replaced by a subsequently selected key when
the quota of keys from that split is satisfied.
|
K[] |
InputSampler.IntervalSampler.getSample(InputFormat<K,V> inf,
Job job)
For each split sampled, emit when the ratio of the number of records
retained to the total record count is less than the specified
frequency.
|
static void |
KeyFieldBasedComparator.setKeyFieldComparatorOptions(Job job,
String keySpec)
Set the
KeyFieldBasedComparator options used to compare keys. |
void |
KeyFieldBasedPartitioner.setKeyFieldPartitionerOptions(Job job,
String keySpec)
Set the
KeyFieldBasedPartitioner options used for
Partitioner |
static <K,V> void |
InputSampler.writePartitionFile(Job job,
InputSampler.Sampler<K,V> sampler)
Write a partition file for the given job, using the Sampler provided.
|
Modifier and Type | Method and Description |
---|---|
protected void |
CLI.displayTasks(Job job,
String type,
String state)
Display the information about a job's tasks, of a particular type and
in a particular state
|
Modifier and Type | Method and Description |
---|---|
Job |
DistCp.execute()
Implements the core-execution.
|
Modifier and Type | Method and Description |
---|---|
static Path |
CopyOutputFormat.getCommitDirectory(Job job)
Getter for the final commit-directory.
|
static Path |
CopyOutputFormat.getWorkingDirectory(Job job)
Getter for the working directory.
|
static void |
CopyOutputFormat.setCommitDirectory(Job job,
Path commitDirectory)
Setter for the final directory for DistCp (where files copied will be
moved, atomically.)
|
static void |
CopyOutputFormat.setWorkingDirectory(Job job,
Path workingDirectory)
Setter for the working directory for DistCp (where files will be copied
before they are moved to the final commit-directory.)
|
Copyright © 2013 Apache Software Foundation. All rights reserved.