public class TeraInputFormat extends FileInputFormat<Text,Text>
FileInputFormat.Counter
INPUT_DIR, INPUT_DIR_RECURSIVE, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE
Constructor and Description |
---|
TeraInputFormat() |
Modifier and Type | Method and Description |
---|---|
RecordReader<Text,Text> |
createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split.
|
List<InputSplit> |
getSplits(JobContext job)
Generate the list of files and make them into FileSplits.
|
static void |
writePartitionFile(JobContext job,
Path partFile)
Use the input splits to take samples of the input and generate sample
keys.
|
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, listStatus, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize
public static void writePartitionFile(JobContext job, Path partFile) throws Throwable
job
- the job to samplepartFile
- where to write the output file toThrowable
- if something goes wrongpublic RecordReader<Text,Text> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException
InputFormat
RecordReader.initialize(InputSplit, TaskAttemptContext)
before
the split is used.createRecordReader
in class InputFormat<Text,Text>
split
- the split to be readcontext
- the information about the taskIOException
public List<InputSplit> getSplits(JobContext job) throws IOException
FileInputFormat
getSplits
in class FileInputFormat<Text,Text>
job
- the job contextInputSplit
s for the job.IOException
Copyright © 2013 Apache Software Foundation. All rights reserved.