public class PigOutputFormat extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
Modifier and Type | Class and Description |
---|---|
static class |
PigOutputFormat.PigRecordWriter
Wrapper class which will delegate calls to the actual RecordWriter - this
should only get called in the single store case.
|
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
MAPRED_OUTPUT_DIR
hadoop job output directory
|
static java.lang.String |
MAPRED_TASK_PARTITION
hadoop partition number
|
static java.lang.String |
PIG_MAPRED_OUTPUT_DIR
the temporary directory for the multi store
|
static java.lang.String |
PIG_TMP_PATH
the relative path that can be used to build a temporary
place to store the output from a number of map-reduce tasks
|
Constructor and Description |
---|
PigOutputFormat() |
Modifier and Type | Method and Description |
---|---|
void |
checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobcontext) |
org.apache.hadoop.mapreduce.OutputCommitter |
getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext) |
org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable,Tuple> |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext) |
static void |
setLocation(org.apache.hadoop.mapreduce.JobContext jobContext,
POStore store)
Before delegating calls to underlying OutputFormat or OutputCommitter
Pig needs to ensure the Configuration in the JobContext contains
the output location and StoreFunc
for the specific store - so set these up in the context for this specific
store
|
public static final java.lang.String MAPRED_OUTPUT_DIR
public static final java.lang.String MAPRED_TASK_PARTITION
public static final java.lang.String PIG_MAPRED_OUTPUT_DIR
public static final java.lang.String PIG_TMP_PATH
public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable,Tuple> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext) throws java.io.IOException, java.lang.InterruptedException
getRecordWriter
in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
java.io.IOException
java.lang.InterruptedException
public static void setLocation(org.apache.hadoop.mapreduce.JobContext jobContext, POStore store) throws java.io.IOException
jobContext
- the JobContext
store
- the POStorejava.io.IOException
- on failurepublic void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobcontext) throws java.io.IOException, java.lang.InterruptedException
checkOutputSpecs
in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
java.io.IOException
java.lang.InterruptedException
public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext) throws java.io.IOException, java.lang.InterruptedException
getOutputCommitter
in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
java.io.IOException
java.lang.InterruptedException
Copyright © 2007-2012 The Apache Software Foundation