public class MapReduceJobPropertiesParser extends Object implements JobPropertyParser
MapReduceJobPropertiesParser
is a utility class that
parses MapReduce job configuration properties and converts the value into a
well defined DataType
. Users can use the
parseJobProperty(String, String)
API to
process job configuration parameters. This API will parse a job property
represented as a key-value pair and return the value wrapped inside a
DataType
. Callers can then use the returned DataType
for
further processing.
MapReduceJobPropertiesParser
thrives on the key name to decide which
DataType
to wrap the value with. Values for keys representing
job-name, queue-name, user-name etc are wrapped inside JobName
,
QueueName
, UserName
etc respectively. Keys ending with *dir*
are considered as a directory and hence gets be wrapped inside
FileName
. Similarly key ending with *codec*, *log*, *class* etc are
also handled accordingly. Values representing basic java data-types like
integer, float, double, boolean etc are wrapped inside
DefaultDataType
. If the key represents some jvm-level settings then
only standard settings are extracted and gets wrapped inside
DefaultDataType
. Currently only '-Xmx' and '-Xms' settings are
considered while the rest are ignored.
Note that the parseJobProperty(String,
String)
API maps the keys to a configuration parameter listed in
MRJobConfig
. This not only filters non-framework specific keys thus
ignoring user-specific and hard-to-parse keys but also provides a consistent
view for all possible inputs. So if users invoke the
parseJobProperty(String, String)
API
with either <"mapreduce.job.user.name", "bob"> or <"user.name", "bob">, then
the result would be a UserName
DataType
wrapping the user-name "bob".Constructor and Description |
---|
MapReduceJobPropertiesParser() |
Modifier and Type | Method and Description |
---|---|
static void |
extractMaxHeapOpts(String javaOptions,
List<String> heapOpts,
List<String> others)
Extracts the -Xmx heap option from the specified string.
|
static void |
extractMinHeapOpts(String javaOptions,
List<String> heapOpts,
List<String> others)
Extracts the -Xms heap option from the specified string.
|
DataType<?> |
parseJobProperty(String key,
String value)
Parse the specified job configuration key-value pair.
|
public DataType<?> parseJobProperty(String key, String value)
JobPropertyParser
parseJobProperty
in interface JobPropertyParser
DataType
if this parser can parse this value.
Returns 'null' otherwise.public static void extractMaxHeapOpts(String javaOptions, List<String> heapOpts, List<String> others)
Copyright © 2013 Apache Software Foundation. All rights reserved.