Modifier and Type | Interface and Description |
---|---|
interface |
Reconfigurable
Something whose
Configuration can be changed at run time. |
Modifier and Type | Class and Description |
---|---|
class |
Configured
Base class for things that may be configured with a
Configuration . |
class |
ReconfigurableBase
Utility base class for implementing the Reconfigurable interface.
|
Modifier and Type | Class and Description |
---|---|
class |
BaileyBorweinPlouffe
A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
digits of Pi.
|
class |
DBCountPageView
This is a demonstrative program, which uses DBInputFormat for reading
the input data from a database, and DBOutputFormat for writing the data
to the database.
|
class |
Grep |
class |
Join
Given a set of sorted datasets keyed with the same class and yielding
equal partitions, it is possible to effect a join of those datasets
prior to the map.
|
class |
MultiFileWordCount
MultiFileWordCount is an example to demonstrate the usage of
MultiFileInputFormat.
|
class |
QuasiMonteCarlo
A map/reduce program that estimates the value of Pi
using a quasi-Monte Carlo (qMC) method.
|
class |
RandomTextWriter
This program uses map/reduce to just run a distributed job where there is
no interaction between the tasks and each task writes a large unsorted
random sequence of words.
|
class |
RandomWriter
This program uses map/reduce to just run a distributed job where there is
no interaction between the tasks and each task write a large unsorted
random binary sequence file of BytesWritable.
|
class |
Sort<K,V>
This is the trivial map/reduce program that does absolutely nothing
other than use the framework to fragment and sort the input values.
|
class |
WordMean |
class |
WordMedian |
class |
WordStandardDeviation |
Modifier and Type | Class and Description |
---|---|
class |
DistributedPentomino
Launch a distributed pentomino solver.
|
Modifier and Type | Class and Description |
---|---|
class |
DistBbp
A map/reduce program that uses a BBP-type method to compute exact
binary digits of Pi.
|
class |
DistSum
The main class for computing sums using map/reduce jobs.
|
Modifier and Type | Class and Description |
---|---|
class |
TeraChecksum |
class |
TeraGen
Generate the official GraySort input data set.
|
class |
TeraSort
Generates the sampled split points, launches the job, and waits for it to
finish.
|
static class |
TeraSort.SimplePartitioner
A total order partitioner that assigns keys based on their first
PREFIX_LENGTH bytes, assuming a flat distribution.
|
class |
TeraValidate
Generate 1 mapper per a file that checks to make sure the keys
are sorted within each file.
|
Modifier and Type | Class and Description |
---|---|
class |
ChecksumFileSystem
Abstract Checksumed FileSystem.
|
class |
FileSystem
An abstract base class for a fairly generic filesystem.
|
class |
FilterFileSystem
A
FilterFileSystem contains
some other file system, which it uses as
its basic file system, possibly transforming
the data along the way or providing additional
functionality. |
class |
FsShell
Provide command line access to a FileSystem.
|
protected class |
FsShell.Help
Displays short usage of commands sans the long description
|
protected class |
FsShell.Usage
Display help for commands with their short usage and long description
|
class |
FsShellPermissions
This class is the home for file permissions related commands.
|
static class |
FsShellPermissions.Chgrp
Used to change group of files
|
static class |
FsShellPermissions.Chmod
The pattern is almost as flexible as mode allowed by chmod shell command.
|
static class |
FsShellPermissions.Chown
Used to change owner and/or group of files
|
class |
HarFileSystem
This is an implementation of the Hadoop Archive
Filesystem.
|
class |
LocalFileSystem
Implement the FileSystem API for the checksumed local filesystem.
|
class |
RawLocalFileSystem
Implement the FileSystem API for the raw local filesystem.
|
class |
Trash
Provides a trash facility which supports pluggable Trash policies.
|
class |
TrashPolicy
This interface is used for implementing different Trash policies.
|
class |
TrashPolicyDefault
Provides a trash feature.
|
Modifier and Type | Class and Description |
---|---|
class |
FTPFileSystem
A
FileSystem backed by an FTP client provided by Apache Commons Net. |
Modifier and Type | Class and Description |
---|---|
class |
HttpFSFileSystem
HttpFSServer implementation of the FileSystemAccess FileSystem.
|
Modifier and Type | Class and Description |
---|---|
class |
MigrationTool
This class is a tool for migrating data from an older to a newer version
of an S3 filesystem.
|
class |
S3FileSystem
A block-based
FileSystem backed by
Amazon S3. |
Modifier and Type | Class and Description |
---|---|
class |
NativeS3FileSystem
A
FileSystem for reading and writing files stored on
Amazon S3. |
Modifier and Type | Class and Description |
---|---|
class |
Command
An abstract class for the execution of a file system command
|
class |
CommandFactory
class to search for and register commands
|
class |
Count
Count the number of directories, files, bytes, quota, and remaining quota.
|
class |
FsCommand
Base class for all "hadoop fs" commands
|
Modifier and Type | Class and Description |
---|---|
class |
ViewFileSystem
ViewFileSystem (extends the FileSystem interface) implements a client-side
mount table.
|
Modifier and Type | Class and Description |
---|---|
class |
HAAdmin
A command-line tool for making calls in the HAServiceProtocol.
|
class |
ShellCommandFencer
Fencing method that runs a shell command.
|
class |
SshFenceByTcpPort
This fencing implementation sshes to the target node and uses
fuser to kill the process listening on the service's
TCP port. |
Modifier and Type | Class and Description |
---|---|
class |
DistributedFileSystem
Implementation of the abstract FileSystem for the DFS system.
|
class |
HftpFileSystem
An implementation of a protocol for accessing filesystems over HTTP.
|
class |
HsftpFileSystem
An implementation of a protocol for accessing filesystems over HTTPS.
|
Modifier and Type | Class and Description |
---|---|
class |
JournalNode
The JournalNode is a daemon which allows namenodes using
the QuorumJournalManager to log and retrieve edits stored
remotely.
|
Modifier and Type | Class and Description |
---|---|
class |
DataNode
DataNode is a class (and program) that stores a set of
blocks for a DFS deployment.
|
Modifier and Type | Class and Description |
---|---|
class |
AvailableSpaceVolumeChoosingPolicy<V extends FsVolumeSpi>
A DN volume choosing policy which takes into account the amount of free
space on each of the available volumes when considering where to assign a
new replica allocation.
|
Modifier and Type | Class and Description |
---|---|
class |
BootstrapStandby
Tool which allows the standby node's storage directories to be bootstrapped
by copying the latest namespace snapshot from the active namenode.
|
Modifier and Type | Class and Description |
---|---|
class |
DFSAdmin
This class provides some DFS administrative access shell commands.
|
class |
DFSck
This class provides rudimentary checking of DFS volumes for errors and
sub-optimal conditions.
|
class |
DFSHAAdmin
Class to extend HAAdmin to do a little bit of HDFS-specific configuration.
|
class |
GetConf
Tool for getting configuration information from a configuration file.
|
class |
GetGroups
HDFS implementation of a tool for getting the groups which a given user
belongs to.
|
Modifier and Type | Class and Description |
---|---|
class |
OfflineEditsViewer
This class implements an offline edits viewer, tool that
can be used to view edit logs.
|
Modifier and Type | Class and Description |
---|---|
class |
WebHdfsFileSystem
A FileSystem for HDFS over the web.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractMapWritable
Abstract base class for MapWritable and SortedMapWritable
Unlike org.apache.nutch.crawl.MapWritable, this class allows creation of
MapWritable<Writable, MapWritable> so the CLASS_TO_ID and ID_TO_CLASS
maps travel with the class instead of being static.
|
class |
EnumSetWritable<E extends Enum<E>>
A Writable wrapper for EnumSet.
|
class |
GenericWritable
A wrapper for Writable instances.
|
class |
MapWritable
A Writable Map.
|
class |
ObjectWritable
A polymorphic Writable that writes an instance with it's class name.
|
class |
SortedMapWritable
A Writable SortedMap.
|
Modifier and Type | Class and Description |
---|---|
class |
BZip2Codec
This class provides output and input streams for bzip2 compression
and decompression.
|
class |
DefaultCodec |
class |
DeflateCodec
Alias class for DefaultCodec to enable codec discovery by 'deflate' name.
|
class |
GzipCodec
This class creates gzip compressors/decompressors.
|
class |
Lz4Codec
This class creates lz4 compressors/decompressors.
|
class |
SnappyCodec
This class creates snappy compressors/decompressors.
|
Modifier and Type | Class and Description |
---|---|
class |
SerializationFactory
A factory for
Serialization s. |
class |
WritableSerialization
A
Serialization for Writable s that delegates to
Writable.write(java.io.DataOutput) and
Writable.readFields(java.io.DataInput) . |
Modifier and Type | Class and Description |
---|---|
class |
AvroReflectSerialization
Serialization for Avro Reflect classes.
|
class |
AvroSerialization<T>
Base class for providing serialization to Avro types.
|
class |
AvroSpecificSerialization
Serialization for Avro Specific classes.
|
Modifier and Type | Class and Description |
---|---|
class |
JobClient
JobClient is the primary interface for the user-job to interact
with the cluster. |
class |
MapOutputFile
Manipulate the working area for the transient store for maps and reduces.
|
class |
MapTask
A Map task.
|
class |
MROutputFiles
Manipulate the working area for the transient store for maps and reduces.
|
class |
ReduceTask
A Reduce task.
|
class |
Task
Base class for tasks.
|
class |
YarnOutputFiles
Manipulate the working area for the transient store for maps and reduces.
|
Modifier and Type | Class and Description |
---|---|
class |
Gridmix
Driver class for the Gridmix3 benchmark.
|
Modifier and Type | Class and Description |
---|---|
class |
Submitter
The main entry point and job submitter.
|
Modifier and Type | Class and Description |
---|---|
class |
DataDrivenDBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table.
|
class |
DBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table.
|
class |
OracleDataDrivenDBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table in an Oracle db.
|
Modifier and Type | Interface and Description |
---|---|
static interface |
SequenceFileInputFilter.Filter
filter interface
|
Modifier and Type | Class and Description |
---|---|
static class |
SequenceFileInputFilter.FilterBase
base class for Filters
|
static class |
SequenceFileInputFilter.MD5Filter
This class returns a set of records by examing the MD5 digest of its
key against a filtering frequency f.
|
static class |
SequenceFileInputFilter.PercentFilter
This class returns a percentage of records
The percentage is determined by a filtering frequency f using
the criteria record# % f == 0.
|
static class |
SequenceFileInputFilter.RegexFilter
Records filter by matching key to regex
|
Modifier and Type | Class and Description |
---|---|
class |
CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>
A RecordReader that can effect joins of RecordReaders sharing a common key
type and partitioning.
|
class |
InnerJoinRecordReader<K extends WritableComparable<?>>
Full inner join.
|
class |
JoinRecordReader<K extends WritableComparable<?>>
Base class for Composite joins returning Tuples of arbitrary Writables.
|
class |
MultiFilterRecordReader<K extends WritableComparable<?>,V extends Writable>
Base class for Composite join returning values derived from multiple
sources, but generally not tuples.
|
class |
OuterJoinRecordReader<K extends WritableComparable<?>>
Full outer join.
|
class |
OverrideRecordReader<K extends WritableComparable<?>,V extends Writable>
Prefer the "rightmost" data source for this key.
|
Modifier and Type | Class and Description |
---|---|
class |
BinaryPartitioner<V>
Partition
BinaryComparable keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes() . |
class |
InputSampler<K,V>
Utility for collecting samples and writing a partition file for
TotalOrderPartitioner . |
class |
KeyFieldBasedComparator<K,V>
This comparator implementation provides a subset of the features provided
by the Unix/GNU Sort.
|
class |
KeyFieldBasedPartitioner<K2,V2>
Defines a way to partition keys based on certain key fields (also see
KeyFieldBasedComparator . |
class |
TotalOrderPartitioner<K extends WritableComparable<?>,V>
Partitioner effecting a total order by reading split points from
an externally generated source.
|
Modifier and Type | Class and Description |
---|---|
class |
CLI
Interprets the map reduce cli options
|
Modifier and Type | Class and Description |
---|---|
class |
JobEndNotifier
This class handles job end notification.
|
Modifier and Type | Class and Description |
---|---|
class |
MapReduceTrackingUriPlugin |
Modifier and Type | Class and Description |
---|---|
class |
AbstractDNSToSwitchMapping
This is a base class for DNS to Switch mappings.
|
class |
CachedDNSToSwitchMapping
A cached implementation of DNSToSwitchMapping that takes an
raw DNSToSwitchMapping and stores the resolved network location in
a cache.
|
class |
ScriptBasedMapping
This class implements the
DNSToSwitchMapping interface using a
script configured via the
CommonConfigurationKeysPublic.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY option. |
class |
SocksSocketFactory
Specialized SocketFactory to create sockets with a SOCKS proxy
|
class |
TableMapping
Simple
DNSToSwitchMapping implementation that reads a 2 column text
file. |
Modifier and Type | Class and Description |
---|---|
class |
LdapGroupsMapping
An implementation of
GroupMappingServiceProvider which
connects directly to an LDAP server for determining group membership. |
Modifier and Type | Interface and Description |
---|---|
interface |
KeyStoresFactory
Interface that gives access to
KeyManager and TrustManager
implementations. |
Modifier and Type | Class and Description |
---|---|
class |
FileBasedKeyStoresFactory
KeyStoresFactory implementation that reads the certificates from
keystore files. |
Modifier and Type | Class and Description |
---|---|
class |
DumpTypedBytes
Utility program that fetches all files that match a given pattern and dumps
their content to stdout as typed bytes.
|
class |
LoadTypedBytes
Utility program that reads typed bytes from standard input and stores them in
a sequence file for which the path is given as an argument.
|
class |
StreamJob
All the client-side work happens here.
|
Modifier and Type | Class and Description |
---|---|
class |
CopyListing
The CopyListing abstraction is responsible for how the list of
sources and targets is constructed, for DistCp's copy function.
|
class |
DistCh
A Map-reduce program to recursively change files properties
such as owner, group and permission.
|
class |
DistCp
DistCp is the main driver-class for DistCpV2.
|
class |
DistCpV1
A Map-reduce program to recursively copy directories between
different file-systems.
|
class |
FileBasedCopyListing
FileBasedCopyListing implements the CopyListing interface,
to create the copy-listing for DistCp,
by iterating over all source paths mentioned in a specified input-file.
|
class |
GetGroupsBase
Base class for the HDFS and MR implementations of tools which fetch and
display the groups that users belong to.
|
class |
GlobbedCopyListing
GlobbedCopyListing implements the CopyListing interface, to create the copy
listing-file by "globbing" all specified source paths (wild-cards and all.)
|
class |
HadoopArchives
a archive creation utility.
|
static class |
Logalyzer.LogComparator
A WritableComparator optimized for UTF8 keys of the logs.
|
class |
SimpleCopyListing
The SimpleCopyListing is responsible for making the exhaustive list of
all files/directories under its specified list of input-paths.
|
Modifier and Type | Class and Description |
---|---|
class |
Anonymizer |
class |
Folder |
class |
HadoopLogsAnalyzer
Deprecated.
|
class |
TraceBuilder
The main driver of the Rumen Parser.
|
Modifier and Type | Class and Description |
---|---|
class |
TypedBytesWritableInput
Provides functionality for reading typed bytes as Writable objects.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Tool
A tool interface that supports handling of generic command-line options.
|
Modifier and Type | Class and Description |
---|---|
class |
ApplicationCLI |
class |
LogsCLI |
class |
NodeCLI |
class |
RMAdminCLI |
class |
YarnCLI |
Modifier and Type | Class and Description |
---|---|
class |
LogCLIHelpers |
Modifier and Type | Class and Description |
---|---|
class |
ContainerExecutor |
class |
DefaultContainerExecutor |
class |
LinuxContainerExecutor |
Modifier and Type | Interface and Description |
---|---|
interface |
LCEResourcesHandler |
Modifier and Type | Class and Description |
---|---|
class |
CgroupsLCEResourcesHandler |
class |
DefaultLCEResourcesHandler |
Modifier and Type | Class and Description |
---|---|
class |
CapacityScheduler |
Modifier and Type | Class and Description |
---|---|
class |
NewAppWeightBooster
A
WeightAdjuster implementation that gives a weight boost to new jobs
for a certain amount of time -- by default, a 3x weight boost for 60 seconds. |
Modifier and Type | Class and Description |
---|---|
class |
FifoScheduler |
Modifier and Type | Class and Description |
---|---|
class |
LinuxResourceCalculatorPlugin
Plugin to calculate resource information on Linux systems.
|
class |
ProcfsBasedProcessTree
A Proc file-system based ProcessTree.
|
class |
ResourceCalculatorPlugin
Plugin to calculate resource information on the system.
|
class |
ResourceCalculatorProcessTree
Interface class to obtain process resource usage
|
class |
TrackingUriPlugin
Plugin to derive a tracking URL from a Yarn Application ID
|
class |
WindowsBasedProcessTree |
class |
WindowsResourceCalculatorPlugin |
Copyright © 2013 Apache Software Foundation. All rights reserved.