Modifier and Type | Class and Description |
---|---|
class |
BufferedFSInputStream
A class optimizes reading from FSInputStream by bufferring
|
class |
ChecksumFs
Abstract Checksumed Fs.
|
class |
CommonConfigurationKeys
This class contains constants for configuration keys used
in the common code.
|
class |
DelegateToFileSystem
Implementation of AbstractFileSystem based on the existing implementation of
FileSystem . |
class |
DelegationTokenRenewer
A daemon thread that waits for the next file system to renew.
|
class |
FileSystemLinkResolver<T>
FileSystem-specific class used to operate on and resolve symlinks in a path.
|
class |
FilterFs
A
FilterFs contains some other file system, which it uses as its
basic file system, possibly transforming the data along the way or providing
additional functionality. |
class |
FSLinkResolver<T>
Used primarily by
FileContext to operate on and resolve
symlinks in a path. |
class |
FsShell
Provide command line access to a FileSystem.
|
class |
FsShellPermissions
This class is the home for file permissions related commands.
|
class |
FsUrlStreamHandlerFactory
Factory for URL stream handlers.
|
class |
GlobPattern
A class for POSIX glob pattern with brace expansions.
|
interface |
HasFileDescriptor
Having a FileDescriptor
|
class |
Hdfs |
class |
HdfsBlockLocation
Wrapper for
BlockLocation that also includes a LocatedBlock ,
allowing more detailed queries to the datanode about a block. |
class |
LocalFileSystemConfigKeys
This class contains constants for configuration keys used
in the local file system, raw local fs and checksum fs.
|
class |
TrashPolicyDefault
Provides a trash feature.
|
Modifier and Type | Field and Description |
---|---|
static long |
DelegationTokenRenewer.renewCycle |
Modifier and Type | Method and Description |
---|---|
AbstractFileSystem |
FileContext.getDefaultFileSystem() |
Token<?> |
FileSystem.getDelegationToken(String renewer)
Get a new delegation token for this file system.
|
void |
FsServerDefaults.readFields(DataInput in) |
void |
ContentSummary.readFields(DataInput in) |
boolean |
Seekable.seekToNewSource(long targetPos)
Seeks a different copy of the data.
|
void |
FsServerDefaults.write(DataOutput out) |
void |
ContentSummary.write(DataOutput out) |
Modifier and Type | Class and Description |
---|---|
class |
FtpConfigKeys
This class contains constants for configuration keys used
in the ftp file system.
|
class |
FtpFs
The FtpFs implementation of AbstractFileSystem.
|
class |
FTPInputStream |
Modifier and Type | Class and Description |
---|---|
class |
HttpFSFileSystem
HttpFSServer implementation of the FileSystemAccess FileSystem.
|
static class |
HttpFSFileSystem.Operation |
class |
HttpFSKerberosAuthenticator
A
KerberosAuthenticator subclass that fallback to
HttpFSPseudoAuthenticator . |
static class |
HttpFSKerberosAuthenticator.DelegationTokenOperation
DelegationToken operations.
|
class |
HttpFSPseudoAuthenticator
A
PseudoAuthenticator subclass that uses FileSystemAccess's
UserGroupInformation to obtain the client user name (the UGI's login user). |
class |
HttpFSUtils
Utility methods used by HttpFS classes.
|
Modifier and Type | Class and Description |
---|---|
class |
CheckUploadContentTypeFilter
Filter that Enforces the content-type to be application/octet-stream for
POST and PUT requests.
|
class |
FSOperations
FileSystem operation executors used by
HttpFSServer . |
static class |
FSOperations.FSAppend
Executor that performs an append FileSystemAccess files system operation.
|
static class |
FSOperations.FSConcat
Executor that performs an append FileSystemAccess files system operation.
|
static class |
FSOperations.FSContentSummary
Executor that performs a content-summary FileSystemAccess files system operation.
|
static class |
FSOperations.FSCreate
Executor that performs a create FileSystemAccess files system operation.
|
static class |
FSOperations.FSDelete
Executor that performs a delete FileSystemAccess files system operation.
|
static class |
FSOperations.FSFileChecksum
Executor that performs a file-checksum FileSystemAccess files system operation.
|
static class |
FSOperations.FSFileStatus
Executor that performs a file-status FileSystemAccess files system operation.
|
static class |
FSOperations.FSHomeDir
Executor that performs a home-dir FileSystemAccess files system operation.
|
static class |
FSOperations.FSListStatus
Executor that performs a list-status FileSystemAccess files system operation.
|
static class |
FSOperations.FSMkdirs
Executor that performs a mkdirs FileSystemAccess files system operation.
|
static class |
FSOperations.FSOpen
Executor that performs a open FileSystemAccess files system operation.
|
static class |
FSOperations.FSRename
Executor that performs a rename FileSystemAccess files system operation.
|
static class |
FSOperations.FSSetOwner
Executor that performs a set-owner FileSystemAccess files system operation.
|
static class |
FSOperations.FSSetPermission
Executor that performs a set-permission FileSystemAccess files system operation.
|
static class |
FSOperations.FSSetReplication
Executor that performs a set-replication FileSystemAccess files system operation.
|
static class |
FSOperations.FSSetTimes
Executor that performs a set-times FileSystemAccess files system operation.
|
class |
HttpFSAuthenticationFilter
Subclass of hadoop-auth
AuthenticationFilter that obtains its configuration
from HttpFSServer's server configuration. |
class |
HttpFSExceptionProvider
JAX-RS
ExceptionMapper implementation that maps HttpFSServer's
exceptions to HTTP status codes. |
class |
HttpFSKerberosAuthenticationHandler
Server side
AuthenticationHandler that authenticates requests
using the incoming delegation token as a 'delegation' query string parameter. |
class |
HttpFSParametersProvider
HttpFS ParametersProvider.
|
static class |
HttpFSParametersProvider.AccessTimeParam
Class for access-time parameter.
|
static class |
HttpFSParametersProvider.BlockSizeParam
Class for block-size parameter.
|
static class |
HttpFSParametersProvider.DataParam
Class for data parameter.
|
static class |
HttpFSParametersProvider.DestinationParam
Class for to-path parameter.
|
static class |
HttpFSParametersProvider.DoAsParam
Class for do-as parameter.
|
static class |
HttpFSParametersProvider.FilterParam
Class for filter parameter.
|
static class |
HttpFSParametersProvider.GroupParam
Class for group parameter.
|
static class |
HttpFSParametersProvider.LenParam
Class for len parameter.
|
static class |
HttpFSParametersProvider.ModifiedTimeParam
Class for modified-time parameter.
|
static class |
HttpFSParametersProvider.OffsetParam
Class for offset parameter.
|
static class |
HttpFSParametersProvider.OperationParam
Class for operation parameter.
|
static class |
HttpFSParametersProvider.OverwriteParam
Class for overwrite parameter.
|
static class |
HttpFSParametersProvider.OwnerParam
Class for owner parameter.
|
static class |
HttpFSParametersProvider.PermissionParam
Class for permission parameter.
|
static class |
HttpFSParametersProvider.RecursiveParam
Class for delete's recursive parameter.
|
static class |
HttpFSParametersProvider.ReplicationParam
Class for replication parameter.
|
static class |
HttpFSParametersProvider.SourcesParam
Class for concat sources parameter.
|
class |
HttpFSReleaseFilter
Filter that releases FileSystemAccess filesystem instances upon HTTP request
completion.
|
class |
HttpFSServer
Main class of HttpFSServer server.
|
class |
HttpFSServerWebApp
Bootstrap class that manages the initialization and destruction of the
HttpFSServer server, it is a
javax.servlet.ServletContextListener
implementation that is wired in HttpFSServer's WAR
WEB-INF/web.xml . |
Modifier and Type | Class and Description |
---|---|
class |
LocalConfigKeys
This class contains constants for configuration keys used
in the local file system, raw local fs and checksum fs.
|
class |
LocalFs
The LocalFs implementation of ChecksumFs.
|
class |
RawLocalFs
The RawLocalFs implementation of AbstractFileSystem.
|
Modifier and Type | Class and Description |
---|---|
class |
ChmodParser
Parse a permission mode passed in from a chmod command and apply that
mode against an existing file.
|
Modifier and Type | Class and Description |
---|---|
class |
Block
Holds metadata about a block of data being stored in a
FileSystemStore . |
interface |
FileSystemStore
|
class |
INode
Holds file metadata including type (regular file, or directory),
and the list of blocks that are pointers to the data.
|
class |
S3Credentials
Extracts AWS credentials from the filesystem URI or configuration.
|
class |
S3FileSystemConfigKeys
This class contains constants for configuration keys used
in the s3 file system.
|
Modifier and Type | Class and Description |
---|---|
class |
S3NativeFileSystemConfigKeys
This class contains constants for configuration keys used
in the s3 file system.
|
Modifier and Type | Class and Description |
---|---|
class |
Command
An abstract class for the execution of a file system command
|
class |
CommandFactory
class to search for and register commands
|
class |
Count
Count the number of directories, files, bytes, quota, and remaining quota.
|
class |
FsCommand
Base class for all "hadoop fs" commands
|
class |
PathData
Encapsulates a Path (path), its FileStatus (stat), and its FileSystem (fs).
|
Modifier and Type | Class and Description |
---|---|
class |
ActiveStandbyElector
This class implements a simple library to perform leader election on top of
Apache Zookeeper.
|
class |
FailoverController
The FailOverController is responsible for electing an active service
on startup or when the current active is changing (eg due to failure),
monitoring the health of a service, and performing a fail-over when a
new active service is either manually selected by a user or elected.
|
class |
HAAdmin
A command-line tool for making calls in the HAServiceProtocol.
|
class |
HAServiceStatus |
class |
HealthMonitor
This class is a daemon which runs in a loop, periodically heartbeating
with an HA service.
|
static class |
HealthMonitor.State |
class |
NodeFencer
This class parses the configured list of fencing methods, and
is responsible for trying each one in turn while logging informative
output.
|
interface |
ZKFCProtocol
Protocol exposed by the ZKFailoverController, allowing for graceful
failover.
|
Modifier and Type | Class and Description |
---|---|
class |
HAServiceProtocolClientSideTranslatorPB
This class is the client side translator to translate the requests made on
HAServiceProtocol interfaces to the RPC server implementing
HAServiceProtocolPB . |
class |
HAServiceProtocolServerSideTranslatorPB
This class is used on the server side.
|
class |
ZKFCProtocolServerSideTranslatorPB |
Modifier and Type | Class and Description |
---|---|
class |
BlockMissingException
This exception is thrown when a read encounters a block that has no locations
associated with it.
|
class |
BlockReaderFactory
Utility class to create BlockReader implementations.
|
class |
DeprecatedUTF8
A simple wrapper around
UTF8 . |
class |
DFSClient
DFSClient can connect to a Hadoop Filesystem and
perform basic file tasks.
|
static class |
DFSClient.Renewer |
class |
DFSClientFaultInjector
Used for injecting faults in DFSClient and DFSOutputStream tests.
|
class |
DFSConfigKeys
This class contains constants for configuration keys used
in hdfs.
|
class |
DFSInputStream
DFSInputStream provides bytes from a named file.
|
class |
DFSOutputStream
DFSOutputStream creates files from a stream of bytes.
|
class |
DFSUtil |
static class |
DFSUtil.DecomStaleComparator
Comparator for sorting DataNodeInfo[] based on decommissioned/stale states.
|
static class |
DistributedFileSystem.DiskStatus
Deprecated.
Use
FsStatus instead |
class |
HdfsConfiguration
Adds deprecated keys into the configuration.
|
class |
HDFSPolicyProvider
PolicyProvider for HDFS protocols. |
class |
HftpFileSystem
An implementation of a protocol for accessing filesystems over HTTP.
|
static class |
HftpFileSystem.TokenManager |
class |
HsftpFileSystem
An implementation of a protocol for accessing filesystems over HTTPS.
|
class |
RemoteBlockReader
Deprecated.
this is an old implementation that is being left around
in case any issues spring up with the new
RemoteBlockReader2 implementation.
It will be removed in the next release. |
class |
RemoteBlockReader2
This is a wrapper around connection to datanode
and understands checksum, offset etc.
|
Modifier and Type | Method and Description |
---|---|
void |
DFSClient.clearDataEncryptionKey() |
DFSClient |
DistributedFileSystem.getClient() |
DataEncryptionKey |
DFSClient.getDataEncryptionKey() |
Modifier and Type | Class and Description |
---|---|
class |
DomainPeer
Represents a peer that we communicate with by using blocking I/O
on a UNIX domain socket.
|
class |
DomainPeerServer |
class |
EncryptedPeer
Represents a peer that we communicate with by using an encrypted
communications medium.
|
interface |
Peer
Represents a connection to a peer.
|
interface |
PeerServer |
class |
TcpPeerServer |
Modifier and Type | Class and Description |
---|---|
class |
AlreadyBeingCreatedException
The exception that happens when you ask to create a file that already
is being created, but is not closed yet.
|
class |
Block
A Block is a Hadoop FS primitive, identified by a
long.
|
class |
BlockListAsLongs
This class provides an interface for accessing list of blocks that
has been implemented as long[].
|
class |
BlockListAsLongs.BlockReportIterator
Iterates over blocks in the block report.
|
class |
BlockLocalPathInfo
A block and the full path information to the block data file and
the metadata file stored on the local file system.
|
interface |
ClientDatanodeProtocol
An client-datanode protocol for block recovery
|
interface |
ClientProtocol
ClientProtocol is used by user code via
DistributedFileSystem class to communicate
with the NameNode. |
class |
DatanodeID
This class represents the primary identifier for a Datanode.
|
class |
DatanodeInfo
This class extends the primary identifier of a Datanode with ephemeral
state, eg usage information, current administrative state, and the
network location that is communicated to clients.
|
class |
DirectoryListing
This class defines a partial listing of a directory to support
iterative directory listing.
|
class |
DSQuotaExceededException |
class |
ExtendedBlock
Identifies a Block uniquely across the block pools
|
class |
FSLimitException |
class |
HdfsBlocksMetadata
Augments an array of blocks on a datanode with additional information about
where the block is stored.
|
class |
HdfsConstants
Some handy constants
|
class |
HdfsFileStatus
Interface that represents the over the wire information for a file.
|
class |
HdfsLocatedFileStatus
Interface that represents the over the wire information
including block locations for a file.
|
class |
LayoutVersion
This class tracks changes in the layout version of HDFS.
|
class |
LocatedBlock
Associates a block with the Datanodes that contain its replicas
and other block metadata (E.g.
|
class |
LocatedBlocks
Collection of blocks with their locations and the file length.
|
class |
NSQuotaExceededException |
class |
QuotaExceededException
This exception is thrown when modification to HDFS results in violation
of a directory quota.
|
class |
RecoveryInProgressException
Exception indicating that a replica is already being recovery.
|
class |
SnapshotInfo
SnapshotInfo maintains information for a snapshot
|
class |
UnregisteredNodeException
This exception is thrown when a node that has not previously
registered is trying to access the name node.
|
class |
UnresolvedPathException
Thrown when a symbolic link is encountered in a path.
|
Modifier and Type | Class and Description |
---|---|
class |
BlockConstructionStage
Block Construction Stage
|
class |
DataTransferEncryptor
A class which, given connected input/output streams, will perform a
handshake using those streams based on SASL to produce new Input/Output
streams which will encrypt/decrypt all data written/read from said streams.
|
static class |
DataTransferEncryptor.InvalidMagicNumberException |
interface |
DataTransferProtocol
Transfer data to/from datanode using a streaming protocol.
|
class |
DataTransferProtoUtil
Static utilities for dealing with the protocol buffers used by the
Data Transfer Protocol.
|
class |
InvalidEncryptionKeyException
Encryption key verification failed.
|
class |
IOStreamPair
A little struct class to wrap an InputStream and an OutputStream.
|
class |
Op
Operation
|
class |
PacketHeader
Header data for each packet that goes through the read/write pipelines.
|
class |
PacketReceiver
Class to handle reading packets one-at-a-time from the wire.
|
class |
PipelineAck
Pipeline Acknowledgment
|
class |
Receiver
Receiver
|
class |
ReplaceDatanodeOnFailure
The setting of replace-datanode-on-failure feature.
|
class |
Sender
Sender
|
Modifier and Type | Interface and Description |
---|---|
interface |
ClientDatanodeProtocolPB |
class |
ClientDatanodeProtocolServerSideTranslatorPB
Implementation for protobuf service that forwards requests
received on
ClientDatanodeProtocolPB to the
ClientDatanodeProtocol server implementation. |
class |
ClientDatanodeProtocolTranslatorPB
This class is the client side translator to translate the requests made on
ClientDatanodeProtocol interfaces to the RPC server implementing
ClientDatanodeProtocolPB . |
interface |
ClientNamenodeProtocolPB |
class |
ClientNamenodeProtocolServerSideTranslatorPB
This class is used on the server side.
|
class |
ClientNamenodeProtocolTranslatorPB
This class forwards NN's ClientProtocol calls as RPC calls to the NN server
while translating from the parameter types used in ClientProtocol to the
new PB types.
|
class |
DatanodeProtocolClientSideTranslatorPB
This class is the client side translator to translate the requests made on
DatanodeProtocol interfaces to the RPC server implementing
DatanodeProtocolPB . |
interface |
DatanodeProtocolPB |
interface |
InterDatanodeProtocolPB |
class |
InterDatanodeProtocolServerSideTranslatorPB
Implementation for protobuf service that forwards requests
received on
InterDatanodeProtocolPB to the
InterDatanodeProtocol server implementation. |
class |
InterDatanodeProtocolTranslatorPB
This class is the client side translator to translate the requests made on
InterDatanodeProtocol interfaces to the RPC server implementing
InterDatanodeProtocolPB . |
interface |
JournalProtocolPB
Protocol used to journal edits to a remote node.
|
class |
JournalProtocolServerSideTranslatorPB
Implementation for protobuf service that forwards requests
received on
JournalProtocolPB to the
JournalProtocol server implementation. |
class |
JournalProtocolTranslatorPB
This class is the client side translator to translate the requests made on
JournalProtocol interfaces to the RPC server implementing
JournalProtocolPB . |
interface |
NamenodeProtocolPB
Protocol that a secondary NameNode uses to communicate with the NameNode.
|
class |
NamenodeProtocolTranslatorPB
This class is the client side translator to translate the requests made on
NamenodeProtocol interfaces to the RPC server implementing
NamenodeProtocolPB . |
Modifier and Type | Class and Description |
---|---|
class |
IPCLoggerChannel
Channel to a remote JournalNode using Hadoop IPC.
|
class |
QuorumJournalManager
A JournalManager that writes to a set of remote JournalNodes,
requiring a quorum of nodes to ack each write.
|
Modifier and Type | Class and Description |
---|---|
class |
JournalNotFormattedException
Exception indicating that a call has been made to a JournalNode
which is not yet formatted.
|
class |
JournalOutOfSyncException |
interface |
QJournalProtocol
Protocol used to communicate between
QuorumJournalManager
and each JournalNode . |
class |
RequestInfo |
Modifier and Type | Interface and Description |
---|---|
interface |
QJournalProtocolPB
Protocol used to journal edits to a JournalNode participating
in the quorum journal.
|
class |
QJournalProtocolServerSideTranslatorPB
Implementation for protobuf service that forwards requests
received on
JournalProtocolPB to the
JournalProtocol server implementation. |
class |
QJournalProtocolTranslatorPB
This class is the client side translator to translate the requests made on
JournalProtocol interfaces to the RPC server implementing
JournalProtocolPB . |
Modifier and Type | Class and Description |
---|---|
class |
GetJournalEditServlet
This servlet is used in two cases:
The QuorumJournalManager, when reading edits, fetches the edit streams
from the journal nodes.
During edits synchronization, one journal node will fetch edits from
another journal node.
|
class |
JournalFaultInjector
Used for injecting faults in QuorumJournalManager tests.
|
class |
JournalNode
The JournalNode is a daemon which allows namenodes using
the QuorumJournalManager to log and retrieve edits stored
remotely.
|
class |
JournalNodeHttpServer
Encapsulates the HTTP server started by the Journal Service.
|
Modifier and Type | Class and Description |
---|---|
class |
BlockKey
Key used for generating and verifying block tokens
|
class |
BlockTokenIdentifier |
static class |
BlockTokenIdentifier.Renewer |
class |
BlockTokenSecretManager
BlockTokenSecretManager can be instantiated in 2 modes, master mode and slave
mode.
|
class |
BlockTokenSelector
A block token selector for HDFS
|
class |
DataEncryptionKey
A little struct class to contain all fields required to perform encryption of
the DataTransferProtocol.
|
class |
ExportedBlockKeys
Object for passing block keys
|
class |
InvalidBlockTokenException
Access token verification failed.
|
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenIdentifier
A delegation token identifier that is specific to HDFS.
|
class |
DelegationTokenSecretManager
A HDFS specific delegation token secret manager.
|
class |
DelegationTokenSelector
A delegation token that is specialized for HDFS
|
Modifier and Type | Class and Description |
---|---|
class |
Balancer
The balancer is a tool that balances disk space usage on an HDFS cluster
when some datanodes become full or when new empty nodes join the cluster.
|
Modifier and Type | Interface and Description |
---|---|
interface |
BlockCollection
This interface is used by the block manager to expose a
few characteristics of a collection of Block/BlockUnderConstruction.
|
class |
BlockInfo
BlockInfo class maintains for a given block
the
INodeFile it is part of and datanodes where the replicas of
the block are stored. |
class |
BlockManager
Keeps information related to the blocks stored in the Hadoop cluster.
|
class |
BlockPlacementPolicy
This interface is used for choosing the desired number of targets
for placing block replicas.
|
static class |
BlockPlacementPolicy.NotEnoughReplicasException |
class |
BlockPlacementPolicyDefault
The class is responsible for choosing the desired number of targets
for placing block replicas.
|
class |
CorruptReplicasMap
Stores information about all corrupt blocks in the File System.
|
class |
DatanodeDescriptor
This class extends the DatanodeInfo class with ephemeral information (eg
health, capacity, what blocks are associated with the Datanode) that is
private to the Namenode, ie this class is not exposed to clients.
|
static class |
DatanodeDescriptor.BlockTargetPair
Block and targets pair
|
class |
DatanodeManager
Manage datanodes, include decommission and other activities.
|
class |
OutOfV1GenerationStampsException
This exception is thrown when the name node runs out of V1 generation
stamps.
|
Modifier and Type | Class and Description |
---|---|
class |
GenerationStamp
A GenerationStamp is a Hadoop FS primitive, identified by a long.
|
class |
HdfsServerConstants
Some handy internal HDFS constants
|
class |
InconsistentFSStateException
The exception is thrown when file system state is inconsistent
and is not recoverable.
|
class |
IncorrectVersionException
The exception is thrown when external version does not match
current version of the application.
|
class |
JspHelper |
class |
Storage
Storage information file.
|
static interface |
Storage.FormatConfirmable
Interface for classes which need to have the user confirm their
formatting during NameNode -format and other similar operations.
|
static class |
Storage.StorageDirectory
One of the storage directories.
|
static interface |
Storage.StorageDirType
An interface to denote storage directory type
Implementations can define a type for storage directory by implementing
this interface.
|
interface |
StorageErrorReporter
Interface which implementations of
JournalManager can use to report
errors on underlying storage directories. |
class |
StorageInfo
Common class for storage information.
|
class |
UpgradeStatusReport
Base upgrade upgradeStatus class.
|
class |
Util |
Modifier and Type | Class and Description |
---|---|
class |
BlockMetadataHeader
BlockMetadataHeader manages metadata for data blocks on Datanodes.
|
class |
BlockPoolSliceStorage
Manages storage for the set of BlockPoolSlices which share a particular
block pool id, on this DataNode.
|
class |
DataBlockScanner
DataBlockScanner manages block scanning for all the block pools.
|
static class |
DataBlockScanner.Servlet |
class |
DataNode
DataNode is a class (and program) that stores a set of
blocks for a DFS deployment.
|
class |
DatanodeJspHelper |
class |
DatanodeUtil
Provide utility methods for Datanode.
|
class |
DataStorage
Data storage information file.
|
class |
DirectoryScanner
Periodically scans the data directories for block and block metadata files.
|
class |
DNConf
Simple class encapsulating all of the configuration that the DataNode
loads at startup time.
|
interface |
Replica
This represents block replicas which are stored in DataNode.
|
class |
ReplicaInfo
This class is used by datanodes to maintain meta data of its replicas.
|
Modifier and Type | Method and Description |
---|---|
static DataNode |
DataNode.createDataNode(String[] args,
Configuration conf,
SecureDataNodeStarter.SecureResources resources)
Instantiate & Start a single datanode daemon and wait for it to finish.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FsDatasetSpi<V extends FsVolumeSpi>
This is a service provider interface for the underlying storage that
stores replicas for a data node.
|
interface |
VolumeChoosingPolicy<V extends FsVolumeSpi>
This interface specifies the policy for choosing volumes to store replicas.
|
Modifier and Type | Class and Description |
---|---|
class |
FsDatasetUtil
Utility methods.
|
Modifier and Type | Class and Description |
---|---|
class |
DataNodeMetrics
This class is for maintaining the various DataNode statistics
and publishing them through the metrics interfaces.
|
interface |
FSDatasetMBean
This Interface defines the methods to get the status of a the FSDataset of
a data node.
|
Modifier and Type | Class and Description |
---|---|
class |
BackupImage
Extension of FSImage for the backup node.
|
class |
BackupNode
BackupNode.
|
class |
BackupState |
class |
CheckpointConf |
class |
CheckpointSignature
A unique signature intended to identify checkpoint transactions.
|
class |
ContentSummaryServlet
Servlets for file checksum
|
class |
EditLogFileInputStream
An implementation of the abstract class
EditLogInputStream , which
reads edits from a file. |
class |
EditLogFileOutputStream
An implementation of the abstract class
EditLogOutputStream , which
stores edits in a local file. |
class |
EditLogInputException
Thrown when there's a failure to read an edit log op from disk when loading
edits.
|
class |
EditLogInputStream
A generic abstract class to support reading edits log data from
persistent storage.
|
class |
EditLogOutputStream
A generic abstract class to support journaling of edits logs into
a persistent storage.
|
class |
EditsDoubleBuffer
A double-buffer for edits.
|
class |
FileChecksumServlets
Servlets for file checksum
|
static class |
FileChecksumServlets.GetServlet
Get FileChecksum
|
static class |
FileChecksumServlets.RedirectServlet
Redirect file checksum queries to an appropriate datanode.
|
class |
FileDataServlet
Redirect queries about the hosted filesystem to an appropriate datanode.
|
class |
FileJournalManager
Journal manager for the common case of edits files being written
to a storage directory.
|
static class |
FileJournalManager.EditLogFile
Record of an edit log that has been located and had its filename parsed.
|
class |
FsckServlet
This class is used in Namesystem's web server to do fsck on namenode.
|
interface |
FSClusterStats
This interface is used for retrieving the load related statistics of
the cluster.
|
class |
FSEditLog
FSEditLog maintains a log of the namespace modifications.
|
class |
FSEditLogLoader |
class |
FSEditLogOp
Helper classes for reading the ops from an InputStream.
|
class |
FSEditLogOpCodes
Op codes for edits file
|
class |
FSImage
FSImage handles checkpointing and logging of the namespace edits.
|
class |
FSImageFormat
Contains inner classes for reading or writing the on-disk format for
FSImages.
|
class |
FSImageSerialization
Static utility functions for serializing various pieces of data in the correct
format for the FSImage file.
|
class |
FSNamesystem
FSNamesystem does the actual bookkeeping work for the
DataNode.
|
class |
GetImageServlet
This class is used in Namesystem's jetty to retrieve a file.
|
class |
INode
We keep an in-memory representation of the file/block hierarchy.
|
interface |
INodeAttributes
The attributes of an inode.
|
interface |
INodeDirectoryAttributes
The attributes of an inode.
|
class |
INodeFile
I-node for closed file.
|
interface |
INodeFileAttributes
The attributes of a file.
|
class |
INodeFileUnderConstruction
I-node for file being written.
|
class |
INodeId
An id which uniquely identifies an inode.
|
class |
INodeSymlink
An
INode representing a symbolic link. |
class |
INodeWithAdditionalFields
INode with additional fields including id, name, permission,
access time and modification time. |
interface |
JournalManager
A JournalManager is responsible for managing a single place of storing
edit logs.
|
class |
LeaseExpiredException
The lease that was being used to create this file has expired.
|
class |
LeaseManager
LeaseManager does the lease housekeeping for writing on files.
|
class |
ListPathsServlet
Obtain meta-information about a filesystem.
|
class |
MetaRecoveryContext
Context data for an ongoing NameNode metadata recovery process.
|
class |
NameNode
NameNode serves as both directory namespace manager and
"inode table" for the Hadoop DFS.
|
class |
NamenodeFsck
This class provides rudimentary checking of DFS volumes for errors and
sub-optimal conditions.
|
class |
NameNodeHttpServer
Encapsulates the HTTP server started by the NameNode.
|
class |
NameNodeResourceChecker
NameNodeResourceChecker provides a method -
hasAvailableDiskSpace - which will return true if and only if
the NameNode has disk space available on all required volumes, and any volume
which is configured to be redundant. |
interface |
Namesystem
Namesystem operations.
|
class |
NNStorage
NNStorage is responsible for management of the StorageDirectories used by
the NameNode.
|
class |
NotReplicatedYetException
The file has not finished being written to enough datanodes yet.
|
interface |
SafeMode
SafeMode related operations.
|
class |
SafeModeException
This exception is thrown when the name node is in safe mode.
|
class |
SaveNamespaceCancelledException |
class |
SecondaryNameNode
The Secondary NameNode is a helper to the primary NameNode.
|
class |
SequentialBlockIdGenerator
Generate the next valid block ID by incrementing the maximum block
ID allocated so far, starting at 2^30+1.
|
class |
StartupProgressServlet
Servlet that provides a JSON representation of the namenode's current startup
progress.
|
class |
StreamFile |
class |
TransferFsImage
This class provides fetching a specified file from the NameNode.
|
class |
UnsupportedActionException
This exception is thrown when an operation is not supported.
|
Modifier and Type | Class and Description |
---|---|
class |
ActiveState
Active state of the namenode.
|
class |
BootstrapStandby
Tool which allows the standby node's storage directories to be bootstrapped
by copying the latest namespace snapshot from the active namenode.
|
class |
EditLogTailer
EditLogTailer represents a thread which periodically reads from edits
journals and applies the transactions contained within to a given
FSNamesystem.
|
interface |
HAContext
Context that is to be used by
HAState for getting/setting the
current state and performing required operations. |
class |
HAState
Namenode base state to implement state machine pattern.
|
class |
StandbyCheckpointer
Thread which runs inside the NN when it's in Standby state,
periodically waking up to take a checkpoint of the namespace.
|
class |
StandbyState
Namenode standby state.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FSNamesystemMBean
This Interface defines the methods to get the status of a the FSNamesystem of
a name node.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FileWithSnapshot
An interface for
INodeFile to support snapshot. |
class |
INodeDirectorySnapshottable
Directories where taking snapshots is allowed.
|
class |
INodeFileUnderConstructionWithSnapshot
Represent an
INodeFileUnderConstruction that is snapshotted. |
class |
INodeFileWithSnapshot
Represent an
INodeFile that is snapshotted. |
class |
Snapshot
Snapshot of a sub-tree in the namesystem.
|
Modifier and Type | Class and Description |
---|---|
class |
Phase
Indicates a particular phase of the namenode startup sequence.
|
class |
StartupProgress
StartupProgress is used in various parts of the namenode codebase to indicate
startup progress.
|
class |
StartupProgressMetrics
Links
StartupProgress to a MetricsSource to expose its
information via JMX. |
class |
StartupProgressView
StartupProgressView is an immutable, consistent, read-only view of namenode
startup progress.
|
class |
Status
Indicates run status of a
Phase . |
class |
Step
A step performed by the namenode during a
Phase of startup. |
class |
StepType
Indicates a particular type of
Step . |
Modifier and Type | Class and Description |
---|---|
class |
BlockCommand
A BlockCommand is an instruction to a datanode
regarding some blocks under its control.
|
class |
BlockRecoveryCommand
BlockRecoveryCommand is an instruction to a data-node to recover
the specified blocks.
|
static class |
BlockRecoveryCommand.RecoveringBlock
This is a block with locations from which it should be recovered
and the new generation stamp, which the block will have after
successful recovery.
|
class |
BlocksWithLocations
Maintains an array of blocks and their corresponding storage IDs.
|
static class |
BlocksWithLocations.BlockWithLocations
A class to keep track of a block and its locations
|
class |
CheckpointCommand
Checkpoint command.
|
class |
DatanodeCommand
Base class for data-node command.
|
interface |
DatanodeProtocol
Protocol that a DFS datanode uses to communicate with the NameNode.
|
class |
DatanodeRegistration
DatanodeRegistration class contains all information the name-node needs
to identify and verify a data-node when it contacts the name-node.
|
class |
DisallowedDatanodeException
This exception is thrown when a datanode tries to register or communicate
with the namenode when it does not appear on the list of included nodes,
or has been specifically excluded.
|
class |
FenceResponse
Response to a journal fence request.
|
class |
FinalizeCommand
A BlockCommand is an instruction to a datanode to register with the namenode.
|
class |
HeartbeatResponse |
interface |
InterDatanodeProtocol
An inter-datanode protocol for updating generation stamp
|
class |
JournalInfo
Information that describes a journal
|
interface |
JournalProtocol
Protocol used to journal edits to a remote node.
|
class |
KeyUpdateCommand |
class |
NamenodeCommand
Base class for name-node command.
|
interface |
NamenodeProtocol
Protocol that a secondary NameNode uses to communicate with the NameNode.
|
interface |
NamenodeProtocols
The full set of RPC methods implemented by the Namenode.
|
class |
NamenodeRegistration
Information sent by a subordinate name-node to the active name-node
during the registration process.
|
class |
NamespaceInfo
NamespaceInfo is returned by the name-node in reply
to a data-node handshake.
|
class |
NNHAStatusHeartbeat |
interface |
NodeRegistration
Generic class specifying information, which need to be sent to the name-node
during the registration process.
|
class |
RegisterCommand
A BlockCommand is an instruction to a datanode to register with the namenode.
|
class |
ReplicaRecoveryInfo
Replica recovery information.
|
class |
ServerCommand
Base class for a server command.
|
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenFetcher
Fetch a DelegationToken from the current Namenode and store it in the
specified file.
|
class |
DFSAdmin
This class provides some DFS administrative access shell commands.
|
class |
DFSck
This class provides rudimentary checking of DFS volumes for errors and
sub-optimal conditions.
|
class |
DFSZKFailoverController |
class |
GetGroups
HDFS implementation of a tool for getting the groups which a given user
belongs to.
|
class |
HDFSConcat |
class |
JMXGet
tool to get data from NameNode or DataNode using MBeans currently the
following MBeans are available (under hadoop domain):
hadoop:service=NameNode,name=FSNamesystemState (static)
hadoop:service=NameNode,name=NameNodeActivity (dynamic)
hadoop:service=NameNode,name=RpcActivityForPort9000 (dynamic)
hadoop:service=DataNode,name=RpcActivityForPort50020 (dynamic)
hadoop:name=service=DataNode,FSDatasetState-UndefinedStorageId663800459
(static)
hadoop:service=DataNode,name=DataNodeActivity-UndefinedStorageId-520845215
(dynamic)
implementation note: all logging is sent to System.err (since it is a command
line tool)
|
class |
NNHAServiceTarget
One of the NN NameNodes acting as the target of an administrative command
(e.g.
|
Modifier and Type | Class and Description |
---|---|
class |
BinaryEditsVisitor
BinaryEditsVisitor implements a binary EditsVisitor
|
class |
OfflineEditsViewer
This class implements an offline edits viewer, tool that
can be used to view edit logs.
|
interface |
OfflineEditsVisitor
An implementation of OfflineEditsVisitor can traverse the structure of an
Hadoop edits log and respond to each of the structures within the file.
|
class |
OfflineEditsVisitorFactory
EditsVisitorFactory for different implementations of EditsVisitor
|
class |
StatisticsEditsVisitor
StatisticsEditsVisitor implements text version of EditsVisitor
that aggregates counts of op codes processed
|
class |
XmlEditsVisitor
An XmlEditsVisitor walks over an EditLog structure and writes out
an equivalent XML document that contains the EditLog's components.
|
Modifier and Type | Class and Description |
---|---|
class |
DepthCounter
Utility class for tracking descent into the structure of the
Visitor class (ImageVisitor, EditsVisitor etc.)
|
class |
NameDistributionVisitor
File name distribution visitor.
|
class |
OfflineImageViewer
OfflineImageViewer to dump the contents of an Hadoop image file to XML
or the console.
|
Modifier and Type | Class and Description |
---|---|
class |
LsSnapshottableDir
A tool used to list all snapshottable directories that are owned by the
current user.
|
class |
SnapshotDiff
A tool used to get the difference report between two snapshots, or between
a snapshot and the current status of a directory.
|
Modifier and Type | Class and Description |
---|---|
class |
BestEffortLongFile
Class that represents a file on disk which stores a single
long
value, but does not make any effort to make it truly durable. |
class |
ByteArray
Wrapper for byte[] to use byte[] as key in HashMap
|
class |
ByteBufferOutputStream
OutputStream that writes into a
ByteBuffer . |
class |
Canceler
Provides a simple interface where one thread can mark an operation
for cancellation, and another thread can poll for whether the
cancellation has occurred.
|
class |
CyclicIteration<K,V>
Provide an cyclic
Iterator for a NavigableMap . |
class |
DirectBufferPool
A simple class for pooling direct ByteBuffers.
|
class |
ExactSizeInputStream
An InputStream implementations which reads from some other InputStream
but expects an exact number of bytes.
|
class |
PersistentLongFile
Class that represents a file on disk which persistently stores
a single
long value. |
interface |
ReadOnlyList<E>
A
ReadOnlyList is a unmodifiable list,
which supports read-only operations. |
class |
XMLUtils
General xml utilities.
|
Modifier and Type | Class and Description |
---|---|
class |
HttpConfig
Singleton to get access to Http related configuration.
|
Modifier and Type | Method and Description |
---|---|
static org.eclipse.jetty.server.ServerConnector |
HttpServer.createDefaultChannelConnector(org.eclipse.jetty.server.Server server) |
static org.eclipse.jetty.server.Server |
HttpServer.createServer(Configuration conf) |
Modifier and Type | Class and Description |
---|---|
class |
ReadaheadPool
Manages a pool of threads which can issue readahead requests on file descriptors.
|
class |
WritableName
Utility to permit renaming of Writable implementation classes without
invalidiating files that contain their class name.
|
Modifier and Type | Class and Description |
---|---|
class |
ByteArray
Adaptor class to wrap byte-array backed objects (including java byte array)
as RawComparable objects.
|
Modifier and Type | Class and Description |
---|---|
class |
NativeIO
JNI wrappers for various native IO-related calls not available in Java.
|
Modifier and Type | Class and Description |
---|---|
class |
LossyRetryInvocationHandler<T>
A dummy invocation handler extending RetryInvocationHandler.
|
class |
RetryInvocationHandler<T>
This class implements RpcInvocationHandler and supports retry on the client
side.
|
Modifier and Type | Method and Description |
---|---|
boolean |
JavaSerialization.accept(Class<?> c) |
boolean |
WritableSerialization.accept(Class<?> c) |
int |
JavaSerializationComparator.compare(T o1,
T o2) |
Deserializer<Serializable> |
JavaSerialization.getDeserializer(Class<Serializable> c) |
Deserializer<Writable> |
WritableSerialization.getDeserializer(Class<Writable> c) |
Serializer<Serializable> |
JavaSerialization.getSerializer(Class<Serializable> c) |
Serializer<Writable> |
WritableSerialization.getSerializer(Class<Writable> c) |
Constructor and Description |
---|
JavaSerializationComparator() |
Modifier and Type | Field and Description |
---|---|
static String |
AvroReflectSerialization.AVRO_REFLECT_PACKAGES
Key to configure packages that contain classes to be serialized and
deserialized using this class.
|
static String |
AvroSerialization.AVRO_SCHEMA_KEY |
Modifier and Type | Method and Description |
---|---|
boolean |
AvroSpecificSerialization.accept(Class<?> c) |
boolean |
AvroReflectSerialization.accept(Class<?> c) |
Deserializer<T> |
AvroSerialization.getDeserializer(Class<T> c) |
org.apache.avro.io.DatumReader |
AvroReflectSerialization.getReader(Class<Object> clazz) |
org.apache.avro.io.DatumReader |
AvroSpecificSerialization.getReader(Class<org.apache.avro.specific.SpecificRecord> clazz) |
abstract org.apache.avro.io.DatumReader<T> |
AvroSerialization.getReader(Class<T> clazz)
Create and return Avro DatumReader for the given class.
|
org.apache.avro.Schema |
AvroReflectSerialization.getSchema(Object t) |
org.apache.avro.Schema |
AvroSpecificSerialization.getSchema(org.apache.avro.specific.SpecificRecord t) |
abstract org.apache.avro.Schema |
AvroSerialization.getSchema(T t)
Return an Avro Schema instance for the given class.
|
Serializer<T> |
AvroSerialization.getSerializer(Class<T> c) |
org.apache.avro.io.DatumWriter |
AvroReflectSerialization.getWriter(Class<Object> clazz) |
org.apache.avro.io.DatumWriter |
AvroSpecificSerialization.getWriter(Class<org.apache.avro.specific.SpecificRecord> clazz) |
abstract org.apache.avro.io.DatumWriter<T> |
AvroSerialization.getWriter(Class<T> clazz)
Create and return Avro DatumWriter for the given class.
|
Modifier and Type | Class and Description |
---|---|
class |
ClientId
A class defining a set of static helper methods to provide conversion between
bytes and string for UUID-based client Id.
|
class |
ProtobufHelper
Helper methods for protobuf related RPC implementation
|
interface |
ProtocolMetaInterface
This interface is implemented by the client side translators and can be used
to obtain information about underlying protocol e.g.
|
interface |
ProtocolTranslator
An interface implemented by client-side protocol translators to get the
underlying proxy object the translator is operating on.
|
class |
RetryCache
Maintains a cache of non-idempotent requests that have been successfully
processed by the RPC server implementation, to handle the retries.
|
class |
RpcConstants |
static class |
Server.AuthProtocol |
Modifier and Type | Class and Description |
---|---|
class |
RpcDetailedMetrics
This class is for maintaining RPC method related statistics
and publishing them through the metrics interfaces.
|
class |
RpcMetrics
This class is for maintaining the various RPC statistics
and publishing them through the metrics interfaces.
|
Modifier and Type | Class and Description |
---|---|
class |
RunnableCallable
Adapter class that allows
Runnable s and Callable s to
be treated as the other. |
class |
XException
Generic exception that requires error codes and uses the a message
template from the error code.
|
Modifier and Type | Class and Description |
---|---|
class |
BaseService
Convenience class implementing the
Service interface. |
class |
Server
A Server class provides standard configuration, logging and
Service
lifecyle management. |
static class |
Server.Status
Enumeration that defines the server status.
|
class |
ServerException
Exception thrown by the
Server class. |
static class |
ServerException.ERROR
Error codes use by the
Server class. |
interface |
Service
Service interface for components to be managed by the
Server class. |
class |
ServiceException
Exception thrown by
Service implementations. |
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenIdentifier
HttpFS
DelegationTokenIdentifier implementation. |
interface |
DelegationTokenManager
Service interface to manage HttpFS delegation tokens.
|
class |
DelegationTokenManagerException
Exception thrown by the
DelegationTokenManager service implementation. |
interface |
FileSystemAccess |
class |
FileSystemAccessException |
interface |
Groups |
interface |
Instrumentation |
interface |
ProxyUser |
interface |
Scheduler |
Modifier and Type | Class and Description |
---|---|
class |
FileSystemAccessService |
Modifier and Type | Class and Description |
---|---|
class |
InstrumentationService |
Modifier and Type | Class and Description |
---|---|
class |
SchedulerService |
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenManagerService
DelegationTokenManager service implementation.
|
class |
GroupsService |
class |
ProxyUserService |
static class |
ProxyUserService.ERROR |
Modifier and Type | Class and Description |
---|---|
class |
FileSystemReleaseFilter
|
class |
HostnameFilter
Filter that resolves the requester hostname.
|
class |
MDCFilter
Filter that sets request contextual information for the slf4j MDC.
|
class |
ServerWebApp
Server subclass that implements ServletContextListener
and uses its lifecycle to start and stop the server. |
Modifier and Type | Class and Description |
---|---|
class |
Check
Utility methods to check preconditions.
|
class |
ConfigurationUtils
Configuration utilities.
|
Modifier and Type | Class and Description |
---|---|
class |
BooleanParam |
class |
ByteParam |
class |
EnumParam<E extends Enum<E>> |
class |
ExceptionProvider |
class |
InputStreamEntity |
class |
IntegerParam |
class |
JSONMapProvider |
class |
JSONProvider |
class |
LongParam |
class |
Param<T> |
class |
Parameters
Class that contains all parsed JAX-RS parameters.
|
class |
ParametersProvider
Jersey provider that parses the request parameters based on the
given parameter definition.
|
class |
ShortParam |
class |
StringParam |
class |
UserProvider |
Modifier and Type | Method and Description |
---|---|
static long |
EventCounter.getError() |
static long |
EventCounter.getFatal() |
static long |
EventCounter.getInfo() |
static long |
EventCounter.getWarn() |
Modifier and Type | Class and Description |
---|---|
class |
BackupStore<K,V>
BackupStore is an utility class that is used to support
the mark-reset functionality of values iterator |
class |
IFile
IFile is the simple |
static class |
IFile.Reader<K,V>
IFile.Reader to read intermediate map-outputs. |
static class |
IFile.Writer<K,V>
IFile.Writer to write out intermediate map-outputs. |
class |
IFileInputStream
A checksum input stream, used for IFiles.
|
class |
IFileOutputStream
A Checksum output stream.
|
class |
JobACLsManager |
class |
JobContextImpl |
class |
JobEndNotifier |
class |
JobInProgress |
class |
JvmTask
Task abstraction that can be serialized, implements the writable interface.
|
class |
LocalClientProtocolProvider |
class |
LocalJobRunner
Implements MapReduce locally, in-process, for debugging.
|
class |
MapOutputFile
Manipulate the working area for the transient store for maps and reduces.
|
class |
MapTaskCompletionEventsUpdate
A class that represents the communication between the tasktracker and child
tasks w.r.t the map task completion events.
|
class |
Master |
class |
Merger
Merger is an utility class used by the Map and Reduce tasks for merging
both their memory and disk segments
|
static class |
Merger.Segment<K,V> |
interface |
MRConstants
Some handy constants
|
class |
MROutputFiles
Manipulate the working area for the transient store for maps and reduces.
|
class |
Operation
Generic operation that maps to the dependent set of ACLs that drive the
authorization of the operation.
|
class |
PeriodicStatsAccumulator
This abstract class that represents a bucketed series of
measurements of a quantity being measured in a running task
attempt.
|
class |
ProgressSplitsBlock |
class |
QueueACL
Enum representing an AccessControlList that drives set of operations that
can be performed on a queue.
|
class |
QueueManager
Class that exposes information about queues maintained by the Hadoop
Map/Reduce framework.
|
interface |
RawKeyValueIterator
RawKeyValueIterator is an iterator used to iterate over
the raw keys and values during sort/merge of intermediate data. |
class |
ReduceTask
A Reduce task.
|
static class |
Task.CombineOutputCollector<K,V>
OutputCollector for the combiner.
|
static class |
Task.CombineValuesIterator<KEY,VALUE>
Iterator to return Combined values
|
protected static class |
Task.NewCombinerRunner<K,V> |
protected static class |
Task.OldCombinerRunner<K,V> |
class |
TaskAttemptContextImpl |
class |
TaskLog
A simple logger to handle the task-specific user logs.
|
static class |
TaskLog.LogName
The filter for userlogs.
|
class |
TaskStatus
Describes the current status of a task.
|
static class |
TaskStatus.Phase |
static class |
TaskStatus.State |
interface |
TaskUmbilicalProtocol
Protocol that task child process uses to contact its parent process.
|
class |
TIPStatus
The states of a Tasks.
|
class |
YarnOutputFiles
Manipulate the working area for the transient store for maps and reduces.
|
Modifier and Type | Field and Description |
---|---|
protected YarnClient |
ResourceMgrDelegate.client |
static boolean |
JobClient.MAPREDUCE_CLIENT_RETRY_POLICY_ENABLED_DEFAULT |
static String |
JobClient.MAPREDUCE_CLIENT_RETRY_POLICY_ENABLED_KEY |
static String |
JobClient.MAPREDUCE_CLIENT_RETRY_POLICY_SPEC_DEFAULT |
static String |
JobClient.MAPREDUCE_CLIENT_RETRY_POLICY_SPEC_KEY |
Modifier and Type | Method and Description |
---|---|
static TaskCompletionEvent |
TaskCompletionEvent.downgrade(TaskCompletionEvent event) |
Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context) |
void |
JobQueueInfo.setChildren(List<JobQueueInfo> children) |
void |
JobConf.setCredentials(Credentials credentials) |
void |
TaskCompletionEvent.setEventId(int eventId)
set event Id.
|
void |
JobQueueInfo.setJobStatuses(JobStatus[] stats) |
void |
JobQueueInfo.setProperties(Properties props) |
void |
JobQueueInfo.setQueueName(String queueName)
Set the queue name of the JobQueueInfo
|
void |
JobQueueInfo.setQueueState(String state)
Set the state of the queue
|
void |
YARNRunner.setResourceMgrDelegate(ResourceMgrDelegate resMgrDelegate) |
void |
JobStatus.setRunState(int state)
Change the current run state of the job.
|
void |
JobStatus.setSchedulingInfo(String schedulingInfo)
Used to set the scheduling information associated to a particular Job.
|
void |
JobQueueInfo.setSchedulingInfo(String schedulingInfo)
Set the scheduling information associated to particular job queue
|
void |
TaskCompletionEvent.setTaskRunTime(int taskCompletionTime)
Set the task completion time
|
void |
TaskCompletionEvent.setTaskStatus(TaskCompletionEvent.Status status)
Set task status.
|
void |
TaskCompletionEvent.setTaskTrackerHttp(String taskTrackerHttp)
Set task tracker http location.
|
static void |
FileOutputFormat.setWorkOutputPath(JobConf conf,
Path outputDir)
Set the
Path of the task's temporary output directory
for the map-reduce job. |
RunningJob |
JobClient.submitJobInternal(JobConf conf) |
Modifier and Type | Class and Description |
---|---|
class |
JobCreator |
interface |
UserResolver
Maps users in the trace to a set of valid target users on the test cluster.
|
Modifier and Type | Class and Description |
---|---|
class |
DelegatingInputFormat<K,V>
An
InputFormat that delegates behaviour of paths to multiple other
InputFormats. |
class |
DelegatingMapper<K1,V1,K2,V2>
An
Mapper that delegates behaviour of paths to multiple other
mappers. |
Modifier and Type | Method and Description |
---|---|
protected boolean |
CombineFileInputFormat.isSplitable(JobContext context,
Path file)
Subclasses should avoid overriding this method and should instead only
override
CombineFileInputFormat.isSplitable(FileSystem, Path) . |
Modifier and Type | Class and Description |
---|---|
class |
FileSystemCounter |
class |
JobACL
Job related ACLs
|
class |
JobSubmissionFiles
A utility to manage job submission files.
|
interface |
MRConfig
Place holder for cluster level configuration keys.
|
interface |
MRJobConfig |
class |
StatusReporter |
class |
TaskReport
A report on the state of a task.
|
Modifier and Type | Method and Description |
---|---|
Cluster |
Job.getCluster() |
static Job |
Job.getInstance(Cluster cluster,
JobStatus status,
Configuration conf)
|
org.apache.hadoop.mapreduce.JobSubmitter |
Job.getJobSubmitter(FileSystem fs,
ClientProtocol submitClient)
Only for mocking via unit tests.
|
Counter |
Counter.getUnderlyingCounter() |
boolean |
Job.killTask(TaskAttemptID taskId,
boolean shouldFail)
Kill indicated task attempt.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractCounter
An abstract counter class to provide common implementation of
the counter interface in both mapred and mapreduce packages.
|
class |
AbstractCounterGroup<T extends Counter>
An abstract class to provide common implementation of the
generic counter group in both mapred and mapreduce package.
|
class |
CounterGroupFactory<C extends Counter,G extends CounterGroupBase<C>>
An abstract class to provide common implementation of the
group factory in both mapred and mapreduce packages.
|
class |
FileSystemCounterGroup<C extends Counter>
An abstract class to provide common implementation of the filesystem
counter group in both mapred and mapreduce packages.
|
static class |
FileSystemCounterGroup.FSCounter |
class |
FrameworkCounterGroup<T extends Enum<T>,C extends Counter>
An abstract class to provide common implementation for the framework
counter group in both mapred and mapreduce packages.
|
static class |
FrameworkCounterGroup.FrameworkCounter<T extends Enum<T>>
A counter facade for framework counters.
|
class |
GenericCounter
A generic counter implementation
|
class |
LimitExceededException |
class |
Limits |
Modifier and Type | Method and Description |
---|---|
G |
AbstractCounters.addGroup(G group)
Add a group.
|
G |
AbstractCounters.addGroup(String name,
String displayName)
Add a new group
|
C |
AbstractCounters.findCounter(String scheme,
FileSystemCounter key)
Find the file system counter for the given scheme and enum.
|
FileSystemCounter |
FileSystemCounterGroup.FSCounter.getFileSystemCounter() |
String |
FrameworkCounterGroup.FrameworkCounter.getGroupName() |
T |
FrameworkCounterGroup.FrameworkCounter.getKey() |
String |
FileSystemCounterGroup.FSCounter.getScheme() |
CounterGroupBase<T> |
CounterGroupBase.getUnderlyingGroup() |
boolean |
AbstractCounters.getWriteAllCounters()
Get the "writeAllCounters" option
|
Limits |
AbstractCounters.limits() |
void |
AbstractCounters.setWriteAllCounters(boolean send)
Set the "writeAllCounters" option to true or false
|
Constructor and Description |
---|
AbstractCounters(AbstractCounters<C1,G1> counters,
CounterGroupFactory<C,G> groupFactory)
Construct from another counters object.
|
AbstractCounters(CounterGroupFactory<C,G> gf) |
Modifier and Type | Class and Description |
---|---|
class |
ClientDistributedCacheManager
Manages internal configuration of the cache by the client for job submission.
|
class |
DistributedCache
Deprecated.
|
Modifier and Type | Class and Description |
---|---|
class |
AMStartedEvent
Event to record start of a task attempt
|
class |
EventReader |
interface |
HistoryEvent
Interface for event wrapper classes.
|
class |
HistoryViewer
HistoryViewer is used to parse and view the JobHistory files
|
class |
JobFinishedEvent
Event to record successful completion of job
|
class |
JobHistoryParser
Default Parser for the JobHistory files.
|
class |
JobInfoChangeEvent
Event to record changes in the submit and launch time of
a job
|
class |
JobInitedEvent
Event to record the initialization of a job
|
class |
JobPriorityChangeEvent
Event to record the change of priority of a job
|
class |
JobStatusChangedEvent
Event to record the change of status for a job
|
class |
JobSubmittedEvent
Event to record the submission of a job
|
class |
JobUnsuccessfulCompletionEvent
Event to record Failed and Killed completion of jobs
|
class |
MapAttemptFinishedEvent
Event to record successful completion of a map attempt
|
class |
NormalizedResourceEvent
Event to record the normalized map/reduce requirements.
|
class |
ReduceAttemptFinishedEvent
Event to record successful completion of a reduce attempt
|
class |
TaskAttemptFinishedEvent
Event to record successful task completion
|
class |
TaskAttemptStartedEvent
Event to record start of a task attempt
|
class |
TaskAttemptUnsuccessfulCompletionEvent
Event to record unsuccessful (Killed/Failed) completion of task attempts
|
class |
TaskFailedEvent
Event to record the failure of a task
|
class |
TaskFinishedEvent
Event to record the successful completion of a task
|
class |
TaskStartedEvent
Event to record the start of a task
|
class |
TaskUpdatedEvent
Event to record updates to a task
|
Modifier and Type | Method and Description |
---|---|
JobHistoryParser.JobInfo |
JobHistoryParser.parse(EventReader reader)
Only used for unit tests.
|
void |
JobHistoryParser.parse(EventReader reader,
HistoryEventHandler handler)
Only used for unit tests.
|
Modifier and Type | Class and Description |
---|---|
class |
Chain
The Chain class provides all the common functionality for the
ChainMapper and the ChainReducer classes. |
Modifier and Type | Class and Description |
---|---|
class |
DelegatingInputFormat<K,V>
An
InputFormat that delegates behavior of paths to multiple other
InputFormats. |
class |
DelegatingMapper<K1,V1,K2,V2>
An
Mapper that delegates behavior of paths to multiple other
mappers. |
class |
DelegatingRecordReader<K,V>
This is a delegating RecordReader, which delegates the functionality to the
underlying record reader in
TaggedInputSplit |
Modifier and Type | Method and Description |
---|---|
void |
FileOutputCommitter.abortTask(TaskAttemptContext context,
Path taskAttemptPath) |
void |
FileOutputCommitter.commitTask(TaskAttemptContext context,
Path taskAttemptPath) |
boolean |
FileOutputCommitter.needsTaskCommit(TaskAttemptContext context,
Path taskAttemptPath) |
Constructor and Description |
---|
FileOutputCommitter(Path outputPath,
JobContext context)
Create a file output committer
|
Modifier and Type | Interface and Description |
---|---|
interface |
ClientProtocol
Protocol that a JobClient and the central JobTracker use to communicate.
|
class |
ClientProtocolProvider |
Modifier and Type | Class and Description |
---|---|
class |
SecureShuffleUtils
utilities for generating kyes, hashes and verifying them for shuffle
|
Modifier and Type | Field and Description |
---|---|
static String |
TokenCache.JOB_TOKEN_HDFS_FILE
file name used on HDFS for generated job token
|
static String |
TokenCache.JOB_TOKENS_FILENAME
conf setting for job tokens cache file name
|
Modifier and Type | Method and Description |
---|---|
static Token<?> |
TokenCache.getDelegationToken(Credentials credentials,
String namenode)
Deprecated.
Use
Credentials.getToken(org.apache.hadoop.io.Text)
instead, this method is included for compatibility against Hadoop-1 |
static Token<JobTokenIdentifier> |
TokenCache.getJobToken(Credentials credentials) |
static byte[] |
TokenCache.getShuffleSecretKey(Credentials credentials) |
static Credentials |
TokenCache.loadTokens(String jobTokenFile,
Configuration conf)
Deprecated.
Use
Credentials.readTokenStorageFile(org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration) instead,
this method is included for compatibility against Hadoop-1. |
static Credentials |
TokenCache.loadTokens(String jobTokenFile,
JobConf conf)
Deprecated.
Use
Credentials.readTokenStorageFile(org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration) instead,
this method is included for compatibility against Hadoop-1. |
static void |
TokenCache.setJobToken(Token<? extends TokenIdentifier> t,
Credentials credentials)
store job token
|
static void |
TokenCache.setShuffleSecretKey(byte[] key,
Credentials credentials) |
Modifier and Type | Class and Description |
---|---|
class |
JobTokenIdentifier
The token identifier for job token
|
static class |
JobTokenIdentifier.Renewer |
class |
JobTokenSecretManager
SecretManager for job token.
|
class |
JobTokenSelector
Look through tokens to find the first job token that matches the service
and return it.
|
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenIdentifier
A delegation token identifier that is specific to MapReduce.
|
class |
DelegationTokenSecretManager
A MapReduce specific delegation token secret manager.
|
class |
DelegationTokenSelector
A delegation token that is specialized for MapReduce
|
Modifier and Type | Interface and Description |
---|---|
interface |
JTConfig
Place holder for JobTracker server-level configuration.
|
Modifier and Type | Interface and Description |
---|---|
interface |
TTConfig
Place holder for TaskTracker server-level configuration.
|
Modifier and Type | Class and Description |
---|---|
class |
JobSplit
This class groups the fundamental classes associated with
reading/writing splits.
|
class |
JobSplitWriter
The class that is used by the Job clients to write splits (both the meta
and the raw bytes parts)
|
class |
SplitMetaInfoReader
A utility that reads the split meta info and creates
split meta info objects
|
Modifier and Type | Class and Description |
---|---|
class |
JobContextImpl
A read-only view of the job that is provided to the tasks while they
are running.
|
class |
MapContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper . |
class |
ReduceContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer . |
class |
TaskAttemptContextImpl
The context for task attempts.
|
class |
TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
Modifier and Type | Class and Description |
---|---|
class |
InMemoryReader<K,V>
IFile.InMemoryReader to read map-outputs present in-memory. |
class |
InMemoryWriter<K,V> |
interface |
MergeManager<K,V>
An interface for a reduce side merge that works with the default Shuffle
implementation.
|
class |
ShuffleHeader
Shuffle Header information that is sent by the TaskTracker and
deciphered by the Fetcher thread of Reduce task
|
interface |
ShuffleScheduler<K,V> |
class |
ShuffleSchedulerImpl<K,V> |
Modifier and Type | Field and Description |
---|---|
static String |
CLI.dataPattern |
static String |
CLI.headerPattern |
Modifier and Type | Method and Description |
---|---|
void |
CLI.displayJobList(JobStatus[] jobs,
PrintWriter writer) |
Modifier and Type | Class and Description |
---|---|
class |
ConfigUtil
Place holder for deprecated keys in the framework
|
class |
CountersStrings
String conversion utilities for counters.
|
class |
HostUtil |
class |
ProcessTree
Process tree related operations
|
Modifier and Type | Class and Description |
---|---|
class |
MRDelegationTokenIdentifier
TokenIdentifier that identifies delegation tokens
issued by JobHistoryServer to delegate
MR tasks talking to the JobHistoryServer. |
static class |
MRDelegationTokenIdentifier.Renewer |
Modifier and Type | Interface and Description |
---|---|
interface |
AppContext
Context interface for sharing information across components in YARN App.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskAttemptStateInternal
TaskAttemptImpl internal state machine states.
|
Modifier and Type | Method and Description |
---|---|
void |
JobImpl.constructFinalFullcounters() |
JobStateInternal |
JobImpl.getInternalState() |
Modifier and Type | Method and Description |
---|---|
int |
RMContainerAllocator.getMemLimit() |
void |
RMContainerAllocator.rampDownReduces(int rampDown) |
void |
RMContainerAllocator.rampUpReduces(int rampUp) |
void |
RMContainerAllocator.scheduleAllReduces() |
void |
RMContainerAllocator.scheduleReduces(int totalMaps,
int completedMaps,
int scheduledMaps,
int scheduledReduces,
int assignedMaps,
int assignedReduces,
int mapResourceReqt,
int reduceResourceReqt,
int numPendingReduces,
float maxReduceRampupLimit,
float reduceSlowStart) |
Modifier and Type | Class and Description |
---|---|
class |
ClientHSPolicyProvider
PolicyProvider for YARN MapReduce protocols. |
class |
MRAMPolicyProvider
PolicyProvider for YARN MapReduce protocols. |
Modifier and Type | Class and Description |
---|---|
class |
JHSDelegationTokenSecretManager
A MapReduce specific delegation token secret manager.
|
Modifier and Type | Method and Description |
---|---|
InetSocketAddress |
HistoryClientService.getBindAddress() |
MRClientProtocol |
HistoryClientService.getClientHandler() |
HistoryClientService |
JobHistoryServer.getClientService() |
Modifier and Type | Class and Description |
---|---|
class |
JHAdminConfig
Stores Job History configuration keys that can be set by administrators of
the Job History server.
|
class |
JobHistoryUtils |
Modifier and Type | Class and Description |
---|---|
class |
MRDelegationTokenRenewer |
Modifier and Type | Class and Description |
---|---|
class |
MRApps
Helper class for MR applications
|
class |
MRWebAppUtil |
Modifier and Type | Interface and Description |
---|---|
interface |
MetricsContext
The main interface to the metrics package.
|
interface |
MetricsRecord
A named and optionally tagged set of records to be sent to the metrics
system.
|
class |
MetricsServlet
A servlet to print out metrics data.
|
Modifier and Type | Field and Description |
---|---|
protected static String |
FileContext.FILE_NAME_PROPERTY
Deprecated.
|
protected static String |
FileContext.PERIOD_PROPERTY
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
void |
FileContext.emitRecord(String contextName,
String recordName,
OutputRecord outRec)
Deprecated.
Emits a metrics record to a file.
|
void |
FileContext.flush()
Deprecated.
Flushes the output writer, forcing updates to disk.
|
String |
FileContext.getFileName()
Deprecated.
Returns the configured file name, or null.
|
void |
FileContext.init(String contextName,
ContextFactory factory)
Deprecated.
|
void |
FileContext.startMonitoring()
Deprecated.
Starts or restarts monitoring, by opening in append-mode, the
file specified by the
fileName attribute,
if specified. |
void |
FileContext.stopMonitoring()
Deprecated.
Stops monitoring, closing the file.
|
Constructor and Description |
---|
FileContext()
Deprecated.
Creates a new instance of FileContext
|
Modifier and Type | Method and Description |
---|---|
void |
GangliaContext.emitRecord(String contextName,
String recordName,
OutputRecord outRec) |
void |
GangliaContext.init(String contextName,
ContextFactory factory) |
Constructor and Description |
---|
GangliaContext()
Creates a new instance of GangliaContext
|
Modifier and Type | Class and Description |
---|---|
class |
JvmMetrics
Singleton class which reports Java Virtual Machine metrics to the metrics API.
|
Modifier and Type | Class and Description |
---|---|
static class |
AbstractMetricsContext.MetricMap |
static class |
AbstractMetricsContext.TagMap |
Modifier and Type | Method and Description |
---|---|
void |
CompositeContext.close() |
protected void |
CompositeContext.emitRecord(String contextName,
String recordName,
OutputRecord outRec) |
protected void |
NullContextWithUpdateThread.emitRecord(String contextName,
String recordName,
OutputRecord outRec)
Do-nothing version of emitRecord
|
protected void |
NoEmitMetricsContext.emitRecord(String contextName,
String recordName,
OutputRecord outRec)
Do-nothing version of emitRecord
|
protected void |
NullContext.emitRecord(String contextName,
String recordName,
OutputRecord outRec)
Do-nothing version of emitRecord
|
protected void |
CompositeContext.flush() |
void |
CompositeContext.init(String contextName,
ContextFactory factory) |
void |
NullContextWithUpdateThread.init(String contextName,
ContextFactory factory) |
void |
NoEmitMetricsContext.init(String contextName,
ContextFactory factory) |
boolean |
CompositeContext.isMonitoring()
Return true if all subcontexts are monitoring.
|
MetricsRecord |
CompositeContext.newRecord(String recordName) |
void |
CompositeContext.registerUpdater(Updater updater) |
protected void |
NullContextWithUpdateThread.remove(MetricsRecordImpl record)
Do-nothing version of remove
|
protected void |
NullContext.remove(MetricsRecordImpl record)
Do-nothing version of remove
|
void |
CompositeContext.startMonitoring() |
void |
NullContext.startMonitoring()
Do-nothing version of startMonitoring
|
void |
CompositeContext.stopMonitoring() |
void |
CompositeContext.unregisterUpdater(Updater updater) |
protected void |
NullContextWithUpdateThread.update(MetricsRecordImpl record)
Do-nothing version of update
|
protected void |
NullContext.update(MetricsRecordImpl record)
Do-nothing version of update
|
Constructor and Description |
---|
CompositeContext() |
NoEmitMetricsContext()
Creates a new instance of NullContextWithUpdateThread
|
NullContext()
Creates a new instance of NullContext
|
NullContextWithUpdateThread()
Creates a new instance of NullContextWithUpdateThread
|
Modifier and Type | Class and Description |
---|---|
class |
MetricsBase
This is base class for all metrics
|
Modifier and Type | Method and Description |
---|---|
abstract MetricsSource |
MetricsSystem.getSource(String name) |
abstract MetricsSystem |
MetricsSystem.init(String prefix) |
Modifier and Type | Class and Description |
---|---|
class |
AbstractPatternFilter
Base class for pattern based filters
|
Modifier and Type | Class and Description |
---|---|
class |
MetricsSystemImpl
A base class for metrics system singletons
|
class |
MsInfo
Metrics system related metrics info instances
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultMetricsFactory
Experimental interface to extend metrics dynamically
|
class |
MetricsAnnotations
Metrics annotation helpers.
|
class |
MetricsSourceBuilder
Helper class to build metrics source object from annotations
|
class |
MutableMetricsFactory |
class |
UniqueNames
Generates predictable and user-friendly unique names
|
Modifier and Type | Method and Description |
---|---|
static boolean |
DefaultMetricsSystem.inMiniClusterMode() |
static ObjectName |
DefaultMetricsSystem.newMBeanName(String name) |
MutableRate |
MetricsRegistry.newRate(String name,
String desc,
boolean extended,
boolean returnExisting) |
static MetricsSystem |
DefaultMetricsSystem.setInstance(MetricsSystem ms) |
static void |
DefaultMetricsSystem.setMiniClusterMode(boolean choice) |
static String |
DefaultMetricsSystem.sourceName(String name,
boolean dupOK) |
Modifier and Type | Method and Description |
---|---|
void |
GangliaSink30.appendPrefix(MetricsRecord record,
StringBuilder sb) |
Modifier and Type | Class and Description |
---|---|
class |
JvmMetrics
JVM and logging related metrics.
|
class |
JvmMetricsInfo
JVM and logging related metrics info instances
|
Modifier and Type | Class and Description |
---|---|
class |
Contracts
Additional helpers (besides guava Preconditions) for programming by contract
|
class |
Quantile
Specifies a quantile (with error bounds) to be watched by a
SampleQuantiles object. |
class |
SampleQuantiles
Implementation of the Cormode, Korn, Muthukrishnan, and Srivastava algorithm
for streaming calculation of targeted high-percentile epsilon-approximate
quantiles.
|
class |
SampleStat
Helper to compute running sample stats
|
Modifier and Type | Class and Description |
---|---|
class |
SaslPlainServer |
static class |
UserGroupInformation.HadoopLoginModule
A login module that looks at the Kerberos, Unix, or Windows principal and
adds the corresponding UserName.
|
Modifier and Type | Field and Description |
---|---|
SaslRpcServer.AuthMethod |
SaslRpcServer.authMethod |
Modifier and Type | Method and Description |
---|---|
SaslServer |
SaslRpcServer.create(Server.Connection connection,
SecretManager<TokenIdentifier> secretManager) |
SaslRpcServer.AuthMethod |
SaslRpcClient.getAuthMethod() |
static InetAddress |
SecurityUtil.getByName(String hostname)
Resolves a host subject to the security requirements determined by
hadoop.security.token.service.use_ip.
|
Object |
SaslRpcClient.getNegotiatedProperty(String key) |
static void |
UserGroupInformation.setLoginUser(UserGroupInformation ugi) |
static void |
SecurityUtil.setSecurityInfoProviders(SecurityInfo... providers)
Test setup method to register additional providers.
|
Constructor and Description |
---|
SaslRpcServer(SaslRpcServer.AuthMethod authMethod) |
Modifier and Type | Class and Description |
---|---|
class |
ProxyUsers |
Modifier and Type | Class and Description |
---|---|
class |
FileBasedKeyStoresFactory
KeyStoresFactory implementation that reads the certificates from
keystore files. |
interface |
KeyStoresFactory
Interface that gives access to
KeyManager and TrustManager
implementations. |
class |
ReloadingX509TrustManager
A
TrustManager implementation that reloads its configuration when
the truststore file on disk changes. |
class |
SSLFactory
Factory that creates SSLEngine and SSLSocketFactory instances using
Hadoop configuration information.
|
static class |
SSLFactory.Mode |
interface |
SSLHostnameVerifier
Copied from the not-yet-commons-ssl project at
http://juliusdavies.ca/commons-ssl/
This project is not yet in Apache, but it is Apache 2.0 licensed.
|
Modifier and Type | Class and Description |
---|---|
static class |
Token.PrivateToken<T extends TokenIdentifier>
Indicates whether the token is a clone.
|
Modifier and Type | Method and Description |
---|---|
void |
Token.setKind(Text newKind)
Set the token kind.
|
Modifier and Type | Class and Description |
---|---|
class |
Environment
This is a class used to get the current environment
on the host machines running the map/reduce.
|
class |
PathFinder
Maps a relative pathname to an absolute pathname using the PATH environment.
|
class |
StreamUtil
Utilities used in streaming
|
Modifier and Type | Class and Description |
---|---|
class |
ClassUtil |
class |
DiskChecker
Class that provides utility functions for checking disk problem
|
class |
GenericOptionsParser
GenericOptionsParser is a utility to parse command line
arguments generic to the Hadoop framework. |
class |
GenericsUtil
Contains utility methods for dealing with Java Generics.
|
interface |
GSet<K,E extends K>
A
GSet is set,
which supports the GSet.get(Object) operation. |
class |
GSetByHashMap<K,E extends K>
|
class |
HeapSort
An implementation of the core algorithm of HeapSort.
|
interface |
IdGenerator
Generic ID generator
used for generating various types of number sequences.
|
class |
LightWeightCache<K,E extends K>
A low memory footprint Cache which extends
LightWeightGSet . |
class |
LightWeightGSet<K,E extends K>
A low memory footprint
GSet implementation,
which uses an array for storing the elements
and linked lists for collision resolution. |
class |
NativeCodeLoader
A helper to load the native hadoop code i.e.
|
class |
NativeLibraryChecker |
class |
PrintJarMainClass
A micro-application that prints the main class name out of a jar file.
|
class |
PriorityQueue<T>
A PriorityQueue maintains a partial ordering of its elements such that the
least element can always be found in constant time.
|
class |
QuickSort
An implementation of the core algorithm of QuickSort.
|
class |
RunJar
Run a Hadoop job jar.
|
class |
SequentialNumber
Sequential number generator.
|
interface |
ServicePlugin
Service plug-in interface.
|
class |
ServletUtil |
class |
SignalLogger
This class logs a message whenever we're about to exit on a UNIX signal.
|
class |
StringUtils
General string utils
|
class |
UTF8ByteArrayUtils |
class |
VersionInfo
This class returns build information about Hadoop components.
|
class |
VersionUtil |
class |
XMLUtils
General xml utilities.
|
class |
ZKUtil
Utilities for working with ZooKeeper.
|
static class |
ZKUtil.BadAclFormatException |
static class |
ZKUtil.BadAuthFormatException |
static class |
ZKUtil.ZKAuthInfo
An authentication token passed to ZooKeeper.addAuthInfo
|
Modifier and Type | Class and Description |
---|---|
class |
Hash
This class represents a common API for hashing functions.
|
class |
JenkinsHash
Produces 32-bit hash for hash table lookup.
|
class |
MurmurHash
This is a very fast, non-cryptographic hash suitable for general hash-based
lookup.
|
Modifier and Type | Interface and Description |
---|---|
interface |
ApplicationClientProtocolPB |
interface |
ApplicationMasterProtocolPB |
interface |
ContainerManagementProtocolPB |
Modifier and Type | Method and Description |
---|---|
CancelDelegationTokenResponse |
ApplicationClientProtocol.cancelDelegationToken(CancelDelegationTokenRequest request)
Cancel an existing delegation
Token . |
RenewDelegationTokenResponse |
ApplicationClientProtocol.renewDelegationToken(RenewDelegationTokenRequest request)
Renew an existing delegation
Token . |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationClientProtocolPBClientImpl |
class |
ApplicationMasterProtocolPBClientImpl |
class |
ContainerManagementProtocolPBClientImpl |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationClientProtocolPBServiceImpl |
class |
ApplicationMasterProtocolPBServiceImpl |
class |
ContainerManagementProtocolPBServiceImpl |
Modifier and Type | Class and Description |
---|---|
class |
CancelDelegationTokenRequest
The request issued by the client to the
ResourceManager to cancel a
delegation token. |
class |
CancelDelegationTokenResponse
The response from the
ResourceManager to a cancelDelegationToken
request. |
class |
RenewDelegationTokenRequest
The request issued by the client to renew a delegation token from
the
ResourceManager . |
class |
RenewDelegationTokenResponse
The response to a renewDelegationToken call to the
ResourceManager . |
Modifier and Type | Method and Description |
---|---|
abstract Token |
RenewDelegationTokenRequest.getDelegationToken()
Get the delegation token requested to be renewed by the client.
|
abstract Token |
CancelDelegationTokenRequest.getDelegationToken()
Get the delegation token requested to be cancelled.
|
abstract long |
RenewDelegationTokenResponse.getNextExpirationTime() |
static KillApplicationResponse |
KillApplicationResponse.newInstance() |
static CancelDelegationTokenResponse |
CancelDelegationTokenResponse.newInstance() |
static SubmitApplicationResponse |
SubmitApplicationResponse.newInstance() |
static GetNewApplicationResponse |
GetNewApplicationResponse.newInstance(ApplicationId applicationId,
Resource minCapability,
Resource maxCapability) |
static GetApplicationReportResponse |
GetApplicationReportResponse.newInstance(ApplicationReport ApplicationReport) |
static FinishApplicationMasterResponse |
FinishApplicationMasterResponse.newInstance(boolean isRemovedFromRMStateStore) |
static GetApplicationsResponse |
GetApplicationsResponse.newInstance(List<ApplicationReport> applications) |
static StopContainersResponse |
StopContainersResponse.newInstance(List<ContainerId> succeededRequests,
Map<ContainerId,SerializedException> failedRequests) |
static GetContainerStatusesResponse |
GetContainerStatusesResponse.newInstance(List<ContainerStatus> statuses,
Map<ContainerId,SerializedException> failedRequests) |
static GetClusterNodesResponse |
GetClusterNodesResponse.newInstance(List<NodeReport> nodeReports) |
static GetQueueUserAclsInfoResponse |
GetQueueUserAclsInfoResponse.newInstance(List<QueueUserACLInfo> queueUserAclsList) |
static RenewDelegationTokenResponse |
RenewDelegationTokenResponse.newInstance(long expTime) |
static StartContainersResponse |
StartContainersResponse.newInstance(Map<String,ByteBuffer> servicesMetaData,
List<ContainerId> succeededContainers,
Map<ContainerId,SerializedException> failedContainers) |
static GetQueueInfoResponse |
GetQueueInfoResponse.newInstance(QueueInfo queueInfo) |
static RegisterApplicationMasterResponse |
RegisterApplicationMasterResponse.newInstance(Resource minCapability,
Resource maxCapability,
Map<ApplicationAccessType,String> acls,
ByteBuffer key) |
static RenewDelegationTokenRequest |
RenewDelegationTokenRequest.newInstance(Token dToken) |
static GetDelegationTokenResponse |
GetDelegationTokenResponse.newInstance(Token rmDTToken) |
static CancelDelegationTokenRequest |
CancelDelegationTokenRequest.newInstance(Token dToken) |
static GetClusterMetricsResponse |
GetClusterMetricsResponse.newInstance(YarnClusterMetrics metrics) |
abstract void |
AllocateResponse.setAllocatedContainers(List<Container> containers)
Set the list of newly allocated
Container by the
ResourceManager . |
abstract void |
StartContainersResponse.setAllServicesMetaData(Map<String,ByteBuffer> allServicesMetaData)
Set to the list of auxiliary services which have been started on the
NodeManager . |
abstract void |
AllocateResponse.setAMCommand(AMCommand command) |
abstract void |
RegisterApplicationMasterResponse.setApplicationACLs(Map<ApplicationAccessType,String> acls)
Set the
ApplicationACL s for the application. |
abstract void |
GetNewApplicationResponse.setApplicationId(ApplicationId applicationId) |
abstract void |
GetApplicationsResponse.setApplicationList(List<ApplicationReport> applications) |
abstract void |
GetApplicationReportResponse.setApplicationReport(ApplicationReport ApplicationReport) |
abstract void |
GetApplicationsRequest.setApplicationStates(EnumSet<YarnApplicationState> applicationStates)
Set the application states to filter applications on
|
abstract void |
GetApplicationsRequest.setApplicationTypes(Set<String> applicationTypes)
Set the application types to filter applications on
|
abstract void |
AllocateResponse.setAvailableResources(Resource limit) |
abstract void |
GetClusterMetricsResponse.setClusterMetrics(YarnClusterMetrics metrics) |
abstract void |
AllocateResponse.setCompletedContainersStatuses(List<ContainerStatus> containers) |
abstract void |
GetContainerStatusesResponse.setContainerStatuses(List<ContainerStatus> statuses)
Set the
ContainerStatus es of the requested containers. |
abstract void |
RenewDelegationTokenRequest.setDelegationToken(Token dToken) |
abstract void |
CancelDelegationTokenRequest.setDelegationToken(Token dToken) |
abstract void |
StopContainersResponse.setFailedRequests(Map<ContainerId,SerializedException> failedRequests)
Set the containerId-to-exception map in which the exception indicates error
from per container for failed requests
|
abstract void |
StartContainersResponse.setFailedRequests(Map<ContainerId,SerializedException> failedContainers)
Set the containerId-to-exception map in which the exception indicates error
from per container for failed requests
|
abstract void |
GetContainerStatusesResponse.setFailedRequests(Map<ContainerId,SerializedException> failedContainers)
Set the containerId-to-exception map in which the exception indicates error
from per container for failed requests
|
abstract void |
FinishApplicationMasterResponse.setIsUnregistered(boolean isUnregistered)
Set the flag which indicates that the application has successfully
unregistered with the RM and the application can safely stop.
|
abstract void |
GetNewApplicationResponse.setMaximumResourceCapability(Resource capability) |
abstract void |
RegisterApplicationMasterResponse.setMaximumResourceCapability(Resource capability) |
abstract void |
RenewDelegationTokenResponse.setNextExpirationTime(long expTime) |
abstract void |
AllocateResponse.setNMTokens(List<NMToken> nmTokens) |
abstract void |
GetClusterNodesResponse.setNodeReports(List<NodeReport> nodeReports) |
abstract void |
AllocateResponse.setNumClusterNodes(int numNodes) |
abstract void |
AllocateResponse.setPreemptionMessage(PreemptionMessage request) |
abstract void |
GetQueueInfoResponse.setQueueInfo(QueueInfo queueInfo) |
abstract void |
AllocateResponse.setResponseId(int responseId) |
abstract void |
GetDelegationTokenResponse.setRMDelegationToken(Token rmDTToken) |
abstract void |
StartContainersResponse.setSuccessfullyStartedContainers(List<ContainerId> succeededContainers) |
abstract void |
StopContainersResponse.setSuccessfullyStoppedContainers(List<ContainerId> succeededRequests)
Set the list of containerIds of successfully stopped containers.
|
abstract void |
AllocateResponse.setUpdatedNodes(List<NodeReport> updatedNodes) |
abstract void |
GetQueueUserAclsInfoResponse.setUserAclsInfoList(List<QueueUserACLInfo> queueUserAclsList) |
Modifier and Type | Class and Description |
---|---|
class |
SerializedException |
Modifier and Type | Field and Description |
---|---|
static String |
ApplicationAttemptId.appAttemptIdStrPrefix |
static String |
ApplicationId.appIdStrPrefix |
Modifier and Type | Method and Description |
---|---|
abstract Throwable |
SerializedException.deSerialize()
Deserialize the exception to a new Throwable.
|
abstract SerializedException |
SerializedException.getCause()
Get the cause of this exception or null if the cause is nonexistent or
unknown.
|
abstract String |
SerializedException.getMessage()
Get the detail message string of this exception.
|
abstract int |
NodeReport.getNumContainers()
Get the number of allocated containers on the node.
|
abstract int |
ApplicationResourceUsageReport.getNumReservedContainers()
Get the number of reserved containers
|
abstract String |
ApplicationReport.getOriginalTrackingUrl()
Get the original not-proxied tracking url for the application.
|
abstract String |
SerializedException.getRemoteTrace()
Get the backtrace of this exception.
|
abstract void |
SerializedException.init(String message)
Constructs a new
SerializedException with the specified detail
message. |
abstract void |
SerializedException.init(String message,
Throwable cause)
Constructs a new
SerializedException with the specified detail
message and cause. |
abstract void |
SerializedException.init(Throwable cause)
Constructs a new
SerializedException with the specified cause. |
static ContainerId |
ContainerId.newInstance(ApplicationAttemptId appAttemptId,
int containerId) |
static ApplicationReport |
ApplicationReport.newInstance(ApplicationId applicationId,
ApplicationAttemptId applicationAttemptId,
String user,
String queue,
String name,
String host,
int rpcPort,
Token clientToAMToken,
YarnApplicationState state,
String diagnostics,
String url,
long startTime,
long finishTime,
FinalApplicationStatus finalStatus,
ApplicationResourceUsageReport appResources,
String origTrackingUrl,
float progress,
String applicationType,
Token amRmToken) |
static ApplicationAttemptId |
ApplicationAttemptId.newInstance(ApplicationId appId,
int attemptId) |
static Token |
Token.newInstance(byte[] identifier,
String kind,
byte[] password,
String service) |
static PreemptionContainer |
PreemptionContainer.newInstance(ContainerId id) |
static ContainerStatus |
ContainerStatus.newInstance(ContainerId containerId,
ContainerState containerState,
String diagnostics,
int exitStatus) |
static Container |
Container.newInstance(ContainerId containerId,
NodeId nodeId,
String nodeHttpAddress,
Resource resource,
Priority priority,
Token containerToken) |
static YarnClusterMetrics |
YarnClusterMetrics.newInstance(int numNodeManagers) |
static ApplicationResourceUsageReport |
ApplicationResourceUsageReport.newInstance(int numUsedContainers,
int numReservedContainers,
Resource usedResources,
Resource reservedResources,
Resource neededResources) |
static PreemptionContract |
PreemptionContract.newInstance(List<PreemptionResourceRequest> req,
Set<PreemptionContainer> containers) |
static ApplicationId |
ApplicationId.newInstance(long clusterTimestamp,
int id) |
static NodeReport |
NodeReport.newInstance(NodeId nodeId,
NodeState nodeState,
String httpAddress,
String rackName,
Resource used,
Resource capability,
int numContainers,
String healthReport,
long lastHealthReportTime) |
static NMToken |
NMToken.newInstance(NodeId nodeId,
Token token) |
static PreemptionResourceRequest |
PreemptionResourceRequest.newInstance(ResourceRequest req) |
static StrictPreemptionContract |
StrictPreemptionContract.newInstance(Set<PreemptionContainer> containers) |
static PreemptionMessage |
PreemptionMessage.newInstance(StrictPreemptionContract set,
PreemptionContract contract) |
static QueueInfo |
QueueInfo.newInstance(String queueName,
float capacity,
float maximumCapacity,
float currentCapacity,
List<QueueInfo> childQueues,
List<ApplicationReport> applications,
QueueState queueState) |
static NodeId |
NodeId.newInstance(String host,
int port) |
static QueueUserACLInfo |
QueueUserACLInfo.newInstance(String queueName,
List<QueueACL> acls) |
static SerializedException |
SerializedException.newInstance(Throwable e) |
abstract void |
ApplicationReport.setAMRMToken(Token amRmToken) |
protected abstract void |
ContainerId.setApplicationAttemptId(ApplicationAttemptId atId) |
abstract void |
ApplicationReport.setApplicationId(ApplicationId applicationId) |
protected abstract void |
ApplicationAttemptId.setApplicationId(ApplicationId appID) |
abstract void |
ApplicationReport.setApplicationResourceUsageReport(ApplicationResourceUsageReport appResources)
Store the structure containing the job resources for this application
|
abstract void |
QueueInfo.setApplications(List<ApplicationReport> applications) |
abstract void |
ApplicationReport.setApplicationType(String applicationType) |
protected abstract void |
ApplicationAttemptId.setAttemptId(int attemptId) |
abstract void |
NodeReport.setCapability(Resource capability) |
abstract void |
QueueInfo.setCapacity(float capacity) |
abstract void |
QueueInfo.setChildQueues(List<QueueInfo> childQueues) |
abstract void |
ApplicationReport.setClientToAMToken(Token clientToAMToken) |
protected abstract void |
ApplicationId.setClusterTimestamp(long clusterTimestamp) |
abstract void |
ContainerStatus.setContainerId(ContainerId containerId) |
abstract void |
PreemptionContract.setContainers(Set<PreemptionContainer> containers) |
abstract void |
StrictPreemptionContract.setContainers(Set<PreemptionContainer> containers) |
abstract void |
Container.setContainerToken(Token containerToken) |
abstract void |
PreemptionMessage.setContract(PreemptionContract contract) |
abstract void |
ApplicationReport.setCurrentApplicationAttemptId(ApplicationAttemptId applicationAttemptId) |
abstract void |
QueueInfo.setCurrentCapacity(float currentCapacity) |
abstract void |
ApplicationReport.setDiagnostics(String diagnostics) |
abstract void |
ContainerStatus.setDiagnostics(String diagnostics) |
abstract void |
ContainerStatus.setExitStatus(int exitStatus) |
abstract void |
ApplicationReport.setFinalApplicationStatus(FinalApplicationStatus finishState) |
abstract void |
ApplicationReport.setFinishTime(long finishTime) |
abstract void |
NodeReport.setHealthReport(String healthReport) |
abstract void |
ApplicationReport.setHost(String host) |
protected abstract void |
NodeId.setHost(String host) |
abstract void |
NodeReport.setHttpAddress(String httpAddress) |
abstract void |
Container.setId(ContainerId id) |
abstract void |
PreemptionContainer.setId(ContainerId id) |
protected abstract void |
ApplicationId.setId(int id) |
protected abstract void |
ContainerId.setId(int id) |
abstract void |
Token.setIdentifier(ByteBuffer identifier) |
abstract void |
Token.setKind(String kind) |
abstract void |
NodeReport.setLastHealthReportTime(long lastHealthReport) |
abstract void |
QueueInfo.setMaximumCapacity(float maximumCapacity) |
abstract void |
ApplicationReport.setName(String name) |
abstract void |
ApplicationResourceUsageReport.setNeededResources(Resource needed_resources) |
abstract void |
Container.setNodeHttpAddress(String nodeHttpAddress) |
abstract void |
NodeReport.setNodeId(NodeId nodeId) |
abstract void |
Container.setNodeId(NodeId nodeId) |
abstract void |
NodeReport.setNodeState(NodeState nodeState) |
abstract void |
NodeReport.setNumContainers(int numContainers) |
abstract void |
YarnClusterMetrics.setNumNodeManagers(int numNodeManagers) |
abstract void |
ApplicationResourceUsageReport.setNumReservedContainers(int num_reserved_containers)
Set the number of reserved containers
|
abstract void |
ApplicationResourceUsageReport.setNumUsedContainers(int num_containers)
Set the number of used containers
|
abstract void |
ApplicationReport.setOriginalTrackingUrl(String url) |
abstract void |
Token.setPassword(ByteBuffer password) |
protected abstract void |
NodeId.setPort(int port) |
abstract void |
ApplicationSubmissionContext.setPriority(Priority priority)
Set the
Priority of the application. |
abstract void |
Container.setPriority(Priority priority) |
abstract void |
ApplicationReport.setProgress(float progress) |
abstract void |
ApplicationReport.setQueue(String queue) |
abstract void |
QueueUserACLInfo.setQueueName(String queueName) |
abstract void |
QueueInfo.setQueueName(String queueName) |
abstract void |
QueueInfo.setQueueState(QueueState queueState) |
abstract void |
NodeReport.setRackName(String rackName) |
abstract void |
ApplicationResourceUsageReport.setReservedResources(Resource reserved_resources) |
abstract void |
Container.setResource(Resource resource) |
abstract void |
PreemptionContract.setResourceRequest(List<PreemptionResourceRequest> req) |
abstract void |
PreemptionResourceRequest.setResourceRequest(ResourceRequest req) |
abstract void |
ApplicationReport.setRpcPort(int rpcPort) |
abstract void |
Token.setService(String service) |
abstract void |
ApplicationReport.setStartTime(long startTime) |
abstract void |
ContainerStatus.setState(ContainerState state) |
abstract void |
PreemptionMessage.setStrictContract(StrictPreemptionContract set) |
abstract void |
ApplicationReport.setTrackingUrl(String url) |
abstract void |
NodeReport.setUsed(Resource used) |
abstract void |
ApplicationResourceUsageReport.setUsedResources(Resource resources) |
abstract void |
ApplicationReport.setUser(String user) |
abstract void |
QueueUserACLInfo.setUserAcls(List<QueueACL> acls) |
abstract void |
ApplicationReport.setYarnApplicationState(YarnApplicationState state) |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationAttemptIdPBImpl |
class |
ApplicationIdPBImpl |
class |
ApplicationReportPBImpl |
class |
ApplicationResourceUsageReportPBImpl |
class |
ApplicationSubmissionContextPBImpl |
class |
ContainerIdPBImpl |
class |
ContainerLaunchContextPBImpl |
class |
ContainerPBImpl |
class |
ContainerStatusPBImpl |
class |
LocalResourcePBImpl |
class |
NMTokenPBImpl |
class |
NodeIdPBImpl |
class |
NodeReportPBImpl |
class |
PreemptionContainerPBImpl |
class |
PreemptionContractPBImpl |
class |
PreemptionMessagePBImpl |
class |
PreemptionResourceRequestPBImpl |
class |
PriorityPBImpl |
class |
ProtoBase<T extends com.google.protobuf.Message> |
class |
ProtoUtils |
class |
QueueInfoPBImpl |
class |
QueueUserACLInfoPBImpl |
class |
ResourceBlacklistRequestPBImpl |
class |
ResourcePBImpl |
class |
ResourceRequestPBImpl |
class |
StrictPreemptionContractPBImpl |
class |
TokenPBImpl |
class |
URLPBImpl |
class |
YarnClusterMetricsPBImpl |
Modifier and Type | Method and Description |
---|---|
static RetryPolicy |
RMProxy.createRetryPolicy(Configuration conf) |
Modifier and Type | Method and Description |
---|---|
static void |
NMTokenCache.clearCache()
It will remove all the nm tokens from its cache
|
static boolean |
NMTokenCache.containsNMToken(String nodeAddr)
Returns true if NMToken is present in cache.
|
static int |
NMTokenCache.numberOfNMTokensInCache()
Returns the number of NMTokens present in cache.
|
static void |
NMTokenCache.removeNMToken(String nodeAddr)
Removes NMToken for specified node manager
|
Constructor and Description |
---|
AMRMClient(String name) |
NMClient(String name) |
YarnClient(String name) |
Constructor and Description |
---|
AMRMClientAsync(AMRMClient<T> client,
int intervalMs,
AMRMClientAsync.CallbackHandler callbackHandler) |
NMClientAsync(String name,
NMClient client,
NMClientAsync.CallbackHandler callbackHandler) |
Modifier and Type | Class and Description |
---|---|
class |
AMRMClientAsyncImpl<T extends AMRMClient.ContainerRequest> |
class |
NMClientAsyncImpl |
Constructor and Description |
---|
AMRMClientAsyncImpl(AMRMClient<T> client,
int intervalMs,
AMRMClientAsync.CallbackHandler callbackHandler) |
NMClientAsyncImpl(String name,
NMClient client,
NMClientAsync.CallbackHandler callbackHandler) |
Modifier and Type | Class and Description |
---|---|
class |
AMRMClientImpl<T extends AMRMClient.ContainerRequest> |
class |
NMClientImpl
This class implements
NMClient . |
class |
YarnClientImpl |
Modifier and Type | Method and Description |
---|---|
protected ContainerManagementProtocol |
ContainerManagementProtocolProxy.ContainerManagementProtocolProxyData.newProxy(YarnRPC rpc,
String containerManagerBindAddr,
ContainerId containerId,
Token token) |
protected void |
AMRMClientImpl.populateNMTokens(AllocateResponse allocateResponse) |
void |
YarnClientImpl.setRMClient(ApplicationClientProtocol rmClient) |
Constructor and Description |
---|
ContainerManagementProtocolProxy.ContainerManagementProtocolProxyData(YarnRPC rpc,
String containerManagerBindAddr,
ContainerId containerId,
Token token) |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationCLI |
class |
NodeCLI |
class |
RMAdminCLI |
class |
YarnCLI |
Modifier and Type | Class and Description |
---|---|
class |
RecordFactoryPBImpl |
class |
RpcClientFactoryPBImpl |
class |
RpcServerFactoryPBImpl |
Modifier and Type | Class and Description |
---|---|
class |
AggregatedLogDeletionService
A service that periodically deletes aggregated logs.
|
static class |
AggregatedLogFormat.ContainerLogsReader |
static class |
AggregatedLogFormat.LogValue |
static class |
AggregatedLogFormat.LogWriter
The writer that writes out the aggregated logs.
|
class |
ContainerLogsRetentionPolicy |
class |
LogAggregationUtils |
Modifier and Type | Method and Description |
---|---|
int |
LogCLIHelpers.dumpAContainerLogs(String containerIdStr,
AggregatedLogFormat.LogReader reader,
PrintStream out) |
int |
LogCLIHelpers.dumpAContainersLogs(String appId,
String containerId,
String nodeId,
String jobOwner) |
int |
LogCLIHelpers.dumpAllContainersLogs(ApplicationId appId,
String appOwner,
PrintStream out) |
AggregatedLogFormat.ContainerLogsReader |
AggregatedLogFormat.LogReader.getContainerLogsReader(ContainerId containerId)
Get a ContainerLogsReader to read the logs for
the specified container.
|
void |
AggregatedLogFormat.LogKey.readFields(DataInput in) |
void |
AggregatedLogFormat.LogKey.write(DataOutput out) |
Modifier and Type | Class and Description |
---|---|
class |
AdminACLsManager |
static class |
AMRMTokenIdentifier.Renewer |
static class |
ContainerTokenIdentifier.Renewer |
Modifier and Type | Method and Description |
---|---|
ApplicationAttemptId |
AMRMTokenIdentifier.getApplicationAttemptId() |
Modifier and Type | Class and Description |
---|---|
static class |
ClientToAMTokenIdentifier.Renewer |
Modifier and Type | Method and Description |
---|---|
ClientToAMTokenIdentifier |
BaseClientToAMTokenSecretManager.createIdentifier() |
byte[] |
BaseClientToAMTokenSecretManager.createPassword(ClientToAMTokenIdentifier identifier) |
abstract SecretKey |
BaseClientToAMTokenSecretManager.getMasterKey(ApplicationAttemptId applicationAttemptId) |
byte[] |
BaseClientToAMTokenSecretManager.retrievePassword(ClientToAMTokenIdentifier identifier) |
static void |
RMDelegationTokenIdentifier.Renewer.setSecretManager(AbstractDelegationTokenSecretManager<RMDelegationTokenIdentifier> secretManager,
InetSocketAddress serviceAddress) |
Modifier and Type | Interface and Description |
---|---|
interface |
ResourceManagerAdministrationProtocol |
interface |
ResourceManagerAdministrationProtocolPB |
Constructor and Description |
---|
ApplicationInitializationContext(String user,
ApplicationId applicationId,
ByteBuffer appDataForService) |
ApplicationTerminationContext(ApplicationId applicationId) |
ContainerContext(String user,
ContainerId containerId,
Resource resource) |
ContainerInitializationContext(String user,
ContainerId containerId,
Resource resource) |
ContainerTerminationContext(String user,
ContainerId containerId,
Resource resource) |
Modifier and Type | Class and Description |
---|---|
class |
ResourceManagerAdministrationProtocolPBClientImpl |
Modifier and Type | Class and Description |
---|---|
class |
ResourceManagerAdministrationProtocolPBServiceImpl |
Modifier and Type | Class and Description |
---|---|
class |
RefreshAdminAclsRequest |
class |
RefreshAdminAclsResponse |
class |
RefreshNodesRequest |
class |
RefreshNodesResponse |
class |
RefreshQueuesRequest |
class |
RefreshQueuesResponse |
class |
RefreshServiceAclsRequest |
class |
RefreshServiceAclsResponse |
class |
RefreshSuperUserGroupsConfigurationRequest |
class |
RefreshSuperUserGroupsConfigurationResponse |
class |
RefreshUserToGroupsMappingsRequest |
class |
RefreshUserToGroupsMappingsResponse |
Modifier and Type | Method and Description |
---|---|
static RefreshAdminAclsResponse |
RefreshAdminAclsResponse.newInstance() |
static RefreshQueuesResponse |
RefreshQueuesResponse.newInstance() |
static RefreshServiceAclsResponse |
RefreshServiceAclsResponse.newInstance() |
static RefreshUserToGroupsMappingsResponse |
RefreshUserToGroupsMappingsResponse.newInstance() |
static RefreshNodesResponse |
RefreshNodesResponse.newInstance() |
static RefreshSuperUserGroupsConfigurationResponse |
RefreshSuperUserGroupsConfigurationResponse.newInstance() |
Modifier and Type | Class and Description |
---|---|
class |
RefreshAdminAclsRequestPBImpl |
class |
RefreshAdminAclsResponsePBImpl |
class |
RefreshNodesRequestPBImpl |
class |
RefreshNodesResponsePBImpl |
class |
RefreshQueuesRequestPBImpl |
class |
RefreshQueuesResponsePBImpl |
class |
RefreshServiceAclsRequestPBImpl |
class |
RefreshServiceAclsResponsePBImpl |
class |
RefreshSuperUserGroupsConfigurationRequestPBImpl |
class |
RefreshSuperUserGroupsConfigurationResponsePBImpl |
class |
RefreshUserToGroupsMappingsRequestPBImpl |
class |
RefreshUserToGroupsMappingsResponsePBImpl |
Modifier and Type | Method and Description |
---|---|
static NodeHealthStatus |
NodeHealthStatus.newInstance(boolean isNodeHealthy,
String healthReport,
long lastHealthReport) |
abstract void |
NodeHealthStatus.setHealthReport(String healthReport) |
abstract void |
NodeHealthStatus.setIsNodeHealthy(boolean isNodeHealthy) |
abstract void |
NodeHealthStatus.setLastHealthReportTime(long lastHealthReport) |
Modifier and Type | Method and Description |
---|---|
void |
NodeStatusUpdaterImpl.addStoppedContainersToCache(ContainerId containerId) |
NodeStatusUpdater |
NodeManager.getNodeStatusUpdater() |
boolean |
DeletionService.isTerminated()
Determine if the service has completely stopped.
|
protected boolean |
NodeStatusUpdaterImpl.isTokenKeepAliveEnabled(Configuration conf) |
void |
NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() |
Modifier and Type | Interface and Description |
---|---|
interface |
ResourceLocalizationSpec |
Modifier and Type | Method and Description |
---|---|
protected void |
ContainerManagerImpl.authorizeGetAndStopContainerRequest(ContainerId containerId,
Container container,
boolean stopRequest,
NMTokenIdentifier identifier) |
protected void |
ContainerManagerImpl.authorizeStartRequest(NMTokenIdentifier nmTokenIdentifier,
ContainerTokenIdentifier containerTokenIdentifier) |
boolean |
ContainerManagerImpl.getBlockNewContainerRequestsStatus() |
protected NMTokenIdentifier |
ContainerManagerImpl.selectNMTokenIdentifier(UserGroupInformation remoteUgi) |
protected void |
ContainerManagerImpl.updateNMTokenIdentifier(NMTokenIdentifier nmTokenIdentifier) |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.LocalizerRunner |
ResourceLocalizationService.getLocalizerRunner(String locId) |
Map<String,org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.LocalizerRunner> |
ResourceLocalizationService.getPrivateLocalizers() |
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.PublicLocalizer |
ResourceLocalizationService.getPublicLocalizer() |
Modifier and Type | Method and Description |
---|---|
NodeId |
NMTokenSecretManagerInNM.getNodeId() |
boolean |
NMTokenSecretManagerInNM.isAppAttemptNMTokenKeyPresent(ApplicationAttemptId appAttemptId) |
void |
NMContainerTokenSecretManager.setMasterKey(MasterKey masterKeyRecord)
Used by NodeManagers to create a token-secret-manager with the key obtained
from the RM.
|
void |
NMTokenSecretManagerInNM.setMasterKey(MasterKey masterKey)
Used by NodeManagers to create a token-secret-manager with the key
obtained from the RM.
|
Modifier and Type | Class and Description |
---|---|
class |
NMPolicyProvider
PolicyProvider for YARN NodeManager protocols. |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationMasterService |
class |
ClusterMetrics |
static class |
ResourceManager.ApplicationAttemptEventDispatcher |
static class |
ResourceManager.ApplicationEventDispatcher |
static class |
ResourceManager.NodeEventDispatcher |
static class |
ResourceManager.RMContainerPreemptEventDispatcher |
static class |
ResourceManager.SchedulerEventDispatcher |
Modifier and Type | Method and Description |
---|---|
AMRMTokenSecretManager |
ResourceManager.getAMRMTokenSecretManager() |
ApplicationACLsManager |
ResourceManager.getApplicationACLsManager() |
ApplicationMasterService |
ResourceManager.getApplicationMasterService() |
InetSocketAddress |
ApplicationMasterService.getBindAddress() |
InetSocketAddress |
ClientRMService.getBindAddress() |
ClientRMService |
ResourceManager.getClientRMService() |
QueueACLsManager |
ResourceManager.getQueueACLsManager() |
ResourceScheduler |
ResourceManager.getResourceScheduler()
return the scheduler.
|
ResourceTrackerService |
ResourceManager.getResourceTrackerService()
return the resource tracking component.
|
RMContainerTokenSecretManager |
ResourceManager.getRMContainerTokenSecretManager() |
NMTokenSecretManagerInRM |
ResourceManager.getRMNMTokenSecretManager() |
Modifier and Type | Class and Description |
---|---|
class |
FileSystemRMStateStore |
class |
MemoryRMStateStore |
class |
RMStateStore |
Modifier and Type | Class and Description |
---|---|
class |
ResourceType |
class |
ResourceWeights |
Modifier and Type | Class and Description |
---|---|
class |
RMNodeImpl
This class is used to keep track of all the applications/containers
running on a node.
|
Modifier and Type | Class and Description |
---|---|
class |
ActiveUsersManager
ActiveUsersManager tracks active users in the system. |
class |
AppSchedulingInfo
This class keeps track of all the consumption of an application.
|
class |
NodeReport
Node usage report.
|
class |
QueueMetrics |
class |
SchedulerApplication
Represents an Application from the viewpoint of the scheduler.
|
class |
SchedulerNode
Represents a YARN Cluster Node from the viewpoint of the scheduler.
|
class |
SchedulerNodeReport
Node usage report.
|
class |
SchedulerUtils
Utilities shared by schedulers.
|
Modifier and Type | Method and Description |
---|---|
static void |
QueueMetrics.clearQueueMetrics()
Helper method to clear cache - used only for unit tests.
|
Modifier and Type | Class and Description |
---|---|
class |
CSAssignment |
interface |
CSQueue
CSQueue represents a node in the tree of
hierarchical queues in the CapacityScheduler . |
class |
LeafQueue |
class |
ParentQueue |
Modifier and Type | Field and Description |
---|---|
static String |
CapacitySchedulerConfiguration.ALL_ACL |
static String |
CapacitySchedulerConfiguration.CAPACITY |
static boolean |
CapacitySchedulerConfiguration.DEFAULT_ENABLE_USER_METRICS |
static float |
CapacitySchedulerConfiguration.DEFAULT_MAXIMUM_APPLICATIONMASTERS_RESOURCE_PERCENT |
static float |
CapacitySchedulerConfiguration.DEFAULT_MAXIMUM_CAPACITY_VALUE |
static int |
CapacitySchedulerConfiguration.DEFAULT_MAXIMUM_SYSTEM_APPLICATIIONS |
static int |
CapacitySchedulerConfiguration.DEFAULT_NODE_LOCALITY_DELAY |
static Class<? extends ResourceCalculator> |
CapacitySchedulerConfiguration.DEFAULT_RESOURCE_CALCULATOR_CLASS |
static int |
CapacitySchedulerConfiguration.DEFAULT_USER_LIMIT |
static float |
CapacitySchedulerConfiguration.DEFAULT_USER_LIMIT_FACTOR |
static String |
CapacitySchedulerConfiguration.DOT |
static String |
CapacitySchedulerConfiguration.ENABLE_USER_METRICS |
static String |
CapacitySchedulerConfiguration.MAXIMUM_AM_RESOURCE_SUFFIX |
static String |
CapacitySchedulerConfiguration.MAXIMUM_APPLICATION_MASTERS_RESOURCE_PERCENT |
static String |
CapacitySchedulerConfiguration.MAXIMUM_APPLICATIONS_SUFFIX |
static String |
CapacitySchedulerConfiguration.MAXIMUM_CAPACITY |
static float |
CapacitySchedulerConfiguration.MAXIMUM_CAPACITY_VALUE |
static String |
CapacitySchedulerConfiguration.MAXIMUM_SYSTEM_APPLICATIONS |
static float |
CapacitySchedulerConfiguration.MINIMUM_CAPACITY_VALUE |
static String |
CapacitySchedulerConfiguration.NODE_LOCALITY_DELAY |
static String |
CapacitySchedulerConfiguration.NONE_ACL |
static String |
CapacitySchedulerConfiguration.PREFIX |
static String |
CapacitySchedulerConfiguration.QUEUES |
static String |
CapacitySchedulerConfiguration.RESOURCE_CALCULATOR_CLASS
ResourceComparator for scheduling.
|
static String |
CapacitySchedulerConfiguration.ROOT |
static String |
CapacityScheduler.ROOT_QUEUE |
static String |
CapacitySchedulerConfiguration.STATE |
static float |
CapacitySchedulerConfiguration.UNDEFINED |
static String |
CapacitySchedulerConfiguration.USER_LIMIT |
static String |
CapacitySchedulerConfiguration.USER_LIMIT_FACTOR |
Modifier and Type | Method and Description |
---|---|
float |
LeafQueue.getMaxAMResourcePerQueuePercent()
Used only by tests.
|
Resource |
LeafQueue.getMaximumAllocation()
Used only by tests.
|
Resource |
LeafQueue.getMinimumAllocation()
Used only by tests.
|
float |
LeafQueue.getMinimumAllocationFactor()
Used only by tests.
|
int |
LeafQueue.getNodeLocalityDelay() |
int |
LeafQueue.getNumActiveApplications(String user) |
int |
LeafQueue.getNumApplications(String user) |
int |
LeafQueue.getNumPendingApplications(String user) |
int |
LeafQueue.getUserLimit() |
float |
LeafQueue.getUserLimitFactor() |
Modifier and Type | Class and Description |
---|---|
class |
FiCaSchedulerApp
Represents an Application from the viewpoint of the scheduler.
|
Modifier and Type | Method and Description |
---|---|
Resource |
FiCaSchedulerApp.getCurrentReservation()
Get total current reservations.
|
Modifier and Type | Class and Description |
---|---|
class |
AllocationConfigurationException
Thrown when the allocation file for
QueueManager is malformed. |
class |
AppSchedulable |
class |
FairSchedulerConfiguration |
class |
FifoAppComparator
Order
AppSchedulable objects by priority and then by submit time, as
in the default scheduler in Hadoop. |
class |
FSLeafQueue |
class |
FSParentQueue |
class |
FSQueue |
class |
FSSchedulerApp |
class |
FSSchedulerNode |
class |
NewAppWeightBooster
A
WeightAdjuster implementation that gives a weight boost to new jobs
for a certain amount of time -- by default, a 3x weight boost for 60 seconds. |
class |
QueueManager
Maintains a list of queues as well as scheduling parameters for each queue,
such as guaranteed share allocations, from the fair scheduler config file.
|
class |
Schedulable
A Schedulable represents an entity that can launch tasks, such as a job
or a queue.
|
interface |
WeightAdjuster
A pluggable object for altering the weights of apps in the fair scheduler,
which is used for example by
NewAppWeightBooster to give higher
weight to new jobs so that short jobs finish faster. |
Modifier and Type | Class and Description |
---|---|
class |
DominantResourceFairnessPolicy
Makes scheduling decisions by trying to equalize dominant resource usage.
|
class |
FairSharePolicy
Makes scheduling decisions by trying to equalize shares of memory.
|
class |
FifoPolicy |
Modifier and Type | Class and Description |
---|---|
class |
DelegationTokenRenewer
Service to renew application delegation tokens.
|
class |
RMDelegationTokenSecretManager
A ResourceManager specific delegation token secret manager.
|
Modifier and Type | Method and Description |
---|---|
void |
NMTokenSecretManagerInRM.activateNextMasterKey()
Activate the new master-key
|
void |
RMContainerTokenSecretManager.activateNextMasterKey()
Activate the new master-key
|
Set<DelegationKey> |
RMDelegationTokenSecretManager.getAllMasterKeys() |
Map<RMDelegationTokenIdentifier,Long> |
RMDelegationTokenSecretManager.getAllTokens() |
int |
RMDelegationTokenSecretManager.getLatestDTSequenceNumber() |
SecretKey |
AMRMTokenSecretManager.getMasterKey() |
MasterKey |
NMTokenSecretManagerInRM.getNextKey() |
MasterKey |
RMContainerTokenSecretManager.getNextKey() |
boolean |
NMTokenSecretManagerInRM.isApplicationAttemptNMTokenPresent(ApplicationAttemptId appAttemptId,
NodeId nodeId) |
boolean |
NMTokenSecretManagerInRM.isApplicationAttemptRegistered(ApplicationAttemptId appAttemptId) |
void |
NMTokenSecretManagerInRM.rollMasterKey()
Creates a new master-key and sets it as the primary.
|
void |
RMContainerTokenSecretManager.rollMasterKey()
Creates a new master-key and sets it as the primary.
|
void |
AMRMTokenSecretManager.setMasterKey(SecretKey masterKey) |
Modifier and Type | Class and Description |
---|---|
class |
RMPolicyProvider
PolicyProvider for YARN ResourceManager protocols. |
Modifier and Type | Class and Description |
---|---|
class |
ApplicationACLsManager |
Modifier and Type | Method and Description |
---|---|
MasterKey |
BaseContainerTokenSecretManager.getCurrentKey() |
MasterKey |
BaseNMTokenSecretManager.getCurrentKey() |
Modifier and Type | Method and Description |
---|---|
static Token |
BuilderUtils.newContainerToken(NodeId nodeId,
byte[] password,
ContainerTokenIdentifier tokenIdentifier) |
Modifier and Type | Class and Description |
---|---|
class |
Graph |
class |
VisualizeStateMachine |
Modifier and Type | Class and Description |
---|---|
class |
Apps
Yarn application related utilities
|
class |
ConverterUtils
This class contains a set of utilities which help converting data structures
from/to 'serializableFormat' to/from hadoop/nativejava data structures.
|
class |
LinuxResourceCalculatorPlugin
Plugin to calculate resource information on Linux systems.
|
class |
ProcfsBasedProcessTree
A Proc file-system based ProcessTree.
|
class |
ResourceCalculatorProcessTree
Interface class to obtain process resource usage
|
class |
StringHelper
Common string manipulation helpers
|
class |
Times |
class |
WindowsBasedProcessTree |
class |
WindowsResourceCalculatorPlugin |
class |
YarnVersionInfo
This class finds the package info for Yarn.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultResourceCalculator |
class |
DominantResourceCalculator
A
ResourceCalculator which uses the concept of
dominant resource to compare multi-dimensional resources. |
class |
ResourceCalculator
A set of
Resource comparison and manipulation interfaces. |
class |
Resources |
Modifier and Type | Method and Description |
---|---|
void |
DefaultWrapperServlet.doGet(javax.servlet.http.HttpServletRequest req,
javax.servlet.http.HttpServletResponse resp) |
Modifier and Type | Class and Description |
---|---|
class |
HamletImpl
A simple unbuffered generic hamlet implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
WebAppUtils |
Copyright © 2013 Apache Software Foundation. All rights reserved.