Modifier and Type | Method and Description |
---|---|
Path |
Configuration.getLocalPath(String dirsProp,
String path)
Get a local file under a directory named by dirsProp with
the given path.
|
Modifier and Type | Method and Description |
---|---|
void |
Configuration.addResource(Path file)
Add a configuration resource.
|
Modifier and Type | Method and Description |
---|---|
static BigDecimal |
QuasiMonteCarlo.estimatePi(int numMaps,
long numPoints,
Path tmpDir,
Configuration conf)
Run a map/reduce job for estimating Pi.
|
Modifier and Type | Method and Description |
---|---|
static void |
TeraInputFormat.writePartitionFile(JobContext job,
Path partFile)
Use the input splits to take samples of the input and generate sample
keys.
|
Modifier and Type | Field and Description |
---|---|
protected Path |
FSInputChecker.file
The file name from which data is read from
|
protected Path |
TrashPolicy.trash |
Modifier and Type | Method and Description |
---|---|
Path |
FileSystem.createSnapshot(Path path)
Create a snapshot with a default name.
|
Path |
FilterFileSystem.createSnapshot(Path path,
String snapshotName) |
Path |
FileSystem.createSnapshot(Path path,
String snapshotName)
Create a snapshot
|
protected Path |
FileSystem.fixRelativePart(Path p)
|
Path |
ChecksumFileSystem.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
Path |
ChecksumFs.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
Path |
TrashPolicyDefault.getCurrentTrashDir() |
Path |
FsShell.getCurrentTrashDir()
Returns the Trash object associated with this shell.
|
abstract Path |
TrashPolicy.getCurrentTrashDir()
Get the current working directory of the Trash Policy
|
Path |
FilterFileSystem.getHomeDirectory() |
Path |
AbstractFileSystem.getHomeDirectory()
Return the current user's home directory in this file system.
|
Path |
FilterFs.getHomeDirectory() |
Path |
FileContext.getHomeDirectory()
Return the current user's home directory in this file system.
|
Path |
RawLocalFileSystem.getHomeDirectory() |
Path |
FileSystem.getHomeDirectory()
Return the current user's home directory in this filesystem.
|
Path |
HarFileSystem.getHomeDirectory()
return the top level archive path.
|
Path |
DelegateToFileSystem.getHomeDirectory() |
protected Path |
FilterFileSystem.getInitialWorkingDirectory() |
Path |
AbstractFileSystem.getInitialWorkingDirectory()
Some file systems like LocalFileSystem have an initial workingDir
that is used as the starting workingDir.
|
Path |
FilterFs.getInitialWorkingDirectory() |
protected Path |
RawLocalFileSystem.getInitialWorkingDirectory() |
protected Path |
FileSystem.getInitialWorkingDirectory()
Note: with the new FilesContext class, getWorkingDirectory()
will be removed.
|
Path |
DelegateToFileSystem.getInitialWorkingDirectory() |
Path |
Hdfs.getLinkTarget(Path p) |
Path |
FilterFileSystem.getLinkTarget(Path f) |
Path |
AbstractFileSystem.getLinkTarget(Path f)
Partially resolves the path.
|
Path |
FilterFs.getLinkTarget(Path f) |
Path |
FileContext.getLinkTarget(Path f)
Returns the target of the given symbolic link as it was specified
when the link was created.
|
Path |
FileSystem.getLinkTarget(Path f)
|
Path |
DelegateToFileSystem.getLinkTarget(Path f) |
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
Configuration conf)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
long size,
Configuration conf)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
long size,
Configuration conf,
boolean checkWrite)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathToRead(String pathStr,
Configuration conf)
Get a path from the local FS for reading.
|
Path |
Path.getParent()
Returns the parent of a path or null if at root.
|
Path |
FileStatus.getPath() |
Path |
PathIOException.getPath() |
static Path |
Path.getPathWithoutSchemeAndAuthority(Path path) |
Path |
FileStatus.getSymlink() |
Path |
PathIOException.getTargetPath() |
Path |
FilterFileSystem.getWorkingDirectory()
Get the current working directory for the given file system
|
Path |
FileContext.getWorkingDirectory()
Gets the working directory for wd-relative names (such a "foo/bar").
|
Path |
RawLocalFileSystem.getWorkingDirectory() |
abstract Path |
FileSystem.getWorkingDirectory()
Get the current working directory for the given file system
|
Path |
HarFileSystem.getWorkingDirectory()
return the top level archive.
|
Path |
Path.makeQualified(FileSystem fs)
Deprecated.
|
Path |
FilterFileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
AbstractFileSystem.makeQualified(Path path)
Make the path fully qualified to this file system
|
Path |
FilterFs.makeQualified(Path path) |
Path |
FileContext.makeQualified(Path path)
Make the path fully qualified if it is isn't.
|
Path |
FileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
HarFileSystem.makeQualified(Path path) |
Path |
Path.makeQualified(URI defaultUri,
Path workingDir)
Returns a qualified path object.
|
static Path |
Path.mergePaths(Path path1,
Path path2)
Merge 2 paths such that the second path is appended relative to the first.
|
static Path |
FSLinkResolver.qualifySymlinkTarget(URI pathURI,
Path pathWithLink,
Path target)
Return a fully-qualified version of the given symlink target if it
has no scheme and authority.
|
protected Path |
FileContext.resolve(Path f)
Resolves all symbolic links in the specified path.
|
protected Path |
FileContext.resolveIntermediate(Path f)
Resolves all symbolic links in the specified path leading up
to, but not including the final path component.
|
protected Path |
FilterFileSystem.resolveLink(Path f) |
protected Path |
FileSystem.resolveLink(Path f)
|
Path |
FilterFileSystem.resolvePath(Path p) |
Path |
AbstractFileSystem.resolvePath(Path p)
Return the fully-qualified path of path f resolving the path
through any internal symlinks or mount point
|
Path |
FilterFs.resolvePath(Path p) |
Path |
FileContext.resolvePath(Path f)
Resolve the path following any symlinks or mount points
|
Path |
FileSystem.resolvePath(Path p)
Return the fully-qualified path of path f resolving the path
through any symlinks or mount point
|
Path |
FilterFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
RawLocalFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
Path |
FileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
HarFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
Path |
ChecksumFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats)
convert an array of FileStatus to an array of Path
|
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats,
Path path)
convert an array of FileStatus to an array of Path.
|
Path |
Path.suffix(String suffix)
Adds a suffix to the final name in the path.
|
Modifier and Type | Method and Description |
---|---|
Iterable<Path> |
LocalDirAllocator.getAllLocalPathsToRead(String pathStr,
Configuration conf)
Get all of the paths that currently exist in the working directories.
|
RemoteIterator<Path> |
Hdfs.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FilterFileSystem.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
AbstractFileSystem.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FilterFs.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FileContext.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FileSystem.listCorruptFileBlocks(Path path) |
Modifier and Type | Method and Description |
---|---|
boolean |
GlobFilter.accept(Path path) |
boolean |
PathFilter.accept(Path path)
Tests whether or not the specified abstract pathname should be
included in a pathname list.
|
FSDataOutputStream |
FileSystem.append(Path f)
Append to an existing file (optional operation).
|
FSDataOutputStream |
HarFileSystem.append(Path f) |
FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FilterFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
FSDataOutputStream |
RawLocalFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
abstract FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
HarFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
FSDataOutputStream |
ChecksumFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
boolean |
FileSystem.cancelDeleteOnExit(Path f)
Cancel the deletion of the path when the FileSystem is closed
|
protected void |
FilterFileSystem.checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
AbstractFileSystem.checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
FilterFs.checkPath(Path path) |
protected void |
FileSystem.checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
FilterFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Called when we're all done writing to the target.
|
void |
RawLocalFileSystem.completeLocalOutput(Path fsWorkingFile,
Path tmpLocalFile) |
void |
FileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Called when we're all done writing to the target.
|
void |
HarFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
void |
ChecksumFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
void |
FilterFileSystem.concat(Path f,
Path[] psrcs) |
void |
FilterFileSystem.concat(Path f,
Path[] psrcs) |
void |
FileSystem.concat(Path trg,
Path[] psrcs)
Concat existing files together.
|
void |
FileSystem.concat(Path trg,
Path[] psrcs)
Concat existing files together.
|
static boolean |
FileUtil.copy(File src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
Configuration conf)
Copy local files to a FileSystem.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path[] srcs,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf) |
static boolean |
FileUtil.copy(FileSystem srcFS,
Path[] srcs,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf) |
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
File dst,
boolean deleteSource,
Configuration conf)
Copy FileSystem files to local files.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf)
Copy files between FileSystems.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
Configuration conf)
Copy files between FileSystems.
|
boolean |
FileContext.Util.copy(Path src,
Path dst)
Copy file from src to dest.
|
boolean |
FileContext.Util.copy(Path src,
Path dst,
boolean deleteSource,
boolean overwrite)
Copy from src to dst, optionally deleting src and overwriting dst.
|
void |
FilterFileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FilterFileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FilterFileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
The src file is on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
The src file is on the local disk.
|
void |
FilterFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is on the local disk.
|
void |
HarFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
not implemented.
|
void |
ChecksumFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
LocalFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
FileSystem.copyFromLocalFile(Path src,
Path dst)
The src file is on the local disk.
|
static boolean |
FileUtil.copyMerge(FileSystem srcFS,
Path srcDir,
FileSystem dstFS,
Path dstFile,
boolean deleteSource,
Configuration conf,
String addString)
Copy all files in a directory to one output file (merge).
|
void |
FilterFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
FileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
HarFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
copies the file in the har filesystem to a local file.
|
void |
ChecksumFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
LocalFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
FileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst,
boolean useRawLocalFileSystem)
The src file is under FS, and the dst is on the local disk.
|
void |
FileSystem.copyToLocalFile(Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
ChecksumFileSystem.copyToLocalFile(Path src,
Path dst,
boolean copyCrc)
The src file is under FS, and the dst is on the local disk.
|
static FSDataOutputStream |
FileSystem.create(FileSystem fs,
Path file,
FsPermission permission)
create a file with the provided permission
The permission of the file is set to be the provided permission as in
setPermission, not permission&~umask
It is implemented using two RPCs.
|
FSDataOutputStream |
FileSystem.create(Path f)
Create an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite)
Create an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize)
Create an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize)
Create an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
AbstractFileSystem.create(Path f,
EnumSet<CreateFlag> createFlag,
Options.CreateOpts... opts)
The specification of this method matches that of
FileContext.create(Path, EnumSet, Options.CreateOpts...) except
that the Path f must be fully qualified and the permission is absolute
(i.e. |
FSDataOutputStream |
FileContext.create(Path f,
EnumSet<CreateFlag> createFlag,
Options.CreateOpts... opts)
Create or overwrite file on indicated path and returns an output stream for
writing into the file.
|
FSDataOutputStream |
FilterFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
abstract FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
HarFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
ChecksumFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt)
Create an FSDataOutputStream at the indicated Path with a custom
checksum option
|
FSDataOutputStream |
FileSystem.create(Path f,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication)
Create an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
HdfsDataOutputStream |
Hdfs.createInternal(Path f,
EnumSet<CreateFlag> createFlag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
abstract FSDataOutputStream |
AbstractFileSystem.createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent)
The specification of this method matches that of
AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...) except that the opts
have been declared explicitly. |
FSDataOutputStream |
FilterFs.createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
FSDataOutputStream |
DelegateToFileSystem.createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
FSDataOutputStream |
ChecksumFs.createInternal(Path f,
EnumSet<CreateFlag> createFlag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
static String |
FileUtil.createJarWithClassPath(String inputClassPath,
Path pwd,
Map<String,String> callerEnv)
Create a jar file at the given path, containing a manifest with a classpath
that references all specified entries.
|
boolean |
FileSystem.createNewFile(Path f)
Creates the given Path as a brand-new zero-length file.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Deprecated.
API only for 0.20-append
|
FSDataOutputStream |
RawLocalFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Deprecated.
API only for 0.20-append
|
FSDataOutputStream |
ChecksumFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FilterFileSystem.createNonRecursive(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Deprecated.
|
FSDataOutputStream |
RawLocalFileSystem.createNonRecursive(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Deprecated.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Deprecated.
API only for 0.20-append
|
Path |
FileSystem.createSnapshot(Path path)
Create a snapshot with a default name.
|
Path |
FilterFileSystem.createSnapshot(Path path,
String snapshotName) |
Path |
FileSystem.createSnapshot(Path path,
String snapshotName)
Create a snapshot
|
void |
Hdfs.createSymlink(Path target,
Path link,
boolean createParent) |
void |
FilterFileSystem.createSymlink(Path target,
Path link,
boolean createParent) |
void |
AbstractFileSystem.createSymlink(Path target,
Path link,
boolean createParent)
The specification of this method matches that of
FileContext.createSymlink(Path, Path, boolean) ; |
void |
FilterFs.createSymlink(Path target,
Path link,
boolean createParent) |
void |
FileContext.createSymlink(Path target,
Path link,
boolean createParent)
Creates a symbolic link to an existing file.
|
void |
FileSystem.createSymlink(Path target,
Path link,
boolean createParent)
|
void |
DelegateToFileSystem.createSymlink(Path target,
Path link,
boolean createParent) |
boolean |
FileSystem.delete(Path f)
Deprecated.
Use
FileSystem.delete(Path, boolean) instead. |
boolean |
Hdfs.delete(Path f,
boolean recursive) |
boolean |
FilterFileSystem.delete(Path f,
boolean recursive)
Delete a file
|
abstract boolean |
AbstractFileSystem.delete(Path f,
boolean recursive)
The specification of this method matches that of
FileContext.delete(Path, boolean) except that Path f must be for
this file system. |
boolean |
FilterFs.delete(Path f,
boolean recursive) |
boolean |
FileContext.delete(Path f,
boolean recursive)
Delete a file.
|
boolean |
RawLocalFileSystem.delete(Path p,
boolean recursive)
Delete the given path to a file or directory.
|
abstract boolean |
FileSystem.delete(Path f,
boolean recursive)
Delete a file.
|
boolean |
HarFileSystem.delete(Path f,
boolean recursive)
Not implemented.
|
boolean |
ChecksumFileSystem.delete(Path f,
boolean recursive)
Implement the delete(Path, boolean) in checksum
file system.
|
boolean |
DelegateToFileSystem.delete(Path f,
boolean recursive) |
boolean |
ChecksumFs.delete(Path f,
boolean recursive)
Implement the delete(Path, boolean) in checksum
file system.
|
boolean |
FileContext.deleteOnExit(Path f)
Mark a path to be deleted on JVM shutdown.
|
boolean |
FileSystem.deleteOnExit(Path f)
Mark a path to be deleted when FileSystem is closed.
|
void |
FilterFileSystem.deleteSnapshot(Path path,
String snapshotName) |
void |
FileSystem.deleteSnapshot(Path path,
String snapshotName)
Delete a snapshot of a directory
|
abstract T |
FileSystemLinkResolver.doCall(Path p)
FileSystem subclass-specific implementation of superclass method.
|
boolean |
FileContext.Util.exists(Path f)
Does the file exist?
Note: Avoid using this method if you already have FileStatus in hand.
|
boolean |
FileSystem.exists(Path f)
Check if exists.
|
protected Path |
FileSystem.fixRelativePart(Path p)
|
static void |
FileUtil.fullyDelete(FileSystem fs,
Path dir)
Deprecated.
|
long |
FileSystem.getBlockSize(Path f)
Deprecated.
Use getFileStatus() instead
|
Path |
ChecksumFileSystem.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
Path |
ChecksumFs.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
long |
ChecksumFileSystem.getChecksumFileLength(Path file,
long fileSize)
Return the length of the checksum file given the size of the
actual file.
|
long |
ChecksumFs.getChecksumFileLength(Path file,
long fileSize)
Return the length of the checksum file given the size of the
actual file.
|
ContentSummary |
FilterFileSystem.getContentSummary(Path f) |
ContentSummary |
FileContext.Util.getContentSummary(Path f)
Return the
ContentSummary of path f. |
ContentSummary |
FileSystem.getContentSummary(Path f)
Return the
ContentSummary of a given Path . |
long |
FilterFileSystem.getDefaultBlockSize(Path f) |
long |
FileSystem.getDefaultBlockSize(Path f)
Return the number of bytes that large input files should be optimally
be split into to minimize i/o time.
|
short |
FilterFileSystem.getDefaultReplication(Path f) |
short |
FileSystem.getDefaultReplication(Path path)
Get the default replication for a path.
|
List<Token<?>> |
FileContext.getDelegationTokens(Path p,
String renewer)
Get delegation tokens for the file systems accessed for a given
path.
|
BlockLocation[] |
Hdfs.getFileBlockLocations(Path p,
long start,
long len) |
abstract BlockLocation[] |
AbstractFileSystem.getFileBlockLocations(Path f,
long start,
long len)
The specification of this method matches that of
FileContext.getFileBlockLocations(Path, long, long) except that
Path f must be for this file system. |
BlockLocation[] |
FilterFs.getFileBlockLocations(Path f,
long start,
long len) |
BlockLocation[] |
FileContext.getFileBlockLocations(Path f,
long start,
long len)
Return blockLocation of the given file for the given offset and len.
|
BlockLocation[] |
FileSystem.getFileBlockLocations(Path p,
long start,
long len)
Return an array containing hostnames, offset and size of
portions of the given file.
|
BlockLocation[] |
DelegateToFileSystem.getFileBlockLocations(Path f,
long start,
long len) |
FileChecksum |
Hdfs.getFileChecksum(Path f) |
FileChecksum |
FilterFileSystem.getFileChecksum(Path f) |
abstract FileChecksum |
AbstractFileSystem.getFileChecksum(Path f)
The specification of this method matches that of
FileContext.getFileChecksum(Path) except that Path f must be for
this file system. |
FileChecksum |
FilterFs.getFileChecksum(Path f) |
FileChecksum |
FileContext.getFileChecksum(Path f)
Get the checksum of a file.
|
FileChecksum |
FileSystem.getFileChecksum(Path f)
Get the checksum of a file.
|
FileChecksum |
HarFileSystem.getFileChecksum(Path f) |
FileChecksum |
DelegateToFileSystem.getFileChecksum(Path f) |
FileStatus |
Hdfs.getFileLinkStatus(Path f) |
FileStatus |
FilterFileSystem.getFileLinkStatus(Path f) |
FileStatus |
AbstractFileSystem.getFileLinkStatus(Path f)
The specification of this method matches that of
FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component. |
FileStatus |
FilterFs.getFileLinkStatus(Path f) |
FileStatus |
FileContext.getFileLinkStatus(Path f)
Return a file status object that represents the path.
|
FileStatus |
FileSystem.getFileLinkStatus(Path f)
|
FileStatus |
DelegateToFileSystem.getFileLinkStatus(Path f) |
FileStatus |
Hdfs.getFileStatus(Path f) |
FileStatus |
FilterFileSystem.getFileStatus(Path f)
Get file status.
|
abstract FileStatus |
AbstractFileSystem.getFileStatus(Path f)
The specification of this method matches that of
FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
FileStatus |
FilterFs.getFileStatus(Path f) |
FileStatus |
FileContext.getFileStatus(Path f)
Return a file status object that represents the path.
|
FileStatus |
RawLocalFileSystem.getFileStatus(Path f) |
abstract FileStatus |
FileSystem.getFileStatus(Path f)
Return a file status object that represents the path.
|
FileStatus |
HarFileSystem.getFileStatus(Path f)
return the filestatus of files in har archive.
|
FileStatus |
DelegateToFileSystem.getFileStatus(Path f) |
protected AbstractFileSystem |
FileContext.getFSofPath(Path absOrFqPath)
Get the file system of supplied path.
|
protected static FileSystem |
FileSystem.getFSofPath(Path absOrFqPath,
Configuration conf) |
FsStatus |
AbstractFileSystem.getFsStatus(Path f)
The specification of this method matches that of
FileContext.getFsStatus(Path) except that Path f must be for this
file system. |
FsStatus |
FilterFs.getFsStatus(Path f) |
FsStatus |
FileContext.getFsStatus(Path f)
Returns a status object describing the use and capacity of the
file system denoted by the Parh argument p.
|
static int |
HarFileSystem.getHarHash(Path p)
the hash of the path p inside the filesystem
|
static TrashPolicy |
TrashPolicy.getInstance(Configuration conf,
FileSystem fs,
Path home)
Get an instance of the configured TrashPolicy based on the value
of the configuration parameter fs.trash.classname.
|
long |
FileSystem.getLength(Path f)
Deprecated.
Use getFileStatus() instead
|
Path |
Hdfs.getLinkTarget(Path p) |
Path |
FilterFileSystem.getLinkTarget(Path f) |
Path |
AbstractFileSystem.getLinkTarget(Path f)
Partially resolves the path.
|
Path |
FilterFs.getLinkTarget(Path f) |
Path |
FileContext.getLinkTarget(Path f)
Returns the target of the given symbolic link as it was specified
when the link was created.
|
Path |
FileSystem.getLinkTarget(Path f)
|
Path |
DelegateToFileSystem.getLinkTarget(Path f) |
static Path |
Path.getPathWithoutSchemeAndAuthority(Path path) |
short |
FileSystem.getReplication(Path src)
Deprecated.
Use getFileStatus() instead
|
FsServerDefaults |
FilterFileSystem.getServerDefaults(Path f) |
FsServerDefaults |
FileSystem.getServerDefaults(Path p)
Return a set of server default configuration values
|
FsStatus |
FilterFileSystem.getStatus(Path p) |
FsStatus |
RawLocalFileSystem.getStatus(Path p) |
FsStatus |
FileSystem.getStatus(Path p)
Returns a status object describing the use and capacity of the
file system.
|
String |
AbstractFileSystem.getUriPath(Path p)
Get the path-part of a pathname.
|
String |
FilterFs.getUriPath(Path p) |
FileStatus[] |
FileContext.Util.globStatus(Path pathPattern)
Return all the files that match filePattern and are not checksum
files.
|
FileStatus[] |
FileSystem.globStatus(Path pathPattern)
Return all the files that match filePattern and are not checksum
files.
|
FileStatus[] |
FileContext.Util.globStatus(Path pathPattern,
PathFilter filter)
Return an array of FileStatus objects whose path names match pathPattern
and is accepted by the user-supplied path filter.
|
FileStatus[] |
FileSystem.globStatus(Path pathPattern,
PathFilter filter)
Return an array of FileStatus objects whose path names match pathPattern
and is accepted by the user-supplied path filter.
|
void |
TrashPolicyDefault.initialize(Configuration conf,
FileSystem fs,
Path home) |
abstract void |
TrashPolicy.initialize(Configuration conf,
FileSystem fs,
Path home)
Used to setup the trash policy.
|
static boolean |
ChecksumFileSystem.isChecksumFile(Path file)
Return true iff file is a checksum file name.
|
static boolean |
ChecksumFs.isChecksumFile(Path file)
Return true iff file is a checksum file name.
|
boolean |
FileSystem.isDirectory(Path f)
True iff the named path is a directory.
|
boolean |
FileSystem.isFile(Path f)
True iff the named path is a regular file.
|
RemoteIterator<Path> |
Hdfs.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FilterFileSystem.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
AbstractFileSystem.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FilterFs.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FileContext.listCorruptFileBlocks(Path path) |
RemoteIterator<Path> |
FileSystem.listCorruptFileBlocks(Path path) |
RemoteIterator<LocatedFileStatus> |
FileContext.Util.listFiles(Path f,
boolean recursive)
List the statuses and block locations of the files in the given path.
|
RemoteIterator<LocatedFileStatus> |
FileSystem.listFiles(Path f,
boolean recursive)
List the statuses and block locations of the files in the given path.
|
RemoteIterator<LocatedFileStatus> |
Hdfs.listLocatedStatus(Path p) |
RemoteIterator<LocatedFileStatus> |
FilterFileSystem.listLocatedStatus(Path f)
List files and its block locations in a directory.
|
RemoteIterator<LocatedFileStatus> |
AbstractFileSystem.listLocatedStatus(Path f)
The specification of this method matches that of
FileContext.listLocatedStatus(Path) except that Path f
must be for this file system. |
RemoteIterator<LocatedFileStatus> |
FileContext.listLocatedStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
RemoteIterator<LocatedFileStatus> |
FileSystem.listLocatedStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
RemoteIterator<LocatedFileStatus> |
ChecksumFileSystem.listLocatedStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
protected RemoteIterator<LocatedFileStatus> |
FileSystem.listLocatedStatus(Path f,
PathFilter filter)
Listing a directory
The returned results include its block location if it is a file
The results are filtered by the given path filter
|
FileStatus[] |
Hdfs.listStatus(Path f) |
FileStatus[] |
FilterFileSystem.listStatus(Path f)
List files in a directory.
|
abstract FileStatus[] |
AbstractFileSystem.listStatus(Path f)
The specification of this method matches that of
FileContext.Util.listStatus(Path) except that Path f must be
for this file system. |
FileStatus[] |
FilterFs.listStatus(Path f) |
RemoteIterator<FileStatus> |
FileContext.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
FileStatus[] |
FileContext.Util.listStatus(Path f)
List the statuses of the files/directories in the given path
if the path is a directory.
|
FileStatus[] |
RawLocalFileSystem.listStatus(Path f) |
abstract FileStatus[] |
FileSystem.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
FileStatus[] |
HarFileSystem.listStatus(Path f)
liststatus returns the children of a directory
after looking up the index files.
|
FileStatus[] |
ChecksumFileSystem.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
FileStatus[] |
DelegateToFileSystem.listStatus(Path f) |
FileStatus[] |
ChecksumFs.listStatus(Path f) |
FileStatus[] |
FileContext.Util.listStatus(Path[] files)
|
FileStatus[] |
FileSystem.listStatus(Path[] files)
Filter files/directories in the given list of paths using default
path filter.
|
FileStatus[] |
FileContext.Util.listStatus(Path[] files,
PathFilter filter)
Filter files/directories in the given list of paths using user-supplied
path filter.
|
FileStatus[] |
FileSystem.listStatus(Path[] files,
PathFilter filter)
Filter files/directories in the given list of paths using user-supplied
path filter.
|
FileStatus[] |
FileContext.Util.listStatus(Path f,
PathFilter filter)
Filter files/directories in the given path using the user-supplied path
filter.
|
FileStatus[] |
FileSystem.listStatus(Path f,
PathFilter filter)
Filter files/directories in the given path using the user-supplied path
filter.
|
RemoteIterator<FileStatus> |
Hdfs.listStatusIterator(Path f) |
RemoteIterator<FileStatus> |
AbstractFileSystem.listStatusIterator(Path f)
The specification of this method matches that of
FileContext.listStatus(Path) except that Path f must be for this
file system. |
Path |
FilterFileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
AbstractFileSystem.makeQualified(Path path)
Make the path fully qualified to this file system
|
Path |
FilterFs.makeQualified(Path path) |
Path |
FileContext.makeQualified(Path path)
Make the path fully qualified if it is isn't.
|
Path |
FileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
HarFileSystem.makeQualified(Path path) |
Path |
Path.makeQualified(URI defaultUri,
Path workingDir)
Returns a qualified path object.
|
static Path |
Path.mergePaths(Path path1,
Path path2)
Merge 2 paths such that the second path is appended relative to the first.
|
void |
Hdfs.mkdir(Path dir,
FsPermission permission,
boolean createParent) |
abstract void |
AbstractFileSystem.mkdir(Path dir,
FsPermission permission,
boolean createParent)
The specification of this method matches that of
FileContext.mkdir(Path, FsPermission, boolean) except that the Path
f must be fully qualified and the permission is absolute (i.e. |
void |
FilterFs.mkdir(Path dir,
FsPermission permission,
boolean createParent) |
void |
FileContext.mkdir(Path dir,
FsPermission permission,
boolean createParent)
Make(create) a directory and all the non-existent parents.
|
void |
DelegateToFileSystem.mkdir(Path dir,
FsPermission permission,
boolean createParent) |
static boolean |
FileSystem.mkdirs(FileSystem fs,
Path dir,
FsPermission permission)
create a directory with the provided permission
The permission of the directory is set to be the provided permission as in
setPermission, not permission&~umask
|
boolean |
RawLocalFileSystem.mkdirs(Path f)
Creates the specified directory hierarchy.
|
boolean |
FileSystem.mkdirs(Path f)
Call
FileSystem.mkdirs(Path, FsPermission) with default permission. |
boolean |
ChecksumFileSystem.mkdirs(Path f) |
boolean |
FilterFileSystem.mkdirs(Path f,
FsPermission permission) |
boolean |
RawLocalFileSystem.mkdirs(Path f,
FsPermission permission) |
abstract boolean |
FileSystem.mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
boolean |
HarFileSystem.mkdirs(Path f,
FsPermission permission)
not implemented.
|
void |
FileSystem.moveFromLocalFile(Path[] srcs,
Path dst)
The src files is on the local disk.
|
void |
FileSystem.moveFromLocalFile(Path[] srcs,
Path dst)
The src files is on the local disk.
|
void |
RawLocalFileSystem.moveFromLocalFile(Path src,
Path dst) |
void |
FileSystem.moveFromLocalFile(Path src,
Path dst)
The src file is on the local disk.
|
static boolean |
Trash.moveToAppropriateTrash(FileSystem fs,
Path p,
Configuration conf)
In case of the symlinks or mount points, one has to move the appropriate
trashbin in the actual volume of the path p being deleted.
|
void |
FileSystem.moveToLocalFile(Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
boolean |
TrashPolicyDefault.moveToTrash(Path path) |
boolean |
Trash.moveToTrash(Path path)
Move a file or directory to the current trash directory.
|
abstract boolean |
TrashPolicy.moveToTrash(Path path)
Move a file or directory to the current trash directory.
|
abstract T |
FSLinkResolver.next(AbstractFileSystem fs,
Path p)
Generic helper function overridden on instantiation to perform a
specific operation on the given file system using the given path
which may result in an UnresolvedLinkException.
|
abstract T |
FileSystemLinkResolver.next(FileSystem fs,
Path p)
Calls the abstract FileSystem call equivalent to the specialized subclass
implementation in
FileSystemLinkResolver.doCall(Path) . |
FSDataInputStream |
AbstractFileSystem.open(Path f)
The specification of this method matches that of
FileContext.open(Path) except that Path f must be for this
file system. |
FSDataInputStream |
FilterFs.open(Path f) |
FSDataInputStream |
FileContext.open(Path f)
Opens an FSDataInputStream at the indicated Path using
default buffersize.
|
FSDataInputStream |
FileSystem.open(Path f)
Opens an FSDataInputStream at the indicated Path.
|
HdfsDataInputStream |
Hdfs.open(Path f,
int bufferSize) |
FSDataInputStream |
FilterFileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
abstract FSDataInputStream |
AbstractFileSystem.open(Path f,
int bufferSize)
The specification of this method matches that of
FileContext.open(Path, int) except that Path f must be for this
file system. |
FSDataInputStream |
FilterFs.open(Path f,
int bufferSize) |
FSDataInputStream |
FileContext.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
RawLocalFileSystem.open(Path f,
int bufferSize) |
abstract FSDataInputStream |
FileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
HarFileSystem.open(Path f,
int bufferSize)
Returns a har input stream which fakes end of
file.
|
FSDataInputStream |
ChecksumFileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
DelegateToFileSystem.open(Path f,
int bufferSize) |
FSDataInputStream |
ChecksumFs.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
File |
RawLocalFileSystem.pathToFile(Path path)
Convert a path to a File.
|
File |
LocalFileSystem.pathToFile(Path path)
Convert a path to a File.
|
protected FSDataOutputStream |
FilterFileSystem.primitiveCreate(Path f,
FsPermission absolutePermission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt) |
protected FSDataOutputStream |
FileSystem.primitiveCreate(Path f,
FsPermission absolutePermission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt)
Deprecated.
|
protected boolean |
FilterFileSystem.primitiveMkdir(Path f,
FsPermission abdolutePermission) |
protected boolean |
RawLocalFileSystem.primitiveMkdir(Path f,
FsPermission absolutePermission) |
protected boolean |
FileSystem.primitiveMkdir(Path f,
FsPermission absolutePermission)
Deprecated.
|
protected void |
FileSystem.primitiveMkdir(Path f,
FsPermission absolutePermission,
boolean createParent)
Deprecated.
|
static Path |
FSLinkResolver.qualifySymlinkTarget(URI pathURI,
Path pathWithLink,
Path target)
Return a fully-qualified version of the given symlink target if it
has no scheme and authority.
|
boolean |
FilterFileSystem.rename(Path src,
Path dst)
Renames Path src to Path dst.
|
boolean |
RawLocalFileSystem.rename(Path src,
Path dst) |
abstract boolean |
FileSystem.rename(Path src,
Path dst)
Renames Path src to Path dst.
|
boolean |
HarFileSystem.rename(Path src,
Path dst) |
boolean |
ChecksumFileSystem.rename(Path src,
Path dst)
Rename files/dirs
|
void |
AbstractFileSystem.rename(Path src,
Path dst,
Options.Rename... options)
The specification of this method matches that of
FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
void |
FileContext.rename(Path src,
Path dst,
Options.Rename... options)
Renames Path src to Path dst
|
protected void |
FileSystem.rename(Path src,
Path dst,
Options.Rename... options)
Deprecated.
|
void |
Hdfs.renameInternal(Path src,
Path dst) |
abstract void |
AbstractFileSystem.renameInternal(Path src,
Path dst)
The specification of this method matches that of
FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system and NO OVERWRITE is performed. |
void |
FilterFs.renameInternal(Path src,
Path dst) |
void |
DelegateToFileSystem.renameInternal(Path src,
Path dst) |
void |
ChecksumFs.renameInternal(Path src,
Path dst)
Rename files/dirs.
|
void |
Hdfs.renameInternal(Path src,
Path dst,
boolean overwrite) |
void |
AbstractFileSystem.renameInternal(Path src,
Path dst,
boolean overwrite)
The specification of this method matches that of
FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
void |
FilterFs.renameInternal(Path src,
Path dst,
boolean overwrite) |
void |
FilterFileSystem.renameSnapshot(Path path,
String snapshotOldName,
String snapshotNewName) |
void |
FileSystem.renameSnapshot(Path path,
String snapshotOldName,
String snapshotNewName)
Rename a snapshot
|
boolean |
ChecksumFileSystem.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Report a checksum error to the file system.
|
boolean |
LocalFileSystem.reportChecksumFailure(Path p,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Moves files to a bad file directory on the same device, so that their
storage will not be reused.
|
boolean |
ChecksumFs.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Report a checksum error to the file system.
|
T |
FSLinkResolver.resolve(FileContext fc,
Path path)
Performs the operation specified by the next function, calling it
repeatedly until all symlinks in the given path are resolved.
|
T |
FileSystemLinkResolver.resolve(FileSystem filesys,
Path path)
Attempt calling overridden
FileSystemLinkResolver.doCall(Path) method with
specified FileSystem and Path . |
protected Path |
FileContext.resolve(Path f)
Resolves all symbolic links in the specified path.
|
protected Path |
FileContext.resolveIntermediate(Path f)
Resolves all symbolic links in the specified path leading up
to, but not including the final path component.
|
protected Path |
FilterFileSystem.resolveLink(Path f) |
protected Path |
FileSystem.resolveLink(Path f)
|
Path |
FilterFileSystem.resolvePath(Path p) |
Path |
AbstractFileSystem.resolvePath(Path p)
Return the fully-qualified path of path f resolving the path
through any internal symlinks or mount point
|
Path |
FilterFs.resolvePath(Path p) |
Path |
FileContext.resolvePath(Path f)
Resolve the path following any symlinks or mount points
|
Path |
FileSystem.resolvePath(Path p)
Return the fully-qualified path of path f resolving the path
through any symlinks or mount point
|
void |
Hdfs.setOwner(Path f,
String username,
String groupname) |
void |
FilterFileSystem.setOwner(Path p,
String username,
String groupname) |
abstract void |
AbstractFileSystem.setOwner(Path f,
String username,
String groupname)
The specification of this method matches that of
FileContext.setOwner(Path, String, String) except that Path f must
be for this file system. |
void |
FilterFs.setOwner(Path f,
String username,
String groupname) |
void |
FileContext.setOwner(Path f,
String username,
String groupname)
Set owner of a path (i.e.
|
void |
RawLocalFileSystem.setOwner(Path p,
String username,
String groupname)
Use the command chown to set owner.
|
void |
FileSystem.setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e.
|
void |
HarFileSystem.setOwner(Path p,
String username,
String groupname)
not implemented.
|
void |
DelegateToFileSystem.setOwner(Path f,
String username,
String groupname) |
void |
FileStatus.setPath(Path p) |
void |
Hdfs.setPermission(Path f,
FsPermission permission) |
void |
FilterFileSystem.setPermission(Path p,
FsPermission permission) |
abstract void |
AbstractFileSystem.setPermission(Path f,
FsPermission permission)
The specification of this method matches that of
FileContext.setPermission(Path, FsPermission) except that Path f
must be for this file system. |
void |
FilterFs.setPermission(Path f,
FsPermission permission) |
void |
FileContext.setPermission(Path f,
FsPermission permission)
Set permission of a path.
|
void |
RawLocalFileSystem.setPermission(Path p,
FsPermission permission)
Use the command chmod to set permission.
|
void |
FileSystem.setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
void |
HarFileSystem.setPermission(Path p,
FsPermission permission)
Not implemented.
|
void |
DelegateToFileSystem.setPermission(Path f,
FsPermission permission) |
boolean |
Hdfs.setReplication(Path f,
short replication) |
boolean |
FilterFileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
abstract boolean |
AbstractFileSystem.setReplication(Path f,
short replication)
The specification of this method matches that of
FileContext.setReplication(Path, short) except that Path f must be
for this file system. |
boolean |
FilterFs.setReplication(Path f,
short replication) |
boolean |
FileContext.setReplication(Path f,
short replication)
Set replication for an existing file.
|
boolean |
FileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
boolean |
HarFileSystem.setReplication(Path src,
short replication)
Not implemented.
|
boolean |
ChecksumFileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
boolean |
DelegateToFileSystem.setReplication(Path f,
short replication) |
boolean |
ChecksumFs.setReplication(Path src,
short replication)
Set replication for an existing file.
|
void |
FileStatus.setSymlink(Path p) |
void |
Hdfs.setTimes(Path f,
long mtime,
long atime) |
void |
FilterFileSystem.setTimes(Path p,
long mtime,
long atime) |
abstract void |
AbstractFileSystem.setTimes(Path f,
long mtime,
long atime)
The specification of this method matches that of
FileContext.setTimes(Path, long, long) except that Path f must be
for this file system. |
void |
FilterFs.setTimes(Path f,
long mtime,
long atime) |
void |
FileContext.setTimes(Path f,
long mtime,
long atime)
Set access time of a file.
|
void |
RawLocalFileSystem.setTimes(Path p,
long mtime,
long atime)
Sets the
Path 's last modified time only to the given
valid time. |
void |
FileSystem.setTimes(Path p,
long mtime,
long atime)
Set access time of a file
|
void |
DelegateToFileSystem.setTimes(Path f,
long mtime,
long atime) |
void |
FileContext.setVerifyChecksum(boolean verifyChecksum,
Path f)
Set the verify checksum flag for the file system denoted by the path.
|
void |
FilterFileSystem.setWorkingDirectory(Path newDir)
Set the current working directory for the given file system.
|
void |
FileContext.setWorkingDirectory(Path newWDir)
Set the working directory for wd-relative names (such a "foo/bar").
|
void |
RawLocalFileSystem.setWorkingDirectory(Path newDir)
Set the working directory to the given directory.
|
abstract void |
FileSystem.setWorkingDirectory(Path new_dir)
Set the current working directory for the given file system.
|
void |
HarFileSystem.setWorkingDirectory(Path newDir) |
Path |
FilterFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
RawLocalFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
Path |
FileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
HarFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
Path |
ChecksumFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats,
Path path)
convert an array of FileStatus to an array of Path.
|
Constructor and Description |
---|
AvroFSInput(FileContext fc,
Path p)
Construct given a
FileContext and a Path . |
FileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
long access_time,
FsPermission permission,
String owner,
String group,
Path path)
Constructor for file systems on which symbolic links are not supported
|
FileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
long access_time,
FsPermission permission,
String owner,
String group,
Path symlink,
Path path) |
FileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
Path path) |
FSInputChecker(Path file,
int numOfRetries)
Constructor
|
FSInputChecker(Path file,
int numOfRetries,
boolean verifyChecksum,
Checksum sum,
int chunkSize,
int checksumSize)
Constructor
|
LocatedFileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
long access_time,
FsPermission permission,
String owner,
String group,
Path symlink,
Path path,
BlockLocation[] locations)
Constructor
|
Path(Path parent,
Path child)
Resolve a child path against a parent path.
|
Path(Path parent,
String child)
Resolve a child path against a parent path.
|
Path(String parent,
Path child)
Resolve a child path against a parent path.
|
Modifier and Type | Method and Description |
---|---|
Path |
FTPFileSystem.getHomeDirectory() |
Path |
FTPFileSystem.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
FTPFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
FTPFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
A stream obtained via this call must be closed before using other APIs of
this class or else the invocation will block.
|
boolean |
FTPFileSystem.delete(Path file,
boolean recursive) |
FileStatus |
FTPFileSystem.getFileStatus(Path file) |
FileStatus[] |
FTPFileSystem.listStatus(Path file) |
boolean |
FTPFileSystem.mkdirs(Path file,
FsPermission permission) |
FSDataInputStream |
FTPFileSystem.open(Path file,
int bufferSize) |
boolean |
FTPFileSystem.rename(Path src,
Path dst) |
void |
FTPFileSystem.setWorkingDirectory(Path newDir) |
Modifier and Type | Method and Description |
---|---|
Path |
HttpFSFileSystem.getHomeDirectory()
Return the current user's home directory in this filesystem.
|
Path |
HttpFSFileSystem.getWorkingDirectory()
Get the current working directory for the given file system
|
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
HttpFSFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
void |
HttpFSFileSystem.concat(Path f,
Path[] psrcs)
Concat existing files together.
|
void |
HttpFSFileSystem.concat(Path f,
Path[] psrcs)
Concat existing files together.
|
FSDataOutputStream |
HttpFSFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
boolean |
HttpFSFileSystem.delete(Path f)
Deprecated.
Use delete(Path, boolean) instead
|
boolean |
HttpFSFileSystem.delete(Path f,
boolean recursive)
Delete a file.
|
ContentSummary |
HttpFSFileSystem.getContentSummary(Path f) |
FileChecksum |
HttpFSFileSystem.getFileChecksum(Path f) |
FileStatus |
HttpFSFileSystem.getFileStatus(Path f)
Return a file status object that represents the path.
|
FileStatus[] |
HttpFSFileSystem.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
boolean |
HttpFSFileSystem.mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
FSDataInputStream |
HttpFSFileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
boolean |
HttpFSFileSystem.rename(Path src,
Path dst)
Renames Path src to Path dst.
|
void |
HttpFSFileSystem.setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e.
|
void |
HttpFSFileSystem.setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
boolean |
HttpFSFileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
void |
HttpFSFileSystem.setTimes(Path p,
long mtime,
long atime)
Set access time of a file
|
void |
HttpFSFileSystem.setWorkingDirectory(Path newDir)
Set the current working directory for the given file system.
|
Modifier and Type | Method and Description |
---|---|
boolean |
FSOperations.FSListStatus.accept(Path path) |
Modifier and Type | Method and Description |
---|---|
Path |
RawLocalFs.getLinkTarget(Path f) |
Modifier and Type | Method and Description |
---|---|
void |
RawLocalFs.createSymlink(Path target,
Path link,
boolean createParent) |
FileStatus |
RawLocalFs.getFileLinkStatus(Path f)
Return a FileStatus representing the given path.
|
Path |
RawLocalFs.getLinkTarget(Path f) |
Modifier and Type | Method and Description |
---|---|
Path |
S3FileSystem.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
Set<Path> |
FileSystemStore.listDeepSubPaths(Path path) |
Set<Path> |
FileSystemStore.listSubPaths(Path path) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
S3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
S3FileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
S3FileSystem.delete(Path path,
boolean recursive) |
void |
FileSystemStore.deleteINode(Path path) |
FileStatus |
S3FileSystem.getFileStatus(Path f)
FileStatus for S3 file systems.
|
boolean |
FileSystemStore.inodeExists(Path path) |
boolean |
S3FileSystem.isFile(Path path) |
Set<Path> |
FileSystemStore.listDeepSubPaths(Path path) |
FileStatus[] |
S3FileSystem.listStatus(Path f) |
Set<Path> |
FileSystemStore.listSubPaths(Path path) |
boolean |
S3FileSystem.mkdirs(Path path,
FsPermission permission) |
FSDataInputStream |
S3FileSystem.open(Path path,
int bufferSize) |
boolean |
S3FileSystem.rename(Path src,
Path dst) |
INode |
FileSystemStore.retrieveINode(Path path) |
void |
S3FileSystem.setWorkingDirectory(Path dir) |
void |
FileSystemStore.storeINode(Path path,
INode inode) |
Modifier and Type | Method and Description |
---|---|
Path |
NativeS3FileSystem.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
NativeS3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
NativeS3FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
NativeS3FileSystem.delete(Path f,
boolean recurse) |
FileStatus |
NativeS3FileSystem.getFileStatus(Path f) |
FileStatus[] |
NativeS3FileSystem.listStatus(Path f)
If
f is a file, this method will make a single call to S3. |
boolean |
NativeS3FileSystem.mkdirs(Path f,
FsPermission permission) |
FSDataInputStream |
NativeS3FileSystem.open(Path f,
int bufferSize) |
boolean |
NativeS3FileSystem.rename(Path src,
Path dst) |
void |
NativeS3FileSystem.setWorkingDirectory(Path newDir)
Set the working directory to the given directory.
|
Modifier and Type | Field and Description |
---|---|
Path |
PathData.path |
Modifier and Type | Method and Description |
---|---|
protected void |
FsCommand.run(Path path) |
protected abstract void |
Command.run(Path path)
Execute the command on the input path
|
Modifier and Type | Method and Description |
---|---|
Path |
ViewFs.getHomeDirectory() |
Path |
ViewFileSystem.getHomeDirectory() |
Path |
ViewFs.getLinkTarget(Path f) |
Path |
ViewFileSystem.getTrashCanLocation(Path f) |
Path |
ViewFileSystem.getWorkingDirectory() |
Path |
ViewFs.resolvePath(Path f) |
Path |
ViewFileSystem.resolvePath(Path f) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
ViewFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
FSDataOutputStream |
ViewFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
ViewFs.createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
FSDataOutputStream |
ViewFileSystem.createNonRecursive(Path f,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
void |
ViewFs.createSymlink(Path target,
Path link,
boolean createParent) |
boolean |
ViewFileSystem.delete(Path f) |
boolean |
ViewFs.delete(Path f,
boolean recursive) |
boolean |
ViewFileSystem.delete(Path f,
boolean recursive) |
ContentSummary |
ViewFileSystem.getContentSummary(Path f) |
long |
ViewFileSystem.getDefaultBlockSize(Path f) |
short |
ViewFileSystem.getDefaultReplication(Path f) |
BlockLocation[] |
ViewFs.getFileBlockLocations(Path f,
long start,
long len) |
FileChecksum |
ViewFs.getFileChecksum(Path f) |
FileChecksum |
ViewFileSystem.getFileChecksum(Path f) |
FileStatus |
ViewFs.getFileLinkStatus(Path f) |
FileStatus |
ViewFs.getFileStatus(Path f) |
FileStatus |
ViewFileSystem.getFileStatus(Path f) |
Path |
ViewFs.getLinkTarget(Path f) |
FsServerDefaults |
ViewFileSystem.getServerDefaults(Path f) |
Path |
ViewFileSystem.getTrashCanLocation(Path f) |
FileStatus[] |
ViewFs.listStatus(Path f) |
FileStatus[] |
ViewFileSystem.listStatus(Path f) |
RemoteIterator<FileStatus> |
ViewFs.listStatusIterator(Path f) |
void |
ViewFs.mkdir(Path dir,
FsPermission permission,
boolean createParent) |
boolean |
ViewFileSystem.mkdirs(Path dir,
FsPermission permission) |
FSDataInputStream |
ViewFs.open(Path f,
int bufferSize) |
FSDataInputStream |
ViewFileSystem.open(Path f,
int bufferSize) |
boolean |
ViewFileSystem.rename(Path src,
Path dst) |
void |
ViewFs.renameInternal(Path src,
Path dst) |
void |
ViewFs.renameInternal(Path src,
Path dst,
boolean overwrite) |
Path |
ViewFs.resolvePath(Path f) |
Path |
ViewFileSystem.resolvePath(Path f) |
void |
ViewFs.setOwner(Path f,
String username,
String groupname) |
void |
ViewFileSystem.setOwner(Path f,
String username,
String groupname) |
void |
ViewFs.setPermission(Path f,
FsPermission permission) |
void |
ViewFileSystem.setPermission(Path f,
FsPermission permission) |
boolean |
ViewFs.setReplication(Path f,
short replication) |
boolean |
ViewFileSystem.setReplication(Path f,
short replication) |
void |
ViewFs.setTimes(Path f,
long mtime,
long atime) |
void |
ViewFileSystem.setTimes(Path f,
long mtime,
long atime) |
void |
ViewFileSystem.setWorkingDirectory(Path new_dir) |
Constructor and Description |
---|
NotInMountpointException(Path path,
String operation) |
Modifier and Type | Method and Description |
---|---|
Path |
DistributedFileSystem.createSnapshot(Path path,
String snapshotName) |
Path |
DistributedFileSystem.getHomeDirectory() |
Path |
DistributedFileSystem.getLinkTarget(Path f) |
Path |
HftpFileSystem.getWorkingDirectory() |
Path |
DistributedFileSystem.getWorkingDirectory() |
Path |
CorruptFileBlockIterator.next() |
protected Path |
DistributedFileSystem.resolveLink(Path f) |
Modifier and Type | Method and Description |
---|---|
RemoteIterator<Path> |
DistributedFileSystem.listCorruptFileBlocks(Path path) |
Modifier and Type | Method and Description |
---|---|
void |
DistributedFileSystem.allowSnapshot(Path path) |
FSDataOutputStream |
HftpFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
DistributedFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
void |
DistributedFileSystem.concat(Path trg,
Path[] psrcs)
Move blocks from srcs to trg and delete srcs afterwards.
|
void |
DistributedFileSystem.concat(Path trg,
Path[] psrcs)
Move blocks from srcs to trg and delete srcs afterwards.
|
FSDataOutputStream |
HftpFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
DistributedFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
HdfsDataOutputStream |
DistributedFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
InetSocketAddress[] favoredNodes)
Same as
DistributedFileSystem.create(Path, FsPermission, boolean, int, short, long,
Progressable) with the addition of favoredNodes that is a hint to
where the namenode should place the file blocks. |
FSDataOutputStream |
DistributedFileSystem.create(Path f,
FsPermission permission,
EnumSet<CreateFlag> cflags,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt) |
FSDataOutputStream |
DistributedFileSystem.createNonRecursive(Path f,
FsPermission permission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Same as create(), except fails if parent directory doesn't already exist.
|
Path |
DistributedFileSystem.createSnapshot(Path path,
String snapshotName) |
void |
DistributedFileSystem.createSymlink(Path target,
Path link,
boolean createParent) |
boolean |
HftpFileSystem.delete(Path f,
boolean recursive) |
boolean |
DistributedFileSystem.delete(Path f,
boolean recursive) |
void |
DistributedFileSystem.deleteSnapshot(Path snapshotDir,
String snapshotName) |
void |
DistributedFileSystem.disallowSnapshot(Path path) |
ContentSummary |
HftpFileSystem.getContentSummary(Path f) |
ContentSummary |
DistributedFileSystem.getContentSummary(Path f) |
BlockLocation[] |
DistributedFileSystem.getFileBlockLocations(Path p,
long start,
long len) |
FileChecksum |
HftpFileSystem.getFileChecksum(Path f) |
FileChecksum |
DistributedFileSystem.getFileChecksum(Path f) |
FileStatus |
DistributedFileSystem.getFileLinkStatus(Path f) |
FileStatus |
HftpFileSystem.getFileStatus(Path f) |
FileStatus |
DistributedFileSystem.getFileStatus(Path f)
Returns the stat information about the file.
|
Path |
DistributedFileSystem.getLinkTarget(Path f) |
SnapshotDiffReport |
DistributedFileSystem.getSnapshotDiffReport(Path snapshotDir,
String fromSnapshot,
String toSnapshot)
Get the difference between two snapshots, or between a snapshot and the
current tree of a directory.
|
FsStatus |
DistributedFileSystem.getStatus(Path p) |
boolean |
DistributedFileSystem.isFileClosed(Path src)
Get the close status of a file
|
RemoteIterator<Path> |
DistributedFileSystem.listCorruptFileBlocks(Path path) |
protected RemoteIterator<LocatedFileStatus> |
DistributedFileSystem.listLocatedStatus(Path p,
PathFilter filter) |
FileStatus[] |
HftpFileSystem.listStatus(Path f) |
FileStatus[] |
DistributedFileSystem.listStatus(Path p)
List all the entries of a directory
Note that this operation is not atomic for a large directory.
|
boolean |
DistributedFileSystem.mkdir(Path f,
FsPermission permission)
Create a directory, only when the parent directories exist.
|
boolean |
HftpFileSystem.mkdirs(Path f,
FsPermission permission) |
boolean |
DistributedFileSystem.mkdirs(Path f,
FsPermission permission)
Create a directory and its parent directories.
|
FSDataInputStream |
HftpFileSystem.open(Path f,
int buffersize) |
FSDataInputStream |
DistributedFileSystem.open(Path f,
int bufferSize) |
protected HdfsDataOutputStream |
DistributedFileSystem.primitiveCreate(Path f,
FsPermission absolutePermission,
EnumSet<CreateFlag> flag,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt) |
protected boolean |
DistributedFileSystem.primitiveMkdir(Path f,
FsPermission absolutePermission) |
boolean |
DistributedFileSystem.recoverLease(Path f)
Start the lease recovery of a file
|
boolean |
HftpFileSystem.rename(Path src,
Path dst) |
boolean |
DistributedFileSystem.rename(Path src,
Path dst) |
void |
DistributedFileSystem.rename(Path src,
Path dst,
Options.Rename... options)
This rename operation is guaranteed to be atomic.
|
void |
DistributedFileSystem.renameSnapshot(Path path,
String snapshotOldName,
String snapshotNewName) |
boolean |
DistributedFileSystem.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Deprecated.
|
protected Path |
DistributedFileSystem.resolveLink(Path f) |
void |
DistributedFileSystem.setOwner(Path p,
String username,
String groupname) |
void |
DistributedFileSystem.setPermission(Path p,
FsPermission permission) |
void |
DistributedFileSystem.setQuota(Path src,
long namespaceQuota,
long diskspaceQuota)
Set a directory's quotas
|
boolean |
DistributedFileSystem.setReplication(Path src,
short replication) |
void |
DistributedFileSystem.setTimes(Path p,
long mtime,
long atime) |
void |
HftpFileSystem.setWorkingDirectory(Path f) |
void |
DistributedFileSystem.setWorkingDirectory(Path dir) |
Constructor and Description |
---|
CorruptFileBlockIterator(DFSClient dfs,
Path path) |
Modifier and Type | Method and Description |
---|---|
void |
HdfsAdmin.allowSnapshot(Path path)
Allow snapshot on a directory.
|
void |
HdfsAdmin.clearQuota(Path src)
Clear the namespace quota (count of files, directories and sym links) for a
directory.
|
void |
HdfsAdmin.clearSpaceQuota(Path src)
Clear the disk space quota (size of files) for a directory.
|
void |
HdfsAdmin.disallowSnapshot(Path path)
Disallow snapshot on a directory.
|
void |
HdfsAdmin.setQuota(Path src,
long quota)
Set the namespace quota (count of files, directories, and sym links) for a
directory.
|
void |
HdfsAdmin.setSpaceQuota(Path src,
long spaceQuota)
Set the disk space quota (size of files) for a directory.
|
Modifier and Type | Method and Description |
---|---|
Path |
SnapshottableDirectoryStatus.getFullPath() |
Path |
HdfsFileStatus.getFullPath(Path parent)
Get the full path
|
Path |
UnresolvedPathException.getResolvedPath()
Return a path with the link resolved with the target.
|
Modifier and Type | Method and Description |
---|---|
Path |
HdfsFileStatus.getFullPath(Path parent)
Get the full path
|
FileStatus |
HdfsFileStatus.makeQualified(URI defaultUri,
Path path) |
LocatedFileStatus |
HdfsLocatedFileStatus.makeQualifiedLocated(URI defaultUri,
Path path) |
Modifier and Type | Method and Description |
---|---|
Path |
WebHdfsFileSystem.getHomeDirectory() |
Path |
WebHdfsFileSystem.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
WebHdfsFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
void |
WebHdfsFileSystem.concat(Path trg,
Path[] srcs) |
void |
WebHdfsFileSystem.concat(Path trg,
Path[] srcs) |
FSDataOutputStream |
WebHdfsFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
void |
WebHdfsFileSystem.createSymlink(Path destination,
Path f,
boolean createParent)
Create a symlink pointing to the destination path.
|
boolean |
WebHdfsFileSystem.delete(Path f) |
boolean |
WebHdfsFileSystem.delete(Path f,
boolean recursive) |
ContentSummary |
WebHdfsFileSystem.getContentSummary(Path p) |
BlockLocation[] |
WebHdfsFileSystem.getFileBlockLocations(Path p,
long offset,
long length) |
MD5MD5CRC32FileChecksum |
WebHdfsFileSystem.getFileChecksum(Path p) |
FileStatus |
WebHdfsFileSystem.getFileStatus(Path f) |
FileStatus[] |
WebHdfsFileSystem.listStatus(Path f) |
boolean |
WebHdfsFileSystem.mkdirs(Path f,
FsPermission permission) |
FSDataInputStream |
WebHdfsFileSystem.open(Path f,
int buffersize) |
boolean |
WebHdfsFileSystem.rename(Path src,
Path dst) |
void |
WebHdfsFileSystem.rename(Path src,
Path dst,
Options.Rename... options) |
void |
WebHdfsFileSystem.setOwner(Path p,
String owner,
String group) |
void |
WebHdfsFileSystem.setPermission(Path p,
FsPermission permission) |
boolean |
WebHdfsFileSystem.setReplication(Path p,
short replication) |
void |
WebHdfsFileSystem.setTimes(Path p,
long mtime,
long atime) |
void |
WebHdfsFileSystem.setWorkingDirectory(Path dir) |
Constructor and Description |
---|
ConcatSourcesParam(Path[] paths) |
Modifier and Type | Method and Description |
---|---|
SequenceFile.Writer |
SequenceFile.Sorter.cloneFileAttributes(Path inputFile,
Path outputFile,
Progressable prog)
Clones the attributes (like compression of the input file and creates a
corresponding Writer
|
protected SequenceFile.Reader |
MapFile.Reader.createDataFileReader(Path dataFile,
Configuration conf,
SequenceFile.Reader.Option... options)
Override this method to specialize the type of
SequenceFile.Reader returned. |
static SequenceFile.Writer |
SequenceFile.createWriter(FileContext fc,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
SequenceFile.Metadata metadata,
EnumSet<CreateFlag> createFlag,
Options.CreateOpts... opts)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
boolean createParent,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
SequenceFile.Metadata metadata)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Deprecated.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
Progressable progress)
Deprecated.
|
static SequenceFile.Writer.Option |
SequenceFile.Writer.file(Path value) |
static SequenceFile.Reader.Option |
SequenceFile.Reader.file(Path value)
Create an option to specify the path name of the sequence file.
|
static long |
MapFile.fix(FileSystem fs,
Path dir,
Class<? extends Writable> keyClass,
Class<? extends Writable> valueClass,
boolean dryrun,
Configuration conf)
This method attempts to fix a corrupt MapFile by re-creating its index.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(List<SequenceFile.Sorter.SegmentDescriptor> segments,
Path tmpDir)
Merges the list of segments of type
SegmentDescriptor |
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
int factor,
Path tmpDir)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
int factor,
Path tmpDir)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
Path tmpDir)
Merges the contents of files passed in Path[] using a max factor value
that is already set
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
Path tmpDir)
Merges the contents of files passed in Path[] using a max factor value
that is already set
|
void |
SequenceFile.Sorter.merge(Path[] inFiles,
Path outFile)
Merge the provided files.
|
void |
SequenceFile.Sorter.merge(Path[] inFiles,
Path outFile)
Merge the provided files.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
Path tempDir,
boolean deleteInputs)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
Path tempDir,
boolean deleteInputs)
Merges the contents of files passed in Path[]
|
protected void |
MapFile.Reader.open(Path dir,
WritableComparator comparator,
Configuration conf,
SequenceFile.Reader.Option... options) |
protected FSDataInputStream |
SequenceFile.Reader.openFile(FileSystem fs,
Path file,
int bufferSize,
long length)
Override this method to specialize the type of
FSDataInputStream returned. |
void |
SequenceFile.Sorter.sort(Path[] inFiles,
Path outFile,
boolean deleteInput)
Perform a file sort from a set of input files into an output file.
|
void |
SequenceFile.Sorter.sort(Path[] inFiles,
Path outFile,
boolean deleteInput)
Perform a file sort from a set of input files into an output file.
|
void |
SequenceFile.Sorter.sort(Path inFile,
Path outFile)
The backwards compatible interface to sort.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.sortAndIterate(Path[] inFiles,
Path tempDir,
boolean deleteInput)
Perform a file sort from a set of input files and return an iterator.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.sortAndIterate(Path[] inFiles,
Path tempDir,
boolean deleteInput)
Perform a file sort from a set of input files and return an iterator.
|
Constructor and Description |
---|
BloomMapFile.Reader(Path dir,
Configuration conf,
SequenceFile.Reader.Option... options) |
BloomMapFile.Writer(Configuration conf,
Path dir,
SequenceFile.Writer.Option... options) |
MapFile.Reader(Path dir,
Configuration conf,
SequenceFile.Reader.Option... opts) |
MapFile.Writer(Configuration conf,
Path dirName,
SequenceFile.Writer.Option... opts) |
SequenceFile.Reader(FileSystem fs,
Path file,
Configuration conf)
Deprecated.
Use Reader(Configuration, Option...) instead.
|
SequenceFile.Sorter.SegmentDescriptor(long segmentOffset,
long segmentLength,
Path segmentPathName)
Constructs a segment
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass)
Deprecated.
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
SequenceFile.Metadata metadata)
Deprecated.
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
Progressable progress,
SequenceFile.Metadata metadata)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
CompressionCodec |
CompressionCodecFactory.getCodec(Path file)
Find the relevant compression codec for the given file based on its
filename suffix.
|
Modifier and Type | Method and Description |
---|---|
Path |
YarnOutputFiles.getInputFile(int mapId)
Return a local reduce input file created earlier
|
Path |
MROutputFiles.getInputFile(int mapId)
Return a local reduce input file created earlier
|
abstract Path |
MapOutputFile.getInputFile(int mapId)
Return a local reduce input file created earlier
|
Path |
YarnOutputFiles.getInputFileForWrite(TaskID mapId,
long size)
Create a local reduce input file name.
|
Path |
MROutputFiles.getInputFileForWrite(TaskID mapId,
long size)
Create a local reduce input file name.
|
abstract Path |
MapOutputFile.getInputFileForWrite(TaskID mapId,
long size)
Create a local reduce input file name.
|
static Path[] |
FileInputFormat.getInputPaths(JobConf conf)
Get the list of input
Path s for the map-reduce job. |
Path |
JobConf.getLocalPath(String pathString)
Constructs a local file name.
|
Path |
YarnOutputFiles.getOutputFile()
Return the path to local map output file created earlier
|
Path |
MROutputFiles.getOutputFile()
Return the path to local map output file created earlier
|
abstract Path |
MapOutputFile.getOutputFile()
Return the path to local map output file created earlier
|
Path |
YarnOutputFiles.getOutputFileForWrite(long size)
Create a local map output file name.
|
Path |
MROutputFiles.getOutputFileForWrite(long size)
Create a local map output file name.
|
abstract Path |
MapOutputFile.getOutputFileForWrite(long size)
Create a local map output file name.
|
Path |
YarnOutputFiles.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
Path |
MROutputFiles.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
abstract Path |
MapOutputFile.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
Path |
YarnOutputFiles.getOutputIndexFile()
Return the path to a local map output index file created earlier
|
Path |
MROutputFiles.getOutputIndexFile()
Return the path to a local map output index file created earlier
|
abstract Path |
MapOutputFile.getOutputIndexFile()
Return the path to a local map output index file created earlier
|
Path |
YarnOutputFiles.getOutputIndexFileForWrite(long size)
Create a local map output index file name.
|
Path |
MROutputFiles.getOutputIndexFileForWrite(long size)
Create a local map output index file name.
|
abstract Path |
MapOutputFile.getOutputIndexFileForWrite(long size)
Create a local map output index file name.
|
Path |
YarnOutputFiles.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
Path |
MROutputFiles.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
abstract Path |
MapOutputFile.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
static Path |
FileOutputFormat.getOutputPath(JobConf conf)
Get the
Path to the output directory for the map-reduce job. |
Path |
FileSplit.getPath()
The file containing this split's data.
|
static Path |
FileOutputFormat.getPathForCustomFile(JobConf conf,
String name)
Helper function to generate a
Path for a file that is unique for
the task within the job output directory. |
static Path |
SkipBadRecords.getSkipOutputPath(Configuration conf)
Get the directory to which skipped records are written.
|
Path |
YarnOutputFiles.getSpillFile(int spillNumber)
Return a local map spill file created earlier.
|
Path |
MROutputFiles.getSpillFile(int spillNumber)
Return a local map spill file created earlier.
|
abstract Path |
MapOutputFile.getSpillFile(int spillNumber)
Return a local map spill file created earlier.
|
Path |
YarnOutputFiles.getSpillFileForWrite(int spillNumber,
long size)
Create a local map spill file name.
|
Path |
MROutputFiles.getSpillFileForWrite(int spillNumber,
long size)
Create a local map spill file name.
|
abstract Path |
MapOutputFile.getSpillFileForWrite(int spillNumber,
long size)
Create a local map spill file name.
|
Path |
YarnOutputFiles.getSpillIndexFile(int spillNumber)
Return a local map spill index file created earlier
|
Path |
MROutputFiles.getSpillIndexFile(int spillNumber)
Return a local map spill index file created earlier
|
abstract Path |
MapOutputFile.getSpillIndexFile(int spillNumber)
Return a local map spill index file created earlier
|
Path |
YarnOutputFiles.getSpillIndexFileForWrite(int spillNumber,
long size)
Create a local map spill index file name.
|
Path |
MROutputFiles.getSpillIndexFileForWrite(int spillNumber,
long size)
Create a local map spill index file name.
|
abstract Path |
MapOutputFile.getSpillIndexFileForWrite(int spillNumber,
long size)
Create a local map spill index file name.
|
Path |
JobClient.getStagingAreaDir()
Fetch the staging area directory for the application
|
Path |
JobClient.getSystemDir()
Grab the jobtracker system directory path where job-specific files are to be placed.
|
Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context) |
static Path |
FileOutputFormat.getTaskOutputPath(JobConf conf,
String name)
Helper function to create the task's temporary output directory and
return the path to the task's output file.
|
Path |
JobConf.getWorkingDirectory()
Get the current working directory for the default file system.
|
static Path |
FileOutputFormat.getWorkOutputPath(JobConf conf)
Get the
Path to the task's temporary output directory
for the map-reduce job
Tasks' Side-Effect Files |
Path |
FileOutputCommitter.getWorkPath(TaskAttemptContext context,
Path outputPath) |
Modifier and Type | Method and Description |
---|---|
boolean |
Utils.OutputFileUtils.OutputFilesFilter.accept(Path path) |
boolean |
Utils.OutputFileUtils.OutputLogFilter.accept(Path path) |
boolean |
OutputLogFilter.accept(Path path) |
static void |
FileInputFormat.addInputPath(JobConf conf,
Path path)
Add a
Path to the list of inputs for the map-reduce job. |
protected void |
FileInputFormat.addInputPathRecursively(List<FileStatus> result,
FileSystem fs,
Path path,
PathFilter inputFilter)
Add files in the input path recursively into the results.
|
protected static List<FileSystem.Statistics> |
Task.getFsStatistics(Path path,
Configuration conf)
Gets a handle to the Statistics instance based on the scheme associated
with path.
|
Path |
YarnOutputFiles.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
Path |
MROutputFiles.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
abstract Path |
MapOutputFile.getOutputFileForWriteInVolume(Path existing)
Create a local map output file name on the same volume.
|
Path |
YarnOutputFiles.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
Path |
MROutputFiles.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
abstract Path |
MapOutputFile.getOutputIndexFileForWriteInVolume(Path existing)
Create a local map output index file name on the same volume.
|
static SequenceFile.Reader[] |
SequenceFileOutputFormat.getReaders(Configuration conf,
Path dir)
Open the output generated by this format.
|
static MapFile.Reader[] |
MapFileOutputFormat.getReaders(FileSystem ignored,
Path dir,
Configuration conf)
Open the output generated by this format.
|
Path |
FileOutputCommitter.getWorkPath(TaskAttemptContext context,
Path outputPath) |
static boolean |
JobClient.isJobDirValid(Path jobDirPath,
FileSystem fs)
Checks if the job directory is clean and has all the required components
for (re) starting the job
|
protected boolean |
TextInputFormat.isSplitable(FileSystem fs,
Path file) |
protected boolean |
FileInputFormat.isSplitable(FileSystem fs,
Path filename)
Is the given filename splitable? Usually, true, but if the file is
stream compressed, it will not be.
|
protected boolean |
KeyValueTextInputFormat.isSplitable(FileSystem fs,
Path file) |
protected FileSplit |
FileInputFormat.makeSplit(Path file,
long start,
long length,
String[] hosts)
A factory that makes the split for this class.
|
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
int inMemSegments,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Counters.Counter mergedMapOutputsCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Counters.Counter mergedMapOutputsCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Path[] inputs,
boolean deleteInputs,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
int inMemSegments,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
boolean sortSegments,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static <K,V> RawKeyValueIterator |
Merger.merge(Configuration conf,
FileSystem fs,
Class<K> keyClass,
Class<V> valueClass,
List<Merger.Segment<K,V>> segments,
int mergeFactor,
Path tmpDir,
RawComparator<K> comparator,
Progressable reporter,
Counters.Counter readsCounter,
Counters.Counter writesCounter,
Progress mergePhase) |
static void |
FileInputFormat.setInputPaths(JobConf conf,
Path... inputPaths)
Set the array of
Path s as the list of inputs
for the map-reduce job. |
static void |
FileOutputFormat.setOutputPath(JobConf conf,
Path outputDir)
Set the
Path of the output directory for the map-reduce job. |
static void |
SkipBadRecords.setSkipOutputPath(JobConf conf,
Path path)
Set the directory to which skipped records are written.
|
void |
JobConf.setWorkingDirectory(Path dir)
Set the current working directory for the default file system.
|
static void |
FileOutputFormat.setWorkOutputPath(JobConf conf,
Path outputDir)
Set the
Path of the task's temporary output directory
for the map-reduce job. |
void |
SpillRecord.writeToFile(Path loc,
JobConf job)
Write this spill record to the location provided.
|
void |
SpillRecord.writeToFile(Path loc,
JobConf job,
Checksum crc) |
Constructor and Description |
---|
FileSplit(Path file,
long start,
long length,
JobConf conf)
Deprecated.
|
FileSplit(Path file,
long start,
long length,
String[] hosts)
Constructs a split with host information
|
IFile.Reader(Configuration conf,
FileSystem fs,
Path file,
CompressionCodec codec,
Counters.Counter readsCounter)
Construct an IFile Reader.
|
IFile.Writer(Configuration conf,
FileSystem fs,
Path file) |
IFile.Writer(Configuration conf,
FileSystem fs,
Path file,
Class<K> keyClass,
Class<V> valueClass,
CompressionCodec codec,
Counters.Counter writesCounter) |
JobConf(Path config)
Construct a map/reduce configuration.
|
MapTaskAttemptImpl(TaskId taskId,
int attempt,
EventHandler eventHandler,
Path jobFile,
int partition,
JobSplit.TaskSplitMetaInfo splitInfo,
JobConf conf,
TaskAttemptListener taskAttemptListener,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
AppContext appContext) |
Merger.Segment(Configuration conf,
FileSystem fs,
Path file,
CompressionCodec codec,
boolean preserve) |
Merger.Segment(Configuration conf,
FileSystem fs,
Path file,
CompressionCodec codec,
boolean preserve,
Counters.Counter mergedMapOutputsCounter) |
Merger.Segment(Configuration conf,
FileSystem fs,
Path file,
CompressionCodec codec,
boolean preserve,
Counters.Counter mergedMapOutputsCounter,
long rawDataLength) |
Merger.Segment(Configuration conf,
FileSystem fs,
Path file,
long segmentOffset,
long segmentLength,
CompressionCodec codec,
boolean preserve) |
Merger.Segment(Configuration conf,
FileSystem fs,
Path file,
long segmentOffset,
long segmentLength,
CompressionCodec codec,
boolean preserve,
Counters.Counter mergedMapOutputsCounter) |
MultiFileSplit(JobConf job,
Path[] files,
long[] lengths) |
ReduceTaskAttemptImpl(TaskId id,
int attempt,
EventHandler eventHandler,
Path jobFile,
int partition,
int numMapTasks,
JobConf conf,
TaskAttemptListener taskAttemptListener,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
AppContext appContext) |
SpillRecord(Path indexFileName,
JobConf job) |
SpillRecord(Path indexFileName,
JobConf job,
Checksum crc,
String expectedIndexOwner) |
SpillRecord(Path indexFileName,
JobConf job,
String expectedIndexOwner) |
Modifier and Type | Method and Description |
---|---|
abstract org.apache.hadoop.mapred.gridmix.GridmixJob |
JobCreator.createGridmixJob(Configuration conf,
long submissionMillis,
JobStory jobdesc,
Path outRoot,
UserGroupInformation ugi,
int seq)
Create Gridmix simulated job.
|
protected org.apache.hadoop.mapred.gridmix.JobFactory |
Gridmix.createJobFactory(org.apache.hadoop.mapred.gridmix.JobSubmitter submitter,
String traceIn,
Path scratchDir,
Configuration conf,
CountDownLatch startFlag,
UserResolver resolver) |
protected int |
Gridmix.writeInputData(long genbytes,
Path inputDir)
Write random bytes at the path <inputDir> if needed.
|
Constructor and Description |
---|
SerialJobFactory(org.apache.hadoop.mapred.gridmix.JobSubmitter submitter,
JobStoryProducer jobProducer,
Path scratch,
Configuration conf,
CountDownLatch startFlag,
UserResolver resolver)
Creating a new instance does not start the thread.
|
SleepJob(Configuration conf,
long submissionMillis,
JobStory jobdesc,
Path outRoot,
UserGroupInformation ugi,
int seq,
int numLocations,
String[] hosts) |
StressJobFactory(org.apache.hadoop.mapred.gridmix.JobSubmitter submitter,
JobStoryProducer jobProducer,
Path scratch,
Configuration conf,
CountDownLatch startFlag,
UserResolver resolver)
Creating a new instance does not start the thread.
|
Modifier and Type | Method and Description |
---|---|
static String |
CompositeInputFormat.compose(String op,
Class<? extends InputFormat> inf,
Path... path)
Convenience method for constructing composite formats.
|
Modifier and Type | Method and Description |
---|---|
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
protected static FileSplit |
NLineInputFormat.createFileSplit(Path fileName,
long begin,
long length)
NLineInputFormat uses LineRecordReader, which always reads
(and consumes) at least one character out of its upper split
boundary.
|
protected boolean |
CombineFileInputFormat.isSplitable(FileSystem fs,
Path file) |
protected boolean |
CombineFileInputFormat.isSplitable(JobContext context,
Path file)
Subclasses should avoid overriding this method and should instead only
override
CombineFileInputFormat.isSplitable(FileSystem, Path) . |
static void |
TotalOrderPartitioner.setPartitionFile(JobConf job,
Path p)
Deprecated.
|
Constructor and Description |
---|
CombineFileSplit(JobConf job,
Path[] files,
long[] lengths) |
CombineFileSplit(JobConf job,
Path[] files,
long[] start,
long[] lengths,
String[] locations) |
Modifier and Type | Method and Description |
---|---|
Path[] |
JobContext.getArchiveClassPaths()
Get the archive entries in classpath as an array of Path
|
Path[] |
JobContext.getFileClassPaths()
Get the file entries in classpath as an array of Path
|
static Path |
JobSubmissionFiles.getJobConfPath(Path jobSubmitDir)
Get the job conf path.
|
static Path |
JobSubmissionFiles.getJobDistCacheArchives(Path jobSubmitDir)
Get the job distributed cache archives path.
|
static Path |
JobSubmissionFiles.getJobDistCacheFiles(Path jobSubmitDir)
Get the job distributed cache files path.
|
static Path |
JobSubmissionFiles.getJobDistCacheLibjars(Path jobSubmitDir)
Get the job distributed cache libjars path.
|
static Path |
JobSubmissionFiles.getJobJar(Path jobSubmitDir)
Get the job jar path.
|
static Path |
JobSubmissionFiles.getJobSplitFile(Path jobSubmissionDir) |
static Path |
JobSubmissionFiles.getJobSplitMetaFile(Path jobSubmissionDir) |
Path[] |
JobContext.getLocalCacheArchives()
Deprecated.
the array returned only includes the items the were
downloaded. There is no way to map this to what is returned by
JobContext.getCacheArchives() . |
Path[] |
JobContext.getLocalCacheFiles()
Deprecated.
the array returned only includes the items the were
downloaded. There is no way to map this to what is returned by
JobContext.getCacheFiles() . |
Path |
Cluster.getStagingAreaDir()
Grab the jobtracker's view of the staging directory path where
job-specific files will be placed.
|
static Path |
JobSubmissionFiles.getStagingDir(Cluster cluster,
Configuration conf)
Initializes the staging directory and returns the path.
|
Path |
Cluster.getSystemDir()
Grab the jobtracker system directory path where
job-specific files will be placed.
|
Path |
JobContext.getWorkingDirectory()
Get the current working directory for the default file system.
|
Modifier and Type | Method and Description |
---|---|
void |
Job.addArchiveToClassPath(Path archive)
Add an archive path to the current set of classpath entries.
|
void |
Job.addFileToClassPath(Path file)
Add an file path to the current set of classpath entries It adds the file
to cache as well.
|
static Path |
JobSubmissionFiles.getJobConfPath(Path jobSubmitDir)
Get the job conf path.
|
static Path |
JobSubmissionFiles.getJobDistCacheArchives(Path jobSubmitDir)
Get the job distributed cache archives path.
|
static Path |
JobSubmissionFiles.getJobDistCacheFiles(Path jobSubmitDir)
Get the job distributed cache files path.
|
static Path |
JobSubmissionFiles.getJobDistCacheLibjars(Path jobSubmitDir)
Get the job distributed cache libjars path.
|
static Path |
JobSubmissionFiles.getJobJar(Path jobSubmitDir)
Get the job jar path.
|
static Path |
JobSubmissionFiles.getJobSplitFile(Path jobSubmissionDir) |
static Path |
JobSubmissionFiles.getJobSplitMetaFile(Path jobSubmissionDir) |
void |
Job.setWorkingDirectory(Path dir)
Set the current working directory for the default file system.
|
Modifier and Type | Method and Description |
---|---|
static Path[] |
DistributedCache.getArchiveClassPaths(Configuration conf)
Deprecated.
Use
JobContext.getArchiveClassPaths() instead |
static Path[] |
DistributedCache.getFileClassPaths(Configuration conf)
Deprecated.
Use
JobContext.getFileClassPaths() instead |
static Path[] |
DistributedCache.getLocalCacheArchives(Configuration conf)
Deprecated.
Use
JobContext.getLocalCacheArchives() instead |
static Path[] |
DistributedCache.getLocalCacheFiles(Configuration conf)
Deprecated.
Use
JobContext.getLocalCacheFiles() instead |
Modifier and Type | Method and Description |
---|---|
static void |
DistributedCache.addArchiveToClassPath(Path archive,
Configuration conf)
Deprecated.
Use
Job.addArchiveToClassPath(Path) instead |
static void |
DistributedCache.addArchiveToClassPath(Path archive,
Configuration conf,
FileSystem fs)
Deprecated.
Add an archive path to the current set of classpath entries.
|
static void |
DistributedCache.addFileToClassPath(Path file,
Configuration conf)
Deprecated.
Use
Job.addFileToClassPath(Path) instead |
static void |
DistributedCache.addFileToClassPath(Path file,
Configuration conf,
FileSystem fs)
Deprecated.
Add a file path to the current set of classpath entries.
|
Modifier and Type | Method and Description |
---|---|
protected org.apache.hadoop.mapreduce.jobhistory.EventWriter |
JobHistoryEventHandler.createEventWriter(Path historyFilePath) |
Constructor and Description |
---|
EventReader(FileSystem fs,
Path name)
Create a new Event Reader
|
JobHistoryParser(FileSystem fs,
Path historyFile)
Create the job history parser for the given history file using the
given file system
|
Modifier and Type | Method and Description |
---|---|
static Path[] |
FileInputFormat.getInputPaths(JobContext context)
Get the list of input
Path s for the map-reduce job. |
Path |
FileSplit.getPath()
The file containing this split's data.
|
Path |
CombineFileSplit.getPath(int i)
Returns the ith Path
|
Path[] |
CombineFileSplit.getPaths()
Returns all the Paths in the split
|
Modifier and Type | Method and Description |
---|---|
static void |
FileInputFormat.addInputPath(Job job,
Path path)
Add a
Path to the list of inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
protected void |
FileInputFormat.addInputPathRecursively(List<FileStatus> result,
FileSystem fs,
Path path,
PathFilter inputFilter)
Add files in the input path recursively into the results.
|
protected static FileSplit |
NLineInputFormat.createFileSplit(Path fileName,
long begin,
long length)
NLineInputFormat uses LineRecordReader, which always reads
(and consumes) at least one character out of its upper split
boundary.
|
protected boolean |
CombineFileInputFormat.isSplitable(JobContext context,
Path file) |
protected boolean |
TextInputFormat.isSplitable(JobContext context,
Path file) |
protected boolean |
FileInputFormat.isSplitable(JobContext context,
Path filename)
Is the given filename splitable? Usually, true, but if the file is
stream compressed, it will not be.
|
protected boolean |
KeyValueTextInputFormat.isSplitable(JobContext context,
Path file) |
protected FileSplit |
FileInputFormat.makeSplit(Path file,
long start,
long length,
String[] hosts)
A factory that makes the split for this class.
|
static void |
FileInputFormat.setInputPaths(Job job,
Path... inputPaths)
Set the array of
Path s as the list of inputs
for the map-reduce job. |
Constructor and Description |
---|
CombineFileSplit(Path[] files,
long[] lengths) |
CombineFileSplit(Path[] files,
long[] start,
long[] lengths,
String[] locations) |
FileSplit(Path file,
long start,
long length,
String[] hosts)
Constructs a split with host information
|
Modifier and Type | Method and Description |
---|---|
static String |
CompositeInputFormat.compose(String op,
Class<? extends InputFormat> inf,
Path... path)
Convenience method for constructing composite formats.
|
Modifier and Type | Method and Description |
---|---|
Path[] |
WrappedMapper.Context.getArchiveClassPaths() |
Path[] |
WrappedMapper.Context.getFileClassPaths() |
Path[] |
WrappedMapper.Context.getLocalCacheArchives() |
Path[] |
WrappedMapper.Context.getLocalCacheFiles() |
Path |
WrappedMapper.Context.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
Path |
PartialFileOutputCommitter.getCommittedTaskPath(int appAttemptId,
TaskAttemptContext context) |
protected Path |
FileOutputCommitter.getCommittedTaskPath(int appAttemptId,
TaskAttemptContext context)
Compute the path where the output of a committed task is stored until the
entire job is committed for a specific application attempt.
|
Path |
FileOutputCommitter.getCommittedTaskPath(TaskAttemptContext context)
Compute the path where the output of a committed task is stored until
the entire job is committed.
|
static Path |
FileOutputCommitter.getCommittedTaskPath(TaskAttemptContext context,
Path out) |
Path |
FileOutputFormat.getDefaultWorkFile(TaskAttemptContext context,
String extension)
Get the default path and filename for the output format.
|
protected Path |
FileOutputCommitter.getJobAttemptPath(int appAttemptId)
Compute the path where the output of a given job attempt will be placed.
|
Path |
FileOutputCommitter.getJobAttemptPath(JobContext context)
Compute the path where the output of a given job attempt will be placed.
|
static Path |
FileOutputCommitter.getJobAttemptPath(JobContext context,
Path out)
Compute the path where the output of a given job attempt will be placed.
|
static Path |
FileOutputFormat.getOutputPath(JobContext job)
Get the
Path to the output directory for the map-reduce job. |
static Path |
FileOutputFormat.getPathForWorkFile(TaskInputOutputContext<?,?,?,?> context,
String name,
String extension)
Helper function to generate a
Path for a file that is unique for
the task within the job output directory. |
Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context)
Compute the path where the output of a task attempt is stored until
that task is committed.
|
static Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context,
Path out)
Compute the path where the output of a task attempt is stored until
that task is committed.
|
static Path |
FileOutputFormat.getWorkOutputPath(TaskInputOutputContext<?,?,?,?> context)
Get the
Path to the task's temporary output directory
for the map-reduce job
Tasks' Side-Effect Files |
Path |
FileOutputCommitter.getWorkPath()
Get the directory that the task should write results into.
|
Modifier and Type | Method and Description |
---|---|
void |
FileOutputCommitter.abortTask(TaskAttemptContext context,
Path taskAttemptPath) |
void |
FileOutputCommitter.commitTask(TaskAttemptContext context,
Path taskAttemptPath) |
static Path |
FileOutputCommitter.getCommittedTaskPath(TaskAttemptContext context,
Path out) |
static Path |
FileOutputCommitter.getJobAttemptPath(JobContext context,
Path out)
Compute the path where the output of a given job attempt will be placed.
|
static MapFile.Reader[] |
MapFileOutputFormat.getReaders(Path dir,
Configuration conf)
Open the output generated by this format.
|
static Path |
FileOutputCommitter.getTaskAttemptPath(TaskAttemptContext context,
Path out)
Compute the path where the output of a task attempt is stored until
that task is committed.
|
boolean |
FileOutputCommitter.needsTaskCommit(TaskAttemptContext context,
Path taskAttemptPath) |
static void |
FileOutputFormat.setOutputPath(Job job,
Path outputDir)
Set the
Path of the output directory for the map-reduce job. |
Constructor and Description |
---|
FileOutputCommitter(Path outputPath,
JobContext context)
Create a file output committer
|
FileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
PartialFileOutputCommitter(Path outputPath,
JobContext context) |
PartialFileOutputCommitter(Path outputPath,
TaskAttemptContext context) |
Modifier and Type | Method and Description |
---|---|
static void |
TotalOrderPartitioner.setPartitionFile(Configuration conf,
Path p)
Set the path to the SequenceFile storing the sorted partition keyset.
|
Modifier and Type | Method and Description |
---|---|
Path[] |
WrappedReducer.Context.getArchiveClassPaths() |
Path[] |
WrappedReducer.Context.getFileClassPaths() |
Path[] |
WrappedReducer.Context.getLocalCacheArchives() |
Path[] |
WrappedReducer.Context.getLocalCacheFiles() |
Path |
WrappedReducer.Context.getWorkingDirectory() |
Modifier and Type | Method and Description |
---|---|
static void |
TokenCache.obtainTokensForNamenodes(Credentials credentials,
Path[] ps,
Configuration conf)
Convenience method to obtain delegation tokens from namenodes
corresponding to the paths passed.
|
Modifier and Type | Method and Description |
---|---|
static void |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
InputSplit[] splits) |
static <T extends InputSplit> |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
List<InputSplit> splits) |
static <T extends InputSplit> |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
T[] splits) |
static JobSplit.TaskSplitMetaInfo[] |
SplitMetaInfoReader.readSplitMetaInfo(JobID jobId,
FileSystem fs,
Configuration conf,
Path jobSubmitDir) |
Modifier and Type | Method and Description |
---|---|
Path[] |
JobContextImpl.getArchiveClassPaths()
Get the archive entries in classpath as an array of Path
|
Path[] |
JobContextImpl.getFileClassPaths()
Get the file entries in classpath as an array of Path
|
Path[] |
JobContextImpl.getLocalCacheArchives()
Return the path array of the localized caches
|
Path[] |
JobContextImpl.getLocalCacheFiles()
Return the path array of the localized files
|
Path |
JobContextImpl.getWorkingDirectory()
Get the current working directory for the default file system.
|
Modifier and Type | Method and Description |
---|---|
Path |
Job.getConfFile() |
Modifier and Type | Field and Description |
---|---|
protected Path |
TaskImpl.jobFile |
protected Path |
TaskAttemptImpl.jobFile |
Path |
JobImpl.remoteJobConfFile |
Modifier and Type | Method and Description |
---|---|
Path |
JobImpl.getConfFile() |
Constructor and Description |
---|
MapTaskImpl(JobId jobId,
int partition,
EventHandler eventHandler,
Path remoteJobConfFile,
JobConf conf,
JobSplit.TaskSplitMetaInfo taskSplitMetaInfo,
TaskAttemptListener taskAttemptListener,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
int appAttemptId,
MRAppMetrics metrics,
AppContext appContext) |
ReduceTaskImpl(JobId jobId,
int partition,
EventHandler eventHandler,
Path jobFile,
JobConf conf,
int numMapTasks,
TaskAttemptListener taskAttemptListener,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
int appAttemptId,
MRAppMetrics metrics,
AppContext appContext) |
TaskAttemptImpl(TaskId taskId,
int i,
EventHandler eventHandler,
TaskAttemptListener taskAttemptListener,
Path jobFile,
int partition,
JobConf conf,
String[] dataLocalHosts,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
AppContext appContext) |
TaskImpl(JobId jobId,
TaskType taskType,
int partition,
EventHandler eventHandler,
Path remoteJobConfFile,
JobConf conf,
TaskAttemptListener taskAttemptListener,
Token<JobTokenIdentifier> jobToken,
Credentials credentials,
Clock clock,
int appAttemptId,
MRAppMetrics metrics,
AppContext appContext) |
Modifier and Type | Method and Description |
---|---|
Path |
CompletedJob.getConfFile() |
Path |
PartialJob.getConfFile() |
Path |
HistoryFileManager.HistoryFileInfo.getConfFile() |
Modifier and Type | Method and Description |
---|---|
protected void |
CompletedJob.loadFullHistoryData(boolean loadTasks,
Path historyFileAbsolute) |
Constructor and Description |
---|
CompletedJob(Configuration conf,
JobId jobId,
Path historyFile,
boolean loadTasks,
String userName,
HistoryFileManager.HistoryFileInfo info,
JobACLsManager aclsMgr) |
Modifier and Type | Method and Description |
---|---|
static Path |
JobHistoryUtils.getPreviousJobHistoryPath(Configuration conf,
ApplicationAttemptId applicationAttemptId) |
static Path |
JobHistoryUtils.getStagingConfFile(Path logDir,
JobId jobId,
int attempt)
Gets the conf file path for jobs in progress.
|
static Path |
JobHistoryUtils.getStagingJobHistoryFile(Path dir,
JobId jobId,
int attempt)
Get the job history file path for non Done history files.
|
static Path |
JobHistoryUtils.getStagingJobHistoryFile(Path dir,
String jobId,
int attempt)
Get the job history file path for non Done history files.
|
Modifier and Type | Method and Description |
---|---|
static Path |
JobHistoryUtils.getStagingConfFile(Path logDir,
JobId jobId,
int attempt)
Gets the conf file path for jobs in progress.
|
static Path |
JobHistoryUtils.getStagingJobHistoryFile(Path dir,
JobId jobId,
int attempt)
Get the job history file path for non Done history files.
|
static Path |
JobHistoryUtils.getStagingJobHistoryFile(Path dir,
String jobId,
int attempt)
Get the job history file path for non Done history files.
|
static List<FileStatus> |
JobHistoryUtils.localGlobber(FileContext fc,
Path root,
String tail) |
static List<FileStatus> |
JobHistoryUtils.localGlobber(FileContext fc,
Path root,
String tail,
PathFilter filter) |
static List<FileStatus> |
JobHistoryUtils.localGlobber(FileContext fc,
Path root,
String tail,
PathFilter filter,
AtomicBoolean hasFlatFiles) |
Modifier and Type | Method and Description |
---|---|
static Path |
MRApps.getEndJobCommitFailureFile(Configuration conf,
String user,
JobId jobId) |
static Path |
MRApps.getEndJobCommitSuccessFile(Configuration conf,
String user,
JobId jobId) |
static Path |
MRApps.getStagingAreaDir(Configuration conf,
String user) |
static Path |
MRApps.getStartJobCommitFile(Configuration conf,
String user,
JobId jobId) |
Modifier and Type | Method and Description |
---|---|
static Credentials |
Credentials.readTokenStorageFile(Path filename,
Configuration conf)
Convenience method for reading a token storage file, and loading the Tokens
therein in the passed UGI
|
void |
Credentials.writeTokenStorageFile(Path filename,
Configuration conf) |
Modifier and Type | Method and Description |
---|---|
Path |
DistCpOptions.getAtomicWorkPath()
Get work path for atomic commit.
|
Path |
DistCpOptions.getLogPath()
Get output directory for writing distcp logs.
|
Path |
DistCpOptions.getSourceFileListing()
File path (hdfs:// or file://) that contains the list of actual
files to copy
|
Path |
DistCpOptions.getTargetPath()
Getter for the targetPath.
|
Modifier and Type | Method and Description |
---|---|
List<Path> |
DistCpOptions.getSourcePaths()
Getter for sourcePaths.
|
Modifier and Type | Method and Description |
---|---|
void |
CopyListing.buildListing(Path pathToListFile,
DistCpOptions options)
Build listing function creates the input listing that distcp uses to
perform the copy.
|
static void |
DistCpV1.copy(Configuration conf,
String srcPath,
String destPath,
Path logPath,
boolean srcAsList,
boolean ignoreReadFailures)
Deprecated.
|
void |
SimpleCopyListing.doBuildListing(Path pathToListingFile,
DistCpOptions options)
The interface to be implemented by sub-classes, to create the source/target file listing.
|
void |
FileBasedCopyListing.doBuildListing(Path pathToListFile,
DistCpOptions options)
Implementation of CopyListing::buildListing().
|
protected abstract void |
CopyListing.doBuildListing(Path pathToListFile,
DistCpOptions options)
The interface to be implemented by sub-classes, to create the source/target file listing.
|
void |
GlobbedCopyListing.doBuildListing(Path pathToListingFile,
DistCpOptions options)
Implementation of CopyListing::buildListing().
|
void |
DistCpOptions.setAtomicWorkPath(Path atomicWorkPath)
Set the work path for atomic commit
|
void |
DistCpOptions.setLogPath(Path logPath)
Set the log path where distcp output logs are stored
Uses JobStagingDir/_logs by default
|
Modifier and Type | Method and Description |
---|---|
void |
DistCpOptions.setSourcePaths(List<Path> sourcePaths)
Setter for sourcePaths.
|
Constructor and Description |
---|
DistCpOptions(List<Path> sourcePaths,
Path targetPath)
Constructor, to initialize source/target paths.
|
DistCpOptions(Path sourceFileListing,
Path targetPath)
Constructor, to initialize source/target paths.
|
Constructor and Description |
---|
DistCpOptions(List<Path> sourcePaths,
Path targetPath)
Constructor, to initialize source/target paths.
|
Modifier and Type | Method and Description |
---|---|
static Path |
CopyOutputFormat.getCommitDirectory(Job job)
Getter for the final commit-directory.
|
static Path |
CopyOutputFormat.getWorkingDirectory(Job job)
Getter for the working directory.
|
Modifier and Type | Method and Description |
---|---|
static void |
CopyOutputFormat.setCommitDirectory(Job job,
Path commitDirectory)
Setter for the final directory for DistCp (where files copied will be
moved, atomically.)
|
static void |
CopyOutputFormat.setWorkingDirectory(Job job,
Path workingDirectory)
Setter for the working directory for DistCp (where files will be copied
before they are moved to the final commit-directory.)
|
Constructor and Description |
---|
CopyCommitter(Path outputPath,
TaskAttemptContext context)
Create a output committer
|
Modifier and Type | Method and Description |
---|---|
void |
DefaultInputDemuxer.bindTo(Path path,
Configuration conf) |
void |
InputDemuxer.bindTo(Path path,
Configuration conf)
Bind the
InputDemuxer to a particular file. |
void |
DefaultOutputter.init(Path path,
Configuration conf) |
void |
Outputter.init(Path path,
Configuration conf)
Initialize the
Outputter to a specific path. |
Constructor and Description |
---|
ClusterTopologyReader(Path path,
Configuration conf)
Constructor.
|
JobTraceReader(Path path,
Configuration conf)
Constructor.
|
ZombieCluster(Path path,
MachineNode defaultNode,
Configuration conf)
Construct a homogeneous cluster.
|
ZombieJobProducer(Path path,
ZombieCluster cluster,
Configuration conf)
Constructor
|
ZombieJobProducer(Path path,
ZombieCluster cluster,
Configuration conf,
long randomSeed)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static Path |
DistCpUtils.sortListing(FileSystem fs,
Configuration conf,
Path sourceListing)
Sort sequence file containing FileStatus and Text as key and value respecitvely
|
Modifier and Type | Method and Description |
---|---|
static boolean |
DistCpUtils.checksumsAreEqual(FileSystem sourceFS,
Path source,
FileSystem targetFS,
Path target)
Utility to compare checksums for the paths specified.
|
static long |
DistCpUtils.getFileSize(Path path,
Configuration configuration)
Retrieves size of the file at the specified path.
|
static String |
DistCpUtils.getRelativePath(Path sourceRootPath,
Path childPath)
Gets relative path of child path with respect to a root path
For ex.
|
static void |
DistCpUtils.preserve(FileSystem targetFS,
Path path,
FileStatus srcFileStatus,
EnumSet<DistCpOptions.FileAttribute> attributes)
Preserve attribute on file matching that of the file status being sent
as argument.
|
static Path |
DistCpUtils.sortListing(FileSystem fs,
Configuration conf,
Path sourceListing)
Sort sequence file containing FileStatus and Text as key and value respecitvely
|
Modifier and Type | Method and Description |
---|---|
Path |
Options.PathOption.getValue() |
static Path[] |
StringUtils.stringToPath(String[] str) |
Modifier and Type | Method and Description |
---|---|
static void |
DiskChecker.checkDir(LocalFileSystem localFS,
Path dir,
FsPermission expected)
Create the local directory if necessary, check permissions and also ensure
it can be read from and written into.
|
static void |
DiskChecker.mkdirsWithExistsAndPermissionCheck(LocalFileSystem localFS,
Path dir,
FsPermission expected)
Create the directory or check permissions if it already exists.
|
Constructor and Description |
---|
Options.PathOption(Path value) |
Modifier and Type | Method and Description |
---|---|
static Path |
LogAggregationUtils.getRemoteAppLogDir(Path remoteRootLogDir,
ApplicationId appId,
String user,
String suffix)
Gets the remote app log dir.
|
static Path |
LogAggregationUtils.getRemoteLogSuffixedDir(Path remoteRootLogDir,
String user,
String suffix)
Gets the remote suffixed log dir for the user.
|
static Path |
LogAggregationUtils.getRemoteLogUserDir(Path remoteRootLogDir,
String user)
Gets the remote log user dir.
|
static Path |
LogAggregationUtils.getRemoteNodeLogFileForApp(Path remoteRootLogDir,
ApplicationId appId,
String user,
NodeId nodeId,
String suffix)
Constructs the full filename for an application's log file per node.
|
Modifier and Type | Method and Description |
---|---|
static Path |
LogAggregationUtils.getRemoteAppLogDir(Path remoteRootLogDir,
ApplicationId appId,
String user,
String suffix)
Gets the remote app log dir.
|
static Path |
LogAggregationUtils.getRemoteLogSuffixedDir(Path remoteRootLogDir,
String user,
String suffix)
Gets the remote suffixed log dir for the user.
|
static Path |
LogAggregationUtils.getRemoteLogUserDir(Path remoteRootLogDir,
String user)
Gets the remote log user dir.
|
static Path |
LogAggregationUtils.getRemoteNodeLogFileForApp(Path remoteRootLogDir,
ApplicationId appId,
String user,
NodeId nodeId,
String suffix)
Constructs the full filename for an application's log file per node.
|
Constructor and Description |
---|
AggregatedLogFormat.LogReader(Configuration conf,
Path remoteAppLogFile) |
AggregatedLogFormat.LogWriter(Configuration conf,
Path remoteAppLogFile,
UserGroupInformation userUgi) |
Modifier and Type | Method and Description |
---|---|
Path |
LocalDirsHandlerService.getLocalPathForWrite(String pathStr) |
Path |
LocalDirsHandlerService.getLocalPathForWrite(String pathStr,
long size,
boolean checkWrite) |
Path |
LocalDirsHandlerService.getLogPathForWrite(String pathStr,
boolean checkWrite) |
Path |
LocalDirsHandlerService.getLogPathToRead(String pathStr) |
protected Path |
ContainerExecutor.getPidFilePath(ContainerId containerId)
Get the pidFile of the container.
|
Path |
DeletionService.FileDeletionTask.getSubDir() |
Modifier and Type | Method and Description |
---|---|
List<Path> |
DeletionService.FileDeletionTask.getBaseDirs() |
Modifier and Type | Method and Description |
---|---|
void |
ContainerExecutor.activateContainer(ContainerId containerId,
Path pidFilePath)
Mark the container as active
|
DeletionService.FileDeletionTask |
DeletionService.createFileDeletionTask(String user,
Path subDir,
Path[] baseDirs)
Helper method to create file deletion task.
|
DeletionService.FileDeletionTask |
DeletionService.createFileDeletionTask(String user,
Path subDir,
Path[] baseDirs)
Helper method to create file deletion task.
|
void |
DeletionService.delete(String user,
Path subDir,
Path... baseDirs)
/**
Delete the path(s) as this user.
|
void |
DeletionService.delete(String user,
Path subDir,
Path... baseDirs)
/**
Delete the path(s) as this user.
|
void |
LinuxContainerExecutor.deleteAsUser(String user,
Path dir,
Path... baseDirs) |
void |
LinuxContainerExecutor.deleteAsUser(String user,
Path dir,
Path... baseDirs) |
abstract void |
ContainerExecutor.deleteAsUser(String user,
Path subDir,
Path... basedirs) |
abstract void |
ContainerExecutor.deleteAsUser(String user,
Path subDir,
Path... basedirs) |
void |
DefaultContainerExecutor.deleteAsUser(String user,
Path subDir,
Path... baseDirs) |
void |
DefaultContainerExecutor.deleteAsUser(String user,
Path subDir,
Path... baseDirs) |
int |
LinuxContainerExecutor.launchContainer(Container container,
Path nmPrivateCotainerScriptPath,
Path nmPrivateTokensPath,
String user,
String appId,
Path containerWorkDir,
List<String> localDirs,
List<String> logDirs) |
abstract int |
ContainerExecutor.launchContainer(Container container,
Path nmPrivateContainerScriptPath,
Path nmPrivateTokensPath,
String user,
String appId,
Path containerWorkDir,
List<String> localDirs,
List<String> logDirs)
Launch the container on the node.
|
int |
DefaultContainerExecutor.launchContainer(Container container,
Path nmPrivateContainerScriptPath,
Path nmPrivateTokensPath,
String userName,
String appId,
Path containerWorkDir,
List<String> localDirs,
List<String> logDirs) |
void |
LinuxContainerExecutor.startLocalizer(Path nmPrivateContainerTokensPath,
InetSocketAddress nmAddr,
String user,
String appId,
String locId,
List<String> localDirs,
List<String> logDirs) |
abstract void |
ContainerExecutor.startLocalizer(Path nmPrivateContainerTokens,
InetSocketAddress nmAddr,
String user,
String appId,
String locId,
List<String> localDirs,
List<String> logDirs)
Prepare the environment for containers in this application to execute.
|
void |
DefaultContainerExecutor.startLocalizer(Path nmPrivateContainerTokensPath,
InetSocketAddress nmAddr,
String user,
String appId,
String locId,
List<String> localDirs,
List<String> logDirs) |
Modifier and Type | Method and Description |
---|---|
Path |
ContainerResourceLocalizedEvent.getLocation() |
Modifier and Type | Method and Description |
---|---|
Map<Path,List<String>> |
ContainerImpl.getLocalizedResources() |
Map<Path,List<String>> |
Container.getLocalizedResources() |
Constructor and Description |
---|
ContainerResourceLocalizedEvent(ContainerId container,
LocalResourceRequest rsrc,
Path loc) |
Modifier and Type | Method and Description |
---|---|
void |
ContainerLaunch.sanitizeEnv(Map<String,String> environment,
Path pwd,
List<Path> appDirs,
List<String> containerLogDirs,
Map<Path,List<String>> resources) |
Modifier and Type | Method and Description |
---|---|
void |
ContainerLaunch.sanitizeEnv(Map<String,String> environment,
Path pwd,
List<Path> appDirs,
List<String> containerLogDirs,
Map<Path,List<String>> resources) |
void |
ContainerLaunch.sanitizeEnv(Map<String,String> environment,
Path pwd,
List<Path> appDirs,
List<String> containerLogDirs,
Map<Path,List<String>> resources) |
Modifier and Type | Method and Description |
---|---|
Path |
LocalizedResource.getLocalPath() |
Path |
LocalResourceRequest.getPath() |
Modifier and Type | Method and Description |
---|---|
protected void |
ContainerLocalizer.localizeFiles(LocalizationProtocol nodemanager,
CompletionService<Path> cs,
UserGroupInformation ugi) |
Constructor and Description |
---|
ContainerLocalizer(FileContext lfs,
String user,
String appId,
String localizerId,
List<Path> localDirs,
RecordFactory recordFactory) |
Modifier and Type | Method and Description |
---|---|
Path |
ResourceLocalizedEvent.getLocation() |
Constructor and Description |
---|
ResourceLocalizedEvent(LocalResourceRequest rsrc,
Path location,
long size) |
Constructor and Description |
---|
AppLogAggregatorImpl(Dispatcher dispatcher,
DeletionService deletionService,
Configuration conf,
ApplicationId appId,
UserGroupInformation userUgi,
LocalDirsHandlerService dirsHandler,
Path remoteNodeLogFileForApp,
ContainerLogsRetentionPolicy retentionPolicy,
Map<ApplicationAccessType,String> appAcls) |
Modifier and Type | Method and Description |
---|---|
static String |
ProcessIdFileReader.getProcessId(Path path)
Get the process id from specified file path.
|
static ResourceLocalizationSpec |
NodeManagerBuilderUtils.newResourceLocalizationSpec(LocalResource rsrc,
Path path) |
Modifier and Type | Method and Description |
---|---|
Path |
FSDownload.call() |
static Path |
ConverterUtils.getPathFromYarnURL(URL url)
return a hadoop path from a given url
|
Modifier and Type | Method and Description |
---|---|
static URL |
ConverterUtils.getYarnUrlFromPath(Path path) |
Constructor and Description |
---|
FSDownload(FileContext files,
UserGroupInformation ugi,
Configuration conf,
Path destDirPath,
LocalResource resource) |
Copyright © 2013 Apache Software Foundation. All rights reserved.