public class FileBasedCopyListing extends CopyListing
Constructor and Description |
---|
FileBasedCopyListing(Configuration configuration,
Credentials credentials)
Constructor, to initialize base-class.
|
Modifier and Type | Method and Description |
---|---|
void |
doBuildListing(Path pathToListFile,
DistCpOptions options)
Implementation of CopyListing::buildListing().
|
protected long |
getBytesToCopy()
Return the total bytes that distCp should copy for the source paths
This doesn't consider whether file is same should be skipped during copy
|
protected long |
getNumberOfPaths()
Return the total number of paths to distcp, includes directories as well
This doesn't consider whether file/dir is already present and should be skipped during copy
|
protected void |
validatePaths(DistCpOptions options)
Validate input and output paths
|
buildListing, getCopyListing, getCredentials, setCredentials
getConf, setConf
public FileBasedCopyListing(Configuration configuration, Credentials credentials)
configuration
- The input Configuration object.credentials
- - Credentials object on which the FS delegation tokens are cached. If null
delegation token caching is skippedprotected void validatePaths(DistCpOptions options) throws IOException, org.apache.hadoop.tools.CopyListing.InvalidInputException
validatePaths
in class CopyListing
options
- - Input optionsIOException
org.apache.hadoop.tools.CopyListing.InvalidInputException
public void doBuildListing(Path pathToListFile, DistCpOptions options) throws IOException
doBuildListing
in class CopyListing
pathToListFile
- Path on HDFS where the listing file is written.options
- Input Options for DistCp (indicating source/target paths.)IOException
protected long getBytesToCopy()
getBytesToCopy
in class CopyListing
protected long getNumberOfPaths()
getNumberOfPaths
in class CopyListing
Copyright © 2013 Apache Software Foundation. All rights reserved.