@InterfaceAudience.LimitedPrivate(value="HDFS") @InterfaceStability.Unstable public abstract class FSInputChecker extends FSInputStream
Modifier and Type | Field and Description |
---|---|
protected static int |
CHECKSUM_SIZE |
protected Path |
file
The file name from which data is read from
|
static org.apache.commons.logging.Log |
LOG |
Modifier | Constructor and Description |
---|---|
protected |
FSInputChecker(Path file,
int numOfRetries)
Constructor
|
protected |
FSInputChecker(Path file,
int numOfRetries,
boolean verifyChecksum,
Checksum sum,
int chunkSize,
int checksumSize)
Constructor
|
Modifier and Type | Method and Description |
---|---|
int |
available() |
static long |
checksum2long(byte[] checksum)
Deprecated.
|
protected abstract long |
getChunkPosition(long pos)
Return position of beginning of chunk containing pos.
|
long |
getPos()
Return the current offset from the start of the file
|
void |
mark(int readlimit) |
boolean |
markSupported() |
protected boolean |
needChecksum()
Return true if there is a need for checksum verification
|
int |
read()
Read one checksum-verified byte
|
int |
read(byte[] b,
int off,
int len)
Read checksum verified bytes from this byte-input stream into
the specified byte array, starting at the given offset.
|
protected abstract int |
readChunk(long pos,
byte[] buf,
int offset,
int len,
byte[] checksum)
Reads in checksum chunks into
buf at offset
and checksum into checksum . |
protected static int |
readFully(InputStream stm,
byte[] buf,
int offset,
int len)
A utility function that tries to read up to
len bytes from
stm |
void |
reset() |
void |
seek(long pos)
Seek to the given position in the stream.
|
protected void |
set(boolean verifyChecksum,
Checksum sum,
int maxChunkSize,
int checksumSize)
Set the checksum related parameters
|
long |
skip(long n)
Skips over and discards
n bytes of data from the
input stream. |
read, readFully, readFully, seekToNewSource
close, read
public static final org.apache.commons.logging.Log LOG
protected Path file
protected static final int CHECKSUM_SIZE
protected FSInputChecker(Path file, int numOfRetries)
file
- The name of the file to be readnumOfRetries
- Number of read retries when ChecksumError occursprotected FSInputChecker(Path file, int numOfRetries, boolean verifyChecksum, Checksum sum, int chunkSize, int checksumSize)
file
- The name of the file to be readnumOfRetries
- Number of read retries when ChecksumError occurssum
- the type of Checksum enginechunkSize
- maximun chunk sizechecksumSize
- the number byte of each checksumprotected abstract int readChunk(long pos, byte[] buf, int offset, int len, byte[] checksum) throws IOException
buf
at offset
and checksum into checksum
.
Since checksums can be disabled, there are two cases implementors need
to worry about:
(a) needChecksum() will return false:
- len can be any positive value
- checksum will be null
Implementors should simply pass through to the underlying data stream.
or
(b) needChecksum() will return true:
- len >= maxChunkSize
- checksum.length is a multiple of CHECKSUM_SIZE
Implementors should read an integer number of data chunks into
buf. The amount read should be bounded by len or by
checksum.length / CHECKSUM_SIZE * maxChunkSize. Note that len may
be a value that is not a multiple of maxChunkSize, in which case
the implementation may return less than len.
The method is used for implementing read, therefore, it should be optimized
for sequential reading.pos
- chunkPosbuf
- desitination bufferoffset
- offset in buf at which to store datalen
- maximum number of bytes to readchecksum
- the data buffer into which to write checksumsIOException
protected abstract long getChunkPosition(long pos)
pos
- a postion in the fileprotected boolean needChecksum()
public int read() throws IOException
read
in class InputStream
-1
if the end of the
stream is reached.IOException
- if an I/O error occurs.public int read(byte[] b, int off, int len) throws IOException
This method implements the general contract of the corresponding
method of
the read
class. As an additional
convenience, it attempts to read as many bytes as possible by repeatedly
invoking the InputStream
read
method of the underlying stream. This
iterated read
continues until one of the following
conditions becomes true:
read
method of the underlying stream returns
-1
, indicating end-of-file.
read
on the underlying stream returns
-1
to indicate end-of-file then this method returns
-1
. Otherwise this method returns the number of bytes
actually read.read
in class InputStream
b
- destination buffer.off
- offset at which to start storing bytes.len
- maximum number of bytes to read.-1
if the end of
the stream has been reached.IOException
- if an I/O error occurs.
ChecksumException if any checksum error occurs@Deprecated public static long checksum2long(byte[] checksum)
public long getPos() throws IOException
FSInputStream
getPos
in interface Seekable
getPos
in class FSInputStream
IOException
public int available() throws IOException
available
in class InputStream
IOException
public long skip(long n) throws IOException
n
bytes of data from the
input stream.
This method may skip more bytes than are remaining in the backing file. This produces no exception and the number of bytes skipped may include some number of bytes that were beyond the EOF of the backing file. Attempting to read from the stream after skipping past the end will result in -1 indicating the end of the file.
If n
is negative, no bytes are skipped.
skip
in class InputStream
n
- the number of bytes to be skipped.IOException
- if an I/O error occurs.
ChecksumException if the chunk to skip to is corruptedpublic void seek(long pos) throws IOException
This method may seek past the end of the file. This produces no exception and an attempt to read from the stream will result in -1 indicating the end of the file.
seek
in interface Seekable
seek
in class FSInputStream
pos
- the postion to seek to.IOException
- if an I/O error occurs.
ChecksumException if the chunk to seek to is corruptedprotected static int readFully(InputStream stm, byte[] buf, int offset, int len) throws IOException
len
bytes from
stm
stm
- an input streambuf
- destiniation bufferoffset
- offset at which to store datalen
- number of bytes to readIOException
- if there is any IO errorprotected final void set(boolean verifyChecksum, Checksum sum, int maxChunkSize, int checksumSize)
verifyChecksum
- whether to verify checksumsum
- which type of checksum to usemaxChunkSize
- maximun chunk sizechecksumSize
- checksum sizepublic final boolean markSupported()
markSupported
in class InputStream
public final void mark(int readlimit)
mark
in class InputStream
public final void reset() throws IOException
reset
in class InputStream
IOException
Copyright © 2013 Apache Software Foundation. All rights reserved.