public class BookKeeperJournalManager extends Object implements JournalManager
<property>
<name>dfs.namenode.edits.dir</name>
<value>bookkeeper://zk1:2181;zk2:2181;zk3:2181/hdfsjournal</value>
</property>
<property>
<name>dfs.namenode.edits.journal-plugin.bookkeeper</name>
<value>org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager</value>
</property>
The URI format for bookkeeper is bookkeeper://[zkEnsemble]/[rootZnode]
[zookkeeper ensemble] is a list of semi-colon separated, zookeeper host:port
pairs. In the example above there are 3 servers, in the ensemble,
zk1, zk2 & zk3, each one listening on port 2181.
[root znode] is the path of the zookeeper znode, under which the editlog
information will be stored.
Other configuration options are:
JournalManager.CorruptionException
Modifier and Type | Field and Description |
---|---|
static String |
BKJM_BOOKKEEPER_DIGEST_PW |
static String |
BKJM_BOOKKEEPER_DIGEST_PW_DEFAULT |
static String |
BKJM_BOOKKEEPER_ENSEMBLE_SIZE |
static int |
BKJM_BOOKKEEPER_ENSEMBLE_SIZE_DEFAULT |
static String |
BKJM_BOOKKEEPER_QUORUM_SIZE |
static int |
BKJM_BOOKKEEPER_QUORUM_SIZE_DEFAULT |
static String |
BKJM_OUTPUT_BUFFER_SIZE |
static int |
BKJM_OUTPUT_BUFFER_SIZE_DEFAULT |
static String |
BKJM_ZK_LEDGERS_AVAILABLE_PATH |
static String |
BKJM_ZK_LEDGERS_AVAILABLE_PATH_DEFAULT |
static String |
BKJM_ZK_SESSION_TIMEOUT |
static int |
BKJM_ZK_SESSION_TIMEOUT_DEFAULT |
Constructor and Description |
---|
BookKeeperJournalManager(Configuration conf,
URI uri,
NamespaceInfo nsInfo)
Construct a Bookkeeper journal manager.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Close the journal manager, freeing any resources it may hold.
|
void |
finalizeLogSegment(long firstTxId,
long lastTxId)
Finalize a log segment.
|
void |
format(NamespaceInfo ns)
Format the underlying storage, removing any previously
stored data.
|
boolean |
hasSomeData() |
void |
purgeLogsOlderThan(long minTxIdToKeep)
Remove all edit logs with transaction IDs lower than the given transaction
ID.
|
void |
recoverUnfinalizedSegments()
Recover segments which have not been finalized.
|
void |
selectInputStreams(Collection<EditLogInputStream> streams,
long fromTxId,
boolean inProgressOk) |
void |
selectInputStreams(Collection<EditLogInputStream> streams,
long fromTxId,
boolean inProgressOk,
boolean forReading)
Get a list of edit log input streams.
|
void |
setOutputBufferCapacity(int size)
Set the amount of memory that this stream should use to buffer edits.
|
EditLogOutputStream |
startLogSegment(long txId)
Start a new log segment in a BookKeeper ledger.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
toString
public static final String BKJM_OUTPUT_BUFFER_SIZE
public static final int BKJM_OUTPUT_BUFFER_SIZE_DEFAULT
public static final String BKJM_BOOKKEEPER_ENSEMBLE_SIZE
public static final int BKJM_BOOKKEEPER_ENSEMBLE_SIZE_DEFAULT
public static final String BKJM_BOOKKEEPER_QUORUM_SIZE
public static final int BKJM_BOOKKEEPER_QUORUM_SIZE_DEFAULT
public static final String BKJM_BOOKKEEPER_DIGEST_PW
public static final String BKJM_BOOKKEEPER_DIGEST_PW_DEFAULT
public static final String BKJM_ZK_SESSION_TIMEOUT
public static final int BKJM_ZK_SESSION_TIMEOUT_DEFAULT
public static final String BKJM_ZK_LEDGERS_AVAILABLE_PATH
public static final String BKJM_ZK_LEDGERS_AVAILABLE_PATH_DEFAULT
public BookKeeperJournalManager(Configuration conf, URI uri, NamespaceInfo nsInfo) throws IOException
IOException
public void format(NamespaceInfo ns) throws IOException
JournalManager
format
in interface JournalManager
IOException
public boolean hasSomeData() throws IOException
hasSomeData
in interface Storage.FormatConfirmable
IOException
- if the storage cannot be accessed at all.public EditLogOutputStream startLogSegment(long txId) throws IOException
startLogSegment
in interface JournalManager
txId
- First transaction id to be written to the streamIOException
public void finalizeLogSegment(long firstTxId, long lastTxId) throws IOException
finalizeLogSegment
in interface JournalManager
IOException
public void selectInputStreams(Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk) throws IOException
IOException
public void selectInputStreams(Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean forReading) throws IOException
fromTxId
- the first transaction id we want to readinProgressOk
- whether or not in-progress streams should be returnedforReading
- whether or not the caller intends to read from the edit logsIOException
- if the underlying storage has an error or is otherwise
inaccessiblepublic void recoverUnfinalizedSegments() throws IOException
JournalManager
recoverUnfinalizedSegments
in interface JournalManager
IOException
public void purgeLogsOlderThan(long minTxIdToKeep) throws IOException
minTxIdToKeep
- the lowest transaction ID that should be retainedIOException
- in the event of errorpublic void close() throws IOException
JournalManager
close
in interface Closeable
close
in interface AutoCloseable
close
in interface JournalManager
IOException
public void setOutputBufferCapacity(int size)
setOutputBufferCapacity
in interface JournalManager
Copyright © 2013 Apache Software Foundation. All rights reserved.