public class DeltaBinaryPackingValuesWriter extends ValuesWriter
delta-binary-packing: <page-header> <block>*
page-header := <block size in values> <number of miniblocks in a block> <total value count> <first value>
block := <min delta> <list of bitwidths of miniblocks> <miniblocks>
min delta : zig-zag var int encoded
bitWidthsOfMiniBlock : 1 byte little endian
blockSizeInValues,blockSizeInValues,totalValueCount,firstValue : unsigned varint
The algorithm and format is inspired by D. Lemire's paper: http://lemire.me/blog/archives/2012/09/12/fast-integer-compression-decoding-billions-of-integers-per-second/Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_NUM_BLOCK_VALUES |
static int |
DEFAULT_NUM_MINIBLOCKS |
static int |
MAX_BITWIDTH
max bitwidth for a mini block, it is used to allocate miniBlockByteBuffer which is
reused between flushes.
|
Constructor and Description |
---|
DeltaBinaryPackingValuesWriter(int slabSize) |
DeltaBinaryPackingValuesWriter(int blockSizeInValues,
int miniBlockNum,
int slabSize) |
Modifier and Type | Method and Description |
---|---|
long |
getAllocatedSize() |
long |
getBufferedSize()
used to decide if we want to work to the next page
|
BytesInput |
getBytes()
getBytes will trigger flushing block buffer, DO NOT write after getBytes() is called without calling reset()
|
Encoding |
getEncoding()
called after getBytes() and before reset()
|
String |
memUsageString(String prefix) |
void |
reset()
called after getBytes() to reset the current buffer and start writing the next page
|
void |
writeInteger(int v) |
createDictionaryPage, resetDictionary, writeBoolean, writeByte, writeBytes, writeDouble, writeFloat, writeLong
public static final int MAX_BITWIDTH
public static final int DEFAULT_NUM_BLOCK_VALUES
public static final int DEFAULT_NUM_MINIBLOCKS
public DeltaBinaryPackingValuesWriter(int slabSize)
public DeltaBinaryPackingValuesWriter(int blockSizeInValues, int miniBlockNum, int slabSize)
public long getBufferedSize()
ValuesWriter
getBufferedSize
in class ValuesWriter
public void writeInteger(int v)
writeInteger
in class ValuesWriter
public BytesInput getBytes()
getBytes
in class ValuesWriter
public Encoding getEncoding()
ValuesWriter
getEncoding
in class ValuesWriter
public void reset()
ValuesWriter
reset
in class ValuesWriter
public long getAllocatedSize()
getAllocatedSize
in class ValuesWriter
public String memUsageString(String prefix)
memUsageString
in class ValuesWriter
Copyright © 2015. All rights reserved.