public class StoreDirect extends Store
HEADER
| File header, format version and flags
1 | IO_INDEX_SIZE
| Allocated file size of index file in bytes.
2 | IO_PHYS_SIZE
| Allocated file size of physical file in bytes.
3 | IO_FREE_SIZE
| Space occupied by free records in physical file in bytes.
4 | IO_INDEX_SUM
| Checksum of all Index file headers. Checks if store was closed correctly
5..9 | | Reserved for future use
10..14 | | For usage by user
15 | IO_FREE_RECID
|Long Stack of deleted recids, those will be reused and returned by Engine.put(Object, Serializer)
16..4111 | |Long Stack of free physical records. This contains free space released by record update or delete. Each slots corresponds to free record size. TODO check 4111 is right
4112 | IO_USER_START
|Record size and offset in physical file for recid=1
4113 | |Record size and offset in physical file for recid=2
... | ... |... snip ...
N+4111 | |Record size and offset in physical file for recid=N
Long Stack
------------
Long Stack is data structure used to store free records. It is LIFO queue which uses linked records to store 8-byte longs.
Long Stack is identified by slot in Index File, which stores pointer to Long Stack head. The structure of
of index pointer is following:
byte | description
--- |---
0..1 | relative offset in head Long Stack Record to take value from. This value decreases by 8 each take
2..7 | physical file offset of head Long Stack Record, zero if Long Stack is empty
Each Long Stack Record is sequence of 8-byte longs, first slot is header. Long Stack Record structure is following:
byte | description
--- |---
0..1 | length of current Long Stack Record in bytes
2..7 | physical file offset of next Long Stack Record, zero of this record is last
8-15 | Long Stack value
16-23 | Long Stack value
... | and so on until end of Long Stack Record
Physical pointer
----------------
Index slot value typically contains physical pointer (information about record location and size in physical file). First 2 bytes
are record size (max 65536). Then there is 6 byte offset in physical file (max store size is 281 TB).
Physical file offset must always be multiple of 16, so last 4 bites are used to flag extra record information.
Structure of **physical pointer**:
bite | in code | description
--- | --- | ---
0-15 |`val>>>48` | record size
16-59 |`val&MASK_OFFSET
` | physical offset
60 |`val&MASK_LINKED
!=0` | linked record flag
61 |`val&MASK_DISCARD
!=0` | to be discarded while storage is offline flag
62 |`val&MASK_ARCHIVE
!=0` | record modified since last backup flag
63 | | not used yet
Records in Physical File
---------------------------
Records are stored in physical file. Maximal record size size is 64KB, so larger records must
be stored in form of the linked list. Each record starts by Physical Pointer from Index File.
There is flag in Physical Pointer indicating if record is linked. If record is not linked you may
just read ByteBuffer from given size and offset.
If record is linked, each record starts with Physical Pointer to next record. So actual data payload is record size-8.
The last linked record does not have the Physical Pointer header to next record, there is MASK_LINKED flag which
indicates if next record is the last one.Modifier and Type | Field and Description |
---|---|
static String |
DATA_FILE_EXT |
protected boolean |
deleteFilesAfterClose |
protected long |
freeSize |
protected static int |
HEADER
4 byte file header
|
protected Volume |
index |
protected long |
indexSize |
protected static int |
IO_FREE_RECID
index file offset where reference to longstack of free recid is stored
|
protected static int |
IO_FREE_SIZE
index file offset where space occupied by free phys records is stored
|
protected static int |
IO_INDEX_SIZE
index file offset where current size of index file is stored
|
protected static int |
IO_INDEX_SUM
checksum of all index file headers.
|
protected static int |
IO_PHYS_SIZE
index file offset where current size of phys file is stored
|
protected static int |
IO_USER_START
index file offset where first recid available to user is stored
|
protected static int |
LONG_STACK_PREF_COUNT |
protected static int |
LONG_STACK_PREF_COUNT_ALTER |
protected static long |
LONG_STACK_PREF_SIZE |
protected static long |
LONG_STACK_PREF_SIZE_ALTER |
protected static long |
MASK_ARCHIVE |
protected static long |
MASK_DISCARD |
protected static long |
MASK_LINKED |
protected static long |
MASK_OFFSET |
protected static int |
MAX_REC_SIZE
maximal non linked record size
|
protected long |
maxUsedIoList
maximal non zero slot in free phys record, access requires `structuralLock`
|
protected Volume |
phys |
protected static int |
PHYS_FREE_SLOTS_COUNT
number of free physical slots
|
protected long |
physSize |
protected boolean |
readOnly |
protected long |
sizeLimit |
protected boolean |
spaceReclaimReuse |
protected boolean |
spaceReclaimSplit |
protected boolean |
spaceReclaimTrack |
protected static short |
STORE_VERSION
2 byte store version
|
protected boolean |
syncOnCommitDisabled |
checksum, CHECKSUM_FLAG_MASK, CHUNK_SIZE, CHUNK_SIZE_MOD_MASK, compress, COMPRESS_FLAG_MASK, encrypt, ENCRYPT_FLAG_MASK, encryptionXTEA, locks, LOG, LZF, newRecidLock, password, recycledDataOuts, serializerPojo, serializerPojoInitLock, structuralLock
CATALOG_RECID, CHECK_RECORD, CLASS_INFO_RECID, LAST_RESERVED_RECID
Constructor and Description |
---|
StoreDirect(Volume.Factory volFac) |
StoreDirect(Volume.Factory volFac,
boolean readOnly,
boolean deleteFilesAfterClose,
int spaceReclaimMode,
boolean syncOnCommitDisabled,
long sizeLimit,
boolean checksum,
boolean compress,
byte[] password,
boolean disableLocks,
int sizeIncrement) |
Modifier and Type | Method and Description |
---|---|
String |
calculateStatistics()
get some statistics about store.
|
boolean |
canRollback() |
protected void |
checkHeaders() |
void |
clearCache()
clears any underlying cache
|
void |
close()
Close store/cache.
|
void |
commit()
Makes all changes made since the previous commit/rollback permanent.
|
void |
compact() |
protected void |
compactPostUnderLock()
subclasses put additional cleanup after compaction finishes here
|
protected void |
compactPreUnderLock()
subclasses put additional checks before compaction starts here
|
<A> boolean |
compareAndSwap(long recid,
A expectedOldValue,
A newValue,
Serializer<A> serializer)
Updates existing record in atomic (Compare And Swap) manner.
|
protected long |
countLongStackItems(long ioList) |
protected void |
createStructure() |
<A> void |
delete(long recid,
Serializer<A> serializer)
Remove existing record from store/cache
Recid must be a number returned by 'put' method.
|
protected void |
freeIoRecidPut(long ioRecid) |
protected long |
freeIoRecidTake(boolean ensureAvail) |
protected void |
freePhysPut(long indexVal,
boolean recursive) |
protected long |
freePhysTake(int size,
boolean ensureAvail,
boolean recursive) |
<A> A |
get(long recid,
Serializer<A> serializer)
Get existing record.
|
protected <A> A |
get2(long ioRecid,
Serializer<A> serializer) |
long |
getCurrSize()
returns current size occupied by physical store (does not include index).
|
Iterator<Long> |
getFreeRecids() |
long |
getFreeSize()
returns free size in physical store (does not include index).
|
protected long[] |
getLinkedRecordsIndexVals(long indexVal) |
long |
getMaxRecid() |
ByteBuffer |
getRaw(long recid) |
long |
getSizeLimit()
returns maximal store size or `0` if there is no limit
|
protected long |
indexHeaderChecksum() |
boolean |
isClosed()
Checks whether Engine was closed.
|
boolean |
isReadOnly()
Check if you can write into this Engine.
|
protected void |
longStackPut(long ioList,
long offset,
boolean recursive) |
protected long |
longStackTake(long ioList,
boolean recursive) |
protected long[] |
physAllocate(int size,
boolean ensureAvail,
boolean recursive) |
long |
preallocate()
Preallocates recid for not yet created record.
|
void |
preallocate(long[] recids)
Preallocates recids for not yet created record.
|
<A> long |
put(A value,
Serializer<A> serializer)
Insert new record.
|
protected void |
put2(DataOutput2 out,
long ioRecid,
long[] indexVals) |
void |
rollback()
Undoes all changes made in the current transaction.
|
protected static long |
roundTo16(long offset) |
protected static long |
size2ListIoRecid(long size) |
<A> void |
update(long recid,
A value,
Serializer<A> serializer)
Update existing record with new value.
|
protected void |
update2(DataOutput2 out,
long ioRecid) |
void |
updateRaw(long recid,
ByteBuffer data) |
canSnapshot, closeListenerRegister, closeListenerUnregister, deserialize, expectedMasks, forDB, forEngine, getSerializerPojo, lockAllWrite, lockPos, newDataOut2, printStatistics, serialize, snapshot, unlockAllWrite
protected static final long MASK_OFFSET
protected static final long MASK_LINKED
protected static final long MASK_DISCARD
protected static final long MASK_ARCHIVE
protected static final int HEADER
protected static final short STORE_VERSION
protected static final int MAX_REC_SIZE
protected static final int PHYS_FREE_SLOTS_COUNT
protected static final int IO_INDEX_SIZE
protected static final int IO_PHYS_SIZE
protected static final int IO_FREE_SIZE
protected static final int IO_INDEX_SUM
protected static final int IO_FREE_RECID
protected static final int IO_USER_START
public static final String DATA_FILE_EXT
protected static final int LONG_STACK_PREF_COUNT
protected static final long LONG_STACK_PREF_SIZE
protected static final int LONG_STACK_PREF_COUNT_ALTER
protected static final long LONG_STACK_PREF_SIZE_ALTER
protected Volume index
protected Volume phys
protected long physSize
protected long indexSize
protected long freeSize
protected final boolean deleteFilesAfterClose
protected final boolean readOnly
protected final boolean syncOnCommitDisabled
protected final boolean spaceReclaimReuse
protected final boolean spaceReclaimSplit
protected final boolean spaceReclaimTrack
protected final long sizeLimit
protected long maxUsedIoList
public StoreDirect(Volume.Factory volFac, boolean readOnly, boolean deleteFilesAfterClose, int spaceReclaimMode, boolean syncOnCommitDisabled, long sizeLimit, boolean checksum, boolean compress, byte[] password, boolean disableLocks, int sizeIncrement)
public StoreDirect(Volume.Factory volFac)
protected void checkHeaders()
protected void createStructure()
protected long indexHeaderChecksum()
public long preallocate()
Engine
public void preallocate(long[] recids)
Engine
recids
- array to put result intopublic <A> long put(A value, Serializer<A> serializer)
Engine
value
- records to be addedserializer
- used to convert record into/from binary formprotected void put2(DataOutput2 out, long ioRecid, long[] indexVals)
public <A> A get(long recid, Serializer<A> serializer)
Engine
recid
- (record identifier) under which record was persistedserializer
- used to deserialize record from binary formprotected <A> A get2(long ioRecid, Serializer<A> serializer) throws IOException
IOException
public <A> void update(long recid, A value, Serializer<A> serializer)
Engine
recid
- (record identifier) under which record was persisted.value
- new record value to be storedserializer
- used to serialize record into binary formprotected void update2(DataOutput2 out, long ioRecid)
public <A> boolean compareAndSwap(long recid, A expectedOldValue, A newValue, Serializer<A> serializer)
Engine
oldValue==expectedOldValue
when old value is found in instance cacheoldValue
using serializer
and checking oldValue.equals(expectedOldValue)
expectedOldValue
using serializer
and comparing binary array with already serialized oldValue
recid
- (record identifier) under which record was persisted.expectedOldValue
- old value to be compared with existing recordnewValue
- to be written if values are matchingserializer
- used to serialize record into binary formpublic <A> void delete(long recid, Serializer<A> serializer)
Engine
recid
- (record identifier) under which was record persistedserializer
- which may be used in some circumstances to deserialize and store old objectprotected long[] getLinkedRecordsIndexVals(long indexVal)
protected long[] physAllocate(int size, boolean ensureAvail, boolean recursive)
protected static long roundTo16(long offset)
public void close()
Engine
NullPointerException
There is an configuration option DBMaker.closeOnJvmShutdown()
which uses shutdown hook to automatically
close Engine when JVM shutdowns.public boolean isClosed()
Engine
public void commit()
Engine
public void rollback() throws UnsupportedOperationException
Engine
UnsupportedOperationException
.UnsupportedOperationException
- if transactions are disabledpublic boolean isReadOnly()
Engine
public boolean canRollback()
public void clearCache()
Engine
public void compact()
protected void compactPreUnderLock()
protected void compactPostUnderLock()
protected long longStackTake(long ioList, boolean recursive)
protected void longStackPut(long ioList, long offset, boolean recursive)
protected void freeIoRecidPut(long ioRecid)
protected long freeIoRecidTake(boolean ensureAvail)
protected static long size2ListIoRecid(long size)
protected void freePhysPut(long indexVal, boolean recursive)
protected long freePhysTake(int size, boolean ensureAvail, boolean recursive)
public long getMaxRecid()
getMaxRecid
in class Store
public ByteBuffer getRaw(long recid)
public Iterator<Long> getFreeRecids()
getFreeRecids
in class Store
public void updateRaw(long recid, ByteBuffer data)
public long getSizeLimit()
Store
getSizeLimit
in class Store
public long getCurrSize()
Store
getCurrSize
in class Store
public long getFreeSize()
Store
getFreeSize
in class Store
public String calculateStatistics()
Store
calculateStatistics
in class Store
protected long countLongStackItems(long ioList)
Copyright © 2016. All rights reserved.