Class IndexStore
- java.lang.Object
-
- com.mckoi.database.IndexStore
-
public final class IndexStore extends java.lang.Object
A class that manages the storage of a set of transactional index lists in a way that is fast to modify. This class has a number of objectives;- To prevent corruption as best as possible.
- To be able to modify lists of integers very fast and persistantly.
- To have size optimization features such as defragmentation.
- To provide very fast searches on sorted lists (caching features).
- To be able to map a list to an IntegerListInterface interface.
None intuitively, this object also handles unique ids.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private class
IndexStore.IndexIntegerList
The IntegerListInterface implementation that is used to represent a mutable snapshop of the indices at a given point in time.private class
IndexStore.MappedListBlock
An IntegerListBlockInterface implementation that maps a block of a list to an underlying file system representation.private class
IndexStore.SnapshotIndexSet
The implementation of IndexSet which represents a mutable snapshot of the indices stored in this set.
-
Field Summary
Fields Modifier and Type Field Description private int
allocation_length
The current of the allocation information.private int
allocation_sector
The start sector where the block allocation information is currently stored.private int
block_size
The size of a 'block' element of index information in a list.private DebugLogger
debug
A DebugLogger object used to log debug messages to.private static IndexStore.IndexIntegerList[]
EMPTY_INTEGER_LISTS
A convenience static empty integer list array.private java.io.File
file
The name of the index store file.private byte[]
flush_buffer
private java.util.ArrayList
index_set_garbage
The list of SnapshotIndexSet objects that have been deleted and are ready for garbage collection.private FixedSizeDataStore
index_store
The FixedSizeDataStore that contains all the data of the index store.private ByteBuffer
index_table_list
The list of table sector entries that are currently committed.private byte[]
index_table_list_buf
private java.util.ArrayList
memory_index_set_list
The list of SnapshotIndexSet objects returned via the 'getSnapshotIndexSet' method.private Cache
sector_cache
A cache of int[] array blocks that are accessed by this store.private long
SET_ID_KEY
A unique key that is incremented each time a new IndexSet object is created.private long
unique_id
Unique id field that contains a unique number that can be incremented atomically.
-
Constructor Summary
Constructors Constructor Description IndexStore(java.io.File file_name, DebugLogger logger)
Constructs the IndexStore.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addIndexLists(int count, byte type)
Adds a number of blank index tables to the index store.private void
addIndexSetToList(IndexSet index_set)
Adds a SnapshotIndexSet to the list of sets that this store has dispatched.void
close()
Cleanly closes the index store.void
commitIndexSet(IndexSet index_set)
Commits changes made to a snapshop of an IndexSet as being permanent changes to the state of the index store.void
copyTo(java.io.File path)
Copies the persistant part of this to another store.void
create(int block_size)
Creates a new black index store and returns leaving the newly created store in an open state.(package private) long
currentUniqueID()
The current unique id.void
delete()
Deletes the store.boolean
exists()
Returns true if the index store file exists.boolean
fix(UserTerminal terminal)
Performs checks to determine that the index store is stable.void
flush()
Flushes all information in this index store to the file representing this store in the file system.(package private) int
getBlockSize()
Returns the block size of this store.IndexSet
getSnapshotIndexSet()
Returns a current snapshot of the current indexes that are committed in this store.void
hardSynch()
Performs a hard synchronization of this index store.void
init()
Initializes the IndexStore.private void
initBlank()
Initializes the index store to a blank state.boolean
isReadOnly()
Returns true if this store is read only.(package private) long
nextUniqueID()
Atomically returns the next 'unique_id' value from this file.boolean
open(boolean read_only)
Opens this index store.private void
readIndexTableList()
Reads the index table allocation list in to the ByteBuffer object.private void
removeIndexSetFromList(IndexSet index_set)
Removes a SnapshotIndexSet from the list of sets that this store is managing.(package private) void
setUniqueID(long value)
Sets the unique id for this store.java.lang.String
statusString()
Returns a string that contains diagnostic information.
-
-
-
Field Detail
-
debug
private DebugLogger debug
A DebugLogger object used to log debug messages to.
-
file
private java.io.File file
The name of the index store file.
-
block_size
private int block_size
The size of a 'block' element of index information in a list. This has a direct relation to the size of the sectors in the store. This value can be tuned for specific tables. For example, a table that will only ever contain a few items can save disk space by having a smaller block size.
-
index_store
private FixedSizeDataStore index_store
The FixedSizeDataStore that contains all the data of the index store.
-
index_table_list
private ByteBuffer index_table_list
The list of table sector entries that are currently committed. Each entry of this list points to a table index list. The list is formatted as follows;0 (byte) - the type of block. 1 (int) - the number of blocks in this list. 5 (int) - the sector of column status information or -1 if no stats available. 9 to (n * (4 + 4 + 4 + 2)) - the sector (int), the first and last entry of the block and the number of indices in the block (short) of each block in this list. 9 + (n * (4 + 4 + 4 + 2)) .... [ next block ] ....
-
index_table_list_buf
private byte[] index_table_list_buf
-
allocation_sector
private int allocation_sector
The start sector where the block allocation information is currently stored.
-
allocation_length
private int allocation_length
The current of the allocation information.
-
memory_index_set_list
private java.util.ArrayList memory_index_set_list
The list of SnapshotIndexSet objects returned via the 'getSnapshotIndexSet' method. This can be inspected to find all sectors currently being used to store index information.
-
index_set_garbage
private java.util.ArrayList index_set_garbage
The list of SnapshotIndexSet objects that have been deleted and are ready for garbage collection.
-
unique_id
private long unique_id
Unique id field that contains a unique number that can be incremented atomically.
-
sector_cache
private Cache sector_cache
A cache of int[] array blocks that are accessed by this store.
-
flush_buffer
private byte[] flush_buffer
-
SET_ID_KEY
private long SET_ID_KEY
A unique key that is incremented each time a new IndexSet object is created.
-
EMPTY_INTEGER_LISTS
private static IndexStore.IndexIntegerList[] EMPTY_INTEGER_LISTS
A convenience static empty integer list array.
-
-
Constructor Detail
-
IndexStore
public IndexStore(java.io.File file_name, DebugLogger logger)
Constructs the IndexStore.- Parameters:
file_name
- the path to the file of the index store in the file system.
-
-
Method Detail
-
readIndexTableList
private void readIndexTableList() throws java.io.IOException
Reads the index table allocation list in to the ByteBuffer object. The position of the table allocation list can be determined by looking in the reserved area of the index file.- Throws:
java.io.IOException
-
initBlank
private void initBlank() throws java.io.IOException
Initializes the index store to a blank state.- Throws:
java.io.IOException
-
exists
public boolean exists() throws java.io.IOException
Returns true if the index store file exists.- Throws:
java.io.IOException
-
create
public void create(int block_size) throws java.io.IOException
Creates a new black index store and returns leaving the newly created store in an open state. This method initializes the various data in the index store for a blank set of index tables. Must call the 'init' method after this is called.- Parameters:
block_size
- the number of ints stored in each block. This can be optimized for specific use. Must be between 0 and 32768.- Throws:
java.io.IOException
-
open
public boolean open(boolean read_only) throws java.io.IOException
Opens this index store. If 'read_only' is set to true then the store is opened in read only mode.Returns true if opening the store was dirty (was not closed properly) and may need repair.
If the index store does not exist before this method is called then it is created.
- Throws:
java.io.IOException
-
init
public void init() throws java.io.IOException
Initializes the IndexStore. Must be called after it is opened for normal use, however it should not be called if we are fixing or repairing the store.- Throws:
java.io.IOException
-
fix
public boolean fix(UserTerminal terminal) throws java.io.IOException
Performs checks to determine that the index store is stable. If an IndexStore is not stable and can not be fixed cleanly then it deletes all information in the store and returns false indicating the index information must be rebuilt.Assumes the index store has been opened previous to calling this.
Returns true if the IndexStore is stable.
- Throws:
java.io.IOException
-
isReadOnly
public boolean isReadOnly()
Returns true if this store is read only.
-
delete
public void delete()
Deletes the store. Must have been closed before this is called.
-
copyTo
public void copyTo(java.io.File path) throws java.io.IOException
Copies the persistant part of this to another store. Must be open when this is called.- Throws:
java.io.IOException
-
close
public void close() throws java.io.IOException
Cleanly closes the index store.- Throws:
java.io.IOException
-
flush
public void flush() throws java.io.IOException
Flushes all information in this index store to the file representing this store in the file system. This is called to persistantly update the state of the index store.- Throws:
java.io.IOException
-
hardSynch
public void hardSynch() throws java.io.IOException
Performs a hard synchronization of this index store. This will force the OS to synchronize the contents of the data store.For this to be useful, 'flush' should be called before a hardSynch.
- Throws:
java.io.IOException
-
currentUniqueID
long currentUniqueID()
The current unique id.
-
nextUniqueID
long nextUniqueID()
Atomically returns the next 'unique_id' value from this file.
-
setUniqueID
void setUniqueID(long value)
Sets the unique id for this store. This must only be used under extraordinary circumstances, such as restoring from a backup, or converting from one file to another.
-
getBlockSize
int getBlockSize()
Returns the block size of this store.
-
addIndexLists
public void addIndexLists(int count, byte type)
Adds a number of blank index tables to the index store. For example, we may want this store to contain 16 index lists.NOTE: This doesn't write the updated information to the file. You must call 'flush' to write the information to the store.
-
addIndexSetToList
private void addIndexSetToList(IndexSet index_set)
Adds a SnapshotIndexSet to the list of sets that this store has dispatched.
-
removeIndexSetFromList
private void removeIndexSetFromList(IndexSet index_set)
Removes a SnapshotIndexSet from the list of sets that this store is managing.NOTE: This may be called by the finalizer of the IndexSet object if the index_set is not disposed.
-
getSnapshotIndexSet
public IndexSet getSnapshotIndexSet()
Returns a current snapshot of the current indexes that are committed in this store. The returned object can be used to create mutable IntegerListInterface objects. The created index lists are isolated from changes made to the rest of the indexes after this method returns.A transaction must grab an IndexSet object when it opens.
NOTE: We MUST guarentee that the IndexSet is disposed when the transaction finishes.
-
commitIndexSet
public void commitIndexSet(IndexSet index_set)
Commits changes made to a snapshop of an IndexSet as being permanent changes to the state of the index store. This will generate an error if the given IndexSet is not the last set returned from the 'getSnapshotIndexSet' method.For this to be used, during the transaction commit function a 'getSnapshopIndexSet' must be obtained, changes made to it from info in the journal, then a call to this method. There must be a guarentee that 'getSnapshotIndexSet' is not called again during this process.
NOTE: This doesn't write the updated information to the file. You must call 'flush' to write the information to the store.
NOTE: We must be guarenteed that when this method is called no other calls to other methods in this object can be called.
-
statusString
public java.lang.String statusString() throws java.io.IOException
Returns a string that contains diagnostic information.- Throws:
java.io.IOException
-
-