Class IndexSetStore


  • final class IndexSetStore
    extends java.lang.Object
    A class that manages the storage of a set of transactional index lists in a way that is fast to modify. This class has a number of objectives;

    • To prevent corruption as best as possible.
    • To be able to modify lists of integers very fast and persistantly.
    • To have size optimization features such as defragmentation.
    • To provide very fast searches on sorted lists (caching features).
    • To be able to map a list to an IntegerListInterface interface.

    This object uses a com.mckoi.store.Store instance as its backing medium.

    This store manages three types of areas; 'Index header', 'Index block' and 'Index element'.

    Index header: This area type contains an entry for each index being stored. The Index header contains a pointer to an 'Index block' area for each index. The pointer to the 'Index block' in this area changes whenever an index changes, or when new indexes are added or deleted from the store.

    Index block: This area contains a number of pointers to Index element blocks. The number of entries depends on the number of indices in the list. Each entry contains the size of the block, the first and last entry of the block, and a pointer to the element block itself. If an element of the index changes or elements are removed or deleted, this block does NOT change. This should be considered an immutable area.

    Index element: This area simply contains the actual values in a block of the index. An Index element area does not change and should be considered an immutable area.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      private class  IndexSetStore.IndexBlock
      Represents a single 'Index block' area in the store.
      private class  IndexSetStore.IndexIntegerList
      The IntegerListInterface implementation that is used to represent a mutable snapshop of the indices at a given point in time.
      private class  IndexSetStore.MappedListBlock
      An IntegerListBlockInterface implementation that maps a block of a list to an underlying file system representation.
      private class  IndexSetStore.SnapshotIndexSet
      The implementation of IndexSet which represents a mutable snapshot of the indices stored in this set.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addAllAreasUsed​(java.util.ArrayList list)
      Adds to the given ArrayList all the areas in the store that are used by this structure (as Long).
      void addIndexLists​(int count, int type, int block_size)
      Adds a number of blank index tables to the index store.
      void close()
      Closes this index set (cleans up).
      void commitDropIndex​(int index_num)
      Commits a change that drops an index from the index set.
      private void commitIndexHeader()
      Commits the index header with the current values set in 'index_blocks'.
      void commitIndexSet​(IndexSet index_set)
      Commits changes made to a snapshop of an IndexSet as being permanent changes to the state of the index store.
      void copyAllFrom​(IndexSet index_set)
      Overwrites all existing index information in this store and sets it to a copy of the given IndexSet object.
      long create()
      Creates a new black index set store and returns a pointer to a static area that is later used to reference this index set in this store.
      private long createBlankIndexBlock()
      Creates a new blank index block in the store and returns a pointer to the area.
      private void deleteAllAreas​(java.util.ArrayList list)
      Delete all areas specified in the list (as a list of Long).
      IndexSet getSnapshotIndexSet()
      Returns a current snapshot of the current indexes that are committed in this store.
      void init​(long start_p)
      Initializes this index set.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • MAGIC

        private static final int MAGIC
        The magic value that we use to mark the start area.
        See Also:
        Constant Field Values
      • debug

        private final DebugLogger debug
        A DebugLogger object used to log debug messages to.
      • system

        private final TransactionSystem system
        The TransactionSystem for this index set.
      • store

        private Store store
        The Store that contains all the data of the index store.
      • start_area

        private MutableArea start_area
        The starting header of this index set. This is a very small area that simply contains a magic value and a pointer to the index header. This is the only MutableArea object that is required by the index set.
      • index_header_p

        private long index_header_p
        The index header area. The index header area contains an entry for each index being stored. Each entry is 16 bytes in size and has a 16 byte header.

        HEADER: ( version (int), reserved (int), index count (long) )
        ENTRY: ( type (int), block_size (int), index block pointer (long) )

      • index_header_area

        private Area index_header_area
      • index_blocks

        private IndexSetStore.IndexBlock[] index_blocks
        The index blocks - one for each index being stored. An index block area contains an entry for each index element in an index. Each entry is 28 bytes in size and the area has a 16 byte header.

        HEADER: ( version (int), reserved (int), index size (long) )
        ENTRY: ( first entry (long), last entry (long), index element pointer (long), type/element size (int) )

        type/element size contains the number of elements in the block, and the block compaction factor. For example, type 1 means the block contains short sized index values, 2 is int sized index values, and 3 is long sized index values.

    • Constructor Detail

      • IndexSetStore

        public IndexSetStore​(Store store,
                             TransactionSystem system)
        Constructs the IndexSetStore over the given Store object.
    • Method Detail

      • deleteAllAreas

        private void deleteAllAreas​(java.util.ArrayList list)
        Delete all areas specified in the list (as a list of Long).
      • createBlankIndexBlock

        private long createBlankIndexBlock()
                                    throws java.io.IOException
        Creates a new blank index block in the store and returns a pointer to the area.
        Throws:
        java.io.IOException
      • create

        public long create()
                    throws java.io.IOException
        Creates a new black index set store and returns a pointer to a static area that is later used to reference this index set in this store. Remember to synch after this is called.
        Throws:
        java.io.IOException
      • init

        public void init​(long start_p)
                  throws java.io.IOException
        Initializes this index set. This must be called during general initialization of the table object.
        Throws:
        java.io.IOException
      • close

        public void close()
        Closes this index set (cleans up).
      • copyAllFrom

        public void copyAllFrom​(IndexSet index_set)
                         throws java.io.IOException
        Overwrites all existing index information in this store and sets it to a copy of the given IndexSet object. The 'source_index' must be a snapshot as returned by the getSnapshotIndexSet method but not necessarily generated from this index set.

        This will create a new structure within this store that contains the copied index data. This overwrites any existing data in this store so care should be used when using this method.

        This method is an optimized method of copying all the index data in an index set and only requires a small buffer in memory. The index data in 'index_set' is not altered in any way by using this.

        Throws:
        java.io.IOException
      • addAllAreasUsed

        public void addAllAreasUsed​(java.util.ArrayList list)
                             throws java.io.IOException
        Adds to the given ArrayList all the areas in the store that are used by this structure (as Long).
        Throws:
        java.io.IOException
      • addIndexLists

        public void addIndexLists​(int count,
                                  int type,
                                  int block_size)
                           throws java.io.IOException
        Adds a number of blank index tables to the index store. For example, we may want this store to contain 16 index lists.

        NOTE: This doesn't write the updated information to the file. You must call 'flush' to write the information to the store.

        Throws:
        java.io.IOException
      • getSnapshotIndexSet

        public IndexSet getSnapshotIndexSet()
        Returns a current snapshot of the current indexes that are committed in this store. The returned object can be used to create mutable IntegerListInterface objects. The created index lists are isolated from changes made to the rest of the indexes after this method returns.

        A transaction must grab an IndexSet object when it opens.

        NOTE: We MUST guarentee that the IndexSet is disposed when the transaction finishes.

      • commitIndexHeader

        private void commitIndexHeader()
                                throws java.io.IOException
        Commits the index header with the current values set in 'index_blocks'.
        Throws:
        java.io.IOException
      • commitIndexSet

        public void commitIndexSet​(IndexSet index_set)
        Commits changes made to a snapshop of an IndexSet as being permanent changes to the state of the index store. This will generate an error if the given IndexSet is not the last set returned from the 'getSnapshotIndexSet' method.

        For this to be used, during the transaction commit function a 'getSnapshopIndexSet' must be obtained, changes made to it from info in the journal, then a call to this method. There must be a guarentee that 'getSnapshotIndexSet' is not called again during this process.

        NOTE: We must be guarenteed that when this method is called no other calls to other methods in this object can be called.

      • commitDropIndex

        public void commitDropIndex​(int index_num)
                             throws java.io.IOException
        Commits a change that drops an index from the index set. This must be called from within the conglomerate commit. The actual implementation of this overwrites the index with with a 0 length index. This is also useful if you want to reindex a column.
        Throws:
        java.io.IOException