Data::RecordStore internals



Directory Structure
   datastore_root
        |
        +- VERSION     # version file (text)
        |
        +- RECORD_INDEX_SILO # subdirectory with silo that indexes all records, indexed by Record ID
        |         |
        |         +- 0   # silo file with record size 12 and pack template "IL" for Silo Number and index in silo
        |
        +- silos  # subdirectory with silo subdirectories named by Silo Number and _RECORDSTORE
        |    |
        |    +- 12_RECORDSTORE   # subdirectory for silo #12 of record size 4096 (2^12) 
        |    |       |
        |    |       +- 0   # first silo file of record data for store 12 (up to 2GB or configured max file size)
        |    |       |      # has pack template "LZ*" for Record ID and byte data
        |    |       |
        |    |       +- 1   # second silo file of record data for store 12
        |    |       |      # has pack template "LZ*" for Record ID and byte data
        |    |       |
        |    |       |  ... more silo files
        |    |
        |    +- 13_RECORDSTORE   # subdirectory for silo #13 of record size 8192 (2^13) and pack template "LZ*"
        |    |       |
        |    |       +- 0   # silo file with record data with pack template "LZ*" for Record ID and data
        |    |
        |    | ... more silo subdirectories
        |
        +- RECYC_SILO  # subdirectory with silo that keeps track of recycled ids
        |         |
        |         +- 0   # silo file with record size 8 and pack template "L" for Record ID
        |
        +- TRANS  # subdirectory with silo that keeps track of transactions
                  |
                  +- META # subdirectory with silo that keeps track of transactions
                  |    |
                  |    +- 0   # silo file  with size 24 and pack template "ILLI" ( Transaction ID, process id, timestamp, transaction state )
                  |           # states are TRA_ACTIVE, TRA_IN_COMMIT, TRA_IN_ROLLBACK, TRA_CLEANUP_COMMIT, TRA_CLEANUP_ROLLBACK, TRA_DONE
                  |
                  +- instances      # subdirectory of active transaction silo subdirectories
                       |
                       +- trans_id  # silo subdirectory file with transaction action data. The name is the Transaction ID
                       |    |
                       |    +- 0   # silo file with pack template "ALILIL", for transaction type, Record ID, from_silo_number, from_silo_index, to_silo_number, to_silo_index,
                       |           # transaction types : S - stow, D - delete, R - recycle
                       |
                       +- D_trans_id  # 'directory' silo file. The name is the Transaction ID with the suffix "_trans_id"
                            |         # stores Record IDs that have been changed in this transaction. Used to ensure fetch returns
                            |         # the value this has so far in the transaction.
                            |
                            +- 0   # silo file with pack template "L",

Silo Files
   Silo files store byte data in a fixed record format. Each silo has a record size.
   The number of records in a silo file is equal to the file size divided by the record size.
   The indexes into a silo file start counting at 1.
   Entry idx inside a silo file will be at file byte RECORD_SIZE * (idx-1)

Transactions
   

Configurations