Another ArrayIndexOutOfBoundsException

[catbref]

2017-11-11 09:46:30 INFO  Controller:890 - received new valid block
2017-11-11 09:52:04 ERROR Controller:1037 - 136
java.lang.ArrayIndexOutOfBoundsException: 136
	at org.mapdb.Volume$ByteBufferVol.getDataInput(Volume.java:435)
	at org.mapdb.Volume$ByteBufferVol.getDataInput(Volume.java:315)
	at org.mapdb.StoreWAL.get2(StoreWAL.java:344)
	at org.mapdb.StoreWAL.get(StoreWAL.java:320)
	at org.mapdb.Caches$HashTable.get(Caches.java:246)
	at org.mapdb.BTreeMap.get(BTreeMap.java:602)
	at org.mapdb.BTreeMap.containsKey(BTreeMap.java:1506)
	at database.DBMap.contains(DBMap.java:238)
	at database.DBMap.contains(DBMap.java:248)
	at qora.block.Block.getHeight(Block.java:254)
	at qora.block.Block.orphan(Block.java:793)
	at qora.Synchronizer.synchronize(Synchronizer.java:65)
	at qora.Synchronizer.synchronize(Synchronizer.java:224)
	at controller.Controller.update(Controller.java:1034)
	at qora.BlockGenerator.run(BlockGenerator.java:184)
2017-11-11 09:52:04 WARN  Network:117 - Connection error : /208.115.125.68 : 136
2017-11-11 09:52:04 INFO  Controller:193 - STATUS OK
| Last Block Signature: F2s97G7FKrt5B2Ld7sBos3MAGHwJxjz4yqMogft4Rdhst8rTExXchFxSKURFeFbpUDuByDNj97EiBMv25DHPa4i2d4bA5hZCp2tcP9GFs57F4o3LgT63UonmLipYH67e9L3VzT2jYLh97QuvyrfDwUuTX3smM5qhXajFQVHe4NjS866
| Last Block Height: 408875
| Last Block Time: 11/11/2017 9:46:30 AM
| Last Block Found 5m 34s ago.

[catbref]

Working backwards from innermost stack frame:

mapdb/Volume.java:434

        public final DataInput2 getDataInput(long offset, int size) {
            return new DataInput2(chunks[(int)(offset >>> chunkShift)], (int) (offset&chunkSizeModMask));
        }

offset >>> chunkShift equates to 136, which is the non-existent array index in chunks.
chunkShift seems to be 20 so chunks are 1MB in size?

So the offset is wrong when passed at mapdb/StoreWAL.java:344:

            DataInput2 in = (DataInput2) log.getDataInput(r[0]&LOG_MASK_OFFSET, size);

r[0] comes from the modified transaction HashMap (lines 333-334):

        //check if record was modified in current transaction
        long[] r = modified.get(ioRecid);

Is modified somehow out of sync?

• cleared in reloadIndexFile but this shouldn't cause the issue above where there is a record
• set to PREALLOC in preallocate() but also shouldn't cause issue above
• set in put() line 228
• set in update() line 416
• set in compareAndSwap() line 477
• "tombstoned" in delete() line 528

Is the log ByteBufferVol corrupted/reset outside of a write-lock by another thread?

• logReset() only checks for structuralLock
• logReset() called by:
  • rollback() - under full write locks
  • replayLogFile() twice - called by commit() - under full write locks
  • StoreWAL constructor

Does this situation only occur on Windows and hence something to do with mmap/Windows caching?

Is it possible to store more than chunkSize in a chunk when ByteBufferVol should put extra bytes into next chunk? (Maybe add assert()s to mapdb/Volume.java to test this)