#84 - clearing all blocks for a dataCid should remove the data from…

… disk right now (#279)

`leveldb` seems to not immediately delete things from disk when calling `clear` (or `del`)

instead, `leveldb` will periodically remove nonreferenced data when it determines that it needs to (or if it's at an opportune moment, e.g. `open`)

this can be manually tested (we can't do an automated test because `approximateSize` isn't available on all platforms) by doing the following:

1. change any of the `DataStore` tests involving large amounts of data (e.g. `10_000_000`) to be `it.only`
2. repeatedly do `npm run test:node`
3. observe the size of `TEST-DATASTORE` on disk

the size of `TEST-DATASTORE` should not go above ~3× the size of the data (e.g. `30_000_000`)

furthermore, if you then changed that test to not do anything other than `open`, you should see the size of `TEST-DATASTORE` decrease on most runs until it reaches a small enough value that `leveldb` doesn't feel the need to do any compacting

if instead of all that we wanted to delete the data on `clear`, we need to leverage the additional method `compactRange` provided by `classic-level`

in order to make this the nicest API, we only allow this for `sublevel` as that's the easiest way to derive a range to give to `compactRange` (based on the `prefix` of the `sublevel`)

also, for now we're only using `compactRange` inside `clear` as the current structure of `DataStoreLevel` leverages it to delete data (and that's where the large majority of massive data will be)

README.md

@@ -3,7 +3,7 @@
 # Decentralized Web Node (DWN) SDK
 
 Code Coverage
-![Statements](https://img.shields.io/badge/statements-93.59%25-brightgreen.svg?style=flat) ![Branches](https://img.shields.io/badge/branches-93.3%25-brightgreen.svg?style=flat) ![Functions](https://img.shields.io/badge/functions-90.28%25-brightgreen.svg?style=flat) ![Lines](https://img.shields.io/badge/lines-93.59%25-brightgreen.svg?style=flat)
+![Statements](https://img.shields.io/badge/statements-93.64%25-brightgreen.svg?style=flat) ![Branches](https://img.shields.io/badge/branches-92.8%25-brightgreen.svg?style=flat) ![Functions](https://img.shields.io/badge/functions-90.4%25-brightgreen.svg?style=flat) ![Lines](https://img.shields.io/badge/lines-93.64%25-brightgreen.svg?style=flat)
 
 ## Introduction
 

src/store/level-wrapper.ts

@@ -81,7 +81,7 @@ export class LevelWrapper<V> {
   async partition(name: string): Promise<LevelWrapper<V>> {
     await this.createLevelDatabase();
 
-    return new LevelWrapper({ ...this.config, location: '' }, this.db.sublevel(name, {
+    return new LevelWrapper(this.config, this.db.sublevel(name, {
       keyEncoding   : 'utf8',
       valueEncoding : this.config.valueEncoding
     }));
@@ -159,7 +159,9 @@ export class LevelWrapper<V> {
   async clear(): Promise<void> {
     await this.createLevelDatabase();
 
-    return this.db.clear();
+    await this.db.clear();
+
+    await this.compactUnderlyingStorage();
   }
 
   async batch(operations: Array<LevelWrapperBatchOperation<V>>, options?: LevelWrapperOptions): Promise<void> {
@@ -170,6 +172,42 @@ export class LevelWrapper<V> {
     return abortOr(options?.signal, this.db.batch(operations));
   }
 
+  private async compactUnderlyingStorage(options?: LevelWrapperOptions): Promise<void> {
+    options?.signal?.throwIfAborted();
+
+    await abortOr(options?.signal, this.createLevelDatabase());
+
+    const range = this.sublevelRange;
+    if (!range) {
+      return;
+    }
+
+    // additional methods are only available on the root API instance
+    const root = this.root;
+
+    if (root.db.supports.additionalMethods.compactRange) {
+      return abortOr(options?.signal, root.db['compactRange']?.(...range));
+    }
+  }
+
+  private get sublevelRange(): [ string, string ] | undefined {
+    const prefix = this.db['prefix'];
+    if (!prefix) {
+      return undefined;
+    }
+
+    // use the separator to derive an exclusive `end` that will never match to a key (which matches how `abstract-level` creates a `boundary`)
+    return [ prefix, prefix.slice(0, -1) + String.fromCharCode(prefix.charCodeAt(prefix.length - 1) + 1) ];
+  }
+
+  private get root(): LevelWrapper<V> {
+    let db = this.db;
+    for (const parent = db['db']; parent && parent !== db; ) {
+      db = parent;
+    }
+    return new LevelWrapper(this.config, db);
+  }
+
   private async createLevelDatabase(): Promise<void> {
     this.db ??= await this.config.createLevelDatabase<V>(this.config.location, {
       keyEncoding   : 'utf8',

tests/store/data-store.spec.ts

@@ -13,7 +13,7 @@ let store: DataStoreLevel;
 
 describe('DataStore Test Suite', () => {
   before(async () => {
-    store = new DataStoreLevel({ blockstoreLocation: 'TEST-MESSAGESTORE' });
+    store = new DataStoreLevel({ blockstoreLocation: 'TEST-DATASTORE' });
     await store.open();
   });