Skip to content

elmarti/camadb

Folders and files

NameName
Last commit message
Last commit date

Latest commit

6b61adb · Jan 8, 2023
Oct 16, 2021
Jan 4, 2023
Oct 8, 2021
Jan 8, 2023
Oct 7, 2021
Oct 7, 2021
Oct 16, 2021
Oct 7, 2021
Oct 17, 2021
Oct 10, 2021
Oct 16, 2021
Oct 7, 2021
Jan 8, 2023
Oct 16, 2021
Oct 16, 2021
Jan 8, 2023
Oct 16, 2021
Jan 8, 2023

Repository files navigation

CamaDB

CamaDB is a NoSQL embedded database written in pure TypeScript for Node, Electron and browser-based environments.

semantic-release

Why?

I was struggling to find a solution for Electron-based projects that deal with larger datasets in the main thread.

  • I had issues getting SQLite to work with webpack due to its native build
  • SQLite doesn't (by default) return native JS data types (Dates in particular)
  • Other NoSQL embedded databases seem to be largely abandoned
  • Most other NoSQL embedded databases seem to be limited by V8's hard string length limits

Goals

  • Fast querying/insertion/manipulation of data, up to 1 million rows
  • Frictionless integration with all JS runtimes
  • Rich API
  • Full TypeScript support
  • Simplicity and versatility - This is built for storing data in dynamic structures

Current state

This is still under active development and a few features are missing:

  • Indexes - We're finding that this is fast even without these, but every little helps
  • Rich text search

Getting started

Documentation

Installing

yarn add reflect-metadata
yarn add camadb

OR

npm install reflect-metadata --save
npm install camadb --save

Initializing the database

All of these config options are optional:

  • path - Where you want the data to be stored - default is ./.cama or cama for indexeddb and localstorage
  • persistenceAdapter - How you want to persist your data - fs, indexeddb, localstorage or inmemory
  • logLevel - info or debug
  import { Cama } from 'camadb'
  const database = new Cama({
    path: './.cama',
    persistenceAdapter: 'fs',
    logLevel: 'debug'
  });

Initializing a collection

  • Use the columns field to add specific data types for rows. This does need to be done for each column, but is essential for date objects
  • Indexes aren't currently implemented, but the lookup is still very fast across 10 million rows
 const collection = await database.initCollection('test', {
    columns: [{
      type:'date',
      title:'date'
    }],
    indexes: [],
  });

Insert one

 await collection.insertOne({
    _id: 'test',
    name: 'Dummy field',
    description: `Data`,
  });

Insert many

  await collection.insertMany([{
       _id: 'test',
       name: 'Dummy field',
       description: `Data`,
  }]);

Find many

CamaDB uses a MongoDB style query language, powered by SiftJS. Have a look at that project to see the full capabilities of that library.

 const findResult = await collection.findMany({
    _id: {
      $gte: 50000,
    },
  },
    {
      sort:{
        desc: x => x._id
      },
      offset: 100,
      limit: 100
    });

Updating

Again we use a MongoDB style language for data updates, for this we use obop

  await collection.updateMany({
    _id:3
  }, {
    $set: {
      steve:"steve"
    }
  });

Aggregation

We use Mingo for aggregation - currently lookup commands aren't supported.

 const aggregationResult = await collection.aggregate([{
    $match:{
      _id:3
    }
  }]);