var db = new PouchDB('mydb');
db.setSchema([
{singular: 'author', plural: 'authors', relations: { books: {hasMany: 'book'}}},
{singular: 'book', plural: 'books', relations: {author: {belongsTo: 'author'}}}
]);
db.rel.save('author', {
name: 'George R. R. Martin', id: 1, books: [6, 7]
}).then(function () {
return db.rel.save('book', { title: 'A Game of Thrones', id: 6, author: 1});
}).then(function () {
return db.rel.save('book', {title: 'The Hedge Knight', id: 7, author: 1});
}).catch(console.log.bind(console));Relational Pouch is a plugin for PouchDB that allows you to interact with PouchDB/CouchDB like a relational data store, with types and relations.
It provides an enhanced API on top of PouchDB that is probably more familiar to fans of relational databases, and maybe even easier to use. At the same time, though, you still have CouchDB's awesome indexing and sync capabilities.
This plugin also uses clever tricks to avoid creating secondary indexes. This means that even if you have complex entity relations, your database operations should still be very fast.
The main goal of this is to provide an API that is as similar to Ember Data and json:api as possible, while still being performant and Pouch-like.
This plugin is largely what powers Ember Pouch.
Download from GitHub, or use Bower:
bower install relational-pouch
Then include it after pouchdb.js in your HTML page:
<script src="pouchdb.js"></script>
<script src="pouchdb.relational-pouch.js"></script>npm install relational-pouch
And then attach it to the PouchDB object:
var PouchDB = require('pouchdb');
PouchDB.plugin(require('relational-pouch'));db.setSchema(schema)db.rel.save(type, object)db.rel.find(type)db.rel.find(type, id)db.rel.find(type, ids)db.rel.find(type, options)db.rel.del(type, object)db.rel.putAttachment(type, object, attachmentId, attachment, attachmentType)db.rel.getAttachment(type, id, attachmentId)db.rel.removeAttachment(type, object, attachmentId)db.rel.parseDocID(docID)db.rel.makeDocID(docID)db.rel.parseRelDocs(type, pouchDocs)db.rel.findHasMany(type, type, belongsToKey, belongsToId)- Managing relationships
- Managing revisions ("rev")
Call this after you initialize your PouchDB, in order to define your entities and relationships:
var db = new PouchDB('mydb');
db.setSchema([
{
singular: 'post',
plural: 'posts',
relations: {
author: {belongsTo: 'author'},
comments: {hasMany: 'comment'}
}
},
{
singular: 'author',
plural: 'authors',
relations: {
posts: {hasMany: 'post'}
}
},
{
singular: 'comment',
plural: 'comments',
relations: {
post: {belongsTo: 'post'}
}
}
]);This is a synchronous method that does not return a Promise.
You can define one-to-one, one-to-many, and many-to-many relationships using any combination of belongsTo and hasMany that you want. For more examples, read the Ember guide to models, which is what inspired this.
You need to explicitly define the singular and plural forms of your entities, because I'm not a big fan of applying magic Anglocentric defaults to everything.
Once you call setSchema, your db will be blessed with a rel object, which is where you can start using the rest of this plugin's API.
Rarely, you might want to have two different views over the same underlying data. Use documentType to create a view which reads the same data as another type:
var db = new PouchDB('mydb');
db.setSchema([
{
singular: 'post',
plural: 'posts',
relations: {
author: {belongsTo: 'author'},
comments: {hasMany: 'comment'}
}
},
{
singular: 'postSummary',
plural: 'postSummaries',
documentType: 'post'
}
]);Here, when you load a "postSummary", it will return the same core record as "post", but will not resolve the relationships.
Be careful when using this feature — it is probably best to treat a type declaring a documentType as read-only. Do all creates/updates via the main type.
Save an object with a particular type. This returns a Promise.
db.rel.save('post', {
title: 'Rails is Omakase',
text: 'There are a lot of a-la-carte software...'
});Result:
{
"posts": [
{
"title": "Rails is Omakase",
"text": "There are a lot of a-la-carte software...",
"id": "14760983-285C-6D1F-9813-D82E08F1AC29",
"rev": "1-84df2c73028e5b8d0ae1cbb401959370"
}
]
}If you want, you can specify an id. Otherwise an id will be created for you.
db.rel.save('post', {
title: 'Rails is Unagi',
text: 'Delicious unagi. Mmmmmm.',
id: 1
});Result:
{
"posts": [
{
"title": "Rails is Unagi",
"text": "Delicious unagi. Mmmmmm.",
"id": 1,
"rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
}
]
}You'll notice the special field rev, which is a revision identifier. That'll come into play later.
id and rev are reserved fields when you use this plugin. You shouldn't try to use them for something else. An id can be any string or integer.
Find all objects with a given type. Returns a Promise.
db.rel.find('post');Result:
{
"posts": [
{
"title": "Rails is Unagi",
"text": "Delicious unagi. Mmmmmm.",
"id": 1,
"rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
},
{
"title": "Rails is Omakase",
"text": "There are a lot of a-la-carte software...",
"id": "14760983-285C-6D1F-9813-D82E08F1AC29",
"rev": "1-84df2c73028e5b8d0ae1cbb401959370"
}
]
}The list will be empty if it doesn't find anything. The results are sorted by id.
Find an object with the given type and id. Returns a Promise.
db.rel.find('post', 1);Result:
{
"posts": [
{
"title": "Rails is Unagi",
"text": "Delicious unagi. Mmmmmm.",
"id": 1,
"rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
}
]
}Find multiple objects with multiple ids. Returns a Promise.
db.rel.find('post', [3, 2, 1]);Result:
{
"posts": [
{
"title": "Rails is Unagi",
"text": "Delicious unagi. Mmmmmm.",
"id": 1,
"rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
},
{
"title": "Maybe Rails is more like a sushi buffet",
"text": "Heresy!",
"id": 2,
"rev": "1-6d8ac6d86d01b91cfbe2f53e0c81bb86"
}
]
}If an id isn't found, it's simply not returned. Notice that above, there is no object with an id of 3.
find results are always returned ordered by id. The order of your ids array will not necessarily be reflected in the returned array of objects.
Find all objects with a given type and limit the results via the passed in options. Returns a Promise.
db.rel.find('post',{startkey: 1, limit: 2});Result:
{
"posts": [
{
"title": "Rails is Unagi",
"text": "Delicious unagi. Mmmmmm.",
"id": 1,
"rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
},
{
"title": "Maybe Rails is more like a sushi buffet",
"text": "Heresy!",
"id": 2,
"rev": "1-6d8ac6d86d01b91cfbe2f53e0c81bb86"
}
]
}The following options based on the options for PouchDB batch fetch are available:
startkey&endkey: Get documents with IDs in a certain range (inclusive/inclusive).limit: Maximum number of documents to return.skip: Number of docs to skip before returning (warning: poor performance on IndexedDB/LevelDB!).
Deletes the given object. Returns a Promise.
db.rel.del('post', {id:1, rev:"1-0560dbb11ead319c9f5bc1f667ea8e84"});Result:
{"deleted":true}The minimum you need to delete something is an id and a rev. The easiest pattern is to just find it before deleting it:
db.rel.find('post', 1).then(function (post) {
return db.rel.del('post', post);
});Adds an attachment to the given object. Returns a Promise. See PouchDB Attachments but note that ._attachments is instead .attachments in relational-pouch.
var attachment = new Blob(['Is there life on Mars?']);
// Or in Node.js: new Buffer('Is there life on Mars?')
db.rel.putAttachment('post', {id:1, rev:"1-..."}, 'file', attachment, 'text/plain');Result:
{
"posts": [
{
"attachments": {
"file": {
"content_type": "text/plain",
"digest": "md5-1cz9JKh0i+1OLJonmitgiQ==",
"length": 22,
// ... http://pouchdb.com/guides/attachments.html
}
},
"id": 1,
"rev": "2-...."
}
]
}Gets an attachment for the given document id. Returns a Promise to a Blob (or Buffer for Node).
db.rel.getAttachment('post', 1, 'file').then(function (attachment) {
// convert the Blob into an object URL and show it in an image tag
$('img').attr('src', URL.createObjectURL(attachment));
});Remove an attachment from the given object. Returns a Promise.
var attachment = new Blob(['Is there life on Mars?']); // new Buffer('Is there life on Mars?') for node
db.rel.putAttachment('post', {id:1, rev:"1-0560dbb11ead319c9f5bc1f667ea8e84"}, 'file', attachment, 'text/plain').then(function (res) {
var post = res.posts[0];
db.rel.removeAttachment('post', post, 'file');
});Or continuing from the putAttachment example:
db.rel.removeAttachment('post', {id: 1, rev:"2-09d5c5bd86fc170c064b296773044ea9"} , 'file');Result:
{
"posts": [
{
"id": 1,
"rev": "3-...."
}
]
}Parses a raw CouchDB/PouchDB doc _id into an object containing a type and id field. Basically only useful for working with the db.changes() feed, so you can tell what changed from a "relational" perspective rather than from the raw CouchDB/PouchDB perspective.
This method is synchronous, so it directly returns the object rather than a Promise.
db.rel.parseDocID("author_1_0000000000000019");Returns:
{
"type": "author",
"id": 19
}So e.g. with changes() you could do:
db.changes().then(function (changes) {
return changes.results.map(function (change) {
return db.rel.parseDocID(change.id);
});Result is e.g.:
[
{"type": "author", "id": 19},
{"type": "book", "id": 1},
{"type": "book", "id": 2},
{"type": "book", "id": 3}
]Creates a valid _id from an object with type and id properties, such as
parseDocID generates.
db.rel.makeDocID({ "type": "author", "id": 19 });Returns:
"author_1_0000000000000019"Useful if you need to perform operations with the underlying database, e.g.:
var _id = db.rel.makeDocID({ "type": "author", "id": 19 });
db.get(_id).then(function (doc) {
var parsedId = db.parseDocID(doc._id);
doc.data.type = parsedId.type;
doc.data.id = parsedId.id;
return doc.data;
});Parses pouch documents that should be in this schema. Loads extra relations if needed. This function is useful when you are loading data from db.find for example, instead of from db.rel.
Example:
db.find(selector).then(function (data) {
return db.rel.parseRelDocs(type, data.docs);
});Returns data as db.rel.find(type) would return it.
It requires a type parameter, which could be gotten from the id of the first document using db.rel.parseDocID(docID), but most of the time you will know which type you are trying to find with the db.find call.
Uses db.find to build a selector that searches for the documents of type type that have a value of belongsToId set in the field belongsToKey.
This can be used in a higher level API to use asynchronous hasMany relations that don't store the hasMany side.
Example:
db.rel.findHasMany('post', 'author', '1');Returns a Promise that will resolve to an array of posts that have author 1 as db.rel.find(type) would.
Since db.find requires the use of indexes, you need to setup an index for this extra lookup. The fields required by this index are _id and the field specified in the belongsToKey, prefixed with data.. The _id field is used to filter the related items by type. Without it another type with the same field could also be returned.
So the example above would give:
db.createIndex({index: { fields: ['data.author', '_id'] }});
For performance reasons the queried field is used first here. As this is to be expected to return a smaller set that filtering on type first.
If your database however has a lot more document types that has data.author fields too, you may find that switching the order and using id as the first filter will give faster results.
Entity relationships are encoded using the Ember Data Model format, which is a slight simplification of json:api.
An author has exactly one profile, and vice-versa:
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
'profile': {belongsTo: 'profile'}
}
},
{
singular: 'profile',
plural: 'profiles',
relations: {
'author': {belongsTo: 'author'}
}
}
]);
db.rel.save('author', {
name: 'Stephen King',
id: 19,
profile: 21
}).then(function () {
return db.rel.save('profile', {
description: 'nice masculine jawline',
id: 21,
author: 19
});
}).then(function () {
return db.rel.find('author');
});Result:
{
"authors": [
{
"name": "Stephen King",
"profile": 21,
"id": 19,
"rev": "1-bf705a912bf672b30ad262b33a19c5c3"
}
],
"profiles": [
{
"description": "nice masculine jawline",
"author": 19,
"id": 21,
"rev": "1-ef86a08ea3243ea59302ceaa04afd59f"
}
]
}An author has many books:
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
'books': {hasMany: 'book'}
}
},
{
singular: 'book',
plural: 'books',
relations: {
'author': {belongsTo: 'author'}
}
}
]);
db.rel.save('author', {
name: 'Stephen King',
id: 19,
books: [1]
}).then(function () {
return db.rel.save('author', {
name: 'George R. R. Martin',
id: 1,
books: [6, 7]
});
}).then(function () {
return db.rel.save('book', {
title: 'It',
id: 1,
author: 19
});
}).then(function () {
return db.rel.save('book', {
title: 'A Game of Thrones',
id: 6,
author: 1
});
}).then(function () {
return db.rel.save('book', {
title: 'The Hedge Knight',
id: 7,
author: 1
});
}).then(function () {
return db.rel.find('author');
});Result:
{
"authors": [
{
"name": "George R. R. Martin",
"books": [
6,
7
],
"id": 1,
"rev": "1-04e165889a4a9303a6dc07a54cee9741"
},
{
"name": "Stephen King",
"books": [
1
],
"id": 19,
"rev": "1-38580117cb4a1ddb2c7151453a7f9129"
}
],
"books": [
{
"title": "It",
"author": 19,
"id": 1,
"rev": "1-1b7ea74936a8034aee7da27ffd36a63f"
},
{
"title": "A Game of Thrones",
"author": 1,
"id": 6,
"rev": "1-a6f0dc69fc79d5565639074b5defa52d"
},
{
"title": "The Hedge Knight",
"author": 1,
"id": 7,
"rev": "1-4988aa3215070c71e1505a05f90bb60f"
}
]
}Peter Straub actually co-wrote The Talisman with Stephen King. So a book can have many authors, and an author can have many books:
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
'books': {hasMany: 'book'}
}
},
{
singular: 'book',
plural: 'books',
relations: {
'authors': {hasMany: 'author'}
}
}
]);
db.rel.save('author', {
name: 'Stephen King',
id: 19,
books: [1, 2]
}).then(function () {
return db.rel.save('author', {
name: 'Peter Straub',
id: 2,
books: [2, 3]
});
}).then(function () {
return db.rel.save('book', {
title: 'It',
id: 1,
authors: [19]
});
}).then(function () {
return db.rel.save('book', {
title: 'The Talisman',
id: 2,
authors: [19, 2]
});
}).then(function () {
return db.rel.save('book', {
title: 'Ghost Story',
id: 3,
authors: [2]
});
}).then(function () {
return db.rel.find('author');
});Result:
{
"authors": [
{
"name": "Peter Straub",
"books": [
2,
3
],
"id": 2,
"rev": "1-92901c8e3e0775765777bfcbe8f4c2dd"
},
{
"name": "Stephen King",
"books": [
1,
2
],
"id": 19,
"rev": "1-d70d9fe033f583493029372c88ae21d0"
}
],
"books": [
{
"title": "It",
"authors": [
19
],
"id": 1,
"rev": "1-96751a2a5bb7b0fd70564efe6856dbd6"
},
{
"title": "The Talisman",
"authors": [
19,
2
],
"id": 2,
"rev": "1-9faf8c4f72db782dacce16a7849d156b"
},
{
"title": "Ghost Story",
"authors": [
2
],
"id": 3,
"rev": "1-7564a1195f143e24ebf24d914c60d6be"
}
]
}Just like with Ember Data, you can define relationships to be async, which means that dependent objects aren't automatically sideloaded. This can reduce your request time and payload size.
For instance, let's say you want to load all authors, but you don't want to load their books, too. You can do:
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
books: {hasMany: {type: 'book', options: {async: true}}}
}
},
{
singular: 'book',
plural: 'books',
relations: {
author: {belongsTo: {type: 'author', options: {async: true}}}
}
}
]);By default, async is consider false. So this:
...
books: {hasMany: 'book'}
...is equivalent to this:
...
books: {hasMany: {type: 'book', options: {async: false}}}
...Now let's try with {async: true}. You'll notice that, when we fetch the list of authors, only the book ids will be included, not the full books:
return db.rel.save('author', {
name: 'Stephen King',
id: 19,
books: [1, 2, 3]
}).then(function () {
return db.rel.save('book', {
id: 1,
title: 'The Gunslinger'
});
}).then(function () {
return db.rel.save('book', {
id: 2,
title: 'The Drawing of the Three'
});
}).then(function () {
return db.rel.save('book', {
id: 3,
title: 'The Wastelands'
});
}).then(function () {
return db.rel.find('author');
});Result:
{
"authors": [
{
"name": "Stephen King",
"books": [
1,
2,
3
],
"id": 19,
"rev": "1-9faf8c4f72db782dacce16a7849d156b"
}
]
}This can cut down on your request size, if you don't need the full book information when you fetch authors.
Thanks to Lars-Jørgen Kristiansen for implementing this feature!
By default relational-pouch will store the child ids of an Many-to-one relationship as a property on the many side (the parent). This can lead to extra conflicts, since this goes against the normal "documents are changes" way that Couch works best. Edits to documents are best to be self contained, and changing a parent document because a child is inserted can result in problems with multiple users.
A way to fix this is to specify to relational-pouch that the parent actually does not store this array of children. But instead relational-pouch should use a db.find query to search for them.
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
books: {hasMany: {type: 'book', options: {queryInverse: 'author'}}}
}
},
{
singular: 'book',
plural: 'books',
relations: {
author: {belongsTo: 'author'}
}
}
]);This will tell relational-pouch to not save the book ids on the author, and use a query using db.find to look for the related books.
Since this uses db.rel.findHasMany(type, type, belongsToKey, belongsToId) internally, you also need an index as specified there, where belongsToKey is the field specified in the queryInverse option.
For async relations this queryInverse will not work at this moment and the child id array will not be present on the result. You can use the db.rel.findHasMany(type, type, belongsToKey, belongsToId) for this scenario instead.
If you also don't give relational-pouch the child ids when calling db.rel.save you could also completely remove the hasMany side of the relation from the schema .
Deeply nested relationships are also possible. Everything just ends up being sideloaded in the same JSON object response.
{
"lions" : [...],
"tigers" : [...],
"bears" : [...]
}When you save, you must explicitly provide the ids of dependent objects, and they must be saved independently. There is no cascading at all.
You can attach the full entity object with an id to another object, but if you include an object without an id, it will be ignored.
db.setSchema([
{
singular: 'author',
plural: 'authors',
relations: {
profile: {belongsTo: 'profile'},
books: {hasMany: 'books'}
}
},
{
singular: 'profile',
plural: 'profiles',
relations: {
author: {belongsTo: 'author'}
}
},
{
singular: 'book',
plural: 'books',
relations: {
author: {belongsTo: 'author'}
}
}
]);
var profile = {
description: 'nice masculine jawline',
id: 21,
author: 19
};
var book1 = {
id: 1,
title: 'The Gunslinger'
};
var book2 = {
id: 2,
title: 'The Drawing of the Three'
};
var book3 = {
id: 3,
title: 'The Wastelands'
};
db.rel.save('profile', profile).then(function () {
return db.rel.save('book', book1);
}).then(function () {
return db.rel.save('book', book2);
}).then(function () {
return db.rel.save('book', book3);
}).then(function () {
return db.rel.save('author', {
name: 'Stephen King',
id: 19,
profile: profile,
books: [book1, book2, book3]
});
}).then(function () {
return db.rel.find('author');
});Result:
{
"authors": [
{
"name": "Stephen King",
"profile": 21,
"books": [
1,
2,
3
],
"id": 19,
"rev": "1-308a75619dc1b96bece7b6996d36d18b"
}
],
"profiles": [
{
"description": "nice masculine jawline",
"author": 19,
"id": 21,
"rev": "1-7bd39e62046a0816f9c5a3836a548ec8"
}
],
"books": [
{
"title": "The Gunslinger",
"id": 1,
"rev": "1-f3a305eae85642ce74412141ec0ae0bf"
},
{
"title": "The Drawing of the Three",
"id": 2,
"rev": "1-1c94deba48af8c1c2df1c5545246846b"
},
{
"title": "The Wastelands",
"id": 3,
"rev": "1-a4a96e3f9e2cb3d516605fa46bbed080"
}
]
}The plugin is not smart enough to infer bidirectional relationships, so you have to attach the relation to both object. E.g. in the above example, each book explicitly has its author set, and the author explicitly has his books set. If you want to add a new book, you would need to save() the book, add it to the author's list of books, and then save() the author.
When you update an existing object, you'll need to include the rev, or else you'll get a 409 conflict error. This is standard CouchDB/PouchDB behavior, so the common idiom is:
db.rel.find('post', 1).then(function (res) {
// do whatever you want to do to update the post
return db.rel.save('post', res.posts[0]).catch(function (err) {
if (err.code === 409) { // conflict
// handle the conflict somehow. e.g. ask the user to compare the two versions,
// or just try the whole thing again
} else {
throw err; // some other error
}
});
});This also applies to deletions:
db.rel.find('post', 1).then(function (res) {
return db.rel.del('post', res.posts[0]).catch(function (err) {
if (err.code === 409) { // conflict
// handle the conflict
} else {
throw err; // some other error
}
});
});To avoid getting into a long discussion of why you have to do this: suffice it to say, when you build a client-server sync architecture, you are building a distributed system. Distributed systems are hard, and managing conflicts is just a reality when you have multiple computers that aren't perfectly in sync.
You will have to deal with conflicts sooner or later. With PouchDB and CouchDB, you simply pay that cost up-front.
Jan Lenhardt has a nice writeup on this.
Thanks to bterkuile, this plugin also support attachments! Attachments are simply added inline, in the normal PouchDB way, but as
doc.attachments
rather than
doc._attachments
I.e. It follows the same convention as doc.id and doc.rev.
A relational Pouch/Couch is just a regular database that has been partitioned by type.
So for instance, a document with type "pokemon" and id "1" might have an actual _id like "pokemon_1", whereas a "trainer" with id "2" might have an actual _id like "trainer_2". It's not rocket science.
What is important is that this plugin leverages the very efficient allDocs() API, rather than the slower query() API. Also, it joins related documents by simply making extra requests, rather than using native map/reduce joined documents. And it's smart enough to group the requests, so the data is fetched in the fewest posisble number of requests.
Although this method may seem naïve, in practice you get much better performance, because building secondary indexes in Pouch/Couch takes quite a bit of time. Whereas if you just use allDocs(), it uses the built-in index on _id, so you don't have to wait for an index to be built.
This will run the tests in Node using LevelDB:
npm test
You can also check for 100% code coverage using:
npm run coverage
If you don't like the coverage results, change the values from 100 to something else in package.json, or add /*istanbul ignore */ comments.
If you have mocha installed globally you can run single test with:
TEST_DB=local mocha --reporter spec --grep search_phrase
The TEST_DB environment variable specifies the database that PouchDB should use (see package.json).
Run npm run dev and then point your favorite browser to http://127.0.0.1:8001/test/index.html.
The query param ?grep=mysearch will search for tests matching mysearch.
You can run e.g.
CLIENT=selenium:firefox npm test
CLIENT=selenium:phantomjs npm test
This will run the tests automatically and the process will exit with a 0 or a 1 when it's done. Firefox uses IndexedDB, and PhantomJS uses WebSQL.