Skip to content

Commit

Permalink
clarity edits
Browse files Browse the repository at this point in the history
  • Loading branch information
@gritzko
gritzko committed Apr 14, 2024
1 parent d61e79b commit bd797b3
Show file tree
Hide file tree
Showing 5 changed files with 43 additions and 39 deletions.
30 changes: 17 additions & 13 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
[![Build Status](https://github.com/drpcorg/chotki/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/drpcorg/chotki/actions/workflows/test.yml)

<img align="right" width="30%" src="chotki.jpg">

Chotki is a syncable store with really fast counters.
Internally, it is [pebble db][p] running CRDT natively, using
the [Replicated Data Interchange][r] format (RDX). Chotki is
Expand Down Expand Up @@ -61,9 +62,9 @@ is very close to JSON, e.g.
````
Each command returns an ID and/or an error.

Text RDX is different from JSON in several aspects: it has
explicit ID type, arbitrary literals apart from `true`, `false`
and `null`, set collections and some other minor differences.
Text RDX is different from JSON in several aspects: it has the
ID type, arbitrary literals apart from `true`, `false` and
`null`, also set collections and some other minor differences.
On the human readability side, it is pretty much the same thing.

### HTTP
Expand All @@ -79,12 +80,14 @@ serialization/parsing, etc.

## Replicas

The N1 superpower of Chotki is syncing. Replicas may work
The main superpower of Chotki is syncing. Replicas may work
offline, reconnect and resync, or they may sync continuously in
real time. Chotki so far only supports a spanning-tree overlay
network topology. Each replica has a 20-bit "name" (aka
*source*); a replica can only connect to replicas with lesser
src number to prevent loops.
E.g. `a1ece` can connect to `b0b`, but not the other way around
(replica ids are given in hex).

Implementations of client replicas working on mobile devices or
in a browser are planned.
Expand All @@ -93,8 +96,8 @@ in a browser are planned.

Chotki is based on pebble db, which is an LSM database.
A superpower of LSM is "blind writes", i.e. writes with no
preceing read necessary. On a Lenovo Yoga laptop, a Chotki
replica can do about 1mln blind increments to a counter in about
preceding read necessary. On a Lenovo Yoga laptop, a Chotki
replica can do about 1mln blind increments of a counter in about
3 seconds, be it connected to other replicas or not:
````
◌ sinc {fid:b0b-6-2,count:10000000,ms:0}
Expand Down Expand Up @@ -124,7 +127,7 @@ One may check the [first 2017 RON/RDX talk][c] manifesting the
project's goals. In that regard, we may compare RON/RDX to
[Automerge][a], which is a project of exactly the same age. Both
projects started with a columnar-like coding of operations,
which Automerge is using to this day, while RDX followed
which Automerge is using to this day, while RDX followed the
Einstein's maxim: "Everything should be made as simple as
possible, but not simpler". After spending quite some time to
[cram][s] columnar-encoded CRDT into exising databases, RDX was
Expand All @@ -137,17 +140,18 @@ We can also compare Chotki to a number of JavaScript-centric
CRDT databases, such as [RxDB][x] or [SyncedStore][z].
Historically RON/RDX also has it roots in the JavaScript world.
[Swarm.js][j] was likely the first CRDT sync lib in history
(2013-2018); although it was distilled from the earlier Citrea
project (2011-2012). Still, Chotki/RDX has an objective of
creating a production-ready scalable CRDT store, which
JavaScript does not really allow. Still, we will be extremely
happy if some of the JavaScript libs would support RDX
as a unifying format. (Ping us any time!)
(2013-2018); although it was distilled from the earlier
[Citrea][t] project (2011-2012). Still, Chotki/RDX has an
objective of creating a production-ready scalable CRDT store,
which JavaScript does not really allow. Still, we will be
extremely happy if some of the JavaScript libs would consider
supporting RDX as a unifying format. (Ping us any time!)

[j]: https://github.com/gritzko/swarm
[a]: https://automerge.org/
[c]: https://www.youtube.com/watch?v=0Xx9kkTMi10
[s]: https://www.youtube.com/live/M8RRZakZgiI?si=yQVT0Le7FlnpfWXw&t=32187
[t]: https://github.com/gritzko/citrea-model
[x]: https://github.com/pubkey/rxdb
[z]: https://syncedstore.org/docs/

Expand Down
8 changes: 4 additions & 4 deletions rdx/ELM.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ func appendFirstTlvString(tlv []byte, lit byte, bare []byte) []byte {
// parse a text form into a TLV value
func Eparse(txt string) (tlv []byte) {
rdx, err := ParseRDX([]byte(txt))
if err != nil || rdx == nil || rdx.RdxType != ESet {
if err != nil || rdx == nil || rdx.RdxType != Eulerian {
return nil
}
for i := 0; i < len(rdx.Nested); i++ {
Expand Down Expand Up @@ -222,7 +222,7 @@ func Mstring(tlv []byte) (txt string) {
// parse a text form into a TLV value
func Mparse(txt string) (tlv []byte) {
rdx, err := ParseRDX([]byte(txt))
if err != nil || rdx == nil || rdx.RdxType != Map {
if err != nil || rdx == nil || rdx.RdxType != Mapping {
return nil
}
for i := 0; i < len(rdx.Nested); i++ {
Expand Down Expand Up @@ -338,7 +338,7 @@ func Lparse(txt string) (tlv []byte) {
bm, tlv = toytlv.OpenHeader(tlv, 'B')
tlv = append(tlv, '0')
rdx, err := ParseRDX([]byte(txt))
if err != nil || rdx == nil || rdx.RdxType != LArray {
if err != nil || rdx == nil || rdx.RdxType != Linear {
return nil
}
for i := 0; i < len(rdx.Nested); i++ {
Expand Down Expand Up @@ -473,7 +473,7 @@ func (a *LIterator) Merge(bb SortedIterator) int {
}

func Mrdx2tlv(a *RDX) (tlv []byte) {
if a == nil || a.RdxType != Map {
if a == nil || a.RdxType != Mapping {
return nil
}
for i := 0; i+1 < len(a.Nested); i += 2 {
Expand Down
12 changes: 6 additions & 6 deletions rdx/rdx.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ const (
NInc = byte('n')
ZCounter = byte('Z')
ZInc = byte('z')
ESet = byte('E')
LArray = byte('L')
Map = byte('M')
Eulerian = byte('E')
Linear = byte('L')
Mapping = byte('M')
)

type RDX struct {
Expand Down Expand Up @@ -80,7 +80,7 @@ func (rdx *RDX) Feed() (recs toyqueue.Records, err error) {
recs = append(recs, rdx.Text)
case Natural, NInc, ZCounter, ZInc:
recs = append(recs, rdx.Text)
case Map:
case Mapping:
recs = append(recs, RdxSep[RdxOOpen:RdxOOpen+1])
for i := 0; i+1 < len(rdx.Nested); i += 2 {
key, _ := rdx.Nested[i].Feed()
Expand All @@ -93,7 +93,7 @@ func (rdx *RDX) Feed() (recs toyqueue.Records, err error) {
}
}
recs = append(recs, RdxSep[RdxOClose:RdxOClose+1])
case ESet:
case Eulerian:
recs = append(recs, RdxSep[RdxOOpen:RdxOOpen+1])
for i := 0; i < len(rdx.Nested); i++ {
val, _ := rdx.Nested[i].Feed()
Expand All @@ -103,7 +103,7 @@ func (rdx *RDX) Feed() (recs toyqueue.Records, err error) {
}
}
recs = append(recs, RdxSep[RdxOClose:RdxOClose+1])
case LArray:
case Linear:
recs = append(recs, RdxSep[RdxAOpen:RdxAOpen+1])
for i := 0; i < len(rdx.Nested); i++ {
val, _ := rdx.Nested[i].Feed()
Expand Down
18 changes: 9 additions & 9 deletions rdx/rdx.ragel.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {
n := rdx.Nested
n = append(n, RDX{Parent: rdx})
rdx.Nested = n
rdx.RdxType = Map
rdx.RdxType = Mapping
rdx = &n[len(n)-1]
nest++
}
Expand All @@ -208,7 +208,7 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {
}
nest--
rdx = rdx.Parent
if rdx.RdxType != ESet && rdx.RdxType != Map {
if rdx.RdxType != Eulerian && rdx.RdxType != Mapping {
cs = _RDX_error
{
p += 1
Expand All @@ -218,9 +218,9 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {

}
if len(rdx.Nested) == 1 {
rdx.RdxType = ESet
rdx.RdxType = Eulerian
}
if rdx.RdxType == Map {
if rdx.RdxType == Mapping {
if (len(rdx.Nested) & 1) == 1 {
cs = _RDX_error
{
Expand All @@ -238,7 +238,7 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {
n := rdx.Nested
n = append(n, RDX{Parent: rdx})
rdx.Nested = n
rdx.RdxType = LArray
rdx.RdxType = Linear
rdx = &n[len(n)-1]
nest++
}
Expand All @@ -256,7 +256,7 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {
}
nest--
rdx = rdx.Parent
if rdx.RdxType != LArray {
if rdx.RdxType != Linear {
cs = _RDX_error
{
p += 1
Expand All @@ -279,9 +279,9 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {

}
n := rdx.Parent.Nested
if rdx.Parent.RdxType == Map {
if rdx.Parent.RdxType == Mapping {
if len(n) == 1 {
rdx.Parent.RdxType = ESet
rdx.Parent.RdxType = Eulerian
} else if (len(n) & 1) == 1 {
cs = _RDX_error
{
Expand Down Expand Up @@ -309,7 +309,7 @@ func ParseRDX(data []byte) (rdx *RDX, err error) {

}
n := rdx.Parent.Nested
if rdx.Parent.RdxType == Map {
if rdx.Parent.RdxType == Mapping {
if (len(n) & 1) == 0 {
cs = _RDX_error
{
Expand Down
14 changes: 7 additions & 7 deletions repl/commands.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ func (repl *REPL) CommandCreate(arg *rdx.RDX) (id rdx.ID, err error) {
name := "Unnamed replica"
if arg.RdxType == rdx.Reference {
src = rdx.IDFromText(arg.Text)
} else if arg.RdxType == rdx.Map {
} else if arg.RdxType == rdx.Mapping {
for i := 0; i+1 < len(arg.Nested); i += 2 {
key := arg.Nested[i]
val := arg.Nested[i+1]
Expand Down Expand Up @@ -112,7 +112,7 @@ var HelpClass = errors.New(
func (repl *REPL) CommandClass(arg *rdx.RDX) (id rdx.ID, err error) {
id = rdx.BadId
err = HelpClass
if arg == nil || arg.RdxType != rdx.Map || len(arg.Nested) < 2 {
if arg == nil || arg.RdxType != rdx.Mapping || len(arg.Nested) < 2 {
return
}
fields := arg.Nested
Expand Down Expand Up @@ -154,9 +154,9 @@ func (repl *REPL) CommandNew(arg *rdx.RDX) (id rdx.ID, err error) {
tlvs := toyqueue.Records{}
if arg == nil {
return
} else if arg.RdxType == rdx.LArray {
} else if arg.RdxType == rdx.Linear {
return
} else if arg.RdxType == rdx.Map {
} else if arg.RdxType == rdx.Mapping {
pairs := arg.Nested
if len(pairs) >= 2 && pairs[0].String() == "_ref" {
tid = rdx.IDFromText(pairs[1].Text)
Expand Down Expand Up @@ -207,7 +207,7 @@ var HelpEdit = errors.New(
func (repl *REPL) CommandEdit(arg *rdx.RDX) (id rdx.ID, err error) {
id = rdx.BadId
err = HelpEdit
if arg == nil || arg.RdxType != rdx.Map || len(arg.Nested) < 2 {
if arg == nil || arg.RdxType != rdx.Mapping || len(arg.Nested) < 2 {
return
}
if arg.Nested[0].String() == "_id" {
Expand Down Expand Up @@ -458,7 +458,7 @@ func (repl *REPL) CommandTinc(arg *rdx.RDX) (id rdx.ID, err error) {
return
} else if arg.RdxType == rdx.Reference {
id = rdx.IDFromText(arg.Text)
} else if arg.RdxType == rdx.Map {
} else if arg.RdxType == rdx.Mapping {
for i := 0; i+1 < len(arg.Nested); i += 2 {
key := &arg.Nested[i]
val := &arg.Nested[i+1]
Expand Down Expand Up @@ -517,7 +517,7 @@ func (repl *REPL) CommandSinc(arg *rdx.RDX) (id rdx.ID, err error) {
return
} else if arg.RdxType == rdx.Reference {
id = rdx.IDFromText(arg.Text)
} else if arg.RdxType == rdx.Map {
} else if arg.RdxType == rdx.Mapping {
for i := 0; i+1 < len(arg.Nested); i += 2 {
key := &arg.Nested[i]
val := &arg.Nested[i+1]
Expand Down

0 comments on commit bd797b3

Please sign in to comment.