Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull in fixes from across contributors #19

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
d81e566
fix imports
riannucci May 11, 2015
cf7fa95
Make keys able to hold 4GB as well (though I only plan to use it for …
riannucci May 11, 2015
10bbae2
Remove usage of unsafe from gkvlite.
riannucci Oct 20, 2015
1075668
Fix race in gkvlite introduced when removing `unsafe`
riannucci Nov 10, 2015
33184d2
Fix race in item too.
riannucci Nov 11, 2015
7263ff8
Redone fix against one without global locks
cbehopkins Jun 1, 2017
fdbd8b9
Re-worked tests to fix some concurrency termination issues and delete…
cbehopkins Jun 5, 2017
cd6712c
Ran gogmt
cbehopkins Jun 5, 2017
bad1dd4
Broke out race unsafe into separate file to allow race checking durin…
cbehopkins Jun 6, 2017
9e243d0
Tidied up after race detection fixes and go vet fixes
cbehopkins Jun 6, 2017
17a0208
Ran goimports and removed dead code and associated comments
cbehopkins Jan 3, 2018
cb5f1dd
Fix to tests, I'd forgotton a / for the directory separator
cbehopkins Jan 3, 2018
e139dbe
sloppy code in treap started fix
cbehopkins Jan 3, 2018
3be4eb9
Some golint suggested code tidies
cbehopkins Jan 3, 2018
1e2867d
Mereged in luci's fixes to races. Added REVISITs to document my conce…
cbehopkins Jan 4, 2018
cdc793c
(Optionally) Removed mutexs believed to be surplus. Needs testing wit…
cbehopkins Jan 4, 2018
25c89a2
Removing incorrect REVISITs, and gofmt
cbehopkins Jan 4, 2018
4f615e9
Moved store to using an rw mutext for performance
cbehopkins Jan 4, 2018
09734f3
Made size of key programmable
cbehopkins Jan 4, 2018
b75a1e0
Mereged in programmable key size - kept as const to allow compile tim…
cbehopkins Jan 4, 2018
5ea6c4d
ran gofmt -s
cbehopkins Jan 4, 2018
ca7a37f
Made Naming conform to golang norms
cbehopkins Jan 4, 2018
b2563c3
Added in block visitor
cbehopkins Jan 4, 2018
6a6e195
Merge branch 'cbehopkins_mutex_removal'
cbehopkins Jan 4, 2018
cbcb028
Merged in master before more fixes#
cbehopkins Jan 4, 2018
472162c
Fixed Receivers to be common. Now golint clean
cbehopkins Jan 4, 2018
fb03297
Added in tests for random and block readers
cbehopkins Jan 11, 2018
b5341eb
Added in parallel testing of store
cbehopkins Jan 12, 2018
b6f5a2b
Merged master back in
cbehopkins Jan 12, 2018
be411cc
Breaking out some functionality into separate functions to try and in…
cbehopkins Jan 21, 2018
97ea926
Added in universal getters and setters to allow better code paritioni…
cbehopkins Jan 22, 2018
60f41b0
Fixed golint issues after updates
cbehopkins Jan 22, 2018
b04c963
Checking error status from os.Remove
cbehopkins Jan 23, 2018
35b9e46
Test pass after some significant changes
cbehopkins Jan 23, 2018
a813295
Applie gometalinter fixes. Caught all returned error values and checked
cbehopkins Jan 24, 2018
c4f9df5
update README.md
dpavlenkov Apr 25, 2019
e221c8a
ensure file close
dpavlenkov Apr 25, 2019
7c76a35
update README.md
dpavlenkov Apr 25, 2019
a474ee1
Update README.md
dpavlenkov Apr 25, 2019
8fc1d6d
fix tools
dpavsrtrl Apr 25, 2019
db786f7
example, fixes
dpavsrtrl Apr 25, 2019
2935fb5
iterators
dpavsrtrl Apr 25, 2019
270a22b
Updated to use a go.mod that points at this repo
cbehopkins Apr 8, 2021
00fd08c
Updated to say this verion is on 1.16
cbehopkins Apr 8, 2021
dd6ad96
Fix mod file
cbehopkins Jun 17, 2021
634e5da
Minor go vet fixes
cbehopkins Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
*~
*.test
*.tmp
.idea
tmp
tools/view/view
tools/slab/slab
tools/view/view.exe
tools/slab/slab.exe
test.gkvlite
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ gkvlite

gkvlite is a simple, ordered, ACID, key-value persistence library for Go.

[![GoDoc](https://godoc.org/github.com/steveyen/gkvlite?status.svg)](https://godoc.org/github.com/steveyen/gkvlite) [![Build Status](https://drone.io/github.com/steveyen/gkvlite/status.png)](https://drone.io/github.com/steveyen/gkvlite/latest) [![Coverage Status](https://coveralls.io/repos/steveyen/gkvlite/badge.png)](https://coveralls.io/r/steveyen/gkvlite)
[![GoDoc](https://godoc.org/github.com/cbhvn/gkvlite?status.svg)](https://godoc.org/github.com/cbhvn/gkvlite) [![Build Status](https://travis-ci.org/cbhvn/gkvlite.svg?branch=master)](https://travis-ci.org/cbhvn/gkvlite)[![Coverage Status](https://coveralls.io/repos/github/cbhvn/gkvlite/badge.svg?branch=master)](https://coveralls.io/github/cbhvn/gkvlite?branch=master)

Overview
========
Expand All @@ -28,7 +28,7 @@ Key concepts
* A Store can have zero or more Collections.
* A Collection can have zero or more Items.
* An Item is a key and value.
* A key is a []byte, max length 64KB (length is uint16).
* A key is a []byte, max length 4GB/64kB (length is uint32/uint16).
* A value is a []byte, max length 4GB (length is uint32).

ACID properties
Expand Down Expand Up @@ -163,7 +163,7 @@ Examples

import (
"os"
"github.com/steveyen/gkvlite"
"github.com/cbhvn/gkvlite"
)

f, err := os.Create("/tmp/test.gkvlite")
Expand Down
32 changes: 18 additions & 14 deletions alloc.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@ import (
"fmt"
"sync"
"sync/atomic"
"unsafe"
)

// Package global free lists & locks
// Allows memory based lists to be global for efficiency
var freeNodeLock sync.Mutex
var freeNodes *node

Expand All @@ -16,8 +17,11 @@ var freeNodeLocs *nodeLoc
var freeRootNodeLocLock sync.Mutex
var freeRootNodeLocs *rootNodeLoc

// This is maintained as a package global statistics
// allows tracking of stats on a collection vs package
var allocStats AllocStats

// AllocStats holds allocation statistics for profiling
type AllocStats struct {
MkNodes int64
FreeNodes int64 // Number of invocations of the freeNode() API.
Expand Down Expand Up @@ -72,7 +76,7 @@ func (t *Collection) reclaimMarkUpdate(nloc *nodeLoc,
return n
}

func (t *Collection) reclaimNodes_unlocked(n *node,
func (t *Collection) reclaimNodesUnlocked(n *node,
reclaimLater *[3]*node, reclaimMark *node) int64 {
if n == nil {
return 0
Expand All @@ -95,9 +99,9 @@ func (t *Collection) reclaimNodes_unlocked(n *node,
if !n.right.isEmpty() {
right = n.right.Node()
}
t.freeNode_unlocked(n, reclaimMark)
numLeft := t.reclaimNodes_unlocked(left, reclaimLater, reclaimMark)
numRight := t.reclaimNodes_unlocked(right, reclaimLater, reclaimMark)
t.freeNodeUnlocked(n, reclaimMark)
numLeft := t.reclaimNodesUnlocked(left, reclaimLater, reclaimMark)
numRight := t.reclaimNodesUnlocked(right, reclaimLater, reclaimMark)
return 1 + numLeft + numRight
}

Expand Down Expand Up @@ -134,7 +138,7 @@ func (t *Collection) mkNode(itemIn *itemLoc, leftIn *nodeLoc, rightIn *nodeLoc,
return n
}

func (t *Collection) freeNode_unlocked(n *node, reclaimMark *node) {
func (t *Collection) freeNodeUnlocked(n *node, reclaimMark *node) {
if n == nil || n == reclaimMark {
return
}
Expand All @@ -145,9 +149,9 @@ func (t *Collection) freeNode_unlocked(n *node, reclaimMark *node) {
if i != nil {
t.store.ItemDecRef(t, i)
}
n.item = *empty_itemLoc
n.left = *empty_nodeLoc
n.right = *empty_nodeLoc
n.item = itemLoc{}
n.left = nodeLoc{}
n.right = nodeLoc{}
n.numNodes = 0
n.numBytes = 0
n.next = freeNodes
Expand All @@ -173,22 +177,22 @@ func (t *Collection) mkNodeLoc(n *node) *nodeLoc {
allocStats.CurFreeNodeLocs--
freeNodeLocLock.Unlock()
}
nloc.loc = unsafe.Pointer(nil)
nloc.node = unsafe.Pointer(n)
nloc.loc = nil
nloc.node = n
nloc.next = nil
return nloc
}

// Assumes that the caller serializes invocations.
func (t *Collection) freeNodeLoc(nloc *nodeLoc) {
if nloc == nil || nloc == empty_nodeLoc {
if nloc == nil || nloc == &emptyNodeLoc {
return
}
if nloc.next != nil {
panic("double free nodeLoc")
}
nloc.loc = unsafe.Pointer(nil)
nloc.node = unsafe.Pointer(nil)
nloc.loc = nil
nloc.node = nil

freeNodeLocLock.Lock()
nloc.next = freeNodeLocs
Expand Down
Loading