Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fold in rocksdb libs with josh #1974

Closed
wants to merge 285 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
285 commits
Select commit Hold shift + click to select a range
7468483
Fix for memory corruption and segfaults.
milaz Feb 16, 2012
9a92f76
Credit contributors in .cabal file
kim Feb 17, 2012
5976b2c
Adding a mapIter function and building iterItems, iterKeys and iterVa…
Mar 12, 2012
ee456b4
Add myself to the list of authors
Mar 12, 2012
7a56afe
Pull in latest leveldb changes
kim Mar 13, 2012
f5f5d0b
Update example to use iterItem
kim Mar 14, 2012
489e495
Use throwIO instead of ioError
kim Apr 7, 2012
7dd11ad
Add Eq as suggested by PR #7
kim Apr 7, 2012
acdcae8
Release 0.0.3 to get people's contributions on hackage
kim Apr 7, 2012
92b8419
Improve memory management
kim Apr 9, 2012
e3a569e
Inline helpers
kim Apr 9, 2012
f5e54aa
Make _cachePtr strict
kim Apr 10, 2012
04a6e08
Synchronize access to Iterator using MVar
kim Apr 10, 2012
da63363
Unbundle LevelDB, as it now has shared lib support.
kim Apr 11, 2012
f8987d5
Cleanup Makefile & .gitignore
kim Apr 11, 2012
44c254e
WIP: manage (memory) resources using conduit's ResourceT
kim Apr 11, 2012
9835276
Bundle patched version of Control.Monad.Trans.Resource with LIFO deal…
kim Apr 16, 2012
5a074fd
Drop aliasing and re-export Control.Monad.Trans.Resource
kim Apr 17, 2012
afdb81b
Add documentation
kim Apr 17, 2012
fb65eea
Moar documentation
kim Apr 17, 2012
108a8e0
Less messed-up implementation of LIFO deallocation
kim Apr 18, 2012
64071fa
Remove all ResourceT gymnastics, as it now runs finalizers LIFO
kim Apr 19, 2012
93b5eb5
Re-introduce user-defined comparator
kim Apr 21, 2012
aa91fe2
Record syntax for options
kim Apr 21, 2012
930f884
Remove static linking flags
kim Apr 21, 2012
815f820
Remove -debug flag
kim Apr 21, 2012
f759de9
Satisfy iterator invariants
kim Apr 23, 2012
959fe6b
Manage ReadOptions, WriteOptions and WriteBatch in ResourceT
kim Apr 23, 2012
ea79f9e
Document options, expose iterGetError, make mapIter tail recursive
kim May 3, 2012
a090f7b
Additional documentation
kim May 7, 2012
186fd69
Whitespace and more consistent qualified names
kim Jun 11, 2012
c1a0456
Copy ByteStrings into CStringLens
kim Jun 11, 2012
2b69983
Patch release 0.1.1
kim Jun 11, 2012
f2cd264
Add *.hi to .gitignore
kim Nov 11, 2012
e8afc1a
Make GC-safe batch write without copying
kim Nov 11, 2012
842ed05
Add convenient Makefile for examples
kim Nov 11, 2012
017ab0f
Add support for bloom filter ("filter policy")
kim Nov 11, 2012
14ab4fb
Add support for querying the runtime version of libleveldb
kim Nov 11, 2012
832d1c0
Release 0.2.0
kim Nov 11, 2012
a0c0d5e
Try setting up travis build
kim Nov 22, 2012
97497ce
Tell travis about haskell
kim Nov 22, 2012
ea771b0
Use make for install (travis)
kim Nov 22, 2012
2cb7257
Typo
kim Nov 22, 2012
84201f0
Copy leveldb header files to /usr/local/include (travis)
kim Nov 22, 2012
d727232
Add fancy travis build status image
kim Nov 22, 2012
92473e8
Fix build of 'features' example
NicolasT Apr 12, 2013
7130aa3
Optionally build examples through Cabal
NicolasT Apr 13, 2013
4381888
Remove unnecessary imports & dependencies
NicolasT Apr 13, 2013
e362e57
Builds examples during TravisCI run
NicolasT Apr 13, 2013
575f7f1
Prefix example executables with 'leveldb-example-'
kim Apr 13, 2013
390dfeb
Bug: Fix 'fillCache' option setting
NicolasT Apr 13, 2013
3033546
Move Database.LevelDB.Base to Database.LevelDB.C
NicolasT Apr 13, 2013
0e88377
Create low-level API without MonadResource context
NicolasT Apr 13, 2013
cec51a6
Refactor iterKey & iterValue
NicolasT Apr 13, 2013
65eca0d
Drop record syntax for Iterator
kim Apr 19, 2013
670300f
Free options after handle
kim Apr 19, 2013
5e0d499
Don't use default-extensions + "stylish-haskell"-style imports
kim Apr 19, 2013
3b237f5
Formatting
kim Apr 21, 2013
5c20c2a
Fix Iterator locking
kim Apr 21, 2013
6ecf181
Reorganize modules
kim Apr 21, 2013
a95d985
Adjust ghc options
kim Apr 21, 2013
f99905d
Move ResourceT API to it's own module
kim Apr 22, 2013
9380b78
(Re-)introduce withSnapshot combinator in Base
kim Apr 22, 2013
65a756c
Require external synchronization for iterator access
kim May 21, 2013
706da5a
Add note about external synchronization
kim May 21, 2013
3fc44cb
(Re-) introduce 'withIter' bracket
kim Jan 10, 2014
f5802ca
whitespace
kim Jan 10, 2014
9bcd93b
Add a changelog
kim Jan 10, 2014
04b1958
Release 0.3 (finally)
kim Jan 10, 2014
59fa804
first rocksdb version
agrafix Jul 29, 2014
3eff6c0
switched to drone.io
agrafix Jul 29, 2014
35c44fc
Add stack.yaml, update .cabal and .gitignore
gromakovsky Dec 7, 2016
af625cd
Comment out currently broken functionality
gromakovsky Dec 7, 2016
0db279e
Cosmetic: Redundant MonadResource removed, specified imports and remo…
pva701 Dec 9, 2016
41704ad
Removed MonadResource
pva701 Dec 9, 2016
bbd4536
Removed ' from function's name
pva701 Dec 9, 2016
95a090a
Added get and put methods for Binary
pva701 Dec 12, 2016
23d16fe
Add Linux/Windows/OSX CI
domenkozar Dec 18, 2016
af821d1
Also support Windows
domenkozar Dec 19, 2016
92a91e7
No need to install OpenSSL for windows
domenkozar Dec 20, 2016
f3ea378
[CSL-275] Remove not implemented functionality
gromakovsky Jan 11, 2017
76a927d
Remove useless flags from .cabal
gromakovsky Jan 11, 2017
9add539
Increase version
gromakovsky Jan 11, 2017
ac2ad2a
Add basic tests
gromakovsky Jul 17, 2014
fd6b0a3
tests: also run on Windows
domenkozar Jan 14, 2017
bac56a4
README: note windows support
domenkozar Jan 14, 2017
93359a1
Use c_rocksdb_free instead of standard free
gromakovsky Jan 14, 2017
707c432
nix: Use dynamically linked rocksdb
domenkozar Jan 14, 2017
aaf85d0
appveyor: use rocksdb 4.13.5
domenkozar Jan 14, 2017
4338848
appveyor: remove cache as it's unreliable
domenkozar Jan 16, 2017
492563b
appveyor: copy the dll
domenkozar Jan 17, 2017
101bb67
README: correct appveyor link
domenkozar Jan 18, 2017
c2564e1
Readme.md -> README.md
domenkozar Jan 18, 2017
a0a0368
appveyor: add rocksdb.dll as artifact
domenkozar Jan 18, 2017
7920764
Fix extra-source-files in cabal
gromakovsky Jun 18, 2017
ef20e88
Change name to rocksdb-haskell
gromakovsky Jun 18, 2017
310be35
Make version consist of three components
gromakovsky Jun 18, 2017
757e7c0
Remove some leftovers
gromakovsky Jun 18, 2017
b24c894
Remove ghc-prof-options
gromakovsky Jun 18, 2017
5eb8d9d
Set version to 1.0.0
gromakovsky Jun 18, 2017
792ef31
[CSL-1803] On Linux, use UTF-8 for filepath encoding
Nov 17, 2017
5ce3191
[CSL-1803] Fix Unicode problem in Windows
rockbmb Nov 24, 2017
e40524c
[CSL-1803] Bump LTS version to 9.14, so it uses ghc-8.0.2, which is a…
deepfire Dec 1, 2017
df28278
[CSL-1803] Increase package version, update changelog
rockbmb Dec 1, 2017
4567c20
Exported 2 functions for usage in batch writes
Heimdell Mar 28, 2018
1c6b85c
Build rocksdb
edmundnoble Mar 28, 2022
6065b93
Includes no
edmundnoble Mar 30, 2022
71280ee
Move includes
edmundnoble Mar 30, 2022
6526ece
Try again
edmundnoble Mar 30, 2022
9384147
Hm
edmundnoble Mar 30, 2022
b415dce
Attempt to fix mac build
edmundnoble Apr 4, 2022
12247b2
Add back libCrocksdb.so
edmundnoble Apr 4, 2022
a4aa8a6
Remove gflags dep
edmundnoble Apr 4, 2022
db3f8e1
What is with these library suffices??
edmundnoble Apr 4, 2022
f7226fb
Checkpoint using cabal to build C sources
edmundnoble Apr 5, 2022
c0b3dbb
Revert "Remove gflags dep"
edmundnoble Apr 5, 2022
56e7c01
Revert "What is with these library suffices??"
edmundnoble Apr 5, 2022
5668d76
Revert "Checkpoint using cabal to build C sources"
edmundnoble Apr 5, 2022
5f3f13c
Don't even build rocksdb on mac
edmundnoble Apr 6, 2022
6ae9555
Remove includes
edmundnoble Apr 6, 2022
b02b123
Stop bundling it if we don't build it
edmundnoble Apr 6, 2022
1b29f7f
Build on march=ivybridge
edmundnoble Apr 6, 2022
3895b31
Add snappy and bz2 to extra_libs
edmundnoble Apr 6, 2022
676840f
Enable -DSNAPPY
edmundnoble Apr 7, 2022
4c15dfb
Ping
edmundnoble Apr 7, 2022
36b993e
Change around quotes
edmundnoble Apr 7, 2022
5c06a97
Arch generic
edmundnoble Apr 7, 2022
5fcdb6e
Build shared and static libraries separately
edmundnoble Apr 8, 2022
62f605d
-fPIC
edmundnoble Apr 8, 2022
5eb1d7d
Build shared first
edmundnoble Apr 8, 2022
bbe2607
No shared lib
edmundnoble Apr 8, 2022
6d22a53
Clean after building
edmundnoble Apr 8, 2022
9541bb4
Stop building shared
edmundnoble Apr 8, 2022
2679f4f
Hmm
edmundnoble Apr 8, 2022
08573c0
Comment and reorganize Setup.hs
edmundnoble Apr 11, 2022
71ef517
Move FFI from chainweb-storage
edmundnoble Jul 20, 2022
10aab26
Hm
edmundnoble Jul 20, 2022
795ba60
Add CPP files
edmundnoble Jul 20, 2022
94033c6
Fix prefix extractor
edmundnoble Jul 20, 2022
66254e8
Add snappy to extra-libs
edmundnoble Jul 20, 2022
4eee0f0
Native build with GHC
edmundnoble Jul 21, 2022
8ddab46
C++17
edmundnoble Jul 21, 2022
25b1eee
Stop statically linking libstdc++
edmundnoble Jul 21, 2022
6032675
check
edmundnoble Jul 22, 2022
5c03284
correct table prefix extractor signature
edmundnoble Jul 22, 2022
e222a2f
Remove opts_ptr from DB datatype
edmundnoble Jul 22, 2022
99f692e
Put prefix extractor into options
edmundnoble Jul 23, 2022
a956a53
Disable setting prefix extractor again
edmundnoble Jul 23, 2022
a4f00f8
Make the prefix extractor but don't use it
edmundnoble Jul 23, 2022
eaab737
Try less elaborate way to free prefix extractor
edmundnoble Jul 25, 2022
1609c1f
Remove unused Options' field
edmundnoble Jul 26, 2022
50b1e8a
Re-enable TBB, disable AVX2
edmundnoble Jul 29, 2022
a252d0d
Don't expose IteratorPtr
edmundnoble Jul 29, 2022
3f6860d
Fix build so it compiles for aarch64-darwin (M1 Mac)
jwiegley Nov 18, 2022
cb9925e
Flag to disable tbb builds
sirlensalot Dec 7, 2022
32d10bd
positive flag name
sirlensalot Dec 7, 2022
7174a87
disable by default
sirlensalot Dec 10, 2022
6883374
Disable march=native
edmundnoble Jan 5, 2023
0564a84
try march=x86-64
edmundnoble Jan 9, 2023
557d7f1
Remove HAVE_SSE42
edmundnoble Jan 9, 2023
71385fe
Delete -D options add mtune=generic
edmundnoble Jan 9, 2023
fac5aab
Enable SSE3
edmundnoble Jan 9, 2023
aa862f2
add read-only mode
edmundnoble Jun 8, 2023
1baaf67
Only include certain compiler flags if we're on x86-64
jwiegley Feb 8, 2023
f06b8bd
Only define Fast_CRC32 on Linux (for now)
jwiegley Feb 8, 2023
03ab528
Include Fast_CRC32 on Intel Mac machines
jwiegley Feb 14, 2023
72ffc46
Use newer rocksdb
edmundnoble Aug 14, 2023
6b969c0
Use std=c++2a for older gcc
edmundnoble Sep 21, 2023
2d44704
cabal: remove `-Werror` for cxx sources (#12)
thoughtpolice May 8, 2024
5e3bf9c
build rocksdb without -Werror (#11)
chessai May 12, 2024
a8368ff
initial commit
larskuhtz Jul 9, 2019
6789db4
[DedupStore] only insert if dedupHash doesn't already exist
larskuhtz Jul 11, 2019
ea5e0ae
Fix tableIterLast in Data.CAS.RocksDB
larskuhtz Jul 26, 2019
e78da2b
Add a simple test-suite for rocksdb tables
larskuhtz Jul 26, 2019
20871a4
Remove redundant "instance" key word
larskuhtz Aug 13, 2019
659e4e9
add travis-ci script
larskuhtz Aug 13, 2019
0b6f2ba
Add support for atomic write batches
larskuhtz Aug 13, 2019
8f05c59
More rocks db table tests
larskuhtz Aug 15, 2019
852aefd
cas batch calls for RocksDbTable
larskuhtz Aug 16, 2019
653cba0
Add HasCasLookup
larskuhtz Apr 10, 2020
b2f1423
update README
larskuhtz Sep 14, 2020
62281e1
add NoThunks instances to RocksDB types
larskuhtz Sep 14, 2020
4837df6
update travis-ci
larskuhtz Sep 15, 2020
84f188d
github actions workflow
larskuhtz Sep 15, 2020
77043d0
build with ghc-9
larskuhtz May 10, 2021
e213946
Fix caching in GitHub Actions build
larskuhtz Oct 30, 2020
b3eb1ae
GitHub actions update dependencies
larskuhtz Nov 17, 2020
960e027
update ci build
larskuhtz Jun 24, 2021
d31cc8f
Add RocksDB checkpoint capabilities separate from the CAS interface
edmundnoble Dec 20, 2021
9ff5895
Add table size approximation and move checkpoint into the main RocksD…
edmundnoble Dec 21, 2021
090a7ec
Address review
edmundnoble Dec 30, 2021
0099c91
Namespace movements
edmundnoble Dec 30, 2021
3e4b35b
Edmund/rocksdb delete range (#12)
edmundnoble Mar 4, 2022
e7b1f0d
Better ReadOptions API, scaffolding
edmundnoble Mar 15, 2022
7a9141c
set upper and lower bounds on iterator
edmundnoble Mar 15, 2022
7bd9154
Set auto prefix mode
edmundnoble Mar 15, 2022
92e1687
Set prefix extractor
edmundnoble Mar 16, 2022
8f4806f
Enable C++17
edmundnoble Mar 16, 2022
164eb2e
What
edmundnoble Mar 16, 2022
9bee99a
Expose modernDefaultOptions
edmundnoble Mar 17, 2022
0f92786
Enable createIfMissing by default
edmundnoble Mar 17, 2022
687ad74
More flags
edmundnoble Mar 17, 2022
6ab84dc
Disable prefix extractor
edmundnoble Mar 19, 2022
57bb819
Check
edmundnoble Mar 29, 2022
75f9b05
Add include dir
edmundnoble Mar 30, 2022
fa86bb6
Use includes, not install-includes
edmundnoble Mar 30, 2022
c0b97c7
Update rocksdb-haskell
edmundnoble Mar 30, 2022
fedb1e6
implement rocksdb multi-get
larskuhtz Mar 15, 2022
1566b0a
Revert "implement rocksdb multi-get"
edmundnoble Apr 4, 2022
ce5dc32
Alter extra-libraries, rocksdb commit
edmundnoble Apr 11, 2022
534c367
Revert "Revert "implement rocksdb multi-get""
edmundnoble Apr 11, 2022
883625c
Update rocksdb-haskell-kadena
edmundnoble Apr 11, 2022
886cb61
Comments
edmundnoble Apr 11, 2022
380229c
Move FFI to rocksdb-haskell
edmundnoble Jul 20, 2022
9c33957
Fix prefix extractor call
edmundnoble Jul 22, 2022
2af8cdf
Disable prefix extractor memoization
edmundnoble Jul 22, 2022
4339967
Safely set prefix extractor without leaks
edmundnoble Jul 22, 2022
4b670bb
Let withRocksDb take an Options instead of a raw pointer
edmundnoble Jul 23, 2022
40c6671
Update rocksdb
edmundnoble Jul 26, 2022
e608d40
Update rocksdb-haskell
edmundnoble Jul 29, 2022
5c763e5
Review
edmundnoble Jul 30, 2022
45baaa5
Update rocksdb-haskell
edmundnoble Aug 10, 2022
1dbfda4
Update chainweb-storage.cabal
edmundnoble Aug 1, 2022
5d9194d
Update rocksdb-haskell pin
edmundnoble Aug 10, 2022
3b04c7d
add a Table abstraction replacing the Cas abstraction
edmundnoble Sep 26, 2022
b53c21b
Iterator instance for RocksDB, warnings
edmundnoble Sep 26, 2022
5541e1a
fix tests
edmundnoble Sep 27, 2022
c4e9b70
Extra eq constraints for earlier GHC
edmundnoble Sep 27, 2022
435394f
Relax tableLookupM constraint to ReadableTable
edmundnoble Sep 27, 2022
e865642
Add Casify newtype
edmundnoble Sep 28, 2022
e0482f7
Make migration easier
edmundnoble Sep 28, 2022
8a2a303
Add *1 type aliases
edmundnoble Sep 28, 2022
4b9cb7f
Relax casLookupM constraint and fix indentation
edmundnoble Sep 28, 2022
0160f1c
Add unCasify accessor
edmundnoble Sep 28, 2022
4b6b23b
Eachify tableLookupBatch
edmundnoble Oct 15, 2022
fafe3dc
Explicitly use c_rocksdb_free instead of our own free, traversalify t…
edmundnoble Oct 16, 2022
e06ce08
Fix docs and build
edmundnoble Jan 4, 2023
2d82f2d
Attempt mac fix
edmundnoble Jan 5, 2023
7aa139d
s/8.10.5/8.10.7
edmundnoble Jan 5, 2023
21e41df
Include CI updates
edmundnoble Jan 5, 2023
ebd5867
Remove mac fix attempt
edmundnoble Jan 5, 2023
07edf3e
add read-only rocksdb
edmundnoble Jun 8, 2023
1f54d0f
Update rocksdb
edmundnoble Jan 29, 2024
318ec8a
move tests of vendored libraries to canonical folder
chessai Jun 27, 2024
b8b2aba
vendor chainweb-storage and rocksdb-haskell-kadena
chessai Jun 27, 2024
7529dbf
re-add rosetta cabal.project srp
chessai Aug 6, 2024
7c34677
use rocksdb-haskell-kadena as a private library
chessai Aug 6, 2024
3e220fa
update freeze file
chessai Aug 6, 2024
847c210
remove unneeded files from vendored libraries
chessai Aug 6, 2024
a28b8a7
remove packageimports for internal libraries
chessai Aug 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 11 additions & 1 deletion .github/workflows/applications.yml
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,12 @@ jobs:
run: |
ghc --version
cabal --version
- name: Install cabal-plan and confirm its installation
# only do this on master and release branches
if: ${{ startsWith(github.ref, 'refs/heads/release') || startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/master'}}
run: |
cabal install cabal-plan --flags="exe license-report" --constraint="cabal-plan==0.7.3.0" --allow-newer
cabal-plan --version
- name: Install non-Haskell dependencies (ubuntu)
if: contains(matrix.os, 'ubuntu')
run: sudo apt-get install -y libgflags-dev liblz4-dev libzstd-dev libsnappy-dev libbz2-dev
Expand Down Expand Up @@ -340,6 +346,10 @@ jobs:
- name: Build chainweb applications
run: cabal build -j2 --ghc-options=-j2 exe:chainweb-node exe:compact test:chainweb-tests exe:cwtool chainweb:bench:bench

- name: Check long-info output
run: |
$(cabal list-bin chainweb-node) --long-info

# Checks
- name: Check that working directory tree is clean
run: |
Expand Down Expand Up @@ -634,7 +644,7 @@ jobs:
OS: ${{ matrix.os }}
ARTIFACTS_NAME: chainweb-applications.${{ matrix.use-freeze-file }}.${{ matrix.ghc }}.${{ matrix.os }}
ARTIFACTS_ARCHIVE: chainweb.${{ matrix.use-freeze-file }}.${{ matrix.ghc }}.${{ matrix.os }}.${{ needs.config.outputs.git-sha-short }}${{ needs.config.outputs.tag-suffix }}.tar.gz

steps:
- name: Get build artifacts
uses: actions/download-artifact@v4
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Copyright (c) 2018 - 2022 Kadena LLC
Copyright (c) 2018 - 2024 Kadena LLC
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
Loading
Loading