Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync.Pool usage #11

Open
marniks7 opened this issue Sep 8, 2021 · 1 comment
Open

sync.Pool usage #11

marniks7 opened this issue Sep 8, 2021 · 1 comment

Comments

@marniks7
Copy link
Contributor

marniks7 commented Sep 8, 2021

Hi,
I was testing different bitmap libraries (by the way as a replacement for hash map)
The results i received for 8 and operations for 500k entities where first bitmap has 10k entities in a row with 1 bit, Clone of the bitmap is used on each operation

goos: linux
goarch: amd64
pkg: bitmap-usage/benchmark/500k-large-groups/kelindar
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkFindPriceV2_11position            	   26214	     41542 ns/op	   67760 B/op	       8 allocs/op
BenchmarkFindPriceV2_3824position          	   45291	     40293 ns/op	   73904 B/op	       8 allocs/op
BenchmarkFindPriceV2_3824position_OptStats 	   43438	     35956 ns/op	   73952 B/op	      13 allocs/op
BenchmarkFindPriceV2_9701position          	   41796	     36919 ns/op	   65712 B/op	       7 allocs/op
BenchmarkFindPriceV2_MultiplePricesErr     	   44133	     24858 ns/op	   73856 B/op	       6 allocs/op
PASS
ok  	bitmap-usage/benchmark/500k-large-groups/kelindar	51.610s

This is what i received for https://github.com/kelindar/column

goos: linux
goarch: amd64
pkg: bitmap-usage/benchmark/500k-large-groups/kelindar-column
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkFindPriceV2_11position            	   40641	     25217 ns/op	     216 B/op	       8 allocs/op
BenchmarkFindPriceV2_3824position          	   45633	     25901 ns/op	     216 B/op	       8 allocs/op
BenchmarkFindPriceV2_9701position          	   44162	     26637 ns/op	     217 B/op	       8 allocs/op
BenchmarkFindPriceV2_MultiplePricesErr     	   48385	     24083 ns/op	     177 B/op	       6 allocs/op
PASS
ok  	bitmap-usage/benchmark/500k-large-groups/kelindar-column	107.145s

In your kelindar-column you have fill bitmap.Bitmap // The fill-list and Txn with sync.Pool usage which allow to avoid bitmap allocation. I believe the technique is worth mentioning for potential users of this library

Results with sync.Pool:

goos: linux
goarch: amd64
pkg: bitmap-usage/benchmark/500k-large-groups/kelindar
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkFindPriceV2_11position            	   79882	     15122 ns/op	     176 B/op	       6 allocs/op
BenchmarkFindPriceV2_3824position          	   85920	     14792 ns/op	     176 B/op	       6 allocs/op
BenchmarkFindPriceV2_3824position_OptStats 	   68959	     15563 ns/op	     225 B/op	      11 allocs/op
BenchmarkFindPriceV2_9701position          	   56557	     19224 ns/op	     177 B/op	       6 allocs/op
BenchmarkFindPriceV2_MultiplePricesErr     	  106196	     11107 ns/op	     128 B/op	       4 allocs/op
PASS
ok  	bitmap-usage/benchmark/500k-large-groups/kelindar	61.455s

if you are curious here is hashmap results - iteration and comparison for 10k entities

goos: linux
goarch: amd64
pkg: bitmap-usage/benchmark/500k-large-groups/map
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkHttpClientServer_FindPrice            	    5232	    220943 ns/op	    8283 B/op	     112 allocs/op
BenchmarkFindPrice_11position                  	    9979	    118803 ns/op	      64 B/op	       1 allocs/op
BenchmarkFindPrice_3824position                	    9849	    119427 ns/op	      64 B/op	       1 allocs/op
BenchmarkFindPrice_3824position_Optimized      	   25146	     46990 ns/op	      64 B/op	       1 allocs/op
BenchmarkFindPrice_9701position                	    9825	    120534 ns/op	      64 B/op	       1 allocs/op
BenchmarkFindPrice_9701position_Optimized      	    9945	    117033 ns/op	      64 B/op	       1 allocs/op
BenchmarkFindPrice_MultiplePricesErr           	    9799	    119765 ns/op	       0 B/op	       0 allocs/op
BenchmarkFindPrice_MultiplePricesErr_Optimized 	    9814	    119057 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	bitmap-usage/benchmark/500k-large-groups/map	47.843s
@kelindar
Copy link
Owner

kelindar commented Sep 8, 2021

@marniks7 Thanks, I'll add this to documentation. Any other suggestions on what to improve in bitmap or column packages? By the way, great work on marniks7/bitmap-usage!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants