Insert a new Skiplist implementation, repair links and add self-hosted images

README.md

@@ -4,7 +4,7 @@
 
 Here is a brief summary of skip list packages available in Go that you may consider using after a quick Google/Github search. If you know of any others, please contact me so I can add them here.
 
-Some things most of the packages have in common: 
+Some things most of the packages have in common:
 
 - Keys are `int` type, which are 32 or 64 bit depending on GOARCH
 - Values are generally of type `interface {}`, so they accept any datatype (Go does not have generics).
@@ -14,34 +14,40 @@ Some things most of the packages have in common:
 
 Here are some brief notes on each implementation:
 
-- [github.com/mtchavez/skiplist](github.com/mtchavez/skiplist)
+- [github.com/mtchavez/skiplist](https://github.com/mtchavez/skiplist)
   - Values are type `[]byte`, which almost always means conversion.
   - Global constant for *P* value = 0.25, cannot be changed at runtime.
-- [github.com/huandu/skiplist](github.com/huandu/skiplist)
+- [github.com/huandu/skiplist](https://github.com/huandu/skiplist)
   - Globally sets *P* to *almost* 0.25 (using bitmasks and shifting) and can be changed at runtime.
   - You must specify a comparator and type for keys when creating the list.
   - Keys are of type `interface{}`
   - Not threadsafe
-- [github.com/zhenjl/skiplist](github.com/zhenjl/skiplist)
+- [github.com/zhenjl/skiplist](https://github.com/zhenjl/skiplist)
   - Adjustable *P* value and max level per list.
   - Adjustable insert level probability per list.
   - Allows duplicates stored at a single key and therefore does not have an update operation.
   - When creating a list, you specify a comparator. It has many built-in that are generated by running an external script that writes out a Go source file with the interfaces.
   - Uses separate search and insert fingers to speed up finding highly local keys consecutively.
   - Threadsafe but fingers are shared as well across all lists
-- [github.com/golang-collections/go-datastructures/slice/skip](github.com/golang-collections/go-datastructures/)
+- [github.com/golang-collections/go-datastructures/slice/skip](https://github.com/golang-collections/go-datastructures/)
   - Intelligently sets maximum level based on key's datatype (uint8 up to uint64)
   - More complex interface; you must define an Entry type that implements the interface it specifies with a comparator
   - *P* value is a global constant, 0.5
-- [github.com/ryszard/goskiplist](github.com/ryszard/goskiplist)
+- [github.com/ryszard/goskiplist](https://github.com/ryszard/goskiplist)
   - P value is a global constant, 0.25
   - Very straightforward implementation and interface
   - Not threadsafe
-- [github.com/sean-public/fast-skiplist](github.com/sean-public/fast-skiplist)
-  - Fastest concurrent implementation in all benchmarks; `huandu` is very close in every benchmark but it is **not** threadsafe.
+- [github.com/sean-public/fast-skiplist](https://github.com/sean-public/fast-skiplist)
+  - Counting only concurrent implementations, `sean`  is the fastest.
+  - Overall it is always the second fastest in all benchmarks (With a few on-par with `mauriceGit (mt)`) and `huandu` coming very close in most benchmarks.
   - See fast-skiplist's README for details on how this is achieved.
-
-
+- [github.com/MauriceGit/skiplist](https://github.com/MauriceGit/skiplist)
+  - Fastest non-concurrent implementation over all benchmarks. `sean` is on-par regarding **search** and **worstDelete**.
+  - Not threadsafe
+  - Entry type needs to be defined, that implements two simple methods for key extraction and printing.
+  - *P* value can not be set as the calculated level-height is calculated in O(1) instead of normal O(n) coin flips (See repository for detailed implementation).
+  - Only one call to the random number generator is needed per insert! (Instead of the #height generated).
+  - Convenience functions to retrieve next and previous elements are implemented as well as a method to change values in the list without the need of re-insertion.
 
 ### Benchmarks
 
@@ -55,36 +61,30 @@ skiplist-survey > output.csv
 
 The results are in CSV format for easy charting and analysis.
 
-Here is a summary of results I recorded on a Macbook Pro 15 with a 2.7 GHz Intel Core i7 and 16GB RAM. It takes over an hour to run all the benchmarks.
-
-![best inserts chart](http://i.imgur.com/Vo5etzd.png)
-
-The chart above shows the **best-case insert** speeds. The vertical axis is nanoseconds per operation and the horizontal is the number of items in the list. These are the "best" inserts because they happen at the front of the list, which shouldn't require any searching. The difference in speed here demonstrates the overhead each package introduces in even the most basic operations.
-
-
-
-![worst inserts chart](http://i.imgur.com/Z47mCm1.png)
-
-**Worst-case inserts.** These inserts are at the end of the list, requiring searching all the way to the end and then adding the new node. As you can see, `mtchavez` does not scale as well as the other implementations, which only show a small variance even after millions of nodes are added.
+Here is a summary of results I recorded on a Dell Precision with a 2.2 GHz Intel Core i7 (2720QM) and 8GB RAM. It takes over an hour to run all the benchmarks.
 
+The vertical axis is **nanoseconds per operation**, the horizontal is the number of items in the list.
 
 
-![average search chart](http://i.imgur.com/OFgOZQu.png)
+![best inserts chart](graphs/inserts.png)
+**Best-case insert**. These are the "best" inserts because they happen at the front of the list, which shouldn't require any searching. The difference in speed here demonstrates the overhead each package introduces in even the most basic operations.
 
-**Average search speed**. You can see `mtchavez` again is not searching in *O(log n)* or even *O(n)*. `zhenjl` also appears to have a lot of overhead in its search compared to the other implementations.
+![worst inserts chart](graphs/worstInserts.png)
+**Worst-case inserts.** These inserts are at the end of the list, requiring searching all the way to the end and then adding the new node. Even though all implementations only show a small variance even after millions of nodes are added, we can still see very large differences in overall speed because of implementation overhead.
 
+![random inserts chart](graphs/randomInserts.png)
+**Random inserts.** The inserts are at random positions in the skiplist, making this the closest real-world case for inserts. The approximately logarithmic behaviour is clearly visible for all implementations even though the overhead makes `ryszard` take 3x as long as `mauriceGit (mt)`.
 
+![average search chart](graphs/avgSearch.png)
+**Average search speed**. `mtchavez`, `sean` and `mauriceGit (mt)` are approximately equally fast. `zhenjl` seems to introduce some serious overhead, making it more than 8x as slow as the fastest implementations.
 
-![worst case delete chart](http://i.imgur.com/LxSov5E.png)
-
+![worst case delete chart](graphs/worstDelete.png)
 **Worst case deletions**. In this benchmark, a skip list of a given length is created and then every item is removed, starting from the last one and moving to the front.
+`mauriceGit (mt)` and `sean` are around equally fast with around 100ns faster than the next contestant (`huandu`).
 
-
-
-![zoom worse cases deletions](http://i.imgur.com/LQYoXuO.png)
-
-If we zoom in, we can see the speed differences between the fastest implementations a bit better. The fastest overall is `sean`, which averages 10-50ns faster in all operations than the next fastest implementation.
-
+![random delete chart](graphs/randomDelete.png)
+**Random deletions**. Elements are removed from random positions in the skiplist. For Deletions, this is the closest to a real-world case. We can clearly see the the logarithmic behaviour, even though some implementations have a large overhead involved.
+Just like for randomInserts, `mauriceGit (mt)` is the fastest, closely followed by `sean`.
 
 
 ### Todo
@@ -98,6 +98,3 @@ If we zoom in, we can see the speed differences between the fastest implementati
 - Benchmark concurrent inserts on multiple lists to stress globally-locked PRNG in most implementations.
 
   ​
-
-
-

benchmark.go

@@ -55,6 +55,7 @@ func main() {
 	allFunctions = append(allFunctions, huanduFunctions...)
 	allFunctions = append(allFunctions, colFunctions...)
 	allFunctions = append(allFunctions, ryszardFunctions...)
+	allFunctions = append(allFunctions, mtFunctions...)
 
 	for _, f := range allFunctions {
 		runIterations(funcName(f), start, end, step, f)

collections.go

@@ -1,9 +1,9 @@
 package main
 
 import (
-	"time"
-
 	colSkiplist "github.com/golang-collections/go-datastructures/slice/skip"
+	"math/rand"
+	"time"
 )
 
 type mockEntry uint64
@@ -43,6 +43,17 @@ func colWorstInserts(n int) {
 	}
 }
 
+func colRandomInserts(n int) {
+	list := colSkiplist.New(uint(0))
+	rList := rand.Perm(n)
+
+	defer timeTrack(time.Now(), n)
+
+	for _, e := range rList {
+		list.Insert(newMockEntry(uint64(e)))
+	}
+}
+
 func colAvgSearch(n int) {
 	list := colSkiplist.New(uint(0))
 
@@ -99,5 +110,21 @@ func colWorstDelete(n int) {
 	}
 }
 
-var colFunctions = []func(int){colInserts, colWorstInserts,
-	colAvgSearch, colSearchEnd, colDelete, colWorstDelete}
+func colRandomDelete(n int) {
+
+	list := colSkiplist.New(uint(0))
+
+	for i := 0; i < n; i++ {
+		list.Insert(newMockEntry(uint64(i)))
+	}
+
+	rList := rand.Perm(n)
+	defer timeTrack(time.Now(), n)
+
+	for _, e := range rList {
+		_ = list.Delete(newMockEntry(uint64(e)))
+	}
+}
+
+var colFunctions = []func(int){colInserts, colWorstInserts, colRandomInserts,
+	colAvgSearch, colSearchEnd, colDelete, colWorstDelete, colRandomDelete}

graphs/output.csv

@@ -0,0 +1,57 @@
+iterations,100000,200000,300000,400000,500000,600000,700000,800000,900000,1000000,1100000,1200000,1300000,1400000,1500000,1600000,1700000,1800000,1900000,2000000,2100000,2200000,2300000,2400000,2500000,2600000,2700000,2800000,2900000,3000000,
+seanInserts,593,519,535,613,511,514,512,576,506,573,512,488,571,486,580,512,500,561,483,510,546,553,558,522,532,562,535,494,559,524,
+seanWorstInserts,558,579,561,566,570,599,713,704,599,677,674,738,728,685,639,654,754,644,783,683,703,690,663,696,642,679,632,694,650,640,
+seanRandomInserts,1108,1525,1652,1848,2045,2059,2178,2285,2283,2350,2433,2484,2482,2579,2569,2710,2642,2667,2805,2742,2838,2768,2886,2828,2927,2869,3028,2939,2926,2995,
+seanAvgSearch,280,276,289,285,291,288,294,300,301,302,302,305,336,300,296,320,334,296,320,295,296,316,314,307,300,312,303,299,304,308,
+seanSearchEnd,240,281,234,206,258,208,263,179,222,246,255,299,272,223,242,192,236,329,311,354,266,241,203,216,311,395,317,313,320,233,
+seanDelete,192,194,194,196,201,194,195,196,196,195,196,195,204,207,207,195,197,195,205,205,203,203,250,212,208,206,205,209,202,203,
+seanWorstDelete,298,313,311,332,313,326,324,318,323,317,331,323,335,334,331,335,348,326,362,331,335,335,334,342,335,328,334,329,337,350,
+seanRandomDelete,773,1022,1173,1274,1362,1427,1501,1536,1613,1655,1703,1746,1766,1781,1831,1854,1885,1921,1946,1954,1983,2026,2024,2054,2061,2059,2070,2142,2131,2154,
+zhenjlInserts,706,701,695,763,836,797,820,758,821,831,788,782,756,861,791,842,767,859,789,842,779,857,787,853,801,768,804,837,850,842,
+zhenjlWorstInserts,1170,1163,1222,1164,1203,1254,1291,1321,1348,1296,1350,1401,1475,1358,1332,1306,1392,1339,1337,1368,1356,1468,1317,1397,1437,1408,1441,1451,1417,1386,
+zhenjlRandomInserts,1946,2357,2712,2891,3074,3254,3427,3707,3702,3955,3927,4017,4169,4219,4234,4256,4417,4349,4443,4549,4510,4726,4641,4678,4768,4742,4889,4860,4829,4926,
+zhenjlAvgSearch,2250,2229,2266,2308,2320,2571,2367,2342,2424,2422,2373,2422,2398,2478,2515,2437,2499,2459,2404,2444,2537,2453,2584,2483,2412,2486,2443,2446,2461,2550,
+zhenjlSearchEnd,1090,963,1236,1763,1560,1708,1547,1445,1780,1590,1127,1226,1592,1403,1566,2019,1687,1432,1809,1321,2269,1702,1490,1345,1210,1437,1651,1526,1525,1545,
+zhenjlDelete,1234,1526,1459,1521,1494,1485,1459,1468,1530,1526,1495,1468,1471,1477,1525,1528,1480,1534,1525,1540,1488,1551,1573,1507,1533,1529,1520,1496,1493,1541,
+zhenjlWorstDelete,1005,1183,1255,1248,1255,1215,1231,1238,1250,1263,1264,1215,1231,1231,1211,1268,1223,1247,1253,1258,1260,1235,1244,1248,1217,1262,1267,1240,1232,1257,
+zhenjlRandomDelete,2689,2998,3231,3332,3534,3608,3703,3789,3926,3970,3973,4092,4082,4167,4733,4267,4314,4340,4393,4390,4505,4464,4487,4513,4605,4619,4625,4630,4726,4703,
+mtchavezInserts,1009,958,964,1050,949,981,995,1014,933,1055,978,1013,916,1019,976,971,999,999,963,976,1018,956,1010,999,947,1026,1019,943,1023,962,
+mtchavezWorstInserts,830,803,1123,1064,1025,1066,1099,1013,1071,1041,1055,1085,1054,1091,1030,1106,1090,1013,1076,1115,1023,1091,1124,1027,1070,1049,1085,1087,1113,1055,
+mtchavezRandomInserts,1545,1845,2208,2486,2659,2749,2741,2923,2989,3077,3067,3104,3141,3281,3246,3361,3280,3383,3473,3441,3415,3608,3594,3690,3688,3714,3612,3682,3777,3795,
+mtchavezAvgSearch,262,274,281,277,274,310,290,284,280,280,280,290,282,293,279,288,295,296,309,283,295,289,301,323,304,304,303,299,293,304,
+mtchavezSearchEnd,277,173,193,271,161,209,236,236,203,251,218,225,300,210,206,250,266,247,276,205,202,212,260,244,229,249,207,173,210,206,
+mtchavezDelete,419,911,1049,821,956,852,816,691,730,989,980,908,891,751,707,799,804,993,706,819,773,748,1061,1056,807,775,833,679,757,718,
+mtchavezWorstDelete,1143,887,980,923,1025,833,1047,1112,982,848,1017,867,1185,972,894,1041,980,843,877,902,905,841,849,1151,902,764,857,968,971,1117,
+mtchavezRandomDelete,1130,1380,1657,1739,1853,1917,2001,2049,2094,2174,2215,2254,2292,2319,2364,2383,2414,2462,2480,2512,2545,2551,2660,2685,2652,2662,2687,2703,2723,2749,
+huanduInserts,517,513,609,587,609,575,626,553,588,602,635,612,564,608,568,613,574,560,586,588,599,584,599,576,634,554,594,630,572,559,
+huanduWorstInserts,679,655,670,727,758,798,757,763,781,718,771,766,775,800,756,801,731,788,743,827,779,742,776,808,829,816,771,793,823,751,
+huanduRandomInserts,1369,1765,1996,2369,2603,2561,2736,2839,2998,2997,3027,3197,3152,3334,3273,3320,3488,3469,3451,3535,3647,3610,3631,3643,3717,3647,3855,3804,3900,3805,
+huanduAvgSearch,363,376,381,392,388,386,405,401,425,419,409,412,409,395,405,404,445,417,406,398,398,428,416,428,427,419,402,410,422,425,
+huanduSearchEnd,186,274,265,208,236,252,338,258,235,433,304,450,325,243,372,202,482,238,403,239,381,388,280,506,246,339,311,341,360,358,
+huanduDelete,260,262,263,262,261,295,266,263,266,263,267,263,267,272,267,261,266,262,266,358,264,264,267,262,266,312,266,263,264,264,
+huanduWorstDelete,414,407,416,434,416,435,421,440,449,452,440,453,450,461,437,437,465,440,458,477,452,445,461,516,447,445,475,449,451,446,
+huanduRandomDelete,976,1261,1447,1557,1653,1758,1816,1883,1934,2011,2064,2142,2321,2302,2375,2267,2286,2330,2367,2378,2636,2690,2543,2558,2600,2809,2603,2629,2636,2671,
+colInserts,905,930,922,942,936,936,1031,1026,1062,1052,1003,1035,1054,1012,1134,1107,1049,1041,1122,1029,1137,1091,1033,1168,1033,1117,1022,1142,1062,1170,
+colWorstInserts,1040,1027,1024,1112,1039,1224,1256,1230,1113,1267,1308,1157,1226,1302,1236,1249,1206,1260,1150,1176,1230,1199,1190,1195,1248,1216,1197,1271,1285,1164,
+colRandomInserts,2107,2612,2943,3237,3275,3479,3513,3722,3762,3793,3865,4039,4180,4077,4463,4220,4312,4267,4383,4394,4464,4557,4509,4589,4587,4635,4677,4667,4764,4647,
+colAvgSearch,651,690,714,754,736,766,783,748,801,759,828,763,757,827,756,861,799,795,807,794,862,838,889,799,850,808,799,842,828,820,
+colSearchEnd,397,414,476,522,458,322,561,400,424,462,499,582,625,523,466,379,392,429,372,511,559,541,573,502,586,513,683,537,409,684,
+colDelete,523,548,559,572,572,588,597,604,560,655,624,644,647,644,638,570,635,600,614,682,627,602,635,622,634,668,668,600,673,669,
+colWorstDelete,791,664,709,686,739,776,766,740,715,754,735,765,765,822,780,752,765,871,790,874,852,770,760,769,770,812,836,816,799,829,
+colRandomDelete,1590,1944,2243,2305,2426,2525,2607,2789,2945,2934,3025,3045,3048,3064,3263,3221,3241,3229,3334,3303,3331,3380,3456,3632,3510,3460,3492,3486,3581,3605,
+ryszardInserts,863,961,973,1071,1108,1053,943,1012,1048,1135,1030,1131,1028,1008,1144,1078,960,1116,1060,1081,1049,1037,1019,1101,1047,1024,1083,1006,1067,924,
+ryszardWorstInserts,1002,1553,1356,1294,1322,1295,1314,1309,1329,1275,1415,1326,1305,1376,1329,1315,1351,1229,1331,1373,1272,1341,1463,1346,1242,1406,1398,1411,1341,1260,
+ryszardRandomInserts,3661,3223,3534,3935,4088,4210,4529,4612,4683,4604,5013,4826,4981,5104,5005,5416,5116,5601,5442,5621,5563,5888,5684,5796,5876,6018,5725,6306,6203,6337,
+ryszardAvgSearch,491,514,531,578,547,560,584,624,563,593,562,606,583,560,639,601,586,598,614,626,597,588,670,660,615,596,589,604,620,681,
+ryszardSearchEnd,418,297,336,329,261,632,548,410,414,551,534,705,331,416,494,308,254,502,436,323,331,322,392,560,563,305,252,580,538,403,
+ryszardDelete,1117,537,497,504,477,436,436,523,457,575,457,505,582,459,539,547,546,530,496,474,445,573,442,478,536,438,525,440,522,471,
+ryszardWorstDelete,741,783,762,777,706,803,835,908,796,743,860,753,887,858,826,921,847,885,877,756,789,872,854,860,835,776,931,832,794,882,
+ryszardRandomDelete,1819,2222,2597,2617,2798,2936,3144,3289,3352,3312,3352,3560,3588,3628,3677,3688,3931,3963,3807,3913,4189,4179,4043,4352,4333,4333,4609,4761,4840,4822,
+mtInserts,328,326,325,328,367,315,387,350,286,326,307,340,435,309,384,303,314,339,356,283,357,278,395,320,340,309,341,300,314,369,
+mtWorstInserts,252,277,256,256,251,419,343,284,523,320,410,309,427,343,335,351,303,366,368,304,334,373,315,316,343,295,335,401,357,429,
+mtRandomInserts,973,1172,1343,1425,1585,1711,1664,1794,1739,1817,1861,1955,1951,1975,1876,2023,1906,1911,2066,1982,2093,2067,2139,2125,2107,2145,2235,2235,2229,2194,
+mtAvgSearch,287,288,304,303,293,304,301,319,296,323,314,317,314,320,312,318,326,355,319,321,334,314,327,339,326,318,366,321,310,322,
+mtSearchEnd,315,327,307,349,432,374,524,322,344,335,347,371,426,346,567,450,387,409,400,459,354,390,400,297,357,327,452,385,510,328,
+mtDelete,128,132,133,136,135,136,138,141,139,139,141,140,139,144,178,142,140,144,145,146,221,143,143,146,145,146,144,194,146,144,
+mtWorstDelete,318,319,319,321,337,327,328,341,326,330,332,327,345,482,345,376,334,391,345,351,342,347,347,345,357,355,357,344,361,356,
+mtRandomDelete,807,994,1088,1171,1120,1303,1362,1395,1455,1481,1519,1543,1580,1583,1635,1663,1692,1759,1742,1758,1755,1755,1787,1796,1772,1797,1801,1824,1838,1857,

huandu.go

@@ -1,9 +1,9 @@
 package main
 
 import (
-	"time"
-
 	huaSkiplist "github.com/huandu/skiplist"
+	"math/rand"
+	"time"
 )
 
 func huanduInserts(n int) {
@@ -24,6 +24,16 @@ func huanduWorstInserts(n int) {
 	}
 }
 
+func huanduRandomInserts(n int) {
+	list := huaSkiplist.New(huaSkiplist.Int)
+	rList := rand.Perm(n)
+	defer timeTrack(time.Now(), n)
+
+	for _, e := range rList {
+		list.Set(e, testByteString)
+	}
+}
+
 func huanduAvgSearch(n int) {
 	list := huaSkiplist.New(huaSkiplist.Int)
 
@@ -80,5 +90,20 @@ func huanduWorstDelete(n int) {
 	}
 }
 
-var huanduFunctions = []func(int){huanduInserts, huanduWorstInserts,
-	huanduAvgSearch, huanduSearchEnd, huanduDelete, huanduWorstDelete}
+func huanduRandomDelete(n int) {
+	list := huaSkiplist.New(huaSkiplist.Int)
+
+	for i := 0; i < n; i++ {
+		list.Set(i, testByteString)
+	}
+
+	rList := rand.Perm(n)
+	defer timeTrack(time.Now(), n)
+
+	for _, e := range rList {
+		_ = list.Remove(e)
+	}
+}
+
+var huanduFunctions = []func(int){huanduInserts, huanduWorstInserts, huanduRandomInserts,
+	huanduAvgSearch, huanduSearchEnd, huanduDelete, huanduWorstDelete, huanduRandomDelete}