Draft: Better performance for larger inputs #70
Conversation
|
This is interesting for sure. But I don't think adding this complexity for the case of bigger payloads really makes sense. |
| result.append(mod) | ||
|
|
||
| return b'\0' * (origlen - newlen) + bytes(reversed(result)) | ||
| return acc.to_bytes(origlen - newlen + (acc.bit_length() + 7) // 8, "big") |
There was a problem hiding this comment.
This change alone seems to have a pretty good impact and even simplifies the code 😍
|
I am decoding base58 inputs that are ~200 characters long (~150 bytes) that I do not control, and want to mitigate the potential for performance issues if larger values are seen in the future. The nested loop is O(n^2), and becomes closer to O(n^3) for large inputs. Using gmpy2 is consistently fast, but slower for inputs that are about half the size of a bitcoin address. When gmpy is not used, splitting the inputs approximately in half on pre-computed powers of 2, 45, or 58, it moves closer to O(n^2*log(n)). By avoiding the Karatsuba cutoff for the nested loop, Karatsuba large integer multiplications are performed in a divide and conquer manner. The change adds complexity, but it provides better performance for inputs up to ~2MB. |
Add optional gmpy2.mpz for even faster encode/decode Add longer random benchmark
Apply Black code style Quiet mypy errors
Performance for ~30 bytes and larger inputs should be faster. It is much faster for 256 bytes, and should be able to handle inputs in the MB range.