Use WebCrypto API

[7heMech]

I've modified:

• the code to use webcrypto api
• the tests to work properly with promises
• README.md to reflect that library is now async

[7heMech]

Not sure if it works on node <20 🥶 (because of the way crypto api is referenced, might need to change that)

[7heMech]

Btw, I'm already using this in production 😂

[7heMech]

Will make one more commit.

[bellstrand]

I'll have a closer look at the code soon, but before that we'll need to address the NodeJS 18 error (I'm okay with dropping support for Node 16 since it went out of maintenance half a year ago, but Node 18 will need to be supported for another year until it reaches its end of life.

I've also tried running some simple performance tests with the following code (you have to remove the jest clock)

test.only.each([10, 100, 1000, 10000, 50000])("performance, %p", async (index) => {
	const now = Date.now()
	let timestamp = 0
	for (let i = 0; i < index; i++) {
		await TOTP.generate("JBSWY3DPEHPK3PXP", { timestamp: timestamp++ })
	}
	console.log(`${(Date.now() - now) / 1000}s`)
})

And currently this branch takes approximately double the time to execute a TOTP.generate compared to the main branch (I'm running this on node v21.7.2 when trying it out).

I'll gladly help to unravel the performance issues in a few days!

[7heMech]

I'll have a closer look at the code soon, but before that we'll need to address the NodeJS 18 error (I'm okay with dropping support for Node 16 since it went out of maintenance half a year ago, but Node 18 will need to be supported for another year until it reaches its end of life.

I've also tried running some simple performance tests with the following code (you have to remove the jest clock)

test.only.each([10, 100, 1000, 10000, 50000])("performance, %p", async (index) => {
	const now = Date.now()
	let timestamp = 0
	for (let i = 0; i < index; i++) {
		await TOTP.generate("JBSWY3DPEHPK3PXP", { timestamp: timestamp++ })
	}
	console.log(`${(Date.now() - now) / 1000}s`)
})

And currently this branch takes approximately double the time to execute a TOTP.generate compared to the main branch (I'm running this on node v21.7.2 when trying it out).

I'll gladly help to unravel the performance issues in a few days!

Hey, there,
The nodejs <20 support is just a reference issue, it won't be hard to fix.
I've tried to do some performance optimizations, but apparently it wasn't enough, maybe my implementation is not very good, I'll do some debugging as well.

[bellstrand]

I can help you with trying to optimize it, but I'm out travling for another ~10 days, so it will take a little while before being able to do much ;)

[7heMech]

I can help you with trying to optimize it, but I'm out travling for another ~10 days, so it will take a little while before being able to do much ;)

Much appreciated, I'll see what I can do meanwhile.

[7heMech]

@bellstrand I've got a question, so I had in my base32 to buffer function a piece of code which calculates a lookup object with charCode to Index for base32 alphabet, so in order to optimize (even a little) I did this:

/**
 * A precalculated mapping from base32 character codes to their corresponding index values for performance optimization.
 * This mapping is used in the base32ToBuffer method to convert base32 encoded strings to their binary representation.
 */
private static readonly base32: { [key: string]: number } = { "50": 26, "51": 27, "52": 28, "53": 29, "54": 30, "55": 31, "65": 0, "66": 1, "67": 2, "68": 3, "69": 4, "70": 5, "71": 6, "72": 7, "73": 8, "74": 9, "75": 10, "76": 11, "77": 12, "78": 13, "79": 14, "80": 15, "81": 16, "82": 17, "83": 18, "84": 19, "85": 20, "86": 21, "87": 22, "88": 23, "89": 24, "90": 25 }

does this look ok?
It's messing with prettier wanting to format it, maybe I should move it to a separate file?

[7heMech]

Tested on first versions of node 16 and 18, it works!

[bellstrand]

@bellstrand I've got a question, so I had in my base32 to buffer function a piece of code which calculates a lookup object with charCode to Index for base32 alphabet, so in order to optimize (even a little) I did this:

/**
 * A precalculated mapping from base32 character codes to their corresponding index values for performance optimization.
 * This mapping is used in the base32ToBuffer method to convert base32 encoded strings to their binary representation.
 */
private static readonly base32: { [key: string]: number } = { "50": 26, "51": 27, "52": 28, "53": 29, "54": 30, "55": 31, "65": 0, "66": 1, "67": 2, "68": 3, "69": 4, "70": 5, "71": 6, "72": 7, "73": 8, "74": 9, "75": 10, "76": 11, "77": 12, "78": 13, "79": 14, "80": 15, "81": 16, "82": 17, "83": 18, "84": 19, "85": 20, "86": 21, "87": 22, "88": 23, "89": 24, "90": 25 }

does this look ok? It's messing with prettier wanting to format it, maybe I should move it to a separate file?

We can look at moving it to a different file, but might need to do something with the build script to bundle it in one file for that!

[7heMech]

Look at bun branch test speed lol

test.lol.mp4

[7heMech]

@bellstrand I've got a question, so I had in my base32 to buffer function a piece of code which calculates a lookup object with charCode to Index for base32 alphabet, so in order to optimize (even a little) I did this:

/**
 * A precalculated mapping from base32 character codes to their corresponding index values for performance optimization.
 * This mapping is used in the base32ToBuffer method to convert base32 encoded strings to their binary representation.
 */
private static readonly base32: { [key: string]: number } = { "50": 26, "51": 27, "52": 28, "53": 29, "54": 30, "55": 31, "65": 0, "66": 1, "67": 2, "68": 3, "69": 4, "70": 5, "71": 6, "72": 7, "73": 8, "74": 9, "75": 10, "76": 11, "77": 12, "78": 13, "79": 14, "80": 15, "81": 16, "82": 17, "83": 18, "84": 19, "85": 20, "86": 21, "87": 22, "88": 23, "89": 24, "90": 25 }

does this look ok? It's messing with prettier wanting to format it, maybe I should move it to a separate file?

We can look at moving it to a different file, but might need to do something with the build script to bundle it in one file for that!

Oh, fair enough.

[7heMech]

@bellstrand I think I got it, since I await each promise it might be slower, but if I use promise.all?

[bellstrand]

@bellstrand I think I got it, since I await each promise it might be slower, but if I use promise.all?

Well, you use the result of the first promise in the second await, so sadly that won't work!

[7heMech]

If your tests do the same (use promise.all to run all tests "at the same time") instead of each awaited, it will be way faster.
Edit: this is with promise.all of current code

[7heMech]

I got some perf calc going

I guess it really depends on the runtime, this is in bun land

[7heMech]

Well, at least we got isomorphic package now

[7heMech]

@bellstrand I read some stuff on the topic and it seems like because of V8 promises, the webcrypto API is actually slower than third-party libraries for small amounts of data, not sure if it's still relevant, but this might be why.

[7heMech]

@bellstrand I can't seem to be able to get jest to think I have tests for everything

[bellstrand]

It went through the coverage tests on the last run thou! ;)
But as I said, I'm away for ~10 days, so will merge and release a new major version when I get back (since we're doing a breaking change) with moving from sync => async.

[7heMech]

@bellstrand sorry for bothering your travel 😅
I'll leave it at this state till you're available again, today I implemented minification

[7heMech]

I'll run some benchmarks with mitata comparing the libs to check performance.
Edit: yeah, it's a bit slower lol, but at those numbers I'm not sure if it even matters:

[7heMech]

Okay, so I don't really think I can do anything more about this I'd say this is finished state, but do tell me if you want me to change something (or everything 😅)

[7heMech]

@bellstrand not sure if you get notifications without ping.

[bellstrand]

@7heMech Sorry for being slow, I stayed abroad for another week.
I think everything looks great, let's just resolve the yarn.lock conflict and I'll merge everything! :)

[7heMech]

@bellstrand resolved

[7heMech]

@bellstrand I'll do some more changes soon (maybe ASCII encoding key and fix some TS stuff) or at least suggest them as a pr.