From cd954553fae498fdde036b603b837b36dfe2bf1c Mon Sep 17 00:00:00 2001 From: Sindre Sorhus Date: Tue, 24 Oct 2023 04:46:28 +0700 Subject: [PATCH] Buffer blog post --- source/components/blog/Post.astro | 7 +- source/content/apps/ai-actions.md | 4 +- source/content/apps/aiko.md | 7 +- source/content/apps/dato.md | 2 +- source/content/apps/hyperduck.md | 10 ++ source/content/apps/lungo.md | 6 +- source/content/apps/plain-text-editor.md | 4 + source/content/blog/goodbye-nodejs-buffer.md | 180 +++++++++++++++++++ source/content/config.ts | 1 + source/pages/_apps-extra.md | 1 + source/pages/about.md | 1 + source/pages/feedback.astro | 3 +- 12 files changed, 220 insertions(+), 6 deletions(-) create mode 100644 source/content/blog/goodbye-nodejs-buffer.md diff --git a/source/components/blog/Post.astro b/source/components/blog/Post.astro index 09e4c57e..09192de9 100644 --- a/source/components/blog/Post.astro +++ b/source/components/blog/Post.astro @@ -3,6 +3,11 @@ import {getFormattedDate} from '~/utils/utils.js'; const {post} = Astro.props; + +// This `proseCSS` is similiar to the one I use for apps, but not exactly the same. + +// `prose-code:before:hidden prose-code:after:hidden`: https://github.com/tailwindlabs/tailwindcss-typography/issues/18#issuecomment-1280797041 +const proseCSS = 'mx-auto px-6 sm:px-6 py-6 pb-20 max-w-3xl prose prose-lg lg:prose-xl dark:prose-invert dark:prose-headings:text-slate-300 prose-md prose-headings:font-heading prose-headings:leading-tighter prose-headings:tracking-tighter prose-headings:font-bold prose-a:text-primary-600 dark:prose-a:text-primary-400 prose-img:rounded-md prose-img:shadow-lg mt-8 prose-code:before:hidden prose-code:after:hidden prose-a:text-black/75 dark:prose-a:text-white/90 prose-a:underline prose-a:underline-offset-4 prose-a:decoration-primary-500 hover:prose-a:decoration-primary-600 prose-a:decoration-2 hover:prose-a:decoration-4 hover:prose-a:text-black dark:hover:prose-a:text-white'; ---
@@ -26,7 +31,7 @@ const {post} = Astro.props;
diff --git a/source/content/apps/ai-actions.md b/source/content/apps/ai-actions.md index b71b96cb..42b39480 100644 --- a/source/content/apps/ai-actions.md +++ b/source/content/apps/ai-actions.md @@ -16,6 +16,8 @@ For example, an action to interact with the ChatGPT API. Your API key is securely stored in your keychain, not in a shortcut. +**If you are getting a `Missing response` error, it's most likely that the OpenAI safety system prevented your prompt. Try something else.** +
*You may also like my [Actions](/actions) app.* @@ -24,7 +26,7 @@ Your API key is securely stored in your keychain, not in a shortcut. ### macOS version -The macOS version is not yet available on the App Store because App Store review is being difficult. You can get it [here](https://github.com/sindresorhus/meta/files/12800983/AI.Actions.1.0.3.zip) for now. (Requires macOS 13+) +The macOS version is not yet available on the App Store because App Store review is being difficult. You can get it [here](https://dsc.cloud/sindresorhus/AI-Actions-1.0.4-1698250306.zip) for now. (Requires macOS 13+) ### Frequently Asked Questions {#faq} diff --git a/source/content/apps/aiko.md b/source/content/apps/aiko.md index c7ea3e09..b8a49d4a 100644 --- a/source/content/apps/aiko.md +++ b/source/content/apps/aiko.md @@ -19,7 +19,6 @@ My goal is to keep the app simple. If you have more advanced needs, check out [M **Upcoming features** - Batch conversion -- Significantly improved performance on iOS thanks to CoreML - Export to karaoke file
@@ -159,6 +158,10 @@ Known issue: The shortcut has to momentarily show the Aiko app and then hide it ### Frequently Asked Questions {#faq} +#### Can you use the large v3 model for the Mac app? + +The v3 model is [worse](https://github.com/openai/whisper/discussions/1762#discussioncomment-7532295) than v2 in too many cases. I tried releasing v3, but got a lot of emails about the quality being worse, so I ended up reverting it. + #### I have a feature request, bug report, or some feedback [Send it here.](https://sindresorhus.com/feedback?product=Aiko&referrer=Website-FAQ) @@ -194,6 +197,8 @@ This is unfortunately a flaw in the Whisper model. [Workaround.](#tips) This is unfortunately a flaw in the Whisper model. It can sometimes add a sentence like “Thanks for watching!” to the end. There is not much I can do about this. +This issue arises from quirks in the AI's processing, where it sometimes generates off-topic content, often due to data remnants or misinterpreted context. These are not messages or 'whispers' with any underlying meaning; they're random anomalies that OpenAI is actively working to correct. + #### The transcription is in Traditional Chinese while the audio was in Simplified Chinese? The [Whisper AI model](https://github.com/openai/whisper) used by the app does not differentiate between Traditional Chinese and Simplified Chinese, so the result could unfortunately end up with either. [Learn more.](https://github.com/openai/whisper/discussions/277) diff --git a/source/content/apps/dato.md b/source/content/apps/dato.md index d1c85c15..e411ea40 100644 --- a/source/content/apps/dato.md +++ b/source/content/apps/dato.md @@ -80,7 +80,7 @@ Previously, when you clicked the event in the menu bar, you could hide it from t ### Trial -There's a fully functional trial available [here](https://dsc.cloud/sindresorhus/Dato-5.1.1-trial-1697283039.zip). The only limitation is that it will prompt you to buy Dato every 12 hours and it will not receive updates. If you decide to buy Dato on the App Store, all data and settings from the trial version will be preserved (they share the same storage). +There's a fully functional trial available [here](https://dsc.cloud/sindresorhus/Dato-5.1.3-trial-1699456900.zip). The only limitation is that it will prompt you to buy Dato every 12 hours and it will not receive updates. If you decide to buy Dato on the App Store, all data and settings from the trial version will be preserved (they share the same storage). You can also [try it on Setapp](https://go.setapp.com/stp181?_target=https://setapp.com/apps/dato&utm_medium=vendor_program&utm_source=Sindre+Sorhus&utm_content=link) for 7 days for free. diff --git a/source/content/apps/hyperduck.md b/source/content/apps/hyperduck.md index 32347bf7..2283743d 100644 --- a/source/content/apps/hyperduck.md +++ b/source/content/apps/hyperduck.md @@ -192,3 +192,13 @@ I don't have any immediate plans to localize the app. #### Where can I find the changelog? Go [here](https://apps.apple.com/app/id6444667067) and click “Version History”. + +
+ +### Non-App Store Version + +A special version for users that cannot access the App Store. It won't receive automatic updates. I will update it here once a year. + +[Download](https://dsc.cloud/sindresorhus/Hyperduck-1.0.7-1699560532.zip) *(1.0.7)* + +*Requires macOS 13 or later* diff --git a/source/content/apps/lungo.md b/source/content/apps/lungo.md index 99c033fc..b28137e5 100644 --- a/source/content/apps/lungo.md +++ b/source/content/apps/lungo.md @@ -56,10 +56,14 @@ Yes, choose “Activate on Left-click” in the settings. I don't have any immediate plans to support this. It's a lot of work to implement and few people have requested it. -However, you can achieve it using the Shortcuts app. Create a new shortcut with the “Set Enabled State” action provided by Lungo. Shortcuts on iOS has automations, which lets you run a shortcut when a certain app opens. macOS does not yet have automations. I'm confident it will come with macOS 14. In the meantime, you can use the [Shortery app](https://apps.apple.com/us/app/shortery/id1594183810?mt=12) to run a shortcut when a certain app opens. +However, you can achieve it using the Shortcuts app. Create a new shortcut with the “Set Enabled State” action provided by Lungo. Shortcuts on iOS has automations, which lets you run a shortcut when a certain app opens. macOS does not yet have automations. I'm confident it will come with macOS 15. In the meantime, you can use the [Shortery app](https://apps.apple.com/us/app/shortery/id1594183810?mt=12) to run a shortcut when a certain app opens. You could also use BetterTouchTool or Keyboard Maestro instead of Shortery. +#### Can Lungo automatically activate on a set schedule? + +I don't plan to do this built-in, but you can use Shortery for this as explained above. Shortery has a “Time” trigger. + #### Can I use a custom duration? {#custom-duration} Lungo comes with a set of commonly used durations, which should be enough for most users. People love Lungo for its simplicity. Not having custom times also helps prevent decision fatigue. Lungo has tens of thousands of users and the app has existed for 6 years, and only a few people have requested support for custom durations. diff --git a/source/content/apps/plain-text-editor.md b/source/content/apps/plain-text-editor.md index d8ade3a1..13fd6cb8 100644 --- a/source/content/apps/plain-text-editor.md +++ b/source/content/apps/plain-text-editor.md @@ -98,6 +98,10 @@ The app is made using Apple's latest technologies (SwitUI) and they have not add You most likely have an external mouse connected. This is simply how macOS works. You can see the behavior in other text editing apps too. The behavior is completely out of my control. +#### The “Check Spelling While Typing” setting in the context menu does not work + +This is unfortunately a macOS bug and not something I can fix. + #### How can I export, import, sync, or back up the settings? [See this guide.](https://github.com/sindresorhus/guides/blob/main/backup-app-settings.md) diff --git a/source/content/blog/goodbye-nodejs-buffer.md b/source/content/blog/goodbye-nodejs-buffer.md new file mode 100644 index 00000000..d488d42a --- /dev/null +++ b/source/content/blog/goodbye-nodejs-buffer.md @@ -0,0 +1,180 @@ +--- +title: Goodbye, Node.js Buffer +description: It's time to move from Buffer to Uint8Array. +pubDate: 2023-10-24 +tags: + - nodejs + - javascript + - open-source +--- + +The [`Buffer`](https://nodejs.org/api/buffer.html) type has been the cornerstone for binary data handling in Node.js since the beginning. However, these days we have [`Uint8Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array), which is a native JavaScript type and works cross-platform. While `Buffer` is an instance of `Uint8Array`, it introduces numerous methods that are not available in other JavaScript environments. Consequently, code leveraging Buffer-specific methods needs polyfilling, preventing many valuable packages from being browser-compatible. + +`Buffer` also comes with additional caveats. While [`Uint8Array#slice()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/slice) creates an immutable copy, [`Buffer#slice()`](https://nodejs.org/api/buffer.html#bufslicestart-end) creates a mutable segment linked to the original Buffer, resulting in possible unpredictable behavior. The problem is not the behavior of the `Buffer#slice()` method, but the fact that `Buffer` is a subclass of `Uint8Array` and completely changes the behavior of an inherited method. Instead of `Buffer#slice()`, use `Uint8Array#subarray()` or `Buffer#subarray()`. Furthermore, [buffers expose private information](https://github.com/nodejs/node/issues/41588#issuecomment-1016269584) through global variables, a potential security risk. + +It is time to move on. + +## The Plan + +I intend to move [all my packages](https://github.com/search?q=owner%3Asindresorhus+%22node%3Abuffer%22&type=code) from using `Buffer` to `Uint8Array`. If you are a maintainer of a JavaScript package, I encourage you to do the same. + +`Buffer` will never be removed, and probably never even deprecated, but at least the community can slowly move away from it. My hope is that the Node.js team will at least start discouraging the use of `Buffer`. + +## How + +First, familiarize yourself with the [subtle incompatibilities](https://nodejs.org/api/buffer.html#buffers-and-typedarrays) between `Uint8Array` and `Buffer`. + +I have made the [`uint8array-extras` package](https://github.com/sindresorhus/uint8array-extras) to make the transition easier. Pull requests are welcome for additional utilities. + +If your code accepts a `Buffer` and doesn't use any `Buffer`-specific methods, you can simply update your docs and types to `Uint8Array`. Changing the input type from `Buffer` to `Uint8Array` is a non-breaking change since `Buffer` is an instance of `Uint8Array`. + +Changing the return type from `Buffer` to `Uint8Array` is a breaking change, because consumers may use `Buffer`-specific methods. + +If you absolutely need to convert a `Uint8Array` to a `Buffer`, you can use `Buffer.from(uint8Array)` (copies the data) or `Buffer.from(uint8Array.buffer, uint8Array.byteOffset, uint8Array.byteLength)` (does not copy). However, there is usually a better way. + +The primary transition steps are: + +- Remove all `import {Buffer} from 'node:buffer'` imports. +- Remove all occurrences of the `Buffer` global. +- Stop using `Buffer`-specific methods. +- Replace `Buffer` reading/writing methods, like [`Buffer#readInt32BE()`](https://nodejs.org/api/buffer.html#bufreadint32beoffset), with [`DataView`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView). + +### Questions + +#### Why did `Buffer` exist in the first place? + +`Buffer` was created long before `Uint8Array` existed. + +#### How can I convert to and from Base64 with `Uint8Array`? + +You can use my [`uint8array-extras` package](https://github.com/sindresorhus/uint8array-extras) for now. It will most likely eventually be [supported natively](https://github.com/tc39/proposal-arraybuffer-base64/issues) in JavaScript. + +#### How do I handle Node.js APIs that return a `Buffer`, like the `fs` methods? + +Since `Buffer` is a subclass of `Uint8Array`, you can just treat it like a `Uint8Array`. Just make sure you don't use `.slice()` (which differs in behavior) or any Buffer-specific methods. + +### Examples + +#### JavaScript + +```diff ++import {stringToBase64} from 'uint8array-extras'; + +-Buffer.from(string).toString('base64'); ++stringToBase64(string); +``` + +```diff ++import {uint8ArrayToHex} from 'uint8array-extras'; + +-buffer.toString('hex'); ++uint8ArrayToHex(uint8Array); +``` + +```diff + const bytes = getBytes(); + ++const view = new DataView(bytes.buffer); + +-const value = bytes.readInt32BE(1); ++const value = view.getInt32(1); +``` + +```diff + import crypto from 'node:crypto'; +-import {Buffer} from 'node:buffer'; ++import {isUint8Array} from 'uint8array-extras'; + + export default function hash(data) { +- if (!(typeof data === 'string' || Buffer.isBuffer(data))) { ++ if (!(typeof data === 'string' || isUint8Array(data))) { + throw new TypeError('Incorrect type.'); + } + + return crypto.createHash('md5').update(data).digest('hex'); + } +``` + +Most Node.js APIs accept `Uint8Array` too, so no extra work was required. Ideally, this code should also transition to [Web Crypto](https://nodejs.org/api/webcrypto.html), but that's not relevant to this example. + +##### TypeScript + +```diff +-import {Buffer} from 'node:buffer'; + +-export function getSize(input: string | Buffer): number { … } ++export function getSize(input: string | Uint8Array): number { … } +``` + +## Enforcement + +I recommend enforcing `Uint8Array` over `Buffer` with linting. + +Add this to your ESLint config: + +```js +{ + 'no-restricted-globals': [ + 'error', + { + name: 'Buffer', + message: 'Use Uint8Array instead.' + } + ], + 'no-restricted-imports': [ + 'error', + { + name: 'buffer', + message: 'Use Uint8Array instead.' + }, + { + name: 'node:buffer', + message: 'Use Uint8Array instead.' + } + ] +} +``` + +And if you use TypeScript, add this: + +```js +{ + '@typescript-eslint/ban-types': [ + 'error', + { + types: { + Buffer: { + message: 'Use Uint8Array instead.', + suggest: [ + 'Uint8Array' + ] + } + } + } + ] +} +``` + +If you use [XO](https://github.com/xojs/xo), it will soon come with this config by default. + +## How Can I Help? + +[Voice your support](https://github.com/nodejs/node/issues/41588) for Node.js using `Uint8Array` for new APIs. + +Help me move [my packages](https://github.com/search?q=owner%3Asindresorhus+%22node%3Abuffer%22&type=code) to `Uint8Array`. Pick one and give it a go. + +Help us make a [lint rule](https://github.com/sindresorhus/eslint-plugin-unicorn/issues/1808) to prevent the use of `Buffer` methods. + +## Future + +`Uint8Array` (or rather `TypedArray`) need more utility methods! + +For example, there is currently no good built-in way to convert a `Uint8Array` to Base64 or Hex. Although, it looks like this is [most likely coming](https://github.com/tc39/proposal-arraybuffer-base64). + +Consider proposing missing bits to [TC39](https://github.com/tc39/proposals). + +## The End + +Let's make the JavaScript package ecosystem more cross-platform. Thanks for reading. + +[Discuss](https://github.com/sindresorhus/meta/discussions/22) diff --git a/source/content/config.ts b/source/content/config.ts index 98fec30b..0ac1e7db 100644 --- a/source/content/config.ts +++ b/source/content/config.ts @@ -37,6 +37,7 @@ const blogCollection = defineCollection({ tags: z.array(z.enum([ 'open-source', 'javascript', + 'nodejs', ])).optional(), redirectUrl: z.string().url().optional(), }).strict(), diff --git a/source/pages/_apps-extra.md b/source/pages/_apps-extra.md index dd42f822..65f7def9 100644 --- a/source/pages/_apps-extra.md +++ b/source/pages/_apps-extra.md @@ -1,6 +1,7 @@ ## More - [Tiny Apps](/tiny-apps) - Smaller utilities +- [Open Source Apps](https://github.com/search?q=user%3Asindresorhus+language%3Aswift+topic%3Aapp+archived%3Afalse&type=repositories) - My open source apps - [Older Versions](/apps/older-versions) - My apps for older macOS versions - [Terms of Use](/apps/terms) - Guidelines and conditions for using my apps - [RSS Feed for New Apps](/rss-apps.xml) - Get notified about new apps I publish diff --git a/source/pages/about.md b/source/pages/about.md index f1331594..64029a1c 100644 --- a/source/pages/about.md +++ b/source/pages/about.md @@ -20,6 +20,7 @@ My current focus is on macOS app development with Swift and Node.js-based packag - [Twitter](https://twitter.com/sindresorhus) - [Mastodon](https://mastodon.social/@sindresorhus) - [Bluesky](https://bsky.app/profile/sindresorhus.bsky.social) +- [Telegram](https://t.me/sindresorhuschat) - [Newsletter](https://sindresorhus.substack.com) - [GitHub](https://github.com/sindresorhus) - [Instagram](https://instagram.com/sindresorhus) diff --git a/source/pages/feedback.astro b/source/pages/feedback.astro index 9d02bc3c..9b2d6a3d 100644 --- a/source/pages/feedback.astro +++ b/source/pages/feedback.astro @@ -96,7 +96,8 @@ const inputCSS = 'block p-2.5 w-full text-lg text-gray-900 bg-gray-50 rounded-lg if (product === 'Actions') { $('#additional-info').show().append(` -

It is unfortunately not possible to make an action to get the state of the device orientation lock. +

If you get a “com.apple.extensionKit.errorDomain error 2” error when running your shortcut or if the actions don't show up in the Shortcuts app, restart your device. This is caused by a iOS bug.

+

Some actions that are not possible: orientation lock status, flashlight status, ambient sensor info, flight mode status. `); } }