From 863c04c4a0b9eea20a3214457631fc014393a7db Mon Sep 17 00:00:00 2001 From: Jonathan Neal Date: Tue, 15 Feb 2022 09:13:17 -0500 Subject: [PATCH 001/300] chore: merge recursive components rfc --- proposals/0007-recursive-components.md | 161 ------------------------- proposals/0010-recursive-components.md | 8 +- 2 files changed, 5 insertions(+), 164 deletions(-) delete mode 100644 proposals/0007-recursive-components.md diff --git a/proposals/0007-recursive-components.md b/proposals/0007-recursive-components.md deleted file mode 100644 index c99f9578..00000000 --- a/proposals/0007-recursive-components.md +++ /dev/null @@ -1,161 +0,0 @@ -- Start Date: Date: 2021-12-15 -- Reference Issues: https://github.com/withastro/rfcs/discussions/45 -- Implementation PR: - - https://github.com/sgruenholz2/rfcs/blob/recursive-components-patch-1/active-rfcs/0000-recursive-components.md - - https://github.com/withastro/compiler/pull/270 - -# Summary - -Allow a way for astro components to render themselves recursively by exposing a new Astro.self property. - -# Example - -We have a set of items we want to render, stored in an object, indexed by id. -Each item optionally defines an array of childIds. - -``` -const items = { - "1": { - "id": "1", - "childIds": ["1-1","1-2"] - }, - "1-1": { - id: "1-1", - childIds: ["1-1-1", "1-1-2"] - }, - "1-2": { - id: "1-2", - childIds: [] - }, - "1-1-1": { - id: "1-1-1", - childIds: [] - }, - "1-1-2": { - id: "1-1-2", - childIds: [] - }, - "1-1-2-1": { - id: "1-1-2-1", - childIds: [] - }, - "1-1-2-2": { - id: "1-1-2-2", - childIds: [] - }, -}; -``` - -We want to use a recursive component to render the entire tree, -(or one of its branches) by passing in both the entire tree (the items object) -and an itemId like this: - -``` -// src/components/RecursiveItem.astro ---- -const {itemId, items} = Astro.props; -const {childIds} = items[itemId]; ---- -
  • - Item: {itemId} - {childIds.length ? ( -
      - {childIds.map(childId => ( - - ))} -
    - ) : ""} - -``` - -Note that in the above example, the `` component provides -access to this component's render function, allowing it to -reference itself. - -# Motivation - -Nested data structures are everywhere. Common examples include nested blog comments, -file explorer trees, extensive navigation menu trees, or using a headless CMS and nested content blocks to render an entire website. - -Even when the entire tree structure is KNOWN, for anything more than a few levels deep, -using a recursive Component/function for rendering is the most efficient and elegant approach. - -When the tree structure is UNKNOWN, and you want to support N levels deep, using a recursive -Component/function is the ONLY approach that will work. This is often the case -when fetching data from an API. - -Handling this use case lets .astro components do things that other component -frameworks can do, making it a 1st class component framework in its own right. - -The Single File per Component (SFC) pattern that Astro uses is simple, but inflexible. -In other frameworks like React (and presumable Vue and SolidJs) you can create multiple -components within a single file. -This allows you to can create both a function/component to render an `` and another -function/component to render `` have them reference each other, and then expose -either/both as exports. You can't do this with SFC. And, if you try to create these as -2 separate files, you get circular dependencies. The only way for SFC to allow for recursion -is by allowing a component access to reference it's own render function. - -Svelte, which also uses SFC, has already -encountered and solved for this issue by exposing the [svelte:self](https://svelte.dev/docs#svelte_self) -attribute as part of their API. - - -# Detailed design - -The `Astro.self` property exposes the render function of the component. -Defining a constant in the frontmatter and setting it equal to this allows -you to name this component whatever you wish. - -``` -// src/components/RecursiveItem.astro ---- -const {itemId, items} = Astro.props; -const {childIds} = items[itemId]; -const MyRecursiveItem = Astro.self; ---- -
  • - Item: {itemId} - {childIds.length ? ( -
      - {childIds.map(childId => ( - - ))} -
    - ) : ""} - -``` - -# Alternatives - -The other design pattern considered was to mimic the Svelte syntax, using -`` - -This follows an already established precedent (for those familiar with Svelte), -but there's nothing else in Astro yet that leverages `astro:` prefixes as -components. - -By contrast, using `Astro.self` is more explicit, -requiring a bit more work to tap into an advanced feature, which was marginally -preferred by some in the discussion group. Also providing incremental value is -the ability for developers to name the component how they wish (although -this may also be a source of some minor confusion). - -Most importantly, it builds upon existing Astro conventions. We're already -referencing things like `Astro.props` and `Astro.request.params` in the frontmatter. -This is just one more `Astro` property that becomes available there. - -# Adoption strategy - -- No backwards compatibility concerns -- No potential for breaking changes -- Worth noting that recursive rendering can be already achieved with any number of other -platforms: React, Vue, Svelte, etc., so we can approach this at our leisure with -the short term answer to this being: "Do that in your own platform for now". -- Need to update Documentation - -# Unresolved questions - -Svelte has some [additional restrictions](https://svelte.dev/docs#svelte_self) -about how/where theirs can be used. It must be within a conditional or an "each" loop. -They are attempting to protect against infinite loops. Should we build in similar protections? diff --git a/proposals/0010-recursive-components.md b/proposals/0010-recursive-components.md index 8fc4f174..c99f9578 100644 --- a/proposals/0010-recursive-components.md +++ b/proposals/0010-recursive-components.md @@ -1,6 +1,8 @@ -- Start Date: Date: 2021-12-15 -- Reference Issues: https://github.com/withastro/rfcs/discussions/45 -- Implementation PR: https://github.com/sgruenholz2/rfcs/blob/recursive-components-patch-1/active-rfcs/0000-recursive-components.md +- Start Date: Date: 2021-12-15 +- Reference Issues: https://github.com/withastro/rfcs/discussions/45 +- Implementation PR: + - https://github.com/sgruenholz2/rfcs/blob/recursive-components-patch-1/active-rfcs/0000-recursive-components.md + - https://github.com/withastro/compiler/pull/270 # Summary From 2bb7077bf1b6e740bca994e564a67fde861301a3 Mon Sep 17 00:00:00 2001 From: Tony Sullivan Date: Tue, 15 Feb 2022 17:54:06 +0000 Subject: [PATCH 002/300] Update 0006-support-non-html-files.md Adding a link to the merged PR --- proposals/0006-support-non-html-files.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0006-support-non-html-files.md b/proposals/0006-support-non-html-files.md index a8961cc4..0e3b90a4 100644 --- a/proposals/0006-support-non-html-files.md +++ b/proposals/0006-support-non-html-files.md @@ -10,7 +10,7 @@ - Start Date: 2021-09-03 - Reference Issues: N/A -- Implementation PR: N/A +- Implementation PR: [astro #2586](https://github.com/withastro/astro/pull/2586) # Summary From 93fffd2c2e6a121a4c54e0cc7c988e7e827482c2 Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Tue, 8 Mar 2022 07:12:11 +0000 Subject: [PATCH 003/300] draft in progress --- proposals/0015-integrations.md | 101 +++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 proposals/0015-integrations.md diff --git a/proposals/0015-integrations.md b/proposals/0015-integrations.md new file mode 100644 index 00000000..59fd34ce --- /dev/null +++ b/proposals/0015-integrations.md @@ -0,0 +1,101 @@ +- Start Date: 03-07-2022 +- Reference Issues: +- Implementation PR: + +# Summary + +An integration system for extending Astro. + +# Example + +```js +// astro.config.js +export default ({ + integrations: [ + import('@astrojs/vue'), + import('@astrojs/tailwind'), + import('@astrojs/sitemap'), + [import('@astrojs/partytown'), {/* options */}], + ], +}); +``` + +# Motivation + +Astro has no integration or plugin system of its own. Astro uses Vite interanlly, and Vite does have a plugin system that some users have bene able to hook into. However, this experience leaves a lot to be desired (doesn't support all use-cases, touching the `vite` config as a user is considered advanced). + +If we were to add an integration system to Astro, we would unlock two huge wins: +1. Empower users to do more by extending Astro themselves (not blocked by what Astro core can/can't do). +2. Make it easier for users to add common features/libraries to their website + +A great example of this in action is to compare Partytown's documentation for Astro vs. Nuxt: +- Astro: [3 steps across frontmatter, template, and a custom npm script](https://partytown.builder.io/astro) +- Nuxt: [1 step](https://partytown.builder.io/nuxt) + +Tailwind is another example of a tool that is difficult and multi-faceted to add to Astro today, but would be quick and easy if we had an integration. + +# Detailed design + +## User Configuration API + +```js +// astro.config.js +export default ({ + integrations: [ + import('@astrojs/vue'), + // or, with options + [import('@astrojs/vue'), {/* options */}], + ], +}); +``` + +## Integration API + +```ts +export interface AstroIntegration { + name: string; + hooks: { + 'astro:config:setup': (options: { + config: AstroConfig; + command: 'dev' | 'build'; + addRenderer: (renderer: AstroRenderer) => void; + injectScript: (stage: 'beforeHydration' | 'head' | 'bundle', content: string) => void; + injectHtml: (stage: 'head' | 'body', element: string) => void; + }) => void | Partial | Promise | Promise>; + 'astro:config:done': (options: { config: AstroConfig }) => void | Promise; + 'astro:server:setup': (options: { server: vite.ViteDevServer }) => void | Promise; + 'astro:server:start': (options: { address: AddressInfo }) => void | Promise; + 'astro:server:done': () => void | Promise; + 'astro:build:start': () => void | Promise; + 'astro:build:done': (options: { pages: string[]; dir: URL }) => void | Promise; + }; +} +``` + +# Drawbacks + +Why should we *not* do this? Please consider: + +- Implementation cost, both in term of code size and complexity. +- Whether the proposed feature can be implemented in user space. +- Impact on teaching people Astro. +- Integration of this feature with other existing and planned features +- Cost of migrating existing Astro applications (_is it a breaking change?_) + +There are tradeoffs to choosing any path. Attempt to identify them here. + +# Alternatives + +What other designs have been considered? What is the impact of not doing this? + +# Adoption strategy + +- This would be a breaking change for most users. +- We are exploring a codemod + `astro setup` / `astro setup [name]` commands to mitigate this cost + +With enough time and effort we could maybe do this in a backwards-compatible way. However, this is a change that is relatively straightforward to make (only change a single file) AND we actually do want people to know that Astro supports integrations now. As far as breaking changes go, this is a pretty positive one. + + +# Unresolved questions + +TODO \ No newline at end of file From 2346dc98e42e952260a08468a05025d2dc26c7fd Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Thu, 10 Mar 2022 02:39:21 +0000 Subject: [PATCH 004/300] finalize draft --- proposals/0000-integrations.md | 171 +++++++++++++++++++++++++++++++++ proposals/0015-integrations.md | 101 ------------------- 2 files changed, 171 insertions(+), 101 deletions(-) create mode 100644 proposals/0000-integrations.md delete mode 100644 proposals/0015-integrations.md diff --git a/proposals/0000-integrations.md b/proposals/0000-integrations.md new file mode 100644 index 00000000..a7c1a14a --- /dev/null +++ b/proposals/0000-integrations.md @@ -0,0 +1,171 @@ +- Start Date: 03-07-2022 +- Reference Issues: +- Implementation PR: + +# Summary + +An integration system for extending Astro. + +# Example + +```js +// astro.config.js +export default ({ + integrations: [ + import('@astrojs/vue'), + import('@astrojs/tailwind'), + [import('@astrojs/partytown'), {/* options */}], + ], +}); +``` + +```js +// Static imports are also supported, for type checking. +import vuePlugin from '@astrojs/vue'; +import tailwindPlugin from '@astrojs/tailwind'; +import partytownPlugin from '@astrojs/partytown'; + +export default ({ + integrations: [ + vuePlugin(), + tailwindPlugin(), + partytownPlugin({/* options */}) + ], +}); +``` + +# Motivation + +Astro currently has no integration or plugin system of its own. Some users have been able to hook into Astro's internal Vite plugin system to extend Astro, which lets you control the build pipeline. However, this only gives you access to extend one piece of what Astro does (the build). The rest of Astro remains out of touch. + +Adding a first-class integration system would unlock a few huge wins for Astro: +1. Empower users to do more by extending Astro themselves (not blocked by what Astro core can/can't do). +2. Empower users to do more by reusing shared extensions (easy to add "X" to an Astro project). +3. Empower more user-land experimentation, reducing how much Astro core blocks what a user can/can't do with Astro. +4. Organize our codebase by refactoring more logic into internal integrations or moved out of core entirely into external integrations. + +To illustrate this, compare Partytown's documentation for getting started wtih Astro vs. getting started with Nuxt: +- Astro: [3 steps across frontmatter, template, and a custom npm script](https://partytown.builder.io/astro) +- Nuxt: [1 step](https://partytown.builder.io/nuxt) + +Tailwind suffers from a similar difficult setup story in Astro. + +# Detailed design + +**Background:** This API was reached through weeks of experimentation of different designs (see alternatives below). To test the work, I created the following integrations: + +- **Renderers:** `lit`, `svelte`, `react`, `preact`, `vue`, `solid` +- **Libraries:** `tailwind`, `partytown`, `turbolinks` +- **Features:** `sitemap` + +## Integration Usage API + +```js +// astro.config.js +export default ({ + integrations: [ + import('@astrojs/vue'), + // or, with options + [import('@astrojs/vue'), {/* options */}], + // or, as a static ESM import at the top of the file + vuePlugin({/* options */}) + ], +}); +``` + +## Integration Author API + +```ts +export interface AstroIntegration { + name: string; + hooks: { + /** SETUP */ + /** Called on astro startup, lets you modify config */ + 'astro:config:setup': (options: ConfigSetupOptions) => void | Promise; + /** Called after config is finalized, lets you store config object for later */ + 'astro:config:done': (options: { config: Readonly }) => void | Promise; + + /** DEV */ + /** Called on server setup, lets you modify the server */ + 'astro:server:setup': (options: { server: vite.ViteDevServer }) => void | Promise; + /** Called on server startup, lets you read the dev server address/URL */ + 'astro:server:start': (options: { address: AddressInfo }) => void | Promise; + /** Called on server exit */ + 'astro:server:done': () => void | Promise; + + /** BUILD */ + /** Called on build start, lets you modify the server */ + 'astro:build:start': () => void | Promise; + /** Called on build done, lets you read metadata about the build */ + 'astro:build:done': (options: { pages: string[]; dir: URL }) => void | Promise; + }; +} +``` + + +### Integration Author API: Hooks + +- The **Hook** is the main primitive of this proposed integration system. +- Hooks optimize for maximum flexibility for the integration author: you can use our provided helper methods to perform common tasks during each hook, or write your own custom logic for advanced needs. +- Hooks are conceptually aligned with how Rollup and Vite plugins work. This lets us pass some hooks (like 'astro:server:start') to Vite (the Vite `configureServer()` hook) with trivial effort. +- The `hooks: {}` API conceptually matches Rollup & Vite but was designed to avoid the risk of conflict that would have been introduced had we literally extending the Vite plugin idea. + - This is why we prefix all hooks with `astro:`. + + +# Drawbacks + +- **Breaking changes across an integration system are expensive.** This can be mitigated until v1.0.0, see adoption strategy below. + + +# Alternatives + +A previous design used a functional API more like this: + +``` +export default function(integration) { + integration.injectScript(...); + integration.modifyConfig(...); + integration.configureServer(...); + integration.mountDirectory(...); +} +``` + +This produced nice, linear code but at the expense of internal Astro complexity and limited flexibility that eventually blocked some integrations from being possible: + +1. **Complexity:** Astro had to run the entire integration upfront on startup, and then cache these results for later when needed. Many of the methods (like `configureServer`) ended up acting more like hooks anyway. +2. **Inflexible:** Integrations like Partytown couldn't work with a provided `mountDirectory` helper method because because they need to run their own `fs` logic on the final build directory. + +Advanced use-cases like this essentially required hooks to perform custom logic as needed, so the design shifted away from "helpers that do everything for you" and towards "provide hooks with helpers availble if needed". + +# Adoption strategy + +## Experimental Flag + +This proposal suggests only supporting official integrations to start, and mark 3rd-part integrations as experimental via something like a config flag (`--experimental-integrations`) until we hit `v1.0.0-beta`. + +This would let us test the integration system and respond to user feedback before finalizing. + +## Renderers + +The `renderers` API is deprecated by this proposal, with all `renderers` becoming `integrations`: `@astrojs/renderer-vue` -> `@astrojs/vue`. + +With a lot of work, we could do this in a backwards compatible way. However, I would like to avoid that complexity (and the potential for bugs that comes with it) and do this in a breaking change for the following reasons: + +1. **low-effort to upgrade:** updating your renderers to integrations would involve changing your config file only. A codemod could be provided to make this even easier. +1. **easy to assist:** Unlike past breaking changes, this will be fairly easy for Astro to provide helpful output to migrate from one to the other. + +``` +$ astro build + +Astro renderers are no longer supported! +Update your config to use Astros new integration system: + +- renderers: ["@astrojs/vue"] ++ integrations: [import("@astrojs/vue")] +``` + +# Unresolved questions + +- Bikeshedding all the things +- Can we pair this with an `astro add NAME` CLI command? +- Do we use this as a chance to remove the built-in renderers, and force users to install all framework integrations themselves as npm packages? I would like to save that for a later breaking change, since it would make this breaking change more complex (see the adoption strategy defined above). \ No newline at end of file diff --git a/proposals/0015-integrations.md b/proposals/0015-integrations.md deleted file mode 100644 index 59fd34ce..00000000 --- a/proposals/0015-integrations.md +++ /dev/null @@ -1,101 +0,0 @@ -- Start Date: 03-07-2022 -- Reference Issues: -- Implementation PR: - -# Summary - -An integration system for extending Astro. - -# Example - -```js -// astro.config.js -export default ({ - integrations: [ - import('@astrojs/vue'), - import('@astrojs/tailwind'), - import('@astrojs/sitemap'), - [import('@astrojs/partytown'), {/* options */}], - ], -}); -``` - -# Motivation - -Astro has no integration or plugin system of its own. Astro uses Vite interanlly, and Vite does have a plugin system that some users have bene able to hook into. However, this experience leaves a lot to be desired (doesn't support all use-cases, touching the `vite` config as a user is considered advanced). - -If we were to add an integration system to Astro, we would unlock two huge wins: -1. Empower users to do more by extending Astro themselves (not blocked by what Astro core can/can't do). -2. Make it easier for users to add common features/libraries to their website - -A great example of this in action is to compare Partytown's documentation for Astro vs. Nuxt: -- Astro: [3 steps across frontmatter, template, and a custom npm script](https://partytown.builder.io/astro) -- Nuxt: [1 step](https://partytown.builder.io/nuxt) - -Tailwind is another example of a tool that is difficult and multi-faceted to add to Astro today, but would be quick and easy if we had an integration. - -# Detailed design - -## User Configuration API - -```js -// astro.config.js -export default ({ - integrations: [ - import('@astrojs/vue'), - // or, with options - [import('@astrojs/vue'), {/* options */}], - ], -}); -``` - -## Integration API - -```ts -export interface AstroIntegration { - name: string; - hooks: { - 'astro:config:setup': (options: { - config: AstroConfig; - command: 'dev' | 'build'; - addRenderer: (renderer: AstroRenderer) => void; - injectScript: (stage: 'beforeHydration' | 'head' | 'bundle', content: string) => void; - injectHtml: (stage: 'head' | 'body', element: string) => void; - }) => void | Partial | Promise | Promise>; - 'astro:config:done': (options: { config: AstroConfig }) => void | Promise; - 'astro:server:setup': (options: { server: vite.ViteDevServer }) => void | Promise; - 'astro:server:start': (options: { address: AddressInfo }) => void | Promise; - 'astro:server:done': () => void | Promise; - 'astro:build:start': () => void | Promise; - 'astro:build:done': (options: { pages: string[]; dir: URL }) => void | Promise; - }; -} -``` - -# Drawbacks - -Why should we *not* do this? Please consider: - -- Implementation cost, both in term of code size and complexity. -- Whether the proposed feature can be implemented in user space. -- Impact on teaching people Astro. -- Integration of this feature with other existing and planned features -- Cost of migrating existing Astro applications (_is it a breaking change?_) - -There are tradeoffs to choosing any path. Attempt to identify them here. - -# Alternatives - -What other designs have been considered? What is the impact of not doing this? - -# Adoption strategy - -- This would be a breaking change for most users. -- We are exploring a codemod + `astro setup` / `astro setup [name]` commands to mitigate this cost - -With enough time and effort we could maybe do this in a backwards-compatible way. However, this is a change that is relatively straightforward to make (only change a single file) AND we actually do want people to know that Astro supports integrations now. As far as breaking changes go, this is a pretty positive one. - - -# Unresolved questions - -TODO \ No newline at end of file From 169053733d40a405670f748832bcf752b312a569 Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Thu, 10 Mar 2022 02:49:35 +0000 Subject: [PATCH 005/300] add links to existing integrations --- proposals/0000-integrations.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/0000-integrations.md b/proposals/0000-integrations.md index a7c1a14a..1b6aebd3 100644 --- a/proposals/0000-integrations.md +++ b/proposals/0000-integrations.md @@ -52,11 +52,11 @@ Tailwind suffers from a similar difficult setup story in Astro. # Detailed design -**Background:** This API was reached through weeks of experimentation of different designs (see alternatives below). To test the work, I created the following integrations: +**Background:** This API was reached through weeks of experimentation of different designs (see alternatives below). To test the work, I designed and build the following integrations, which are useful for illustrating this RFC: -- **Renderers:** `lit`, `svelte`, `react`, `preact`, `vue`, `solid` -- **Libraries:** `tailwind`, `partytown`, `turbolinks` -- **Features:** `sitemap` +- **Renderers:** [`lit`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/lit/index.js), [`svelte`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/svelte/index.js), [`react`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/react/index.js), [`preact`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/preact/index.js), [`vue`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/vue/index.js), [`solid`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/solid/index.js) +- **Libraries:** [`tailwind`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/tailwind/index.js), [`partytown`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/partytown/index.js), [`turbolinks`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/turbolinks/index.js) +- **Features:** [`sitemap`](https://github.com/withastro/astro/blob/wip-integrations-4/packages/integrations/sitemap/index.js) ## Integration Usage API From 80135989e7aafb3fb245ba762d5acaa3c3c9480a Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Wed, 9 Mar 2022 21:38:07 -0800 Subject: [PATCH 006/300] Update 0000-integrations.md --- proposals/0000-integrations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0000-integrations.md b/proposals/0000-integrations.md index 1b6aebd3..f9ac24bc 100644 --- a/proposals/0000-integrations.md +++ b/proposals/0000-integrations.md @@ -1,6 +1,6 @@ - Start Date: 03-07-2022 - Reference Issues: -- Implementation PR: +- Implementation PR: [(git branch, runnable but not yet a PR)](https://github.com/withastro/astro/compare/wip-integrations-4?expand=1) # Summary @@ -168,4 +168,4 @@ Update your config to use Astros new integration system: - Bikeshedding all the things - Can we pair this with an `astro add NAME` CLI command? -- Do we use this as a chance to remove the built-in renderers, and force users to install all framework integrations themselves as npm packages? I would like to save that for a later breaking change, since it would make this breaking change more complex (see the adoption strategy defined above). \ No newline at end of file +- Do we use this as a chance to remove the built-in renderers, and force users to install all framework integrations themselves as npm packages? I would like to save that for a later breaking change, since it would make this breaking change more complex (see the adoption strategy defined above). From 2da92649de7cd095ca3ee92b9b92fc590ae255fb Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Thu, 10 Mar 2022 22:25:31 +0000 Subject: [PATCH 007/300] initial draft --- proposals/0000-markdown-content-redesign.md | 123 ++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 proposals/0000-markdown-content-redesign.md diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md new file mode 100644 index 00000000..222b0472 --- /dev/null +++ b/proposals/0000-markdown-content-redesign.md @@ -0,0 +1,123 @@ +- Start Date: 03-10-2022 +- Reference Issues: +- Implementation PR: + +# Summary + +A v2.0 API for working with local Markdown files in Astro. + +# Example + +```astro +--- +// BASIC USAGE: + +// Note: Astro.fetchContent() removed in favor of direct `import.meta.globEager` usage +// Note: This works in non-Astro framework components! +const markdownFilesObj = import.meta.globEager('../posts/*.md'); +const markdownFilesArr = Object.values(markdownFilesObj); +const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category === 'blog'); + +// Note: Individual markdown imports are now supported as well. +const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post.md') +--- + +

    {firstPost.data.title}

    +

    Author: {firstPost.data.author}

    +

    Permalink

    + +
    {firstPost.getContent()}
    + +
    +
    +``` + + +# Motivation + +- **Performance:** Markdown rendering continues to be an expensive part of Astro's runtime. While we can look into faster renderers (ex: Goldmark) there is still a big inefficincy on our end with how we hold the tool. Astro currently renders markdown on import, which means that multi-file markdown imports (via both `import.meta.glob` and `Astro.fetchContent()`) block the response until all imported files are rendered, even if only a subset of these files are ever used on the page after filtering. +- **Performance (Memory):** For large projects, this also forces Astro to store all rendered Markdown output in memory at the same time, making it difficult to manage memory efficiently. Chris Bonger reached max memory limits in a project of only 800 markdown pages, and another user reported builds requiring 50+ GB of memory. +- **Infinite Loops:** By rendering during import, we also introduce a risk for infinite loops when we call `fetchContent()` in a Markdown layout used by a fetched page, bouncing between render and `fetchContent()` infinitely. + + +# Detailed design + +- This is implemented in a v2.0 of the internal `'astro:markdown'` Vite plugin +- All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by this plugin. + +1. `src/posts/foo.md?import` + 1. loaded from file system + 2. frontmatter is parsed from the file + 3. a JS module is returned with the following JS: + ```js + ` + // Static: + export const data = ${JSON.stringify(frontmatter)}; + export const file = ${JSON.stringify(id)}; + export const url = ${JSON.stringify(url || undefined)}; + + // Deferred: + export default async function load(...args) { + return (await import(${JSON.stringify(fileId + '?content')})); + }; + export default function getContent() { + return load().then((m: any) => m.default) + } + export default function getHeaders() { + return load().then((m: any) => m.metadata.headers) + } + ` + ``` + +2. `src/posts/foo.md?content` + 1. loaded from file system + 2. render using `config.markdownOptions.render(source)` + 3. return an Astro component representing the markdown content + +3. `src/posts/foo.md` + 1. If we resolve an ID without a query param, we have to decide which to serve + 2. if `importer` is set, then its the user importing via `import.meta.glob` + 1. **result:** resolve to `?import` + 3. if `importer` is null, then its Astro importing via `ssrLoadModule()` or `vite.build()` + 1. **result:** resolve to `?content` since this is a page + +# Drawbacks + +There are few drawbacks to this conceptual approach: its an antipattern to run advanced rendering logic during an import stage for the reasons listed in the "Motivation" section above. + +There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). Vite's automated CI running on Astro should mitigate this somewhat. + +On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.glob` need to use the unmodified import without query params. + + +# Alternatives + +`Astro.fetchContent()` is deprecated here, in favor of the Vite `import.meta.glob` and `import.meta.globEager` APIs. Currently, we use `Astro.fetchContent()` to avoid users having to write `import.meta.glob` themselves. However, this caused the following drawbacks: + +- Only `.astro` files can fetch local markdown files +- `@babel/traverse` used internally to rewrite `Astro.fetchContent()` to `import.meta.glob()`. Doing an AST traversal like this is expensive for such a small change to the AST. +- `Astro.fetchContent()` ended up being too limiting for many users, and today we regularly suggest users use `import.meta.glob` directly. + +An alternative approach could be to either keep `Astro.fetchContent()`, or use `fetchContent` as a much simpler imported helper, similar to the below: + +```js +const markdownFilesObj = import.meta.globEager('../posts/*.md'); +const markdownFilesArr = Object.values(markdownFilesObj); + +// vs. + +import {content} from 'astro/util'; +const markdownFilesArr = content(import.meta.globEager('../posts/*.md')); +``` + +This RFC takes the stance that this most likely not worth the effort, and that users are better served by using the more polished/battle-tested `import.meta.glob` Vite API directly even if it feels more advanced than the current `Astro.fetchContent()`. + + +# Adoption strategy + +1. `Astro.fetchContent()` will throw an error pointing to migration docs. +2. Direct calls to `import.meta.glob()` will change in a breaking way. We would document this in a migration guide. We could mitigate this with some throw errors if you access old properties on the new module interface. + +# Unresolved questions + +- None yet. \ No newline at end of file From e798a441739be2f847b70b351e48317302750080 Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Thu, 10 Mar 2022 14:30:08 -0800 Subject: [PATCH 008/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index 222b0472..70da0b0d 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -1,5 +1,5 @@ - Start Date: 03-10-2022 -- Reference Issues: +- Reference Issues: https://github.com/withastro/rfcs/discussions/5, https://github.com/withastro/rfcs/discussions/118 - Implementation PR: # Summary @@ -36,7 +36,7 @@ const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post. # Motivation - **Performance:** Markdown rendering continues to be an expensive part of Astro's runtime. While we can look into faster renderers (ex: Goldmark) there is still a big inefficincy on our end with how we hold the tool. Astro currently renders markdown on import, which means that multi-file markdown imports (via both `import.meta.glob` and `Astro.fetchContent()`) block the response until all imported files are rendered, even if only a subset of these files are ever used on the page after filtering. -- **Performance (Memory):** For large projects, this also forces Astro to store all rendered Markdown output in memory at the same time, making it difficult to manage memory efficiently. Chris Bonger reached max memory limits in a project of only 800 markdown pages, and another user reported builds requiring 50+ GB of memory. +- **Performance (Memory):** For large projects, this also forces Astro to store all rendered Markdown output in memory at the same time, making it difficult to manage memory efficiently. Our community has reported builds maxing out memory limits in a project of only 800 markdown pages, and builds requiring 50+ GB of memory. - **Infinite Loops:** By rendering during import, we also introduce a risk for infinite loops when we call `fetchContent()` in a Markdown layout used by a fetched page, bouncing between render and `fetchContent()` infinitely. @@ -120,4 +120,4 @@ This RFC takes the stance that this most likely not worth the effort, and that u # Unresolved questions -- None yet. \ No newline at end of file +- None yet. From 23bc5e2454086578001c0066f0141fe18df7a3ee Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Fri, 11 Mar 2022 20:48:53 -0800 Subject: [PATCH 009/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 51 +++++++++++---------- 1 file changed, 27 insertions(+), 24 deletions(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index 70da0b0d..6ec65ec8 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -19,7 +19,7 @@ const markdownFilesArr = Object.values(markdownFilesObj); const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category === 'blog'); // Note: Individual markdown imports are now supported as well. -const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post.md') +const firstPost = await import('../posts/first-post.md'); ---

    {firstPost.data.title}

    @@ -27,9 +27,6 @@ const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post.

    Permalink

    {firstPost.getContent()}
    - -
    -
    ``` @@ -60,10 +57,10 @@ const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post. export default async function load(...args) { return (await import(${JSON.stringify(fileId + '?content')})); }; - export default function getContent() { + export function getContent() { return load().then((m: any) => m.default) } - export default function getHeaders() { + export function getHeaders() { return load().then((m: any) => m.metadata.headers) } ` @@ -76,47 +73,53 @@ const firstPost = markdownFilesFiltered[0] || await import('../posts/first-post. 3. `src/posts/foo.md` 1. If we resolve an ID without a query param, we have to decide which to serve - 2. if `importer` is set, then its the user importing via `import.meta.glob` + 2. if `importer` is set, then its the user importing via `import.meta.glob` or `import.meta.globEager` 1. **result:** resolve to `?import` 3. if `importer` is null, then its Astro importing via `ssrLoadModule()` or `vite.build()` 1. **result:** resolve to `?content` since this is a page # Drawbacks -There are few drawbacks to this conceptual approach: its an antipattern to run advanced rendering logic during an import stage for the reasons listed in the "Motivation" section above. +There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. -There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). Vite's automated CI running on Astro should mitigate this somewhat. - -On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.glob` need to use the unmodified import without query params. +The `import.meta.glob` and `import.meta.globEager` API is more complex to understand, vs. the very literal `Astro.fetchContent()`. This RFC takes the stance that the fact that these APIs are well-documented and battle-tested is worth the complexity cost vs. `Astro.fetchContent()`. However, see the "Alternatives" section below for a few different options for continuing to maintain an additional `Astro.fetchContent()` API as the "simple" interface. # Alternatives -`Astro.fetchContent()` is deprecated here, in favor of the Vite `import.meta.glob` and `import.meta.globEager` APIs. Currently, we use `Astro.fetchContent()` to avoid users having to write `import.meta.glob` themselves. However, this caused the following drawbacks: +`Astro.fetchContent()` is deprecated here, in favor of the Vite `import.meta.glob` and `import.meta.globEager` APIs. + +Currently, we maintain a `Astro.fetchContent()` helper function to avoid users having to write `import.meta.glob` themselves. However, this is more complex than it looks and actually triggers a full Babel parse & traverse to work. In addition, it causes the following drawbacks: -- Only `.astro` files can fetch local markdown files +- Only `.astro` files can use `Astro.fetchContent()` - `@babel/traverse` used internally to rewrite `Astro.fetchContent()` to `import.meta.glob()`. Doing an AST traversal like this is expensive for such a small change to the AST. -- `Astro.fetchContent()` ended up being too limiting for many users, and today we regularly suggest users use `import.meta.glob` directly. +- `Astro.fetchContent()` ended up being too limiting for many users, and today we often suggest users use `import.meta.glob` directly. + +This RFC aims to remove `Astro.fetchContent()` entirely, and that users are better served by using the more polished/battle-tested `import.meta.glob` Vite API directly. Even if it feels more advanced than the current `Astro.fetchContent()`, it's better than maintaining two different APIs to do the same thing. + +However, there are two alternatives that we can also consider: -An alternative approach could be to either keep `Astro.fetchContent()`, or use `fetchContent` as a much simpler imported helper, similar to the below: +1. Continue to support it as-is: `Astro.fetchContent()` +2. Support it as a more flexible helper function: `import {$content} from 'astro/util'`; ```js +// 1. Today +const markdownFilesArr = Astro.fetchContent('../posts/*.md'); + +// 2. Proposed API const markdownFilesObj = import.meta.globEager('../posts/*.md'); const markdownFilesArr = Object.values(markdownFilesObj); -// vs. - -import {content} from 'astro/util'; -const markdownFilesArr = content(import.meta.globEager('../posts/*.md')); +// 3. Alternative API +import {$content} from 'astro/util'; +const markdownFilesArr = $content(import.meta.globEager('../posts/*.md')); ``` -This RFC takes the stance that this most likely not worth the effort, and that users are better served by using the more polished/battle-tested `import.meta.glob` Vite API directly even if it feels more advanced than the current `Astro.fetchContent()`. - - # Adoption strategy -1. `Astro.fetchContent()` will throw an error pointing to migration docs. -2. Direct calls to `import.meta.glob()` will change in a breaking way. We would document this in a migration guide. We could mitigate this with some throw errors if you access old properties on the new module interface. +1. `Astro.fetchContent()` will become deprecated, but not throw an error. It can continue to be a wrapper around `import.meta.globEager` for an easier migration. +2. We update our docs to document `import.meta.globEager` instead of `Astro.fetchContent()` +3. In Astro v1.0, we remove `Astro.fetchContent()`. # Unresolved questions From dfc981838cf883e2659d464ba584d74d2c2b6a3c Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Tue, 15 Mar 2022 15:57:44 -0700 Subject: [PATCH 010/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 62 ++++++++++----------- 1 file changed, 28 insertions(+), 34 deletions(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index 6ec65ec8..f02ca1b3 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -11,19 +11,25 @@ A v2.0 API for working with local Markdown files in Astro. ```astro --- // BASIC USAGE: +// Astro.fetchContent() replaced with a generalized `Astro.glob()` helper. +// This now works for non-MD files as well! +const markdownFilesArr = await Astro.glob('../posts/*.md'); +const markdownFilesFiltered = markdownFilesArr.filter(post => post.frontmatter.category === 'blog'); -// Note: Astro.fetchContent() removed in favor of direct `import.meta.globEager` usage -// Note: This works in non-Astro framework components! +// BASIC USAGE: +// Individual markdown imports are now supported as well! +const firstPost = await import('../posts/first-post.md'); + +// ADVANCED USAGE: +// You can also use the bare-metal `import.meta.glob/globEager` API +// This is useful in non-Astro files, like JSX and Vue. const markdownFilesObj = import.meta.globEager('../posts/*.md'); const markdownFilesArr = Object.values(markdownFilesObj); const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category === 'blog'); - -// Note: Individual markdown imports are now supported as well. -const firstPost = await import('../posts/first-post.md'); --- -

    {firstPost.data.title}

    -

    Author: {firstPost.data.author}

    +

    {firstPost.frontmatter.title}

    +

    Author: {firstPost.frontmatter.author}

    Permalink

    {firstPost.getContent()}
    @@ -39,6 +45,12 @@ const firstPost = await import('../posts/first-post.md'); # Detailed design +## `Astro.glob()` + +TODO + +## Deferred Implementation + - This is implemented in a v2.0 of the internal `'astro:markdown'` Vite plugin - All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by this plugin. @@ -49,7 +61,7 @@ const firstPost = await import('../posts/first-post.md'); ```js ` // Static: - export const data = ${JSON.stringify(frontmatter)}; + export const frontmatter = ${JSON.stringify(frontmatter)}; export const file = ${JSON.stringify(id)}; export const url = ${JSON.stringify(url || undefined)}; @@ -87,39 +99,21 @@ The `import.meta.glob` and `import.meta.globEager` API is more complex to unders # Alternatives -`Astro.fetchContent()` is deprecated here, in favor of the Vite `import.meta.glob` and `import.meta.globEager` APIs. +A previous version of this RFC removed all helpers, and asked the user to use `import.meta.glob` directly themselves. This meant less maintainance/overhead for Astro, but that API suffers from a few problems: -Currently, we maintain a `Astro.fetchContent()` helper function to avoid users having to write `import.meta.glob` themselves. However, this is more complex than it looks and actually triggers a full Babel parse & traverse to work. In addition, it causes the following drawbacks: +1. Not well documented outside of being an advanced Vite API +2. unneccesarily complex (ex: when do I use `import.meta.glob` vs. `import.meta.globEager()`. Also, what is `import.meta`?) -- Only `.astro` files can use `Astro.fetchContent()` -- `@babel/traverse` used internally to rewrite `Astro.fetchContent()` to `import.meta.glob()`. Doing an AST traversal like this is expensive for such a small change to the AST. -- `Astro.fetchContent()` ended up being too limiting for many users, and today we often suggest users use `import.meta.glob` directly. +However, based on feedback from the community I realized that we could keep the idea of a helper while fixing some of the problems of `Astro.fetchContent()`. This new `Astro.glob()` has the following benefits over `Astro.fetchContent()`: -This RFC aims to remove `Astro.fetchContent()` entirely, and that users are better served by using the more polished/battle-tested `import.meta.glob` Vite API directly. Even if it feels more advanced than the current `Astro.fetchContent()`, it's better than maintaining two different APIs to do the same thing. +1. Not just for Markdown, this is a generalized wrapper around `import.meta.globEager()` +2. Easy to understand the connection to `import.meta.glob()`, if your use-case needs that more flexible API -However, there are two alternatives that we can also consider: - -1. Continue to support it as-is: `Astro.fetchContent()` -2. Support it as a more flexible helper function: `import {$content} from 'astro/util'`; - -```js -// 1. Today -const markdownFilesArr = Astro.fetchContent('../posts/*.md'); - -// 2. Proposed API -const markdownFilesObj = import.meta.globEager('../posts/*.md'); -const markdownFilesArr = Object.values(markdownFilesObj); - -// 3. Alternative API -import {$content} from 'astro/util'; -const markdownFilesArr = $content(import.meta.globEager('../posts/*.md')); -``` # Adoption strategy -1. `Astro.fetchContent()` will become deprecated, but not throw an error. It can continue to be a wrapper around `import.meta.globEager` for an easier migration. -2. We update our docs to document `import.meta.globEager` instead of `Astro.fetchContent()` -3. In Astro v1.0, we remove `Astro.fetchContent()`. +1. We update documentation to document `Astro.glob()` over `Astro.fetchContent()`, with helpful migration docs. +1. `Astro.fetchContent()` will be removed and throw an error, telling you to replace with `Astro.glob()`. # Unresolved questions From a4c41867ce4f1b73d85e82c71cf715ae9df43edd Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Tue, 15 Mar 2022 17:16:15 -0700 Subject: [PATCH 011/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 56 ++++++++++++++++----- 1 file changed, 44 insertions(+), 12 deletions(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index f02ca1b3..9ae581f7 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -38,7 +38,7 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category # Motivation -- **Performance:** Markdown rendering continues to be an expensive part of Astro's runtime. While we can look into faster renderers (ex: Goldmark) there is still a big inefficincy on our end with how we hold the tool. Astro currently renders markdown on import, which means that multi-file markdown imports (via both `import.meta.glob` and `Astro.fetchContent()`) block the response until all imported files are rendered, even if only a subset of these files are ever used on the page after filtering. +- **Performance (Speed):** Markdown rendering continues to be an expensive part of Astro's runtime. While we can look into faster renderers (ex: Goldmark) there is still a big inefficincy on our end with how we hold the tool. Astro currently renders markdown on import, which means that multi-file markdown imports (via both `import.meta.glob` and `Astro.fetchContent()`) block the response until all imported files are rendered, even if only a subset of these files are ever used on the page after filtering. - **Performance (Memory):** For large projects, this also forces Astro to store all rendered Markdown output in memory at the same time, making it difficult to manage memory efficiently. Our community has reported builds maxing out memory limits in a project of only 800 markdown pages, and builds requiring 50+ GB of memory. - **Infinite Loops:** By rendering during import, we also introduce a risk for infinite loops when we call `fetchContent()` in a Markdown layout used by a fetched page, bouncing between render and `fetchContent()` infinitely. @@ -47,12 +47,41 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category ## `Astro.glob()` -TODO +```diff +// 1. +// We convert `Astro.glob()` calls at the point of the call, so that Vite +// can do its glob-magic without us re-implementing the complexity on our end. +// This is currently done on Astro.fetchContent(), so no change needed to existing behavior. +- Astro.glob('./foo/*.md'); ++ Astro.glob(import.meta.globEager('./foo/*.md')); +``` + +```ts +// 2. +Astro.glob = function(importMetaGlobResult: Record): Promise { + // Convert the `import.meta.globEager` result into an array. + let allEntries = [...Object.values(importMetaGlobResult)]; + // Report an error if no objects are returned. + // TODO: This may no longer be needed, since we changed Vite logging from error -> warn. + if (allEntries.length === 0) { + throw new Error(`Astro.glob() - no matches found.`); + } + // NOTE: This API was designed to be async, however we convert its argument to a resolve `globEager` + // object at compile time. We fake asynchrony here so that this API can still become async in the + // future if we ever move off of `import.meta.globEager()`. This should not impact users too much. + return Promise.resolve(allEntries); +} +``` + +- This replaces `Astro.fetchContent()` as the new preferred API +- This should support 99% of usage, especially when importing Markdown. +- This is optional: for more advanced use-cases we will still document the lower `import.meta.glob` & `import.meta.globEager` Vite APIs as advanced fallbacks. +- This is Astro-only: Glob imports inside JS and framework components are also considered advanced usage to use the Vite APIs. -## Deferred Implementation +## Deferred Import Implementation -- This is implemented in a v2.0 of the internal `'astro:markdown'` Vite plugin -- All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by this plugin. +- All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by an internal `'astro:markdown'` plugin. +- This logic is implemented as v2.0 of that same internal plugin. 1. `src/posts/foo.md?import` 1. loaded from file system @@ -94,26 +123,29 @@ TODO There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. -The `import.meta.glob` and `import.meta.globEager` API is more complex to understand, vs. the very literal `Astro.fetchContent()`. This RFC takes the stance that the fact that these APIs are well-documented and battle-tested is worth the complexity cost vs. `Astro.fetchContent()`. However, see the "Alternatives" section below for a few different options for continuing to maintain an additional `Astro.fetchContent()` API as the "simple" interface. - # Alternatives -A previous version of this RFC removed all helpers, and asked the user to use `import.meta.glob` directly themselves. This meant less maintainance/overhead for Astro, but that API suffers from a few problems: +A previous version of this RFC removed all helpers, and asked the user to use `import.meta.glob` & `import.meta.globEager` Vite APIs directly themselves. This meant less maintainance/overhead for Astro, but the Vite API suffers from a few problems: 1. Not well documented outside of being an advanced Vite API -2. unneccesarily complex (ex: when do I use `import.meta.glob` vs. `import.meta.globEager()`. Also, what is `import.meta`?) +2. unneccesarily complex (ex: when do I use `import.meta.glob` vs. `import.meta.globEager()`. Also, what is `import.meta` anyhow?) -However, based on feedback from the community I realized that we could keep the idea of a helper while fixing some of the problems of `Astro.fetchContent()`. This new `Astro.glob()` has the following benefits over `Astro.fetchContent()`: +Based on feedback from the community in this RFC, I realized that we could keep the idea of a helper while still fixing some of the problems that plagued the current `Astro.fetchContent()` API. This new `Astro.glob()` has the following benefits over `Astro.fetchContent()`: -1. Not just for Markdown, this is a generalized wrapper around `import.meta.globEager()` +1. Not just for Markdown, this is a generalized wrapper around `import.meta.globEager()` for basic usage 2. Easy to understand the connection to `import.meta.glob()`, if your use-case needs that more flexible API +3. Easy to use, no need to know what `import.meta` and `globEager` do # Adoption strategy 1. We update documentation to document `Astro.glob()` over `Astro.fetchContent()`, with helpful migration docs. -1. `Astro.fetchContent()` will be removed and throw an error, telling you to replace with `Astro.glob()`. +1. `Astro.fetchContent()` will be removed and throw an error, telling you to replace with `Astro.glob()`. + +While this is a breaking adoption strategy, the error message will be clear and `fetchContent` is fairly easy to find-replace across your codebase. + +The larger breaking change will be the fact that frontmatter data is no longer merged onto the main object (`post.title`), and is instead exported as `post.frontmatter.title`). Both our migration docs and the error message should help the user provide TS typings that will make these updates easier with TS errors inline. # Unresolved questions From 8718419fced3d295e0db4e84e7380849c457983f Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Wed, 16 Mar 2022 23:00:18 +0000 Subject: [PATCH 012/300] [accepted] 0015-integrations --- ...0-integrations.md => 0015-integrations.md} | 28 ++++++------------- 1 file changed, 8 insertions(+), 20 deletions(-) rename proposals/{0000-integrations.md => 0015-integrations.md} (93%) diff --git a/proposals/0000-integrations.md b/proposals/0015-integrations.md similarity index 93% rename from proposals/0000-integrations.md rename to proposals/0015-integrations.md index f9ac24bc..735431e8 100644 --- a/proposals/0000-integrations.md +++ b/proposals/0015-integrations.md @@ -9,18 +9,6 @@ An integration system for extending Astro. # Example ```js -// astro.config.js -export default ({ - integrations: [ - import('@astrojs/vue'), - import('@astrojs/tailwind'), - [import('@astrojs/partytown'), {/* options */}], - ], -}); -``` - -```js -// Static imports are also supported, for type checking. import vuePlugin from '@astrojs/vue'; import tailwindPlugin from '@astrojs/tailwind'; import partytownPlugin from '@astrojs/partytown'; @@ -61,14 +49,15 @@ Tailwind suffers from a similar difficult setup story in Astro. ## Integration Usage API ```js -// astro.config.js +import vuePlugin from '@astrojs/vue'; +import tailwindPlugin from '@astrojs/tailwind'; +import partytownPlugin from '@astrojs/partytown'; + export default ({ integrations: [ - import('@astrojs/vue'), - // or, with options - [import('@astrojs/vue'), {/* options */}], - // or, as a static ESM import at the top of the file - vuePlugin({/* options */}) + vuePlugin(), + tailwindPlugin(), + partytownPlugin({/* options */}) ], }); ``` @@ -161,11 +150,10 @@ Astro renderers are no longer supported! Update your config to use Astros new integration system: - renderers: ["@astrojs/vue"] -+ integrations: [import("@astrojs/vue")] ++ integrations: [(await import("@astrojs/vue")).default()] ``` # Unresolved questions -- Bikeshedding all the things - Can we pair this with an `astro add NAME` CLI command? - Do we use this as a chance to remove the built-in renderers, and force users to install all framework integrations themselves as npm packages? I would like to save that for a later breaking change, since it would make this breaking change more complex (see the adoption strategy defined above). From 366b677e60f95feaf0669c40c8b27d20831512da Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Mon, 21 Mar 2022 05:02:07 +0000 Subject: [PATCH 013/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 50 +++++++++++++++------ 1 file changed, 37 insertions(+), 13 deletions(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index 9ae581f7..5fc52a9c 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -38,10 +38,28 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category # Motivation +## Background: Performance Issues + - **Performance (Speed):** Markdown rendering continues to be an expensive part of Astro's runtime. While we can look into faster renderers (ex: Goldmark) there is still a big inefficincy on our end with how we hold the tool. Astro currently renders markdown on import, which means that multi-file markdown imports (via both `import.meta.glob` and `Astro.fetchContent()`) block the response until all imported files are rendered, even if only a subset of these files are ever used on the page after filtering. - **Performance (Memory):** For large projects, this also forces Astro to store all rendered Markdown output in memory at the same time, making it difficult to manage memory efficiently. Our community has reported builds maxing out memory limits in a project of only 800 markdown pages, and builds requiring 50+ GB of memory. - **Infinite Loops:** By rendering during import, we also introduce a risk for infinite loops when we call `fetchContent()` in a Markdown layout used by a fetched page, bouncing between render and `fetchContent()` infinitely. +## Background: Usability Issues + +- `Astro.fetchContent()` currently only supports Markdown files, which is confusing to some users. +- ESM `import` currently supports most file types *except* Markdown, which can work with `import` but you'll get back a different, undocumented object API than if you'd used `fetchContent()`. +- `import.meta.glob()` is another API available to users, but its unclear how `Astro.fetchContent()` and `import.meta.glob` are related. +- Users still have difficulty using Markdown files in their projects due to legacy API decisions that we've been trying to move away from (ex: `.content -> `.Content`) + +## RFC Goals + +- Consolidate all of the existing markdown features into a single API +- Align with Vite and how other file formats are built & imported +- Keep a user-friendly API so that users don't need to use `import.meta.glob/globEager` themselves +- If possible, open up this "user-friendly API" to more file types, not just Markdown +- Solve the mentioned performance issues. + + # Detailed design @@ -80,13 +98,15 @@ Astro.glob = function(importMetaGlobResult: Record): Promise ## Deferred Import Implementation -- All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by an internal `'astro:markdown'` plugin. -- This logic is implemented as v2.0 of that same internal plugin. +- This RFC seeks to refactor how Markdown is loaded internally AND update the Markdown API that users interact with. +- The API updates are captured in `1.` below, while the refactoring is captured in `2.` and `3.` +- The logic that manages all of this lives in the internal `'astro:markdown'` Vite plugin. +- All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by this plugin. 1. `src/posts/foo.md?import` - 1. loaded from file system - 2. frontmatter is parsed from the file - 3. a JS module is returned with the following JS: + 1. loads from file system + 2. parses frontmatter from the file into a JS object + 3. returns a JS module with the following JS: ```js ` // Static: @@ -98,19 +118,21 @@ Astro.glob = function(importMetaGlobResult: Record): Promise export default async function load(...args) { return (await import(${JSON.stringify(fileId + '?content')})); }; - export function getContent() { - return load().then((m: any) => m.default) + export function Content(...args: any) { + return load().then((m: any) => m.default(...args)) } export function getHeaders() { return load().then((m: any) => m.metadata.headers) } - ` + + // Minor Implementation Detail: Needed so that you can do `` in a template. + Content.isAstroComponentFactory = true; ``` 2. `src/posts/foo.md?content` - 1. loaded from file system - 2. render using `config.markdownOptions.render(source)` - 3. return an Astro component representing the markdown content + 1. loads from file system + 2. renders Markdown using `config.markdownOptions.render(source)` + 3. returns an Astro component representing the markdown content 3. `src/posts/foo.md` 1. If we resolve an ID without a query param, we have to decide which to serve @@ -121,7 +143,9 @@ Astro.glob = function(importMetaGlobResult: Record): Promise # Drawbacks -There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. +There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). + +On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. # Alternatives @@ -131,7 +155,7 @@ A previous version of this RFC removed all helpers, and asked the user to use `i 1. Not well documented outside of being an advanced Vite API 2. unneccesarily complex (ex: when do I use `import.meta.glob` vs. `import.meta.globEager()`. Also, what is `import.meta` anyhow?) -Based on feedback from the community in this RFC, I realized that we could keep the idea of a helper while still fixing some of the problems that plagued the current `Astro.fetchContent()` API. This new `Astro.glob()` has the following benefits over `Astro.fetchContent()`: +Based on feedback from the community, I revised this RFC and realized that we could keep the idea of a helper while still fixing some of the problems that plagued the current `Astro.fetchContent()` API. This new `Astro.glob()` has the following benefits over `Astro.fetchContent()`: 1. Not just for Markdown, this is a generalized wrapper around `import.meta.globEager()` for basic usage 2. Easy to understand the connection to `import.meta.glob()`, if your use-case needs that more flexible API From 9238fbfd30924365066a406c8ca0663e0e8b521a Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" Date: Mon, 21 Mar 2022 05:04:00 +0000 Subject: [PATCH 014/300] Update 0000-markdown-content-redesign.md --- proposals/0000-markdown-content-redesign.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0000-markdown-content-redesign.md b/proposals/0000-markdown-content-redesign.md index 5fc52a9c..9804811e 100644 --- a/proposals/0000-markdown-content-redesign.md +++ b/proposals/0000-markdown-content-redesign.md @@ -32,7 +32,7 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category

    Author: {firstPost.frontmatter.author}

    Permalink

    -
    {firstPost.getContent()}
    + ``` From 69cad439ef904b124bbddb454d2f98d6f916b68d Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Mon, 21 Mar 2022 09:28:01 -0400 Subject: [PATCH 015/300] Astro.request RFC --- proposals/astro-request.md | 110 +++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 proposals/astro-request.md diff --git a/proposals/astro-request.md b/proposals/astro-request.md new file mode 100644 index 00000000..8d500d91 --- /dev/null +++ b/proposals/astro-request.md @@ -0,0 +1,110 @@ +- Start Date: 2022-03-21 +- Reference Issues: https://github.com/withastro/rfcs/discussions/151 +- Implementation PR: + +# Summary + +- Change `Astro.request` to become a [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request) object. +- Move `Astro.request.params` to `Astro.params`. +- Move `Astro.request.canonicalURL` to `Astro.canonicalURL`. + +# Example + +```astro +--- +const cookie = Astro.request.headers.get('cookie'); +const loggedIn = parse(cookie).loggedIn; +--- + + + My Blog + + + + Hello, you are {!loggedIn && 'not'} logged in. + + +``` + +# Motivation + +In server-side rendering contexts you will want access to the HTTP request headers to do things such as check cookies. + +[Request](https://developer.mozilla.org/en-US/docs/Web/API/Request) is a standard API to represent an HTTP request that is used by: + +- Service workers +- Cloudflare Workers +- Deno + +As well as other meta-frameworks: + +- SvelteKit +- Remix + +The Request object allows access to headers through the `request.headers` Map-like object. Switching `Astro.request` to a Request will provide a familiar interface for this use-case. + +# Detailed design + +__Astro.request__ is currently an object with this interface: + +```typescript +interface AstroRequest { + /** get the current page URL */ + url: URL; + + /** get the current canonical URL */ + canonicalURL: URL; + + /** get page params (dynamic pages only) */ + params: Params; +} +``` + +This change will move `canonicalURL` and `params` up to the `Astro` object and make `request` a Request. + +```typescript +// Partial +interface Astro { + /** get the current canonical URL */ + canonicalURL: URL; + + /** get page params (dynamic pages only) */ + params: Params; + + /** gets the current request. */ + request: Request; + + /** More here... */ +} +``` + +## SSR vs. SSG + +When designing the initial version of Astro we intentionally omited things like query parameters and headers from the `Astro.*` APIs to prevent users from depending on things that would not work in production (since they are static pages). + +We should continue doing so, which just means: + +- In development mode when not using SSR, the `URL` is made to not contain query parameters. +- In development mode when not using SSR, the `Astro.request.headers` will exist, but not contain any headers (an empty object, essentially). + +With SSR mode enabled, these features will be present in development and production. + +# Drawbacks + +- In the current API `Astro.request.url` is a [URL](https://developer.mozilla.org/en-US/docs/Web/API/URL) object, but `Request.prototype.url` is a string. This is a breaking change. +- Moving things up to the top-level Astro arguably makes that object messier; maybe there is a better approach to cleaning it up. + +# Alternatives + +A few alternatives have been tried: + +- Keeping the existing object but adding a `headers` property that is a [Headers](https://developer.mozilla.org/en-US/docs/Web/API/Headers) (Map-like) object. +- Making Astro.request be a Request but then also adding the `params` and `canonicalURL` properties. + - Feel that doing it this way makes it harder to document and the code becomes slightly more complex. + +Either of these options would be *fine*, but if we were designing Astro from the start with SSR in mind we would probably have made it a Request, so doing so now before 1.0 seems like good timing. + +# Adoption strategy + +- This is a breaking change that would go out before (or during) 1.0. +- Docs would be updated to reflect that `Astro.request.url` is now a string. \ No newline at end of file From dd717d075d19aa25c2eb311a9901c9351dd77f8e Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Tue, 15 Mar 2022 00:13:37 -0500 Subject: [PATCH 016/300] initial draft --- proposals/0000-style-script-defaults.md | 88 +++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 proposals/0000-style-script-defaults.md diff --git a/proposals/0000-style-script-defaults.md b/proposals/0000-style-script-defaults.md new file mode 100644 index 00000000..410601c6 --- /dev/null +++ b/proposals/0000-style-script-defaults.md @@ -0,0 +1,88 @@ +- Start Date: 2022-03-15 +- Reference Issues: [RFC0008](https://github.com/withastro/rfcs/tree/main/proposals/0008-style-script-behavior.md), [#65](https://github.com/withastro/rfcs/discussions/65) +- Implementation PR: + +# Summary + +Astro is inconsist between ` + + + +

    Hello!

    + + + +``` + +# Motivation + +There have been [a few](https://github.com/withastro/rfcs/pull/12) [attempts](https://github.com/withastro/rfcs/discussions/65) to finalize this behavior and address these inconsistent APIs. This RFC aims to settle these questions prior to v1.0. + +- Users are generally in favor of how we currently do scoped, processed, and optimized styles by default. It is logical to extend this feature to scripts. +- Inlining scripts and styles can be useful for anything that must exist inlined into the page, like Google Analytics. Users currently cannot do this for styles. +- Users currently have to remember syntax for styles and scripts that is unlike other Astro directives. +- Encourages our "optimized by default" / "pit of success" mentality. Smart scripts should be easy to use by default. + +# Detailed design + +### `is:inline` + +A new directive, `is:inline`, will be introduced. This will opt both `` 4. `import './X.css'; // in Astro component frontmatter` -4. `import './X.css'; // in a React component` +5. `import './X.css'; // in a React component` It's difficult for us as maintainers to document, test and support so many different way to do things. Each method may support different features, or imply some different behavior. It's equally difficult for users to understand which is the "right" way to do things. @@ -24,29 +24,28 @@ We also need to make sure that our CSS support works with static builds. CSS can #### 1. ✅ `` or `` -- **Usecase:** Referencing a `/public` or external CSS URL. +- **Usecase:** Referencing a `/public` or external CSS URL. - **Usecase:** I need complete control over this `` tag, its attributes, and where it lives in the final output. - No longer supported to use with `Astro.resolve()`! (see other options below for how to reference a style that lives inside of `src/`) -- Not touched by Astro compiler, runtime, or Vite. Both the `` tag and the referenced CSS file will be left as-is in the final output. +- Not touched by Astro compiler, runtime, or Vite. Both the `` tag and the referenced CSS file will be left as-is in the final output. - Must point to a file in `public/` or an external CSS URL. This will not be bundled/optimized. - Must be nested inside of a `` element. -Note: See the [local directive RFC](https://github.com/withastro/rfcs/blob/build-performance-rfc/active-rfcs/0000-build-performance.md#local-directive) for another alternative on referencing a `src/` file that would be processed/built/bundled. - +Note: See the [local directive RFC](https://github.com/withastro/roadmap/blob/build-performance-rfc/active-rfcs/0000-build-performance.md#local-directive) for another alternative on referencing a `src/` file that would be processed/built/bundled. #### 2. ❌ ` // import xUrl from './X.css?url';` -- Note: Vite currently doesn't support a CSS `?url` import. @matthewp fixed a bug where this now returns the correct URL for `xUrl`, but it still won't bundle this CSS file as you would expect. This means that the resolved URL might not actually exist in the final build, if that file had been bundled. +- Note: Vite currently doesn't support a CSS `?url` import. @matthewp fixed a bug where this now returns the correct URL for `xUrl`, but it still won't bundle this CSS file as you would expect. This means that the resolved URL might not actually exist in the final build, if that file had been bundled. - It would be great to support this once Vite fixes their issues, but right now this is considered a Vite limitation. @matthewp to confirm for the final RFC. - reproduction: https://stackblitz.com/edit/vite-1jaulg?file=dist%2Fassets%2Findex.dd7a6b16.js - -#### 3. ✅ `` or `` +#### 3. ✅ `` or `` - **Usecase:** Importing a CSS file that you want optimized with the rest of your site. - Imported CSS is bundled with the rest of the Astro component-level CSS on the page. @@ -64,14 +63,13 @@ Note: See the [local directive RFC](https://github.com/withastro/rfcs/blob/build - **Usecase:** Supported as the only way to reference a CSS file inside of many framework components (React, Preact, Lit, etc). - - # Drawbacks - This proposal is restricting some current behavior that is undefined or very flexible. Some users who are using those behaviors today may need to update their code to follow the more explicit rules of the new proposal. diff --git a/proposals/0002-add-prettier-plugin-to-vscode-extension.md b/proposals/0002-add-prettier-plugin-to-vscode-extension.md index 1c0509b5..4aacecd0 100644 --- a/proposals/0002-add-prettier-plugin-to-vscode-extension.md +++ b/proposals/0002-add-prettier-plugin-to-vscode-extension.md @@ -2,7 +2,7 @@

    ⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️
    This RFC does not meet the requirements of the current RFC process.
    It was accepted before the current process was created. -
    Learn more about the RFC standards. +
    Learn more about the RFC standards.

    diff --git a/proposals/0003-add-test-tool-for-astro-components.md b/proposals/0003-add-test-tool-for-astro-components.md index 6a618a8c..6bfc6f88 100644 --- a/proposals/0003-add-test-tool-for-astro-components.md +++ b/proposals/0003-add-test-tool-for-astro-components.md @@ -2,7 +2,7 @@

    ⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️
    This RFC does not meet the requirements of the current RFC process.
    It was accepted before the current process was created. -
    Learn more about the RFC standards. +
    Learn more about the RFC standards.

    diff --git a/proposals/0004-replace-prism-with-shiki.md b/proposals/0004-replace-prism-with-shiki.md index f845a22b..03b08912 100644 --- a/proposals/0004-replace-prism-with-shiki.md +++ b/proposals/0004-replace-prism-with-shiki.md @@ -2,7 +2,7 @@

    ⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️
    This RFC does not meet the requirements of the current RFC process.
    It was accepted before the current process was created. -
    Learn more about the RFC standards. +
    Learn more about the RFC standards.

    @@ -53,13 +53,13 @@ Phase 2: ✅ - **requires:** a way to customize your markdown syntax highlighter of choice - Move Markdown code blocks to use `` instead of `` (https://github.com/stefanprobst/remark-shiki) - Move our recommendation in docs to use `` over ``, but keep references to ``. -- ~~Add warning when you use `` to use `` instead.~~ This would be jarring to users _intentionally_ sticking with Prism +- ~~Add warning when you use `` to use `` instead.~~ This would be jarring to users _intentionally_ sticking with Prism - **success metric to continue to phase 3:** docs site happy with the new Code component (usage in markdown) Phase 3: Soon - Remove `` entirely. -- Move it into a separate component, for anyone who still wants it. ex:`import Prism from '@astrojs/prism';` +- Move it into a separate component, for anyone who still wants it. ex:`import Prism from '@astrojs/prism';` **Detailed Design**: diff --git a/proposals/0005-support-exports-from-astro-components.md b/proposals/0005-support-exports-from-astro-components.md index 20ff3a44..9e1c7b90 100644 --- a/proposals/0005-support-exports-from-astro-components.md +++ b/proposals/0005-support-exports-from-astro-components.md @@ -2,7 +2,7 @@

    ⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️
    This RFC does not meet the requirements of the current RFC process.
    It was accepted before the current process was created. -
    Learn more about the RFC standards. +
    Learn more about the RFC standards.

    diff --git a/proposals/0006-support-non-html-files.md b/proposals/0006-support-non-html-files.md index 0e3b90a4..1bf02c1b 100644 --- a/proposals/0006-support-non-html-files.md +++ b/proposals/0006-support-non-html-files.md @@ -2,7 +2,7 @@

    ⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️
    This RFC does not meet the requirements of the current RFC process.
    It was accepted before the current process was created. -
    Learn more about the RFC standards. +
    Learn more about the RFC standards.

    @@ -18,7 +18,7 @@ Non-HTML dynamic files. **Background & Motivation**: -There are many reasons to want custom, dynamic files. One of the primary contenders are JSON feeds and other read-only APIs, because currently there is no clean way to make them. But there is also config files like `.htaccess`, `vercel.config.json` and others to for example [set redirects](https://github.com/snowpackjs/astro/issues/708)! +There are many reasons to want custom, dynamic files. One of the primary contenders are JSON feeds and other read-only APIs, because currently there is no clean way to make them. But there is also config files like `.htaccess`, `vercel.config.json` and others to for example [set redirects](https://github.com/snowpackjs/astro/issues/708)! Things like image optimization may also play a role (more so in #965, but there are some use cases here too), with something like a frequently changing logo (think google doodles) fetched and processed seperately from the page itself. @@ -44,14 +44,16 @@ Can be changed to a more astro-specific API, because headers in SSR... (they cou ```js export async function get() { - return "hello world" + return "hello world"; } ``` ```js export async function get() { - const image = await fetch("example2.com/images/dynamic-logo.php").then(x => x.buffer()) - return image //also does buffers + const image = await fetch("example2.com/images/dynamic-logo.php").then((x) => + x.buffer() + ); + return image; //also does buffers } ``` @@ -63,16 +65,17 @@ Discussed in #965, but those would rely on low level components (cough snowpack **Risks, downsides, and/or tradeoffs**: -* User-controlled php file (or just injection attacks in general) -* May be confusing to someone not familiar with sveltekit +- User-controlled php file (or just injection attacks in general) +- May be confusing to someone not familiar with sveltekit **Open Questions**: -* Would it theoretically be possible to use `getstaticpaths` to generate these files? Because `Astro.props` would be exposed and there should be no top-level code, and it should solve most of the usecases presented in #965 and discussed in the RFC call +- Would it theoretically be possible to use `getstaticpaths` to generate these files? Because `Astro.props` would be exposed and there should be no top-level code, and it should solve most of the usecases presented in #965 and discussed in the RFC call **Detailed Design**: > Go back in the git history, right click on the commit that removes the endpoint support, `revert commit` -> - jasikpark +> +> - jasikpark Some discussion about this in the last (as of writing) RFC meeting: https://youtu.be/hhsKS2et8Jk?t=1237 diff --git a/proposals/0007-finalize-head-body.md b/proposals/0007-finalize-head-body.md index 2513d1fe..bb0c8ff7 100644 --- a/proposals/0007-finalize-head-body.md +++ b/proposals/0007-finalize-head-body.md @@ -1,5 +1,5 @@ - Start Date: 11/30/2021 -- Reference Issues: https://github.com/withastro/rfcs/discussions/15 +- Reference Issues: https://github.com/withastro/roadmap/discussions/15 - Implementation PR: # Summary @@ -9,29 +9,30 @@ Finalize behavior around ``, ``, ``, and `` ele # Motivation in Astro v0.21, we have had user reports around `` not acting as expected, breaking some projects from v0.20: + - **Summary:** https://github.com/withastro/astro/issues/2128 - https://github.com/withastro/astro/issues/2046 - https://github.com/withastro/astro/issues/2022 - https://github.com/withastro/astro/issues/2132 - https://github.com/withastro/astro/issues/2151 -Some of these issues stem from undocumented or undefined behaviors in v0.20: was it allowed for `` be conditionally added in a nested expression? +Some of these issues stem from undocumented or undefined behaviors in v0.20: was it allowed for `` be conditionally added in a nested expression? Other issues stem from our attempt to provide implicit `` and `` support in pages and layouts. There was a quick workaround added for v0.21 where `src/pages` and `src/layouts` folder locations must be defined in config so that our parser knows how to parse them as full HTML documents and not smaller HTML fragments. This unblocked users but at the expense of breaking two larger design goals of the project: - - Added `src/layouts` as a new special folder location, when only `src/pages` was intended to be "special". - - Caused Astro to create different output for a component based on the file path of that component. + +- Added `src/layouts` as a new special folder location, when only `src/pages` was intended to be "special". +- Caused Astro to create different output for a component based on the file path of that component. ## Goals 1. Agree on and document the rules and behaviors of the ``, ``, ``, and `` tags. -2. Remove special compiler behavior based on where a file lives in your src directory. +2. Remove special compiler behavior based on where a file lives in your src directory. 3. Remove `src/layouts` as a "special" folder and any config around this. ## Non-Goals 1. **Out of scope:** Support `` injection, defined inside of individual components. Something like [``](https://svelte.dev/docs#svelte_head) for Astro is a commonly requested user feature, but can be tackled as an additional feature, leaving this RFC focused on clarifying our existing API. This RFC does not impact that feature request. - # Detailed design ## Template Changes @@ -55,12 +56,13 @@ Other issues stem from our attempt to provide implicit `` and `` sup Losing `"as": "document"` parse mode will remove some special parser handling, like getting an implicit `` to wrap a standalone `` element. This is intentional to bring us more inline with the RFC, where we respect the HTML authored by the developer as the source of truth and leave mostly as-is in the final output. -In this design it is more "on you" to write valid HTML. This comes from the reality that an imported component can contain unknown HTML, so the compiler can't implicitly assume anything about what is or is not included included the final parent component template. See https://github.com/withastro/astro/issues/2022 for examples of when this breaks down today. We can help with some static linting, and runtime warnings if the final HTML output is invalid. However, this RFC acknowledges the reality that already exists in v0.21 that imported components break any assumptions and help that we previously attempted to provide. +In this design it is more "on you" to write valid HTML. This comes from the reality that an imported component can contain unknown HTML, so the compiler can't implicitly assume anything about what is or is not included included the final parent component template. See https://github.com/withastro/astro/issues/2022 for examples of when this breaks down today. We can help with some static linting, and runtime warnings if the final HTML output is invalid. However, this RFC acknowledges the reality that already exists in v0.21 that imported components break any assumptions and help that we previously attempted to provide. ## Head Injection Changes + - runtime: will remove current post-build head injection, which involves a fragile `'</head>'` string find-and-replace. - compiler: will add a new `head: string` (or: `injectHead(): string`) property to the compiler transform options, which will inject the given HTML string into the bottom of a `<head>` element, if one is return by the compiler. -- runtime: will provide this value head injection value to the compiler, and throw an exception if not used/called exactly once during a page render. +- runtime: will provide this value head injection value to the compiler, and throw an exception if not used/called exactly once during a page render. The Astro runtime will use this new property to inject all CSS styles used by components on the page into the final HTML document. These may be individual file `<link>` or `<style>` tags during development, or a few bundled CSS files in your final production build. @@ -84,11 +86,9 @@ Note: This must handle a `<slot>` containing a `<head>` and/or `<head slot="head <link slot="head" ... /> ``` - - # Drawbacks -- A `<head>` element is always required in your final output. This is because Astro must perform `<head>` injection and parsing the HTML document after generation is considered too expensive for production use (ex: building 10,000+ page websites). +- A `<head>` element is always required in your final output. This is because Astro must perform `<head>` injection and parsing the HTML document after generation is considered too expensive for production use (ex: building 10,000+ page websites). - The `<head>` component itself must be defined inside of an Astro component. You could not, for example, define your `<head>` _inside_ of a React component. This is because Astro cannot inject HTML safely into unknown 3rd-party components. - `<html>` and `<body>` elements are optional in the HTML spec, and therefor optional inside of Astro as well. This ability to output HTML without these two tags may be considered an advantage for spec compliance. However, it means we need to be more diligent with testing this kind of output across our codebase and dependencies. For example, we would need to confirm that Vite does not have trouble with HTML documents that do not include a `<body>`. @@ -104,7 +104,7 @@ Removing `document` mode may break some users relying on certain side-effects an To mitigate this, this RFC proposes the following release plan to remove `as: document` mode from the compiler: -1. In a well-tested PR to Astro core, remove any usage of `as: 'document'` when calling the compiler. +1. In a well-tested PR to Astro core, remove any usage of `as: 'document'` when calling the compiler. 2. Add some checks and warnings to help users migrate. For example, warn in `renderPage()` if anything other than a single `</head>` were found. 3. Release this in a new minor release. This would be the only thing to go out in this release. 4. Explicitly mention the breakage potential in the release notes, and on Discord. @@ -112,9 +112,6 @@ To mitigate this, this RFC proposes the following release plan to remove `as: do Once settled, we would remove the now-unnecessary `layouts` config and remove `as: "document"` support from the compiler in followup minor releases. - - - # Unresolved questions - None yet. diff --git a/proposals/0008-style-script-behavior.md b/proposals/0008-style-script-behavior.md index a555c2cd..e67efc07 100644 --- a/proposals/0008-style-script-behavior.md +++ b/proposals/0008-style-script-behavior.md @@ -1,7 +1,6 @@ - Start Date: 12/09/2021 -- Reference Issues: https://github.com/withastro/astro/issues/1077, https://github.com/withastro/rfcs/discussions/1 -- Implementation PR: - +- Reference Issues: https://github.com/withastro/astro/issues/1077, https://github.com/withastro/roadmap/discussions/1 +- Implementation PR: # Summary @@ -9,12 +8,10 @@ Finalize the component styling & scripting behavior for v1.0. # Important Note for Reviewers -This RFC proposes very little new behavior, and mainly exists to document current behavior and make sure we have a clear, shared understanding about how Astro should work as we head into a v1.0 release. +This RFC proposes very little new behavior, and mainly exists to document current behavior and make sure we have a clear, shared understanding about how Astro should work as we head into a v1.0 release. When reviewing, feel free to ask questions and give feedback on current behavior as well as new behavior. However, be aware that changes to current behavior may be out of scope of this RFC and may require their own RFC seperate from this one. - - # Example - UI Component ```astro @@ -66,7 +63,6 @@ When reviewing, feel free to ask questions and give feedback on current behavior - Non-trivial changes to current behavior. See "Important Note for Reviewers" above. - # Detailed design ## `<style>` @@ -100,7 +96,6 @@ Even though `<style define:vars={...}>` is all about adding dynamic content, it Note that this will continue to have issues with some use-cases. For example, if a component is used twice on the page, the dynamic value may change across different renders but would impact all components on the page. You can see an example of this here, where the last rendered component wins: https://stackblitz.com/edit/github-eww5sz?file=src%2Fcomponents%2FTour.astro&on=stackblitz - ## `<style global>` ```astro @@ -139,7 +134,6 @@ Note that this will continue to have issues with some use-cases. For example, if ## `<script hoist>` - ```astro <!-- INPUT: --> <script hoist> @@ -151,7 +145,6 @@ Note that this will continue to have issues with some use-cases. For example, if <!-- JS is bundled with the rest of your page JavaScript. --> ``` - ### Current Behavior - Script content is processed, ex: TypeScript could potentially be supported. @@ -162,11 +155,10 @@ Note that this will continue to have issues with some use-cases. For example, if ### New RFC Behavior -- `<script hoist>` contents must be static, therefore `define:vars` is not supported. Because the script is bundled ahead-of-time, dynamic values won't exist. This is currently easy to break in v0.21, so this RFC proposes removing the support entirely for `<script hoist>` (still available for `<script>` and `<style>`). +- `<script hoist>` contents must be static, therefore `define:vars` is not supported. Because the script is bundled ahead-of-time, dynamic values won't exist. This is currently easy to break in v0.21, so this RFC proposes removing the support entirely for `<script hoist>` (still available for `<script>` and `<style>`). - Cannot be nested within a template expression or conditional. Bundled scripts must be scanned by the compiler. This is currently easy to break in v0.21 with something like `{alwaysFalse && <script hoist>...` so this RFC moves to remove support for this. This change shouldn't impact many users. - Can only exist top-level in the template (best for components) or nested directly inside of `<head>` or `<body>` (best for pages/layouts). This is to guarentee that scanning and reading hoisted script content is straightforward and bug-free. This shouldn't affect many users. - # Drawbacks - None yet. @@ -177,7 +169,6 @@ This RFC proposes very little new behavior, so this should have little impact on For the new restrictions that the RFC does propose, we will give clear warnings or errors for code that does not match these new restrictions. All new restrictions are meant to explicitly prevent you from writing buggy code, so impact of rolling this out should be overall positive. - # Unresolved questions - None yet. diff --git a/proposals/0009-build-performance.md b/proposals/0009-build-performance.md index 6103e24e..395fd9e0 100644 --- a/proposals/0009-build-performance.md +++ b/proposals/0009-build-performance.md @@ -1,6 +1,6 @@ - Start Date: 2021-12-13 - Reference Issues: - - Previous: https://github.com/withastro/rfcs/pull/44 + - Previous: https://github.com/withastro/roadmap/pull/44 - Implementation PR: https://github.com/withastro/astro/pull/2168 # Summary @@ -53,20 +53,20 @@ const { animal } = Astro.props; <img src={await import(`../images/${animal}.png`)} /> ``` -The above will result in *all* of the images in `../images/` getting built, but only the one you select will be used. +The above will result in _all_ of the images in `../images/` getting built, but only the one you select will be used. # Motivation - Astro is currently only able to build sites with a few hundred pages. Since the introduction of `getStaticPaths` we have known that developers would want to build site into the thousands or tens of thousands of pages. -- Astro's build process relies on scanning the rendered HTML and then *update* the HTML as well, to replace assets with the hashed paths in the build. - - Because of the above, performance has actually regressed in __0.21__, even though it was never the best even before. -- In order to support __SSR__ in the future we have to move away from page-scanning as the way to find and build assets, since SSR apps by their nature *cannot* be rendered ahead of time. +- Astro's build process relies on scanning the rendered HTML and then _update_ the HTML as well, to replace assets with the hashed paths in the build. + - Because of the above, performance has actually regressed in **0.21**, even though it was never the best even before. +- In order to support **SSR** in the future we have to move away from page-scanning as the way to find and build assets, since SSR apps by their nature _cannot_ be rendered ahead of time. # Detailed design ## Enforced static use of client directives -The client directives such as `client:load`, `client:idle`, etc will need to be defined in the hydrated component where they are used, and not rendered dynamically. For example the following is __not allowed__: +The client directives such as `client:load`, `client:idle`, etc will need to be defined in the hydrated component where they are used, and not rendered dynamically. For example the following is **not allowed**: ```astro --- @@ -79,21 +79,23 @@ const attrs = { <Clock {...attrs} /> ``` -We need to know that the site depends on the `client:idle` directive, so that we can *build* the client-side JavaScript needed. +We need to know that the site depends on the `client:idle` directive, so that we can _build_ the client-side JavaScript needed. The implementation will be: 1. In the compiler, include the used directive as part of the exported metadata about the component. 2. In the compiler, mark the component as having statically included a directive. - - How to mark is up to the implementer, but we have other metadata attached to hydrated component usage already, and it would make sense to follow this same method. -3. When rendering, if a component contains a client directive, make sure the directive is matched by the marking in __(2)__. + +- How to mark is up to the implementer, but we have other metadata attached to hydrated component usage already, and it would make sense to follow this same method. + +3. When rendering, if a component contains a client directive, make sure the directive is matched by the marking in **(2)**. 4. If the marking is not there, we know that the directive must have been added statically. Throw an `Error` message for this, letting the user know that the directive must be added statically. ## Deprecate Astro.resolve To deprecate `Astro.resolve` we should: -- Add a warning to the `Astro.resolve` method that says that it is deprecated and links to documentation on alternatives such as the [local: proposal](https://github.com/withastro/rfcs/pull/59) and `import.meta.glob`. +- Add a warning to the `Astro.resolve` method that says that it is deprecated and links to documentation on alternatives such as the [local: proposal](https://github.com/withastro/roadmap/pull/59) and `import.meta.glob`. - After one major version of Astro, replace the warning with an error, preventing its usage in dev or the build. # Drawbacks @@ -106,12 +108,12 @@ No other alternatives have been designed at this time. I do not believe it will # Adoption strategy -- Add the behaviors described in this PR behind a flag, `--experimental-static-build`. A PR that brings partial support for this [already exists](https://github.com/withastro/astro/pull/2168). -- Promote the usage of the [local: directive](https://github.com/withastro/rfcs/pull/59), if that RFC passes, over `Astro.resolve` in documentation and on Discord. +- Add the behaviors described in this PR behind a flag, `--experimental-static-build`. A PR that brings partial support for this [already exists](https://github.com/withastro/astro/pull/2168). +- Promote the usage of the [local: directive](https://github.com/withastro/roadmap/pull/59), if that RFC passes, over `Astro.resolve` in documentation and on Discord. - Add a deprecation warning to `Astro.resolve` that exists for at least 1 major version. - Once the static build becomes the default, leave in the legacy flag behind a flag (such as `--legacy-build`) for 1 major version. - Remove `Astro.resolve` and fully enforce static directives when this feature becomes unflagged. # Unresolved questions -- We don't have data on the performance difference this change will make. Conceptually we believe it will make a big difference, and will get a better indication once the flagged version has been merged in. \ No newline at end of file +- We don't have data on the performance difference this change will make. Conceptually we believe it will make a big difference, and will get a better indication once the flagged version has been merged in. diff --git a/proposals/0010-recursive-components.md b/proposals/0010-recursive-components.md index c99f9578..c1c22eaf 100644 --- a/proposals/0010-recursive-components.md +++ b/proposals/0010-recursive-components.md @@ -1,8 +1,8 @@ -- Start Date: Date: 2021-12-15 -- Reference Issues: https://github.com/withastro/rfcs/discussions/45 -- Implementation PR: - - https://github.com/sgruenholz2/rfcs/blob/recursive-components-patch-1/active-rfcs/0000-recursive-components.md - - https://github.com/withastro/compiler/pull/270 +- Start Date: Date: 2021-12-15 +- Reference Issues: https://github.com/withastro/roadmap/discussions/45 +- Implementation PR: + - https://github.com/sgruenholz2/rfcs/blob/recursive-components-patch-1/active-rfcs/0000-recursive-components.md + - https://github.com/withastro/compiler/pull/270 # Summary @@ -84,23 +84,22 @@ When the tree structure is UNKNOWN, and you want to support N levels deep, using Component/function is the ONLY approach that will work. This is often the case when fetching data from an API. -Handling this use case lets .astro components do things that other component +Handling this use case lets .astro components do things that other component frameworks can do, making it a 1st class component framework in its own right. -The Single File per Component (SFC) pattern that Astro uses is simple, but inflexible. -In other frameworks like React (and presumable Vue and SolidJs) you can create multiple -components within a single file. -This allows you to can create both a function/component to render an `<Item />` and another -function/component to render `<ItemChildren />` have them reference each other, and then expose -either/both as exports. You can't do this with SFC. And, if you try to create these as -2 separate files, you get circular dependencies. The only way for SFC to allow for recursion +The Single File per Component (SFC) pattern that Astro uses is simple, but inflexible. +In other frameworks like React (and presumable Vue and SolidJs) you can create multiple +components within a single file. +This allows you to can create both a function/component to render an `<Item />` and another +function/component to render `<ItemChildren />` have them reference each other, and then expose +either/both as exports. You can't do this with SFC. And, if you try to create these as +2 separate files, you get circular dependencies. The only way for SFC to allow for recursion is by allowing a component access to reference it's own render function. Svelte, which also uses SFC, has already encountered and solved for this issue by exposing the [svelte:self](https://svelte.dev/docs#svelte_self) attribute as part of their API. - # Detailed design The `Astro.self` property exposes the render function of the component. @@ -150,8 +149,8 @@ This is just one more `Astro` property that becomes available there. - No backwards compatibility concerns - No potential for breaking changes - Worth noting that recursive rendering can be already achieved with any number of other -platforms: React, Vue, Svelte, etc., so we can approach this at our leisure with -the short term answer to this being: "Do that in your own platform for now". + platforms: React, Vue, Svelte, etc., so we can approach this at our leisure with + the short term answer to this being: "Do that in your own platform for now". - Need to update Documentation # Unresolved questions diff --git a/proposals/0011-relative-url-scheme.md b/proposals/0011-relative-url-scheme.md index d4039df1..5c375e2b 100644 --- a/proposals/0011-relative-url-scheme.md +++ b/proposals/0011-relative-url-scheme.md @@ -1,15 +1,11 @@ - Start Date: 2021-12-20 -- Reference Issues: [#46](https://github.com/withastro/rfcs/pull/46) +- Reference Issues: [#46](https://github.com/withastro/roadmap/pull/46) - Implementation PR: <!-- leave this empty --> - - # Summary Astro should support source-relative URL resolution in the HTML block of Astro files. - - # Example To resolve the source-relative URL of an image, use a `local:` prefixed attribute. @@ -18,8 +14,6 @@ To resolve the source-relative URL of an image, use a `local:` prefixed attribut <img local:src="kitten.avif" alt="Kitten" /> ``` - - # Motivation In `.astro` files, it’s not always easy to know what a relative URL is relative to. @@ -35,8 +29,6 @@ This ambiguity raises a need for: - An intuitive way to author URLs relative to a source file. - An intuitive way to author URLs within the HTML of an `.astro` file. - - # Detailed design A `local` prefix before an attribute instructs Astro to resolve the attribute value as a URL relative to the source file. @@ -51,20 +43,14 @@ The attribute prefix treats the entire attribute value as the URL. This example <img src={await import('./kitten.avif?url')} alt="Kitten" /> ``` - - # Drawbacks This does not fully replicate `Astro.resolve`, and it does not support attribute values whose value is only partially a URL. Attribute values with partial sources, like `srcset`, `style`, and and `data` attributes, are not handled. They could be handled in a separate, future RFC. While outside the scope of this proposal, a suggestion is a `local:` scheme. - - # Alternatives - - ### Use a `local:` [scheme](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_is_a_URL#scheme) ```astro @@ -73,8 +59,6 @@ Attribute values with partial sources, like `srcset`, `style`, and and `data` at This would support current uses, and allow additional references in `style`, `srcset`, or `data-` attributes, as well as support within wrappers like `url()`. - - ### Limit support to `import` statements and do not support source-relative URLs in the HTML of `.astro` files. ```astro @@ -91,8 +75,6 @@ This is the current functionality and would require no change. - Requires authors to write more JS and less HTML. - Requires authors to name more things. <sup>[1](https://hilton.org.uk/blog/why-naming-things-is-hard)</sup> - - # Adoption strategy The `local` attribute is intended to improve and replace certain usages of `Astro.resolve`. Its addition does not require the removal of `Astro.resolve`. diff --git a/proposals/0013-set-html.md b/proposals/0013-set-html.md index d6b8e97c..b57d0234 100644 --- a/proposals/0013-set-html.md +++ b/proposals/0013-set-html.md @@ -1,7 +1,7 @@ - Start Date: 2021-12-15 - Reference Issues: - - [v1.0 API Unification](https://github.com/withastro/rfcs/discussions/1) - - [Stale RFC](https://github.com/withastro/astro/issues/1827) + - [v1.0 API Unification](https://github.com/withastro/roadmap/discussions/1) + - [Stale RFC](https://github.com/withastro/astro/issues/1827) - Implementation PR: # Summary @@ -49,34 +49,35 @@ const content = await fetch('https://untrusted.com/remote-content.html').then(re These APIs can be considered "escape hatches" for Astro's `{expression}` syntax. Should Astro ever adopt an escaped-by-default approach for expressions (which is on the roadmap for v1.0), these escape hatches will be particularly important. -In the meantime, these APIs still provide value! +In the meantime, these APIs still provide value! 1. Having an explicit prop-based API matches framework user expectations. - - React has [`dangerouslySetInnerHTML`](https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml), which is intentionally verbose to discourage usage. The docs explicitly warn about it being a vector for XSS attacks. - - Preact mirrors React's [`dangerouslySetInnerHTML`](https://github.com/preactjs/preact/issues/29), which is intentionally verbose to discourage usage. The docs do not explicitly mention this feature, except in React compatability guides. - - Vue has [`v-html`](https://v3.vuejs.org/api/directives.html#v-html). The docs explicitly warn about it being a vector for XSS attacks. - - Svelte has [`{@html ...}`](https://svelte.dev/docs#template-syntax-html). The docs explicitly warn about it being a vector for XSS attacks. - - Solid has [`innerHTML` and `textContent`](https://www.solidjs.com/docs/latest/api#innerhtml%2Ftextcontent). The docs explicitly warn about it being a vector for XSS attacks. +- React has [`dangerouslySetInnerHTML`](https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml), which is intentionally verbose to discourage usage. The docs explicitly warn about it being a vector for XSS attacks. +- Preact mirrors React's [`dangerouslySetInnerHTML`](https://github.com/preactjs/preact/issues/29), which is intentionally verbose to discourage usage. The docs do not explicitly mention this feature, except in React compatability guides. +- Vue has [`v-html`](https://v3.vuejs.org/api/directives.html#v-html). The docs explicitly warn about it being a vector for XSS attacks. +- Svelte has [`{@html ...}`](https://svelte.dev/docs#template-syntax-html). The docs explicitly warn about it being a vector for XSS attacks. +- Solid has [`innerHTML` and `textContent`](https://www.solidjs.com/docs/latest/api#innerhtml%2Ftextcontent). The docs explicitly warn about it being a vector for XSS attacks. 2. This API is more explicit, which makes expected behavior clear. - ```astro - <!-- Behavior is intuitive. Dev understands the expected behavior. --> - <article set:html={content} /> - <article set:text={content} /> +```astro +<!-- Behavior is intuitive. Dev understands the expected behavior. --> +<article set:html={content} /> +<article set:text={content} /> + +<!-- Behavior is unintuitive. Dev expectations may vary. --> +<article>{content}</article> +``` - <!-- Behavior is unintuitive. Dev expectations may vary. --> - <article>{content}</article> - ``` 3. This API _could_ allow users to bypass Astro's `script` and `style` restrictions. This can be desirable in some advanced (but potentially dangerous) use cases. - ```astro - <!-- Probably fine! --> - <style set:html={autoGeneratedCss} /> - <!-- ⚠️ DANGER! --> - <script set:html={autoGeneratedJs} /> - ``` + ```astro + <!-- Probably fine! --> + <style set:html={autoGeneratedCss} /> + <!-- ⚠️ DANGER! --> + <script set:html={autoGeneratedJs} /> + ``` # Detailed design @@ -87,24 +88,27 @@ The Astro compiler will need to be updated to detect `set:html` and `set:text` u Currently, built-in HTML elements are expected to be static—they are rendered as strings for performance reasons. In supporting these special props, Astro will need specific handling for elements with these props. The logic for this kind of rendering already exists for `Components` and `custom-elements`. > **Given the following `.astro` code...** -> +> > ```astro > <article set:html={content} /> > ``` -> +> > **Our compiler generates something like this (simplified)** +> > ```js -> html`<article${addAttribute('set:html', content)} />` +> html`<article${addAttribute("set:html", content)} />`; > ``` -> +> > **If this RFC is accepted, our compiler will need to generate something more like this** +> > ```js -> html`${renderElement('article', { 'set:html': content })}` +> html`${renderElement("article", { "set:html": content })}`; > ``` -> +> > **Additionally, `set:text` will need special consideration to be escaped at runtime.** +> > ```js -> html`${renderElement('article', { 'set:text': escapeHTML(content) })}` +> html`${renderElement("article", { "set:text": escapeHTML(content) })}`; > ``` Rather than refactoring all element rendering to `renderElement` function form, our compiler should _only_ handle elements with statically-analyzable `set:html` or `set:text` this way. This balances the performance benefits of storing _most_ HTML content as strings with the dynamic needs of this particular feature. The compiler complexity overhead will be small since there is no _entirely new_ behavior for the printer here. @@ -115,13 +119,13 @@ Injected content will be passed to the `Parent` node as if it were the `default` ### Scenario A: `<element set:html={content} />` and `<element set:text={content} />` -✅ Supported. +✅ Supported. Compiler will inject `content` (for `set:html`) or `escape(content)` (for `set:text`) as the `default` slot of `element`. Renderers will not recieve the `set:html` or `set:text` props—these directives are compiler instructions only. Slots inside of `content` will be output literally (if possible, should warn in dev). ### Scenario B: `<Component set:html={content} />` and `<Component set:text={content} />` -✅ Supported. +✅ Supported. Compiler will inject `content` (for `set:html`) or `escape(content)` (for `set:text`) as the `default` slot of `Component`. Renderers will not recieve the `set:html` or `set:text` props—these directives are compiler instructions only. Slots inside of `content` will be output literally (if possible, should warn in dev). @@ -135,7 +139,7 @@ Compiler will inject `content` (for `set:html`) or `escape(content)` (for `set:t ### Scenario D: `<element set:html={content} set:text={content} />` -⛔️ Not Supported. +⛔️ Not Supported. Compiler will warn about duplicate `set:` directives. @@ -151,9 +155,9 @@ Usage with self-closing tags is supported. Usage with empty start/end tag pairs is supported. -### Scenario G: `<element set:html={content}>text</element>` or `<element set:html={content}><child /></element>` or `<element set:html={content}>{expression}</element>` +### Scenario G: `<element set:html={content}>text</element>` or `<element set:html={content}><child /></element>` or `<element set:html={content}>{expression}</element>` -⛔️ Not Supported. +⛔️ Not Supported. Compiler will warn about duplicate `set:` directive when any text, element, or expression children are passed. @@ -161,13 +165,13 @@ Compiler will warn about duplicate `set:` directive when any text, element, or e **🤔 Open Question!** -Usage with start/end tag pairs containing only whitespace _should_ or _should not be_ supported? +Usage with start/end tag pairs containing only whitespace _should_ or _should not be_ supported? > Whitespace is technically a `text` node, but there are other cases where HTML has special handling for whitespace-only `text` nodes. ### Scenario I: `<element {...{ 'set:html': content } />` (dynamic directive usage) -⛔️ Not Supported. +⛔️ Not Supported. Compiler will not know to compile this element using the dynamic format. We should warn if these directives are detected at runtime. @@ -178,6 +182,7 @@ Compiler will not know to compile this element using the dynamic format. We shou Per **Motivation #3** above, this behavior could allow a user to bypass Astro's `script` and `style` restrictions. The caveat is that the injected contents of `script` and `style` could no longer be scoped, transformed, optimized, or bundled by Astro—the content values would not exist until runtime. Should we support this? **Input** + ```astro --- const css = `div { color: red; }`; @@ -188,6 +193,7 @@ const js = `console.log('uh oh');`; ``` **Output** + ```astro <!-- generates literal output, not scoped or optimized --> <style>div { color: red; }</style> @@ -196,12 +202,14 @@ const js = `console.log('uh oh');`; ``` **Potential Warnings** + ```astro <!-- Will not be scoped, do we require `global`? --> <style set:html={css} /> ``` **Potential Errors** + ```astro <!-- Cannot use `lang` preprocessor with `set:html` --> <style lang="postcss" set:html={css} /> @@ -248,5 +256,4 @@ However, the Astro documentation should push this as _the blessed approach_ for # Unresolved questions -See [**Scenario H**](https://github.com/withastro/rfcs/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-h-element-sethtmlcontent---element) and [**Scenario J**](https://github.com/withastro/rfcs/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-j-script-or-style-usage) in **Detailed Design**. - +See [**Scenario H**](https://github.com/withastro/roadmap/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-h-element-sethtmlcontent---element) and [**Scenario J**](https://github.com/withastro/roadmap/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-j-script-or-style-usage) in **Detailed Design**. diff --git a/proposals/0014-support-draft-markdown-posts.md b/proposals/0014-support-draft-markdown-posts.md index 3963ef8a..cf78a9dd 100644 --- a/proposals/0014-support-draft-markdown-posts.md +++ b/proposals/0014-support-draft-markdown-posts.md @@ -2,7 +2,7 @@ <p align="center"><strong>⚠️⚠️⚠️ Legacy RFC Disclaimer ⚠️⚠️⚠️ <br />This RFC does not meet the requirements of the current RFC process. <br />It was accepted before the current process was created. -<br /><a href="https://github.com/withastro/rfcs#readme">Learn more about the RFC standards.</a> +<br /><a href="https://github.com/withastro/roadmap#readme">Learn more about the RFC standards.</a> </strong></p> <!-- LEGACY RFC --> @@ -18,9 +18,9 @@ Add `draft` support for Markdown posts. **What is Missing from Astro Today?** -- `draft` as a top-level primitive for Markdown posts/files +- `draft` as a top-level primitive for Markdown posts/files - `buildOptions.drafts` as an option to ignore posts/files with `draft: true` -- borrowed from: https://jekyllrb.com/docs/configuration/options/ +- borrowed from: https://jekyllrb.com/docs/configuration/options/ **Proposed Solution** diff --git a/proposals/0016-style-script-defaults.md b/proposals/0016-style-script-defaults.md index 53abecf5..d477f538 100644 --- a/proposals/0016-style-script-defaults.md +++ b/proposals/0016-style-script-defaults.md @@ -1,12 +1,12 @@ - Start Date: 2022-03-15 -- Reference Issues: [RFC0008](https://github.com/withastro/rfcs/tree/main/proposals/0008-style-script-behavior.md), [#65](https://github.com/withastro/rfcs/discussions/65) +- Reference Issues: [RFC0008](https://github.com/withastro/roadmap/tree/main/proposals/0008-style-script-behavior.md), [#65](https://github.com/withastro/roadmap/discussions/65) - Implementation PR: <!-- leave empty --> # Summary -Astro is inconsist between `<style>` and `<script>` default behavior. Currently, `<style>` has opt-out processing, but `<script>` does not. This RFC aims to settle on a good, consistent default for both `<style>` and `<script>`. +Astro is inconsist between `<style>` and `<script>` default behavior. Currently, `<style>` has opt-out processing, but `<script>` does not. This RFC aims to settle on a good, consistent default for both `<style>` and `<script>`. -**This RFC is intended to supercede [RFC0008](https://github.com/withastro/rfcs/tree/main/proposals/0008-style-script-behavior.md).** +**This RFC is intended to supercede [RFC0008](https://github.com/withastro/roadmap/tree/main/proposals/0008-style-script-behavior.md).** - New `is:inline` directive to avoid bundling `style` or `script` - `<style>` => remains scoped, bundled by default. `is:scoped` directive supported for consistency @@ -39,7 +39,7 @@ span { color: green; } # Motivation -There have been [a few](https://github.com/withastro/rfcs/pull/12) [attempts](https://github.com/withastro/rfcs/discussions/65) to finalize this behavior and address these inconsistent APIs. This RFC aims to settle these questions prior to v1.0. +There have been [a few](https://github.com/withastro/roadmap/pull/12) [attempts](https://github.com/withastro/roadmap/discussions/65) to finalize this behavior and address these inconsistent APIs. This RFC aims to settle these questions prior to v1.0. - Users are generally in favor of how we currently do scoped, processed, and optimized styles by default. It is logical to extend this feature to scripts. - Inlining scripts and styles can be useful for anything that must exist inlined into the page, like Google Analytics. Users currently cannot do this for styles. @@ -50,7 +50,7 @@ There have been [a few](https://github.com/withastro/rfcs/pull/12) [attempts](ht ### `is:inline` -A new directive, `is:inline`, will be introduced. This will opt both `<style>` and `<script>` tags out of _bundling_ behavior. +A new directive, `is:inline`, will be introduced. This will opt both `<style>` and `<script>` tags out of _bundling_ behavior. The `is:inline` directive means that `style` and `script` tags: @@ -83,15 +83,16 @@ If any script tag has an attribute (ex: `type="module"`, `type="text/partytown"` A few other attempts have been made at finalizing this API: -- https://github.com/withastro/rfcs/discussions/65 -- https://github.com/withastro/rfcs/blob/style-script-rfc/active-rfcs/0000-style-script-behavior.md#script -- https://github.com/withastro/rfcs/blob/6015b4da8c95258b2136d61d6a6290c7ca989f5a/active-rfcs/0000-component-tag.md +- https://github.com/withastro/roadmap/discussions/65 +- https://github.com/withastro/roadmap/blob/style-script-rfc/active-rfcs/0000-style-script-behavior.md#script +- https://github.com/withastro/roadmap/blob/6015b4da8c95258b2136d61d6a6290c7ca989f5a/active-rfcs/0000-component-tag.md There is a clear need to address this inconsistency but we do not have a clear path forward yet. ## `Script` and `Style` components One potential alternative would be to introduce magical `<Script>` and `<Style>` components. This was considered, but ultimately rejected because... + - Whether these components are available on `globalThis` or must be imported could lead to confusion - These take Astro further away from similar frameworks like Vue and Svelte, which seem to have no problem with "magic by default" `script` and `style`. - These are not actual runtime components that can be renamed or inspected, but rather instructions for the Astro compiler diff --git a/proposals/0017-markdown-content-redesign.md b/proposals/0017-markdown-content-redesign.md index 9804811e..c121a440 100644 --- a/proposals/0017-markdown-content-redesign.md +++ b/proposals/0017-markdown-content-redesign.md @@ -1,5 +1,5 @@ - Start Date: 03-10-2022 -- Reference Issues: https://github.com/withastro/rfcs/discussions/5, https://github.com/withastro/rfcs/discussions/118 +- Reference Issues: https://github.com/withastro/roadmap/discussions/5, https://github.com/withastro/roadmap/discussions/118 - Implementation PR: <!-- leave empty --> # Summary @@ -35,7 +35,6 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category <firstPost.Content /> ``` - # Motivation ## Background: Performance Issues @@ -47,7 +46,7 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category ## Background: Usability Issues - `Astro.fetchContent()` currently only supports Markdown files, which is confusing to some users. -- ESM `import` currently supports most file types *except* Markdown, which can work with `import` but you'll get back a different, undocumented object API than if you'd used `fetchContent()`. +- ESM `import` currently supports most file types _except_ Markdown, which can work with `import` but you'll get back a different, undocumented object API than if you'd used `fetchContent()`. - `import.meta.glob()` is another API available to users, but its unclear how `Astro.fetchContent()` and `import.meta.glob` are related. - Users still have difficulty using Markdown files in their projects due to legacy API decisions that we've been trying to move away from (ex: `.content -> `.Content`) @@ -59,8 +58,6 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category - If possible, open up this "user-friendly API" to more file types, not just Markdown - Solve the mentioned performance issues. - - # Detailed design ## `Astro.glob()` @@ -76,7 +73,9 @@ const markdownFilesFiltered = markdownFilesArr.filter(post => post.data.category ```ts // 2. -Astro.glob = function<T=any>(importMetaGlobResult: Record<string, any>): Promise<T[]> { +Astro.glob = function <T = any>( + importMetaGlobResult: Record<string, any> +): Promise<T[]> { // Convert the `import.meta.globEager` result into an array. let allEntries = [...Object.values(importMetaGlobResult)]; // Report an error if no objects are returned. @@ -84,11 +83,11 @@ Astro.glob = function<T=any>(importMetaGlobResult: Record<string, any>): Promise if (allEntries.length === 0) { throw new Error(`Astro.glob() - no matches found.`); } - // NOTE: This API was designed to be async, however we convert its argument to a resolve `globEager` - // object at compile time. We fake asynchrony here so that this API can still become async in the + // NOTE: This API was designed to be async, however we convert its argument to a resolve `globEager` + // object at compile time. We fake asynchrony here so that this API can still become async in the // future if we ever move off of `import.meta.globEager()`. This should not impact users too much. return Promise.resolve(allEntries); -} +}; ``` - This replaces `Astro.fetchContent()` as the new preferred API @@ -98,59 +97,61 @@ Astro.glob = function<T=any>(importMetaGlobResult: Record<string, any>): Promise ## Deferred Import Implementation -- This RFC seeks to refactor how Markdown is loaded internally AND update the Markdown API that users interact with. +- This RFC seeks to refactor how Markdown is loaded internally AND update the Markdown API that users interact with. - The API updates are captured in `1.` below, while the refactoring is captured in `2.` and `3.` - The logic that manages all of this lives in the internal `'astro:markdown'` Vite plugin. - All Markdown files (ex: `src/posts/foo.md`) are resolved and loaded by this plugin. 1. `src/posts/foo.md?import` - 1. loads from file system - 2. parses frontmatter from the file into a JS object - 3. returns a JS module with the following JS: - ```js - ` - // Static: - export const frontmatter = ${JSON.stringify(frontmatter)}; - export const file = ${JSON.stringify(id)}; - export const url = ${JSON.stringify(url || undefined)}; - - // Deferred: - export default async function load(...args) { - return (await import(${JSON.stringify(fileId + '?content')})); - }; - export function Content(...args: any) { - return load().then((m: any) => m.default(...args)) - } - export function getHeaders() { - return load().then((m: any) => m.metadata.headers) - } - - // Minor Implementation Detail: Needed so that you can do `<Content />` in a template. - Content.isAstroComponentFactory = true; - ``` + + 1. loads from file system + 2. parses frontmatter from the file into a JS object + 3. returns a JS module with the following JS: + + ```js + ` + // Static: + export const frontmatter = ${JSON.stringify(frontmatter)}; + export const file = ${JSON.stringify(id)}; + export const url = ${JSON.stringify(url || undefined)}; + + // Deferred: + export default async function load(...args) { + return (await import(${JSON.stringify(fileId + '?content')})); + }; + export function Content(...args: any) { + return load().then((m: any) => m.default(...args)) + } + export function getHeaders() { + return load().then((m: any) => m.metadata.headers) + } + + // Minor Implementation Detail: Needed so that you can do `<Content />` in a template. + Content.isAstroComponentFactory = true; + ``` 2. `src/posts/foo.md?content` - 1. loads from file system - 2. renders Markdown using `config.markdownOptions.render(source)` - 3. returns an Astro component representing the markdown content + + 1. loads from file system + 2. renders Markdown using `config.markdownOptions.render(source)` + 3. returns an Astro component representing the markdown content 3. `src/posts/foo.md` - 1. If we resolve an ID without a query param, we have to decide which to serve - 2. if `importer` is set, then its the user importing via `import.meta.glob` or `import.meta.globEager` - 1. **result:** resolve to `?import` - 3. if `importer` is null, then its Astro importing via `ssrLoadModule()` or `vite.build()` - 1. **result:** resolve to `?content` since this is a page + 1. If we resolve an ID without a query param, we have to decide which to serve + 2. if `importer` is set, then its the user importing via `import.meta.glob` or `import.meta.globEager` + 1. **result:** resolve to `?import` + 3. if `importer` is null, then its Astro importing via `ssrLoadModule()` or `vite.build()` + 1. **result:** resolve to `?content` since this is a page # Drawbacks -There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). - -On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. +There is a complexity drawback in the implementation details outlined above where the resolved content of `src/posts/foo.md` is dynamic and changes based on the call to `resolveId`. This is a valid use of `resolveId()` (it supports the `importer` argument for this exact reason) BUT Vite's support here is rough and we'd appear to be the first to rely on this less-touched code path (ex: https://github.com/vitejs/vite/issues/5981). +On initial investigation, I don't think an alternate implementation is possible since both `vite.build()` and `import.meta.globEager` need to use the unmodified import without query params. Vite's automated CI running on Astro should mitigate this somewhat. # Alternatives -A previous version of this RFC removed all helpers, and asked the user to use `import.meta.glob` & `import.meta.globEager` Vite APIs directly themselves. This meant less maintainance/overhead for Astro, but the Vite API suffers from a few problems: +A previous version of this RFC removed all helpers, and asked the user to use `import.meta.glob` & `import.meta.globEager` Vite APIs directly themselves. This meant less maintainance/overhead for Astro, but the Vite API suffers from a few problems: 1. Not well documented outside of being an advanced Vite API 2. unneccesarily complex (ex: when do I use `import.meta.glob` vs. `import.meta.globEager()`. Also, what is `import.meta` anyhow?) @@ -161,7 +162,6 @@ Based on feedback from the community, I revised this RFC and realized that we co 2. Easy to understand the connection to `import.meta.glob()`, if your use-case needs that more flexible API 3. Easy to use, no need to know what `import.meta` and `globEager` do - # Adoption strategy 1. We update documentation to document `Astro.glob()` over `Astro.fetchContent()`, with helpful migration docs. diff --git a/proposals/0018-astro-request.md b/proposals/0018-astro-request.md index b33bf233..2c231837 100644 --- a/proposals/0018-astro-request.md +++ b/proposals/0018-astro-request.md @@ -1,5 +1,5 @@ - Start Date: 2022-03-21 -- Reference Issues: https://github.com/withastro/rfcs/discussions/151 +- Reference Issues: https://github.com/withastro/roadmap/discussions/151 - Implementation PR: <!-- leave empty --> # Summary @@ -45,7 +45,7 @@ The Request object allows access to headers through the `request.headers` Map-li # Detailed design -__Astro.request__ is currently an object with this interface: +**Astro.request** is currently an object with this interface: ```typescript interface AstroRequest { @@ -65,7 +65,7 @@ This change will move `canonicalURL` and `params` up to the `Astro` object and m ```typescript // Partial interface Astro { - /** get the current canonical URL */ + /** get the current canonical URL */ canonicalURL: URL; /** get page params (dynamic pages only) */ @@ -102,7 +102,7 @@ A few alternatives have been tried: - Making Astro.request be a Request but then also adding the `params` and `canonicalURL` properties. - Feel that doing it this way makes it harder to document and the code becomes slightly more complex. -Either of these options would be *fine*, but if we were designing Astro from the start with SSR in mind we would probably have made it a Request, so doing so now before 1.0 seems like good timing. +Either of these options would be _fine_, but if we were designing Astro from the start with SSR in mind we would probably have made it a Request, so doing so now before 1.0 seems like good timing. # Adoption strategy diff --git a/proposals/0019-config-finalization.md b/proposals/0019-config-finalization.md index a3894ee0..d46cbfc8 100644 --- a/proposals/0019-config-finalization.md +++ b/proposals/0019-config-finalization.md @@ -1,5 +1,5 @@ - Start Date: 2022-03-22 -- Reference Issues: [[Roadmap] Astro v1.0](https://github.com/withastro/rfcs/discussions/1) +- Reference Issues: [[Roadmap] Astro v1.0](https://github.com/withastro/roadmap/discussions/1) - Implementation PR: <!-- leave empty --> # Summary @@ -117,7 +117,7 @@ This RFC proposes: # Sitemap -This RFC proposes that the `buildOptions.sitemap` and `buildOptions.sitemapFilter` options are removed entirely. +This RFC proposes that the `buildOptions.sitemap` and `buildOptions.sitemapFilter` options are removed entirely. This usecase is handled by the `@astrojs/sitemap` integration, so users should install and configure `@astrojs/sitemap` if they previously used `buildOptions.sitemap`. diff --git a/proposals/0020-deprecate-markdown-component.md b/proposals/0020-deprecate-markdown-component.md index b32d4096..2124ff91 100644 --- a/proposals/0020-deprecate-markdown-component.md +++ b/proposals/0020-deprecate-markdown-component.md @@ -1,5 +1,5 @@ - Start Date: 04-29-2022 -- Reference Issues: https://github.com/withastro/rfcs/discussions/179 +- Reference Issues: https://github.com/withastro/roadmap/discussions/179 - Implementation PR: <!-- leave empty --> # Summary @@ -20,7 +20,6 @@ We've stomached this high maintainence cost up to this point because as a team w ## Why Now? - Even with its high maintainance cost, our `<Markdown>` component continues to be buggy. This causes poor user experiences and taking effort away from other features and improvements that we'd like to ship. At the same time, our `import`/`import()` support for external Markdown files has improved a ton. Importing your markdown is not just possible, but gives much more reliable, tested, feature-complete, type-hints-enabled support: @@ -45,10 +44,9 @@ None of these problems have simple answers, and some of these problems might eve Instead of shipping v1.0 with a broken experience, we are planning to remove the broken experience for now with the hope of revisiting and adding the feature back, post-v1.0. Potentially in a more standard, pluggable way, so that we could support injecting languages other than Markdown into your component. - # Detailed design -1. Disable the `<Markdown />` component *in SSR*. If you use the component with an adapter, it creates a runtime error telling you that this is not supported, and giving you advice on how to upgrade. SSR + `<Markdown />` is already poorly supported today, so this shouldn't impact many users. +1. Disable the `<Markdown />` component _in SSR_. If you use the component with an adapter, it creates a runtime error telling you that this is not supported, and giving you advice on how to upgrade. SSR + `<Markdown />` is already poorly supported today, so this shouldn't impact many users. 2. Before `v1.0.0-rc.1`, move the `<Markdown />` component out into its own package entirely. Call it `@astrojs/markdown`. In the readme of the package, give the SSG-only warning more clearly. 3. Before `v1.0.0-rc.1`, replace references to the Markdown component in our docs with the new package. 4. In `v1.0.0-rc.1`, disable the core `<Markdown />` component entirely. If a developer uses it, point them to the new package or suggest moving the Markdown snippet out into its own file. @@ -58,6 +56,7 @@ The user-land Markdown component will also continue to exist for those who need # Drawbacks & alternatives In practice we've seen the following pattern play out, which gives me hope that most users will be able to make this transition: + 1. **If a block of inline markdown is small,** it's trivial to migrate the Markdown snippet directly to HTML. 2. **If a block of inline markdown is large,** we'd probably recommend anyway that you move it to a separate MD file, based on how unreliable Astro can be when handling it, and how much better your editor/IDE support will be. 3. **If neither is acceptable,** use the new userland `<Markdown />` component. diff --git a/proposals/0021-astro-response.md b/proposals/0021-astro-response.md index 8a8406e5..e55669c7 100644 --- a/proposals/0021-astro-response.md +++ b/proposals/0021-astro-response.md @@ -20,7 +20,7 @@ Astro.response.headers.set('Cache-Control', 'max-age=604800'); When SSR was added to Astro we added the `Astro.request` object which is a [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request), allowing you to examine headers (such as cookies) to dynamically handle page renders. -In order to modify the *response* you are able to return a [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) within your frontmatter like so: +In order to modify the _response_ you are able to return a [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) within your frontmatter like so: ```astro --- @@ -33,7 +33,7 @@ if(!Astro.request.headers.has('cookie')) { <h1>My page</h1> ``` -However, there are many cases where you do want to render your page and *also* modify something about the response such as: +However, there are many cases where you do want to render your page and _also_ modify something about the response such as: - Cache headers such as `Cache-Control` and `ETag`. - Adding cookies via `Set-Cookie` headers. @@ -86,8 +86,8 @@ The initial values of the `Astro.response` will be: ```js Astro.response = { status: 200, - statusText: 'OK', - headers: new Headers() + statusText: "OK", + headers: new Headers(), }; ``` @@ -99,17 +99,17 @@ Astro currently supports returning a [Response](https://developer.mozilla.org/en # Drawbacks -There are other proposals in discussion to add [cookie management](https://github.com/withastro/rfcs/discussions/182) and [cache control](https://github.com/withastro/rfcs/discussions/181) APIs, which are higher-level ways to modify the response. +There are other proposals in discussion to add [cookie management](https://github.com/withastro/roadmap/discussions/182) and [cache control](https://github.com/withastro/roadmap/discussions/181) APIs, which are higher-level ways to modify the response. If those, or similar, proposals go through there will be much less of a use-case for `Astro.response`. However those proposals do not cover: -- Setting *every* possible value of cache headers, for example `ETag` is not covered by the cache control proposal. +- Setting _every_ possible value of cache headers, for example `ETag` is not covered by the cache control proposal. - Setting the `status` or `statusText`. - Setting other types of response headers, such as user-defined headers. # Alternatives -As discussed in the __Drawbacks__ section, one alternative is to provide higher-level APIs for the common use-cases for modifying the response. However it will be impossible to anticipate every need, so providing a lower-level way to modify the response should unblock use-cases we haven't thought about. +As discussed in the **Drawbacks** section, one alternative is to provide higher-level APIs for the common use-cases for modifying the response. However it will be impossible to anticipate every need, so providing a lower-level way to modify the response should unblock use-cases we haven't thought about. Additionally, `Astro.response` could be a [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) instead of an interface we define. The main reason this proposal doesn't do that is because most of the properties on [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) are readonly; you could not modify the `status` or `statusText`. This would be unintuitive to users. @@ -134,4 +134,4 @@ However this is awkward as well because you are setting the response `body` only # Adoption strategy - This is a non-breaking change; `Astro.response` could be added in a single PR. -- In SSG mode the properties would be readonly and ignored in both dev and build. \ No newline at end of file +- In SSG mode the properties would be readonly and ignored in both dev and build. diff --git a/proposals/0022-frontmatter-plugins.md b/proposals/0022-frontmatter-plugins.md index a9248494..460e29a9 100644 --- a/proposals/0022-frontmatter-plugins.md +++ b/proposals/0022-frontmatter-plugins.md @@ -11,19 +11,19 @@ I suggest adding a new `markdown.frontmatterPlugins` config option to unlock som ```js // astro.config.mjs const addImageTools = (frontmatter) => { - frontmatter.setup += `\nimport { Picture } from "astro-imagetools/components";`; - return frontmatter; -} + frontmatter.setup += `\nimport { Picture } from "astro-imagetools/components";`; + return frontmatter; +}; const dynamicLayout = (frontmatter, fileUrl) => { - frontmatter.layout = customLayoutSelector(fileUrl); - return frontmatter; -} + frontmatter.layout = customLayoutSelector(fileUrl); + return frontmatter; +}; export default defineConfig({ - markdown: { - frontmatterPlugins: [addImageTools, dynamicLayout] - } + markdown: { + frontmatterPlugins: [addImageTools, dynamicLayout], + }, }); ``` @@ -32,14 +32,15 @@ export default defineConfig({ Currently, when importing Markdown files, one can modify their content via the `markdown.remarkPlugins` and `markdown.rehypePlugins` config options. However there is currently no supported way to do the same with their frontmatter. This could open up interesting usecases: + - Auto-registering components in imported Markdown files (just append the imports to `frontmatter.setup`!) -- Providing an easier way for people to add layouts conditionally to many Markdown files at once, as mentioned in https://github.com/withastro/rfcs/discussions/161#discussion-3972352 -- Provide a more generic solution to the `frontmatterDefaults` suggestion here: https://github.com/withastro/rfcs/discussions/172#discussioncomment-2558676 +- Providing an easier way for people to add layouts conditionally to many Markdown files at once, as mentioned in https://github.com/withastro/roadmap/discussions/161#discussion-3972352 +- Provide a more generic solution to the `frontmatterDefaults` suggestion here: https://github.com/withastro/roadmap/discussions/172#discussioncomment-2558676 - Probably more that I'm missing! # Detailed design -There is a prototype implementation here: +There is a prototype implementation here: https://github.com/withastro/astro/pull/3411 # Drawbacks @@ -50,10 +51,11 @@ https://github.com/withastro/astro/pull/3411 # Alternatives Considered alternatives included: + - Passing a single `frontmatterUpdate` function in the config. Seemed like a missed -opportunity not to mimic `rehypePlugins` and provide better extensibility. + opportunity not to mimic `rehypePlugins` and provide better extensibility. - Using `unplugin-auto-import` like https://stackblitz.com/edit/github-kzxlce?file=src%2Fpages%2Findex.md -This would solve the usecase of globally registering components but not the others. + This would solve the usecase of globally registering components but not the others. # Adoption strategy @@ -62,4 +64,4 @@ This is a new API and won't require any migrations. # Unresolved questions - What additional parameters beyond the `frontmatter` should be passed to the plugins? -- Should the plugins return a copy or just modify the intial `frontmatter` object? \ No newline at end of file +- Should the plugins return a copy or just modify the intial `frontmatter` object? diff --git a/proposals/0023-inject-route.md b/proposals/0023-inject-route.md index 81ba897a..1c74d42c 100644 --- a/proposals/0023-inject-route.md +++ b/proposals/0023-inject-route.md @@ -1,5 +1,5 @@ - Start Date: 2022-06-01 -- Reference Issues: https://github.com/withastro/rfcs/discussions/201 +- Reference Issues: https://github.com/withastro/roadmap/discussions/201 - Implementation PR: https://github.com/withastro/astro/pull/3457 # Summary @@ -13,25 +13,26 @@ Recently, @FredKSchott made a thread on the discord #feedback-and-suggestions ch export default defineConfig({ integrations: [ { - name: 'my-netlify-integration', + name: "my-netlify-integration", hooks: { - 'astro:config:setup': ({injectRoute}) => { + "astro:config:setup": ({ injectRoute }) => { injectRoute({ /** The route on which to output the entryPoint */ - pattern: '/admin', + pattern: "/admin", /** Bare module specifier pointing to a pre-made admin page */ - entryPoint: 'my-netlify-integration/admin.astro' - }) - } - } - } - ] + entryPoint: "my-netlify-integration/admin.astro", + }); + }, + }, + }, + ], }); ``` # Motivation Some usecases for this could be: + - Adding an `/admin` page for headless CMSes - Implementation of [tailwind-config-viewer](https://github.com/rogden/tailwind-config-viewer) - Authentication providers could very easily ship the required redirectCallback routes etc, e.g. `googleProvider()`, `facebookProvider()` @@ -39,18 +40,18 @@ Some usecases for this could be: # Detailed design -There is a prototype implementation here: +There is a prototype implementation here: https://github.com/withastro/astro/pull/3457 ## Proposed API ```ts export interface InjectedRoute { - pattern: string, - entryPoint: string + pattern: string; + entryPoint: string; } -function injectRoute(injectRoute: InjectedRoute): void {}; +function injectRoute(injectRoute: InjectedRoute): void {} ``` # Drawbacks @@ -69,7 +70,7 @@ This is a new API and won't require any migrations. **Resolved:** ✅ -**Q:** _Should `_`'s in route names be allowed?_ +**Q:** _Should `_`'s in route names be allowed?\_ **A:** Yes. An expected usecase for `injectRoute` is to add "private" routes, like for example `/_admin`, and allowing `_`'s will also help avoid nameclashes. The draft implementation currently already supports this. @@ -84,24 +85,20 @@ This is a new API and won't require any migrations. ```js function myIntegration(config) { return { - name: 'my-integration', + name: "my-integration", hooks: { - 'astro:config:setup': ({injectRoute}) => { + "astro:config:setup": ({ injectRoute }) => { injectRoute({ - pattern: config?.routes?.admin ?? '/admin', - entryPoint: 'my-integration/admin.astro' + pattern: config?.routes?.admin ?? "/admin", + entryPoint: "my-integration/admin.astro", }); - } - } - } + }, + }, + }; } export default defineConfig({ - integrations: [ - myIntegration({routes: - { admin: '/custom-path/admin' } - }) - ] + integrations: [myIntegration({ routes: { admin: "/custom-path/admin" } })], }); ``` @@ -124,19 +121,17 @@ export default defineConfig({ ```js function myIntegration(config) { return { - name: 'my-integration', + name: "my-integration", hooks: { - 'astro:config:setup': ({command, injectRoute}) => { + "astro:config:setup": ({ command, injectRoute }) => { /** This route will only be injected during dev-time */ - if(command === 'dev') injectRoute(routeConfig); - } - } - } + if (command === "dev") injectRoute(routeConfig); + }, + }, + }; } export default defineConfig({ - integrations: [ - myIntegration() - ] + integrations: [myIntegration()], }); -``` \ No newline at end of file +``` diff --git a/proposals/0025-cookie-management.md b/proposals/0025-cookie-management.md index 12ac69fa..280f602e 100644 --- a/proposals/0025-cookie-management.md +++ b/proposals/0025-cookie-management.md @@ -38,8 +38,12 @@ Users in the Astro discord often ask about how to use cookies in Astro and we do ```ts interface AstroCookies { get(key: string): AstroCookie; - set(key: string, value: string | Record<string, any>, options: AstroCookieOptions): void; - delete(key: string, options: { path: string; }): void; + set( + key: string, + value: string | Record<string, any>, + options: AstroCookieOptions + ): void; + delete(key: string, options: { path: string }): void; has(key: string): void; headers(): Array<string>; } @@ -54,7 +58,7 @@ interface AstroCookieOptions { httpOnly?: boolean; maxAge?: number; path?: string; - sameSite?: boolean | 'lax' | 'none' | 'strict'; + sameSite?: boolean | "lax" | "none" | "strict"; secure?: boolean; } ``` @@ -107,14 +111,14 @@ Removes a cookie. This is likely used within an API route. ```js export function post({ request, cookies }) { - cookies.delete('prefs'); + cookies.delete("prefs"); // Set-Cookie headers will be appended. return new Response(null, { status: 302, headers: { - Location: '/' - } + Location: "/", + }, }); } ``` @@ -138,8 +142,8 @@ Provides an iterator of header values that should be set as `Set-Cookie` headers For example, a Node.js implementation would do: ```js -for(const value of cookies.headers()) { - res.setHeader('Set-Cookie', value); +for (const value of cookies.headers()) { + res.setHeader("Set-Cookie", value); } ``` @@ -155,7 +159,7 @@ In .astro files it is available as `Astro.cookies` and in API routes it is a pro ```js export function post({ cookies }) { - const prefs = cookies.get('prefs'); + const prefs = cookies.get("prefs"); // ... } @@ -177,7 +181,7 @@ When setting the headers during the rendering phase we need to take the AstroCoo However: - We only need to `Set-Cookie` if there is a change, such as a cookie value being set or a cookie being deleted. -- If the user has provided their own `Set-Cookie` header we should *not* set the header ourselves. Don't attempt to merge the header, just use the manually set value of the user. +- If the user has provided their own `Set-Cookie` header we should _not_ set the header ourselves. Don't attempt to merge the header, just use the manually set value of the user. ### Deleting cookies @@ -193,7 +197,7 @@ To set an expiration using a string duration value let `30 days` we will use the # Alternatives -- There was a previous [Cookie Management](https://github.com/withastro/rfcs/discussions/182) discussion. This was based on a proposed browser API. That proposal hasn't been adopted by other backend frameworks and has some downsides, such as async get/set that don't make sense for our use-case. +- There was a previous [Cookie Management](https://github.com/withastro/roadmap/discussions/182) discussion. This was based on a proposed browser API. That proposal hasn't been adopted by other backend frameworks and has some downsides, such as async get/set that don't make sense for our use-case. # Adoption strategy @@ -201,5 +205,5 @@ This is a completely additive feature that should have no effect on existing app # Unresolved questions -- The cookie Options mirrors the npm __cookie__ package except is allows spoken-word `expires` option. - - Should we punt in this? It seems useful but there is a cost to extending the options from this library. \ No newline at end of file +- The cookie Options mirrors the npm **cookie** package except is allows spoken-word `expires` option. + - Should we punt in this? It seems useful but there is a cost to extending the options from this library. diff --git a/proposals/0027-content-collections.md b/proposals/0027-content-collections.md index 6b0192cb..28c84223 100644 --- a/proposals/0027-content-collections.md +++ b/proposals/0027-content-collections.md @@ -6,7 +6,8 @@ <aside> -💡 **This RFC is complimented by [the Render Content proposal](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0028-render-content.md).** Our goal is to propose and accept both of these RFCs as a pair before implementing any features discussed. We recommend reading that document *after* reading this to understand how all use cases can be covered. +💡 **This RFC is complimented by [the Render Content proposal](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0028-render-content.md).** Our goal is to propose and accept both of these RFCs as a pair before implementing any features discussed. We recommend reading that document _after_ reading this to understand how all use cases can be covered. + </aside> # Summary @@ -40,7 +41,7 @@ First, **this RFC is focused on Markdown and MDX content only.** We see how this We also expect users to attempt relative paths (i.e. `![image](./image.png)`) in their Markdown and MDX files. Since these files will be query-able by `.astro` files, **we don't expect these paths to resolve correctly without added preprocessing.** -For simplicity, we will consider this use case out-of-scope. This is in-keeping with how Astro handles relative paths in Markdown and MDX today. We will raise an error whenever relative paths are used, and encourage users to use absolute assets paths instead. +For simplicity, we will consider this use case out-of-scope. This is in-keeping with how Astro handles relative paths in Markdown and MDX today. We will raise an error whenever relative paths are used, and encourage users to use absolute assets paths instead. ## Prior Art @@ -96,7 +97,7 @@ And optionally define a schema to enforce frontmatter fields: ```tsx // src/content/config.ts -import { z, defineCollection } from 'astro:content'; +import { z, defineCollection } from "astro:content"; const blog = defineCollection({ schema: { @@ -130,7 +131,7 @@ const posts: Array<MarkdownInstance<{ title: string; ... }>> = await Astro.glob( --- ``` -However, there's no guarantee your frontmatter *actually* matches this `MarkdownInstance` type. +However, there's no guarantee your frontmatter _actually_ matches this `MarkdownInstance` type. Say `blog/columbia.md` is missing the required `title` property. When writing a landing page like this: @@ -147,31 +148,29 @@ Say `blog/columbia.md` is missing the required `title` property. When writing a ...You'll get the ominous error "cannot read property `toUpperCase` of undefined." Stop me if you've had this monologue before: -> *Aw where did I call `toUpperCase` again?* -> -> -> *Right, on the landing page. Probably the `title` property.* -> -> *But which post is missing a title? Agh, better add a `console.log` and scroll through here...* -> -> *Ah finally, it was post #1149. I'll go fix that.* -> +> _Aw where did I call `toUpperCase` again?_ +> +> _Right, on the landing page. Probably the `title` property._ +> +> _But which post is missing a title? Agh, better add a `console.log` and scroll through here..._ +> +> _Ah finally, it was post #1149. I'll go fix that._ **Authors shouldn't have to think like this.** What if instead, they were given a readable error pointing to where the problem is? ![Frontmatter error overlay - Could not parse frontmatter in blog -> columbia.md. "title" is required.](../assets/0027-frontmatter-err.png) -This is why schemas are a *huge* win for a developer's day-to-day. Astro will autocomplete properties that match your schema, and give helpful errors to fix properties that don't. +This is why schemas are a _huge_ win for a developer's day-to-day. Astro will autocomplete properties that match your schema, and give helpful errors to fix properties that don't. ## Importing globs of content can be slow -Second problem: **importing globs of content via `Astro.glob` can be slow at scale.** This is due to a fundamental flaw with importing: even if you *just* need the frontmatter of a post (i.e. for landing pages), you still wait on the *content* of that render and parse to a JS module as well. Though less of a problem with Markdown, globbing hundreds-to-thousands of MDX entries [can slow down dev server HMR updates significantly](https://github.com/withastro/astro/issues/4307). +Second problem: **importing globs of content via `Astro.glob` can be slow at scale.** This is due to a fundamental flaw with importing: even if you _just_ need the frontmatter of a post (i.e. for landing pages), you still wait on the _content_ of that render and parse to a JS module as well. Though less of a problem with Markdown, globbing hundreds-to-thousands of MDX entries [can slow down dev server HMR updates significantly](https://github.com/withastro/astro/issues/4307). To avoid this, Content Collections will focus on processing and returning a post's frontmatter, **not** the post's contents, **and** avoid transforming documents to JS modules. This should make Markdown and MDX equally quick to process, and should make landing pages faster to build and debug. <aside> -💡 Don’t worry, it will still be easy to retrieve a post’s content when you need it! [See the Render Content proposal](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0028-render-content.md) for more. +💡 Don’t worry, it will still be easy to retrieve a post’s content when you need it! [See the Render Content proposal](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0028-render-content.md) for more. </aside> @@ -211,18 +210,18 @@ To clarify, **the user will not view or edit files in the `.astro` directory.** ## The `astro:content` module Content Collections introduces a new virtual module convention for Astro using the `astro:` prefix. Since all types and utilities are generated based on your content, we are free to use whatever name we choose. We chose `astro:` for this proposal since: + 1. It falls in-line with [NodeJS' `node:` convention](https://2ality.com/2021/12/node-protocol-imports.html). 2. It leaves the door open for future `astro:` utility modules based on your project's configuration. The user will import helpers like `getCollection` and `getEntry` from `astro:content` like so: ```tsx -import { getCollection, getEntry } from 'astro:content'; +import { getCollection, getEntry } from "astro:content"; ``` Users can also expect full auto-importing and intellisense from their editor. - ## Creating a collection All entries in `src/content/` **must** be nested in a "collection" directory. This allows you to get a collection of entries based on the directory name, and optionally enforce frontmatter types with a schema. This is similar to creating a new table in a database, or a new content model in a CMS like Contentful. @@ -245,7 +244,7 @@ src/content/ ### Nested directories -Collections are considered **one level deep**, so you cannot nest collections (or collection schemas) within other collections. However, we *will* allow nested directories to better organize your content. This is vital for certain use cases like internationalization: +Collections are considered **one level deep**, so you cannot nest collections (or collection schemas) within other collections. However, we _will_ allow nested directories to better organize your content. This is vital for certain use cases like internationalization: ```bash src/content/ @@ -269,7 +268,7 @@ For instance, say every `blog/` entry should have a `title`, `slug`, a list of ` ```ts // src/content/config.ts -import { z, defineCollection } from 'astro:content'; +import { z, defineCollection } from "astro:content"; const blog = defineCollection({ schema: { @@ -295,8 +294,9 @@ export const collections = { 'my-newsletter': myNewsletter }; ### Why Zod? We chose [Zod](https://github.com/colinhacks/zod) since it offers key benefits over plain TypeScript types. Namely: + - specifying default values for optional fields using `.default()` -- checking the *shape* of string values with built-in regexes, like `.url()` for URLs and `.email()` for emails +- checking the _shape_ of string values with built-in regexes, like `.url()` for URLs and `.email()` for emails ```tsx ... @@ -339,7 +339,7 @@ Assume the `blog` collection schema looks like this: ```tsx // src/content/config.ts -import { defineCollection, z } from 'astro:content'; +import { defineCollection, z } from "astro:content"; const blog = defineCollection({ schema: { @@ -366,7 +366,7 @@ export const collections = { blog }; }; // unique identifier. Today, the file path relative to src/content/[collection] id: '[filePath]'; // union from all entries in src/content/[collection] - // URL-ready slug computed from ID, relative to collection + // URL-ready slug computed from ID, relative to collection // ex. "docs/home.md" -> "home" slug: '[fileBase]'; // union from all entries in src/content/[collection] // raw body of the Markdown or MDX document @@ -376,15 +376,15 @@ export const collections = { blog }; We have purposefully generalized Markdown-specific terms like `frontmatter` and `file` to agnostic names like `data` and `id`. This also follows naming conventions from headless CMSes like Contentful. -Also note that `body` is the *raw* content of the file. This ensures builds remain performant by avoiding expensive rendering pipelines. See [“Moving to `src/pages/`"](#mapping-to-srcpages) to understand how a `<Content />` component could be used to render this file, and pull in that pipeline only where necessary. +Also note that `body` is the _raw_ content of the file. This ensures builds remain performant by avoiding expensive rendering pipelines. See [“Moving to `src/pages/`"](#mapping-to-srcpages) to understand how a `<Content />` component could be used to render this file, and pull in that pipeline only where necessary. ### Nested directories [As noted earlier](#nested-directories), you may organize entries into directories as well. The result will **still be a flat array** when fetching a collection via `getCollection`, with the nested directory reflected in an entry’s `id`: ```tsx -const docsEntries = await getCollection('docs'); -console.log(docsEntries) +const docsEntries = await getCollection("docs"); +console.log(docsEntries); /* -> [ { id: 'en/getting-started.md', slug: 'en/getting-started', data: {...} }, @@ -396,11 +396,11 @@ console.log(docsEntries) */ ``` -This is in-keeping with our database table and CMS collection analogies. Directories are a way to organize your content, but do *not* effect the underlying, flat collection → entry relationship. +This is in-keeping with our database table and CMS collection analogies. Directories are a way to organize your content, but do _not_ effect the underlying, flat collection → entry relationship. ## Mapping to `src/pages/` -We imagine users will want to map their collections onto live URLs on their site. This should be similar to globbing directories outside of `src/pages/` today, using `getStaticPaths` to generate routes dynamically. +We imagine users will want to map their collections onto live URLs on their site. This should be similar to globbing directories outside of `src/pages/` today, using `getStaticPaths` to generate routes dynamically. Say you have a `docs` collection subdivided by locale like so: @@ -452,7 +452,7 @@ Note the `slug` function can access the default generated slug, entry ID, parsed ### Rendering contents -The above example generates routes, but what about rendering our `.md` files on the page? We suggest [reading the Render Content proposal](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0028-render-content.md) for full details on how `getCollection` will compliment that story. +The above example generates routes, but what about rendering our `.md` files on the page? We suggest [reading the Render Content proposal](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0028-render-content.md) for full details on how `getCollection` will compliment that story. # Detailed design @@ -482,14 +482,14 @@ We will add this generation step to the `astro check` command as well, in case u Content Collections will expose Zod as an Astro built-in. This will be available from the `astro:content` module: ```ts -import { z } from 'astro:content'; +import { z } from "astro:content"; ``` This avoids exposing Zod from the base `astro` package, preventing unecessary dependencies in core (SSR builds being the main consideration). However, to populate this virtual module, we will need a separate `astro/zod` module to expose all utilities. Here's an example of how a `zod.mjs` package export may look: ```js // zod.mjs -import * as mod from 'zod'; +import * as mod from "zod"; export { mod as z }; export default mod; ``` @@ -536,8 +536,8 @@ There are alternative solutions to consider across several categories: We considered a few alternatives to using Zod for schemas: - **Generate schemas from a TypeScript type.** This would let users reuse frontmatter types they already have and avoid the learning curve of a new tool. However, TypeScript is missing a few surface-level features that Zod covers: - - Constraining the shape of a given value. For instance, setting a `min` or `max` character length, or testing strings against `email` or `URL` regexes. - - [Transforming](https://github.com/colinhacks/zod#transform) a frontmatter value into a new data type. For example, parsing a date string to a `Date` object, and raising a helpful error for invalid dates. + - Constraining the shape of a given value. For instance, setting a `min` or `max` character length, or testing strings against `email` or `URL` regexes. + - [Transforming](https://github.com/colinhacks/zod#transform) a frontmatter value into a new data type. For example, parsing a date string to a `Date` object, and raising a helpful error for invalid dates. - **Invent our own JSON or YAML-based schema format.** This would fall in-line with a similar open source project, [ContentLayer](https://www.contentlayer.dev/docs/sources/files/mapping-document-types), that specifies types with plain JS. Main drawbacks: replacing one learning curve with another, and increasing the maintenance cost of schemas overtime. In the end, we've chosen Zod since it can scale to complex use cases and takes the maintenance burden off of Astro's shoulders. @@ -546,7 +546,7 @@ In the end, we've chosen Zod since it can scale to complex use cases and takes t We expect most users to compare `getCollection` with `Astro.glob`. There is a notable difference in how each will grab content: -- `Astro.glob` accepts wild cards (i.e. `/posts/**/*.md) to grab entries multiple directories deep, filter by file extension, etc. +- `Astro.glob` accepts wild cards (i.e. `/posts/\*_/_.md) to grab entries multiple directories deep, filter by file extension, etc. - `getCollection` accepts **a collection name only,** with an optional filter function to filter by entry values. The latter limits users to fetching a single collection at a time, and removes nested directories as a filtering option (unless you regex the ID by hand). One alternative could be to [mirror Contentlayer's approach](https://www.contentlayer.dev/docs/sources/files/mapping-document-types#resolving-document-type-with-filepathpattern), wiring schemas to wildcards of any shape: @@ -555,10 +555,10 @@ The latter limits users to fetching a single collection at a time, and removes n // Snippet from Contentlayer documentation // <https://www.contentlayer.dev/docs/sources/files/mapping-document-types#resolving-document-type-with-filepathpattern> const Post = defineDocumentType(() => ({ - name: 'Post', + name: "Post", filePathPattern: `posts/**/*.md`, // ... -})) +})); ``` Still, we've chosen a flat `collection` + schema file approach to a) mirror Astro's file-based routing, and b) establish a familiar database table or headless CMS analogy. @@ -572,13 +572,13 @@ Introducing a new reserved directory (`src/content/`) will be a breaking change This should give us time to address all aspects and corner cases of content Collections. -We intend `src/content/` to be the recommended way to store Markdown and MDX content in your Astro project. Documentation will be *very* important to guide adoption! So, we will speak with the docs team on the best information hierarchy. Not only should we surface the concept of a `src/content/` early for new users, but also guide existing users (who may visit the "Markdown & MDX" and Astro glob documentation) to `src/content/` naturally. A few initial ideas: +We intend `src/content/` to be the recommended way to store Markdown and MDX content in your Astro project. Documentation will be _very_ important to guide adoption! So, we will speak with the docs team on the best information hierarchy. Not only should we surface the concept of a `src/content/` early for new users, but also guide existing users (who may visit the "Markdown & MDX" and Astro glob documentation) to `src/content/` naturally. A few initial ideas: - Expand [Project Structure](https://docs.astro.build/en/core-concepts/project-structure/) to explain `src/content/` - Update Markdown & MDX to reference the Project Structure docs, and expand [Importing Markdown](https://docs.astro.build/en/guides/markdown-content/#importing-markdown) to a more holistic "Using Markdown" section - Add a "local content" section to the [Data Fetching](https://docs.astro.build/en/guides/data-fetching/) page -We can also ease migration for users that *already* have a `src/content/` directory used for other purposes. For instance, we can warn users with a `src/content/` that a) contains other file types or b) does not contain any `schema` files. +We can also ease migration for users that _already_ have a `src/content/` directory used for other purposes. For instance, we can warn users with a `src/content/` that a) contains other file types or b) does not contain any `schema` files. # Unresolved questions @@ -656,4 +656,4 @@ declare module 'astro:content' { type CollectionsConfig = typeof import('/path/to/project/src/content/config'); } -``` \ No newline at end of file +``` diff --git a/proposals/0028-render-content.md b/proposals/0028-render-content.md index d1136f53..64012a92 100644 --- a/proposals/0028-render-content.md +++ b/proposals/0028-render-content.md @@ -1,6 +1,7 @@ - Start Date: 2022-10-14 - Reference Issues: - - https://github.com/withastro/astro/issues/3816 + + - https://github.com/withastro/astro/issues/3816 - Implementation PR: https://github.com/withastro/astro/pull/5291 @@ -8,7 +9,7 @@ <aside> -💡 **This RFC compliments our [Content Collections RFC](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0027-content-collections.md).** We recommend reading that document first to understand the goals of “Content” as a concept, and where rendering content fits into that story. +💡 **This RFC compliments our [Content Collections RFC](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0027-content-collections.md).** We recommend reading that document first to understand the goals of “Content” as a concept, and where rendering content fits into that story. </aside> @@ -59,25 +60,26 @@ There are two major challenges Astro has addressed since the project’s early d Since migrating to Vite, Astro has leaned into its built-in concept for globbing directories of content: `import.meta.glob`. Here’s what that API will output, using both the default “lazy” option and the “eager” option: ```tsx -const lazyPosts = await import.meta.glob('./blog/*.md'); +const lazyPosts = await import.meta.glob("./blog/*.md"); /* { './blog/first.md': () => Promise(module), './blog/second.md': () => Promise(module), } */ -const eagerPosts = await import.meta.glob('./blog/*.md', { eager: true }); +const eagerPosts = await import.meta.glob("./blog/*.md", { eager: true }); /* { './blog/first.md': { frontmatter: {...}, rawContent: '# First...', './blog/second.md': { frontmatter: {...}, rawContent: '# Second...', } */ ``` -You’ll notice that lazily globbing only yields an object of file names. To access any other information about a given post (including frontmatter), you’ll need to call that `() => Promise(module)` function to import the file. You’ll likely call this function across *all* of your modules when tackling the landing page problem **(1)** or the dynamic routes problem **(2)**. +You’ll notice that lazily globbing only yields an object of file names. To access any other information about a given post (including frontmatter), you’ll need to call that `() => Promise(module)` function to import the file. You’ll likely call this function across _all_ of your modules when tackling the landing page problem **(1)** or the dynamic routes problem **(2)**. To make these problems easier to tackle without learning Vite’s nuances, Astro created the `Astro.glob` abstraction. This abstraction is based on the eager example above, but mapping the object to an array of its values. In other words: ```tsx -Astro.glob('stuff') === Object.values(import.meta.glob('stuff', { eager: true })) +Astro.glob("stuff") === + Object.values(import.meta.glob("stuff", { eager: true })); ``` This makes landing pages and dynamic routes fairly trivial to build: @@ -113,11 +115,11 @@ const { Post } = Astro.props; <Post /> ``` -However, there’s a reason that Vite does *not* eagerly load by default, which brings us to… +However, there’s a reason that Vite does _not_ eagerly load by default, which brings us to… ## Where this breaks down -In short, eagerly loading information about every module makes it *tough* to know which resources are actually needed. +In short, eagerly loading information about every module makes it _tough_ to know which resources are actually needed. Take our landing page example above. We’ll assume that none of the entries `blog/**` have style or component imports… well, except for one pesky file. We’ll call this `blog/comic-sans-is-great.mdx`: @@ -135,13 +137,13 @@ body { } ``` -Now that MDX is supported for content authoring, it’s fairly common to pull in one-off styles or components for a given post that aren’t shared by other files. +Now that MDX is supported for content authoring, it’s fairly common to pull in one-off styles or components for a given post that aren’t shared by other files. However, this poses a problem for our landing page. As you may know, you’re free to render the content of a globbed post (styles, components and all) [using the `Content` component](https://docs.astro.build/en/guides/markdown-content/#content). This feature can be a double-edged sword though; Since `Astro.glob` eagerly loads every module, **it will also inject every module’s imports (namely styles) onto the page where it is globbed.** -This means, when `index.astro` globs all of our `blog/**` posts, it will now inject `comic-sans-override.css` onto the page as well. This happens whether we *actually* use the Content component or not. Yikes! +This means, when `index.astro` globs all of our `blog/**` posts, it will now inject `comic-sans-override.css` onto the page as well. This happens whether we _actually_ use the Content component or not. Yikes! This is more jarring in our dynamic routes example. Recall that we’re calling `Astro.glob` to get a list of all paths to generate: @@ -204,9 +206,9 @@ As you might imagine, there’s a bit of trickery needed to selectively add styl ## `renderEntry` API reference - **Param:** `entry: ReturnType<getCollection> | ReturnType<getCollection>['id']` - - Either a complete `getCollection` return type, or the ID of the entry to render. The ID type is a union of all valid IDs in your `src/content` directory (not a generic string) for better type checking. + - Either a complete `getCollection` return type, or the ID of the entry to render. The ID type is a union of all valid IDs in your `src/content` directory (not a generic string) for better type checking. - **Returns: `{ Content: AstroComponentFactory }`** - - A `Content` component for use in Astro or MDX files + - A `Content` component for use in Astro or MDX files ## Usage breakdown @@ -232,18 +234,19 @@ const { Content } = await renderEntry({ id, collection }); In this example: -1. We use `getEntry` or `getCollection` to get references to whatever we want to render ([see the Content Schema RFC example](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0027-content-collections.md#example)) +1. We use `getEntry` or `getCollection` to get references to whatever we want to render ([see the Content Schema RFC example](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0027-content-collections.md#example)) 2. We pass this fetched entry to `renderEntry`. This **imports** our entry processed through the Vite pipeline to retrieve a `Content` component, and **injects** all component resources (styles and nested island dependencies) onto the page. <aside> -💡 For step 2 to work in production builds, we will lazy glob all `src/content/` entries using dynamic imports [via a manifest](#a-new-renderentrymap). This is because we won’t know which entries to bundle until SSR endpoints are run, so we need to prepare all entries in anticipation. Yes, this means build performance will *just* meet the status quo of `Astro.glob`, though it could be optimized in the future. +💡 For step 2 to work in production builds, we will lazy glob all `src/content/` entries using dynamic imports [via a manifest](#a-new-renderentrymap). This is because we won’t know which entries to bundle until SSR endpoints are run, so we need to prepare all entries in anticipation. Yes, this means build performance will _just_ meet the status quo of `Astro.glob`, though it could be optimized in the future. </aside> ### Retrieve `headings` and `injectedFrontmatter` You may also need `renderEntry` to retrieve results from the remark or rehype pipelines. This includes: + - generated headings - [see our existing `getHeadings` utility](https://docs.astro.build/en/guides/integrations-guide/mdx/#getheadings) - injected frontmatter - [see our frontmatter injection example for reading time](https://docs.astro.build/en/guides/markdown-content/#example-injecting-frontmatter) @@ -262,7 +265,7 @@ const blogPosts = await getCollection('blog'); } = await renderEntry(post); const { readingTime } = injectedFrontmatter; const h1 = headings.find(h => h.depth === 1); - + return <p>{h1} - {readingTime} min read</p> })} ``` @@ -271,7 +274,7 @@ const blogPosts = await getCollection('blog'); ## Flagging `src/content` resources -Currently, we crawl **every** module imported by a given page to discover styles and component resources used, and dump all discovered resources into sets of `scripts`, `styles`, and `links`. +Currently, we crawl **every** module imported by a given page to discover styles and component resources used, and dump all discovered resources into sets of `scripts`, `styles`, and `links`. This approach is too naive for our style bleed problem. Instead, we want to flag which resources can be added normally, and which should be deferred until `renderEntry` is called. @@ -281,16 +284,16 @@ Pseudo-code for how this may work: ```tsx function crawlCss(currentModule, discoveredStyles, discoveredSrcContentStyles) { - const { importedModules } = viteServer.getModuleInfo(currentModule); - for (const importedModule of importedModules) { - if (isStyle(currentModule)) { - if (currentModule.endsWith(SPECIAL_FLAG)) { - discoveredSrcContentStyles.add(currentModule); - } else { - discoveredStyles.add(currentModule); - } - } - } + const { importedModules } = viteServer.getModuleInfo(currentModule); + for (const importedModule of importedModules) { + if (isStyle(currentModule)) { + if (currentModule.endsWith(SPECIAL_FLAG)) { + discoveredSrcContentStyles.add(currentModule); + } else { + discoveredStyles.add(currentModule); + } + } + } } ``` @@ -302,7 +305,7 @@ Once we’ve pulled our `src/content` resources for later, we need to inject the // astro-head-inject export function renderEntry(/* ... */) { - // ... + // ... } ``` @@ -311,20 +314,20 @@ This tells the Astro renderer to look for and add propagated HTML into the docum The `createComponent` function takes an object where we can create a component that does head propagation. Pseudo-code for that will look like: ```js -import { createComponent } from 'astro/runtime/server/index.js'; +import { createComponent } from "astro/runtime/server/index.js"; export function renderEntry() { - return createComponent({ - factory(result, props, slots) { - return createHeadAndContent( - renderUniqueStylesheet(result, { - href: '/path/to/these/styles.css' - }), - renderTemplate`${renderComponent(result, 'Other', Other, props, slots)}` - ); - }, - propagation: 'self' - }); + return createComponent({ + factory(result, props, slots) { + return createHeadAndContent( + renderUniqueStylesheet(result, { + href: "/path/to/these/styles.css", + }), + renderTemplate`${renderComponent(result, "Other", Other, props, slots)}` + ); + }, + propagation: "self", + }); } ``` @@ -334,10 +337,12 @@ This means that this component can be used anywhere, such as in layout component ## A new `renderEntryMap` -The Content Collections proposal [presented a new manifest](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0027-content-collections.md) generated from `src/content`, stored in a `.astro` directory as a cache. We expect `renderEntry` to add a lazy `import.meta.glob` call (see background) so we can avoid loading each module until used. +The Content Collections proposal [presented a new manifest](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0027-content-collections.md) generated from `src/content`, stored in a `.astro` directory as a cache. We expect `renderEntry` to add a lazy `import.meta.glob` call (see background) so we can avoid loading each module until used. ```tsx -export const renderEntryMap = import.meta.glob('src/content/**/*.{md,mdx}', { query: { SPECIAL_FLAG: true } }); +export const renderEntryMap = import.meta.glob("src/content/**/*.{md,mdx}", { + query: { SPECIAL_FLAG: true }, +}); ``` # Testing strategy @@ -359,10 +364,10 @@ The main alternative to `renderEntry` would be modifying the internals of `Astro # Adoption strategy -[See content Collections proposal](https://github.com/withastro/rfcs/blob/content-schemas/proposals/0027-content-collections.md#adoption-strategy) +[See content Collections proposal](https://github.com/withastro/roadmap/blob/content-schemas/proposals/0027-content-collections.md#adoption-strategy) # Unresolved questions - Can and **should** we merge `renderEntry`'s behavior into `getCollection`? i.e. making the `Content` component available on all content entries as an async function, rather than using a separate import? - Can `renderEntry` make importing content more performant? i.e. can we explore other avenues to avoid loading content until absolutely necessary, speeding up our MDX processing time to more closely match Markdown? -- Is injecting into compiled code [as presented here](#detailed-design) the best approach for an MVP? Or could our compiler understand `.astro` as a “reserved” import that could modify compiled output safely? \ No newline at end of file +- Is injecting into compiled code [as presented here](#detailed-design) the best approach for an MVP? Or could our compiler understand `.astro` as a “reserved” import that could modify compiled output safely? From a871634aa338d4977882afb933e6cc13b064d272 Mon Sep 17 00:00:00 2001 From: Michael Rienstra <mrienstra@gmail.com> Date: Sun, 29 Jan 2023 19:22:42 -0800 Subject: [PATCH 144/300] `rfcs` top-level directory --> `proposals` --- .github/PULL_REQUEST_TEMPLATE/rfc.yml | 2 +- stage-3--rfc-template.md | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/rfc.yml b/.github/PULL_REQUEST_TEMPLATE/rfc.yml index 02caf631..08f3e2ff 100644 --- a/.github/PULL_REQUEST_TEMPLATE/rfc.yml +++ b/.github/PULL_REQUEST_TEMPLATE/rfc.yml @@ -33,6 +33,6 @@ body: description: | Link to a GitHub-rendered version of your RFC, e.g. You can find this link by navigating to this file on your branch. - placeholder: "https://github.com/<USERNAME>/rfcs/blob/<BRANCH>/rfcs/my-proposal.md" + placeholder: "https://github.com/<USERNAME>/rfcs/blob/<BRANCH>/proposals/my-proposal.md" validations: required: true diff --git a/stage-3--rfc-template.md b/stage-3--rfc-template.md index 5e367d45..3565f36a 100644 --- a/stage-3--rfc-template.md +++ b/stage-3--rfc-template.md @@ -1,11 +1,11 @@ -<!-- +<!-- Note: You are probably looking for `stage-1--discussion-template.md`! This template is reserved for anyone championing an already-approved proposal. - - Community members who would like to propose an idea or feature should begin + + Community members who would like to propose an idea or feature should begin by creating a GitHub Discussion. See the repo README.md for more info. - To use this template: create a new, empty file in the repo under `rfcs/${ID}.md`. + To use this template: create a new, empty file in the repo under `proposals/${ID}.md`. Replace `${ID}` with the official accepted proposal ID, found in the GitHub Issue of the accepted proposal. --> @@ -33,13 +33,13 @@ It can be useful to illustrate your RFC as a user problem in this section. # Goals -A **concise, bulleted-list** outlining the intended goals of this RFC. +A **concise, bulleted-list** outlining the intended goals of this RFC. - What are the exact problems that you are trying to solve with this RFC? - Separate these goals from the solution that you will outline below. - If this RFC isn't approved, these goals can be used to develop alternative solutions. -# Non-Goals +# Non-Goals A **concise, bulleted-list** outlining anything intentionally left out of this RFC: @@ -66,7 +66,7 @@ cases that will be added to cover all of the ways this feature might be used. # Drawbacks -Why should we *not* do this? Please consider: +Why should we _not_ do this? Please consider: - Implementation cost, both in term of code size and complexity. - Whether the proposed feature can be implemented in user space. @@ -92,4 +92,4 @@ Please consider: # Unresolved Questions Optional, but suggested for first drafts. -What parts of the design are still to be determined? \ No newline at end of file +What parts of the design are still to be determined? From c2e21f2628fd68682ee7809ee37d90125fb91630 Mon Sep 17 00:00:00 2001 From: Michael Rienstra <mrienstra@gmail.com> Date: Sun, 29 Jan 2023 19:23:50 -0800 Subject: [PATCH 145/300] `rfcs` --> `roadmap` --- .github/PULL_REQUEST_TEMPLATE/rfc.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/rfc.yml b/.github/PULL_REQUEST_TEMPLATE/rfc.yml index 08f3e2ff..0b8fa36c 100644 --- a/.github/PULL_REQUEST_TEMPLATE/rfc.yml +++ b/.github/PULL_REQUEST_TEMPLATE/rfc.yml @@ -33,6 +33,6 @@ body: description: | Link to a GitHub-rendered version of your RFC, e.g. You can find this link by navigating to this file on your branch. - placeholder: "https://github.com/<USERNAME>/rfcs/blob/<BRANCH>/proposals/my-proposal.md" + placeholder: "https://github.com/<USERNAME>/roadmap/blob/<BRANCH>/proposals/my-proposal.md" validations: required: true From fa022383a24f4632d1a48943d707987f86678ddb Mon Sep 17 00:00:00 2001 From: Michael Rienstra <mrienstra@gmail.com> Date: Sun, 29 Jan 2023 19:39:00 -0800 Subject: [PATCH 146/300] Update old links as needed --- proposals/0001-style-unification.md | 2 +- proposals/0013-set-html.md | 2 +- proposals/0016-style-script-defaults.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0001-style-unification.md b/proposals/0001-style-unification.md index 7a7d2cb8..5f1e0534 100644 --- a/proposals/0001-style-unification.md +++ b/proposals/0001-style-unification.md @@ -31,7 +31,7 @@ We also need to make sure that our CSS support works with static builds. CSS can - Must point to a file in `public/` or an external CSS URL. This will not be bundled/optimized. - Must be nested inside of a `<head>` element. -Note: See the [local directive RFC](https://github.com/withastro/roadmap/blob/build-performance-rfc/active-rfcs/0000-build-performance.md#local-directive) for another alternative on referencing a `src/` file that would be processed/built/bundled. +Note: See the [local directive RFC](https://github.com/withastro/roadmap/blob/aa1229026165709918cc732f2cee6a1d6adfa799/active-rfcs/0000-build-performance.md#local-directive) for another alternative on referencing a `src/` file that would be processed/built/bundled. #### 2. ❌ `<link rel="stylesheet" href={xUrl}> // import xUrl from './X.css?url';` diff --git a/proposals/0013-set-html.md b/proposals/0013-set-html.md index b57d0234..e25d37a5 100644 --- a/proposals/0013-set-html.md +++ b/proposals/0013-set-html.md @@ -256,4 +256,4 @@ However, the Astro documentation should push this as _the blessed approach_ for # Unresolved questions -See [**Scenario H**](https://github.com/withastro/roadmap/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-h-element-sethtmlcontent---element) and [**Scenario J**](https://github.com/withastro/roadmap/blob/set-innerhtml/active-rfcs/0000-set-html.md#scenario-j-script-or-style-usage) in **Detailed Design**. +See [**Scenario H**](https://github.com/withastro/roadmap/blob/main/proposals/0013-set-html.md#scenario-h-element-sethtmlcontent---element) and [**Scenario J**](https://github.com/withastro/roadmap/blob/main/proposals/0013-set-html.md#scenario-j-script-or-style-usage) in **Detailed Design**. diff --git a/proposals/0016-style-script-defaults.md b/proposals/0016-style-script-defaults.md index d477f538..47d9a666 100644 --- a/proposals/0016-style-script-defaults.md +++ b/proposals/0016-style-script-defaults.md @@ -84,7 +84,7 @@ If any script tag has an attribute (ex: `type="module"`, `type="text/partytown"` A few other attempts have been made at finalizing this API: - https://github.com/withastro/roadmap/discussions/65 -- https://github.com/withastro/roadmap/blob/style-script-rfc/active-rfcs/0000-style-script-behavior.md#script +- https://github.com/withastro/roadmap/blob/main/proposals/0008-style-script-behavior.md#script - https://github.com/withastro/roadmap/blob/6015b4da8c95258b2136d61d6a6290c7ca989f5a/active-rfcs/0000-component-tag.md There is a clear need to address this inconsistency but we do not have a clear path forward yet. From 8fe397a93c58149470cc03c60ba3c6aee0876a11 Mon Sep 17 00:00:00 2001 From: Michael Rienstra <mrienstra@gmail.com> Date: Sun, 29 Jan 2023 19:41:51 -0800 Subject: [PATCH 147/300] Whoops, forgot to save `README.md` changes... --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index a700e787..5d535c2f 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,9 @@ **Goal:** Unstructured, low-friction conversations on ideas and improvements to Astro. Useful for gathering early feedback and gauging interest with the community and maintainers. -**Requirements:** None! To suggest an improvement, [create a new Discussion](https://github.com/withastro/rfcs/discussions) using our (completely optional) [proposal template](stage-1--discussion-template.md?plain=1). +**Requirements:** None! To suggest an improvement, [create a new Discussion](https://github.com/withastro/roadmap/discussions) using our (completely optional) [proposal template](stage-1--discussion-template.md?plain=1). -**Location:** GitHub Discussions [(see all open proposals).](https://github.com/withastro/rfcs/discussions) The Astro Discord channel `#feedback-ideas` can also be used to throw an idea out for quick initial feedback, but be warned that chat is short-lived and not designed for longer-lived discussion. +**Location:** GitHub Discussions [(see all open proposals).](https://github.com/withastro/roadmap/discussions) The Astro Discord channel `#feedback-ideas` can also be used to throw an idea out for quick initial feedback, but be warned that chat is short-lived and not designed for longer-lived discussion. ## Stage 2: Accepted Proposal @@ -28,7 +28,7 @@ **Requirements:** An existing proposal (Stage 1). In addition, a proposal is more likely to be accepted if it is detailed and well thought-out, can demonstrate community interest, has at least one champion volunteer, and has buy-in/interest from Astro maintainer(s). -**Location:** GitHub Issues [(see all accepted proposals).](https://github.com/withastro/rfcs/issues) +**Location:** GitHub Issues [(see all accepted proposals).](https://github.com/withastro/roadmap/issues) **What to Expect:** A proposal reaches this stage (aka "is accepted") during a meeting with Maintainers and TSC, following our existing [RFC Proposal](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#voting-rfc-proposals) voting process. @@ -44,7 +44,7 @@ A stale, accepted proposal can be removed (rejected after a previous acceptance) **Requirements:** An accepted proposal (Stage 2) and a proposal champion to author and implement the RFC. -**Location:** GitHub Pull Requests [(see all in-progress RFCs)](https://github.com/withastro/rfcs/pulls) [(see all finished RFCs)](https://github.com/withastro/rfcs/tree/main/rfcs) +**Location:** GitHub Pull Requests [(see all in-progress RFCs)](https://github.com/withastro/roadmap/pulls) [(see all finished RFCs)](https://github.com/withastro/roadmap/tree/main/proposals) **What to Expect:** To create an RFC for an already-accepted proposal, the proposal champion must use our [`stage-3--rfc-template.md`](stage-3--rfc-template.md?plain=1) RFC template in the repo. The initial sections of the RFC template should be copy-pasted from the the accepted proposal (they match 1:1). All remaining sections are left for the champion to complete with the implementation and tradeoff details of the RFC. @@ -62,4 +62,4 @@ Final RFC approval happens by vote, following our existing [RFC Proposal](https: **Prior Art / Special Thanks** -This process is an amalgamation of [Remix's Open Development process](https://remix.run/blog/open-development) and our previous [RFC process](https://github.com/withastro/rfcs/blob/78b736c28fe487ad02ec76bb038ad1087c471057/README.md), which had been based on the RFC processeses of the Vue, React, Rust, and Ember projects. +This process is an amalgamation of [Remix's Open Development process](https://remix.run/blog/open-development) and our previous [RFC process](https://github.com/withastro/roadmap/blob/78b736c28fe487ad02ec76bb038ad1087c471057/README.md), which had been based on the RFC processeses of the Vue, React, Rust, and Ember projects. From 6bb87c83e88a17705ef96d1f7fe9b63d3b8ab786 Mon Sep 17 00:00:00 2001 From: Michael Rienstra <mrienstra@gmail.com> Date: Sun, 29 Jan 2023 20:23:45 -0800 Subject: [PATCH 148/300] Update status as per https://github.com/withastro/astro/pull/3411#issuecomment-1407671566 --- proposals/0022-frontmatter-plugins.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/proposals/0022-frontmatter-plugins.md b/proposals/0022-frontmatter-plugins.md index a9248494..812cf136 100644 --- a/proposals/0022-frontmatter-plugins.md +++ b/proposals/0022-frontmatter-plugins.md @@ -1,6 +1,9 @@ - Start Date: 2022-05-20 -- Reference Issues: https://github.com/withastro/astro/pull/3411 -- Implementation PR: <!-- leave empty --> +- Reference Issues: + - https://github.com/withastro/astro/pull/3411 + - https://github.com/withastro/astro/issues/5099 +- Implementation PR: https://github.com/withastro/astro/pull/5687 +- Docs: https://docs.astro.build/en/guides/markdown-content/#modifying-frontmatter-programmatically ([permalink](https://github.com/withastro/docs/blob/3da9e4a/src/pages/en/guides/markdown-content.mdx#modifying-frontmatter-programmatically)) # Summary From 52781e85ccd4b11218a23ad2884f96f4dabb3011 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Fri, 24 Feb 2023 18:47:00 +0100 Subject: [PATCH 149/300] feat(rfc): Core image story --- proposals/0030-core-image-story.md | 402 +++++++++++++++++++++++++++++ 1 file changed, 402 insertions(+) create mode 100644 proposals/0030-core-image-story.md diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md new file mode 100644 index 00000000..58ddd6e3 --- /dev/null +++ b/proposals/0030-core-image-story.md @@ -0,0 +1,402 @@ +- Start Date: 2023-21-02 +- Reference Issues: #5202 (and all the issues about `@astrojs/image`, really) +- Implementation PR: https://github.com/withastro/astro/pull/6344 + +# Summary + +This RFC aims to simplify the usage, optimization and resizing of images in Astro. + +# Nomenclature + +- **CLS**: Content Layout Shift. A score determining how much content shifted on your page during loading, you want to be as close to 0 (or "no CLS") as possible. + +# Example + +> Note: All the `src` paths in the exemples below represent the final paths at build. During development, those paths may be different due to implementations details. + +> Note 2: All the results below, unless specified otherwise, are done using the default image services provided with Astro. + +## Image Component + +### Image located in `src`, in an Astro file + +```astro +--- +import { Image } from 'astro:assets'; +import myImage from "../assets/my_image.png"; // Image is 1600x900 +--- + +<Image src={myImage} alt="..." /> +``` + +#### Result + +```html +<!-- Image is optimized, proper attributes are enforced --> +<img + src="/_astro/my_image.hash.webp" + width="1600" + height="900" + decoding="async" + loading="lazy" + alt="..." +/> +``` + +### Remote images + +```astro +--- +import { Image } from "astro:image"; +// Image is 1920x1080 +--- + +<Image src="https://example.com/image.png" alt="..." /> +<!-- ERROR! `width` and `height` are required --> + +<Image src="https://example.com/image.png" width={1280} alt="..." /> +<!-- ERROR! `height` is required --> + +<Image src="https://example.com/image.png" width={1280} height={720} alt="..." /> +``` + +#### Result + +```html +<!-- Remote images are not optimized or resized. Nonetheless, you still benefit from the enforced attributes and the guarantee of no CLS, albeit manually. --> +<img + src="https://example.com/image.png" + decoding="async" + loading="lazy" + width="1280" + height="720" + alt="..." +/> +``` + +## Image in Markdown + +```markdown +! [My article cover](./cover.png) +^ Image is 1280x720 +``` + +### Result + +```html +<!-- Image is optimized, proper attributes are enforced --> +<img + src="/_astro/cover.hash.webp" + width="1280" + height="720" + decoding="async" + loading="lazy" + alt="My article cover" +/> +``` + +Remote images are handled as in vanilla Markdown and have no special behaviour. + +## JavaScript API for using optimized images + +```astro +--- +import { getImage } from "astro:assets"; +import image from "../assets/image.png"; // Image is 1920x1080 + +const myImage = await getImage({ src: image, width: 1280, height: 720 }); +--- + +<img src={myImage.src} {...myImage.attributes} /> +``` + +### Result + +```html +<img + src="/_astro/image.hash.webp" + width="1280" + height="720" + loading="lazy" + decoding="async" +/> +``` + +## Defining a new image service + +### Astro Config + +```ts +import { defineConfig } from "astro/config"; + +// https://astro.build/config +export default defineConfig({ + image: { + service: "your-entrypoint", // 'astro/image/services/squoosh' | astro/image/services/sharp | string + }, +}); +``` + +### Export for local services + +```ts +import type { LocalImageService } from "astro"; + +const service: LocalImageService = { + getURL(options: ImageTransform) { + return `/my_super_endpoint_that_transforms_images?w=${options.width}`; + }, + parseURL(url: URL) { + return { + width: url.searchParams.get("w"), + }; + }, + transform(options: ImageTransform) { + const { buffer } = mySuperLibraryThatEncodesImages(options); + + return { + data: buffer, + format: options.format, + }; + }, +}; + +export default service; +``` + +### Export for external services + +```ts +import type { ExternalImageService } from "astro"; + +const service: ExternalImageService = { + getURL(options: ImageTransform) { + return `https://mysupercdn.com/${options.src}?q=${options.quality}`; + }, +}; + +export default service; +``` + +The kind of service exposed is completely invisible to users, the user API is always the same: the `Image` component and `getImage`. Refer to previous examples for usage. + +### Additional method + +```ts +getHTMLAttributes(options: ImageTransform) { + return { + width: options.width, + height: options.height + // ... + } +} +``` + +This method is available on both local and external services. + +# Background & Motivation + +The currently available `@astrojs/image` is great, however, users ultimately find it very confusing for multiple reasons: + +- Error messages are confusing and unhelpful +- Types historically haven't been super great +- Missing documentation +- Due to not being a core integration, it requires manual setup +- Partly due to the previous points, but also for other reasons, it wasn't always clear for users how the integration behaved. +- Hard to use in Markdown / MDX. + +In this RFC, we'd like to outline a plan / API for a core story for images. The main motivation is to create a story that feels fitting to the Astro core, in that it's easy to understand, intuitive to use, and extendable while also being kind and respectful of the user. + +# Goals + +- Make an image component that is easy to use and intuitive to understand. +- Make it easy to use optimized images in Markdown, MDX and future formats. +- Good, core-worthy, DX with awesome error messages, good types, good documentation +- No more integration to install! + +# Non-Goals of this RFC + +- Advanced usage in Markdown (ability to set width, height, quality etc) +- Using optimized images inside client-side framework components +- Automatic generation of `srcset` in the integrated services +- Automatic `loading="eager"` for above the fold images +- Placeholders generation +- Background generation +- Picture component +- Optimizing & resizing remote images +- `.svg` support + +To be clear, we hope to tackle many of those points in the future, in separate, more precise RFCs. Images are hard. + +# Detailed Design + +## `astro:assets` + +A new virtual module will be exported from a Vite plugin (similar to `astro:content`) exposing the different tools the user can access. + +- `Image` (see [[#Image component]]) +- `getImage` (see [[#JavaScript API]]) +- `getImageService` (see [[#Image Services]]) + +We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive that more things might get exposed from there over time. (maybe a crossover with a way to create generic assets for users?) + +## `src/assets` + +A new reserved folders for assets. Assets do not have to live in this folder, however putting them in this folder will unlock several benefits. Notably, we consider this folder to be a safe place for git-based CMSes to put any uploaded assets in. + +Through an included alias `~/assets` pointing to `src/assets`, it'll be easy for users to refer to any uploaded assets there. + +### Content Collections integration + +It is fairly common for one of the property of a Markdown piece of content to need to be a reference to an asset (think, cover image for an article, picture for an author etc). + +In tandem with the `src/assets` folder, we'd like to introduce a way for users to specify that a specific property needs to refer to a valid asset from the `src/assets` folder: + +```ts +import { asset, defineCollection, z } from "astro:content"; + +const blogCollection = defineCollection({ + schema: z.object({ + title: z.string(), + image: asset({ width: 1120, height: 1120 }), + }), +}); +``` + +Additionally, assets referred this way will be transformed automatically to the same shape as if the image was imported (see section below.), so they're ready to be used optimally. However, optimization / resizing is left to the user to do using the JavaScript API if they desire to. + +## New ESM shape for images imports + +Currently, importing images in Astro returns a simple `string` with the path of the image. The `@astrojs/image` integration and this RFC enhance this by instead returning the following shape: `{src: string, width: number, height: number, format: string}`. + +This shape has multiple benefits: + +- It allows users to easily construct `img` tags with no CLS, without needing to opt-in the entire `Image` pipeline if they don't require it: + +```astro +--- +import image from "../my_image.png"; +--- + +<img src={image.src} width={image.width} height={image.height} /> +``` + +- This shapes gives all the information needed to build your own image integration. Depending on the transformer used, you'll often need the original metadata (ex: width and height) to resize the image while keeping the aspect ratio. Or, you might need a different encoder depending on the file format. With this output, all of this is given without needing further steps. + +## Image component + +A component that outputs a simple `<img> `tag with all the required attributes needed to show the image. + +### Properties + +#### width and height + +Those properties are used for resizing the image and ensuring there's no CLS. In most cases, the `width` and `height` would be automatically inferred from the source file and not passing them would just result in the same dimensions. + +The only time those properties are required is for remote images. + +#### format + +`format` is used to set the resulting wanted format for the image. + +Default is left to the services to choose. But for the services exposed by Astro, this property is always optional and is set to `webp`. + +#### quality + +`quality` inform the quality to use when transforming the image. It can either be a number from `0` to `100`, or a preset from `low`, `medium`, `high`, `max`. Specific implementation here is left to the image service used, as they do not necessarily all abide by the same rules. + +Default is left to the services to choose. But for the services exposed by Astro, this property is always optional. + +#### alt + +`alt` is a required property in all cases when using the component. It can be explicitly set to `""`, for cases where an alt-text is not required. + +### Facts + +- For the services exposed by Astro: For ESM images, the only required property is `src`. Remote images (ex: `http://example.com/image.png`, `/image.png` or `${import.meta.env.BASE_URL}/image.png`) however require `width` and `height` to be set manually. +- `src` can be a dynamic import, but be aware that it must respect [Vite's limitations on dynamic imports](https://vitejs.dev/guide/features.html#dynamic-import) + +## JavaScript API + +A `getImage` function taking the same parameters as the image component is also supported, for use cases inside the frontmatter or outside `img` tag (ex: `background-image`) + +This function returns an object with all the properties needed to use / show an image. + +```ts +// Example interface describing the content of `myImage` +interface getImageResult { + // Contain the original options passed to `getImage` + options: Record<string, any>; + // Contain a path you can use to render the image + src: string; + // Contain additional HTML attributes needed to render the image (ex: `width`, `height`, `loading`) + attributes: Record<string, any>; +} +``` + +> Specific types not necessarily accurate to implementation. A more or less open-ended type for `options` and `attributes` is nonetheless required, as different services are allowed to return different things (including non-standard HTML attributes.) + +## Image Services + +By default, Astro will ship with two services can users can choose to transform their images: Squoosh, and Sharp. Both services more or less offer the same results and mostly differ in where they can run. Squoosh will be the default, because albeit slower than Sharp, it supports more platforms. + +Keeping in line with how you can extend Astro in various ways (remark, rehype, integrations etc), it's possible for users to create their own services. + +Two types of services exists: Local and External. + +- Local services handle the image transformation directly at build in SSG / runtime in dev / SSR. You can think of those as wrapper around librairies like Sharp, ImageMagick or Squoosh. +- External services point to URLs and can be used for adding support for services such as Cloudinary, Vercel or any RIAPI-compliant server. + +Services definitions take the shape of an exported default object with various methods ("hooks") used to create all the required properties. The major difference, API-wise, between Local and External services is the presence of a `transform` method doing the actual transformation. + +The different methods available are the following: + +**Required methods** + +- `getURL(options: ImageTransform): string` + - For local services, return the URL of the endpoint managing your transformation (in SSR and dev). + - For external services, return the final URL of the image. + - `options` contain the parameters passed by the user + +**Required for Local services only** + +- `parseURL(url: URL): ImageTransform` + - For SSR and dev, parses the generated URLs by `getURL` back into an ImageTransform to be used by `transform`. +- `transform(buffer: Buffer, options: ImageTransform): { data: Buffer, format: OutputFormat }` + - Transform and return the image. It is necessary to return a `format` to ensure that the proper MIME type is served to users. + +Ultimately, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform`. Those two methods are exposed as convention and used by the two base services, but your endpoint is free to achieve this in a different way if needed. + +**Optional** + +- `getHTMLAttributes(options: ImageTransform): Record<string, any>` + - Return all additional attributes needed to render the image in HTML. For instance, you might want to return a specific `class` or `style`, or `width` and `height`. + +Overall, it's important to remember that 99% of users won't create services, especially local ones. Adapters (think: Vercel's Image Optimization) and integrations (ex: A Cloudinary integration) will supply and configure services for the users. + +# Testing Strategy + +Much like the current `@astrojs/image` integration, this can be tested in many ways, ensuring full coverage of the feature set: + +- Each of the core methods exposed (ex: `getImage`) and the methods they rely on (ex: `imageService.getURL`) can be tested mostly independently through unit tests. +- Integration tests can be developed to ensure the images are properly built in static mode. +- E2E tests can be developed to make sure the images are properly served in dev and SSR. + +Certain parts can be hard to fully E2E tests, such as "Was this image really generated with a quality of 47?", nonetheless, we can test that we at least provided the correct parameters down the chain. + +Overall, I do not expect this feature to be particularly hard, as the `@astrojs/image` testing suite has already proven to work correctly. + +# Drawbacks + +- Images are complicated! We've had many attempts in the community to build a perfect image components, but none of the current offering solved everyone's use case. Some people are also (understandably) cautious of using a third-party integration for this. +- Part of this is breaking! We'll offer flags and write migration guides. However, it's still possible that we'll get reports about behaviour changing unexpectedly / the experience being worse because it needs to be opt-in. + +# Alternatives + +While I believe there may be alternatives in the technical sense, I think it is required for us to eventually have a solid story for images. Have you ever visited a website with no images? Me neither. This is especially true for content websites. + +# Adoption strategy + +For the `Image` component, the JS API and the Content Collection integration, adoption is completely opt-in. You don't use it, it's not included and your image are treated normally. + +The ESM shape change and Markdown integration are both breaking, as such both will have to be under a flag until Astro 3.0. The types will be configured automatically for the user similar to how Content Collections update your `env.d.ts` to avoid typing issues in the editor. From 04191da6c47100165081bc36f590273e523f62f2 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Fri, 24 Feb 2023 18:54:40 +0100 Subject: [PATCH 150/300] fix(rfc): Update anchor links --- proposals/0030-core-image-story.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 58ddd6e3..2cfec59e 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -234,9 +234,9 @@ To be clear, we hope to tackle many of those points in the future, in separate, A new virtual module will be exported from a Vite plugin (similar to `astro:content`) exposing the different tools the user can access. -- `Image` (see [[#Image component]]) -- `getImage` (see [[#JavaScript API]]) -- `getImageService` (see [[#Image Services]]) +- `Image` (see [Image Component](#image-component-1)) +- `getImage` (see [JavaScript API](#javascript-api)) +- `getImageService` (see [Image Services](#image-services)) We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive that more things might get exposed from there over time. (maybe a crossover with a way to create generic assets for users?) From 2f3650b4826c5e78787968508e8388e63d055444 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Fri, 24 Feb 2023 18:55:14 +0100 Subject: [PATCH 151/300] fix(rfc): Remove maybe section --- proposals/0030-core-image-story.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 2cfec59e..33a2d59e 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -238,7 +238,7 @@ A new virtual module will be exported from a Vite plugin (similar to `astro:cont - `getImage` (see [JavaScript API](#javascript-api)) - `getImageService` (see [Image Services](#image-services)) -We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive that more things might get exposed from there over time. (maybe a crossover with a way to create generic assets for users?) +We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive that more things might get exposed from there over time. ## `src/assets` From 5e9b22bed95b8d7de5902f99ec4d08c109e60701 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Mon, 27 Feb 2023 15:45:15 +0000 Subject: [PATCH 152/300] astro check --watch proposal --- .gitignore | 5 ++ proposals/0031-astro-watch-check.md | 96 +++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 .gitignore create mode 100644 proposals/0031-astro-watch-check.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..298d2b7d --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +# ignore top-level vscode settings +.vscode/settings.json + +# ignore IDEA folders +.idea \ No newline at end of file diff --git a/proposals/0031-astro-watch-check.md b/proposals/0031-astro-watch-check.md new file mode 100644 index 00000000..6d0cf93d --- /dev/null +++ b/proposals/0031-astro-watch-check.md @@ -0,0 +1,96 @@ +<!-- + Note: You are probably looking for `stage-1--discussion-template.md`! + This template is reserved for anyone championing an already-approved proposal. + + Community members who would like to propose an idea or feature should begin + by creating a GitHub Discussion. See the repo README.md for more info. + + To use this template: create a new, empty file in the repo under `proposals/${ID}.md`. + Replace `${ID}` with the official accepted proposal ID, found in the GitHub Issue + of the accepted proposal. +--> + +- Start Date: 2023-02-27 +- Reference Issues: https://github.com/withastro/roadmap/issues/473 +- Implementation PR: https://github.com/withastro/astro/pull/6356 + +# Summary + +Add a `--watch` flag to `astro check` command. + +# Background & Motivation + +From the relevant issue: + +> In development mode there isn't a way to know if your .astro files contain no errors, +> aside from editor feedback. +> +> If you use astro check you will know about errors, but might not see them until CI unless you remember to run the command before pushing. + +# Goals + +- A way to check TS errors in .astro files during development workflow. + +# Non-Goals + +- Having errors reported in the UI. This could be a nice enhancement in the future but would come with some downsides (such as possibly slowing down dev) that we want to punt on. + +# Detailed Design + +The new feature will use the existing tools that `astro` has. `vite` exposes a [`watcher` option](https://vitejs.dev/config/server-options.html#server-watch) +that allows users to watch files and react accordingly. + +The suggested solution would be to be spawn a `vite` server instance, watch `.astro` files changes and +check for diagnostics. + +The output emitted will be the same, compared to `astro check`. + +```shell +astro check --watch +``` + +Will output: + +```block +13:46:46 [check] Checking files in watch mode +``` + +The rest of the output will be the same as the command `astro check`. + +# Testing Strategy + +This feature can be unit tested. The plan is to launch a process and listen to its `stdout` and `stderr`. + +This will allow to us inspect the output of the process and test with the correct assertions. + +Although, these kinds of tests are very brittle and unstable, because they require +listening to the output of a command, and this command might take a different amount of time based on the OS. + +In absence of "stable" unit tests, I only see manual testing as an alternative. + +# Drawbacks + +- this feature could be implemented in user-land, but it wouldn't offer the same DX to + developers; +- the command is a long-lived process, which means the user needs to kill the command + manually; + +# Alternatives + +I don't think there's a valid alternative, other than running `astro check` at every change, manually. + +Not having such a feature will impact the overall DX of users. + +Astro already has its own LSP and VSCode extension, but there might be users who prefer other IDEs, +but this means that users need to rely on IDEs and their LSP protocol. + +This proposal will offer a new tool! + +# Adoption strategy + +- offer a new preview release +- make a release with the flag called `--experimental-watch` and make semever-free + +# Unresolved Questions + +N/A From e6393f1ce22b3c4c1b5f922ed19918160589e5ab Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Tue, 28 Feb 2023 15:51:28 +0000 Subject: [PATCH 153/300] chore: feedback --- proposals/0031-astro-watch-check.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0031-astro-watch-check.md b/proposals/0031-astro-watch-check.md index 6d0cf93d..2c36748e 100644 --- a/proposals/0031-astro-watch-check.md +++ b/proposals/0031-astro-watch-check.md @@ -77,7 +77,8 @@ In absence of "stable" unit tests, I only see manual testing as an alternative. # Alternatives -I don't think there's a valid alternative, other than running `astro check` at every change, manually. +The user can use third party tools such as [wireit](https://github.com/google/wireit), although it would mean +that the user wouldn't benefit of future improvements around `astro check --watch`. Not having such a feature will impact the overall DX of users. @@ -89,7 +90,6 @@ This proposal will offer a new tool! # Adoption strategy - offer a new preview release -- make a release with the flag called `--experimental-watch` and make semever-free # Unresolved Questions From 73c4624c2fedd4a4b07e2d6a5615ce6fe8873d62 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Tue, 28 Feb 2023 20:47:30 +0100 Subject: [PATCH 154/300] chore: misc typo fixes --- proposals/0030-core-image-story.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 33a2d59e..0bb64d5f 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -12,7 +12,7 @@ This RFC aims to simplify the usage, optimization and resizing of images in Astr # Example -> Note: All the `src` paths in the exemples below represent the final paths at build. During development, those paths may be different due to implementations details. +> Note: All the `src` paths in the examples below represent the final paths at build. During development, those paths may be different due to implementations details. > Note 2: All the results below, unless specified otherwise, are done using the default image services provided with Astro. @@ -77,7 +77,7 @@ import { Image } from "astro:image"; ## Image in Markdown ```markdown -! [My article cover](./cover.png) +![My article cover](./cover.png) ^ Image is 1280x720 ``` From 4ef1818f6c97e9e869ba63d2200103c9459948ff Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Wed, 1 Mar 2023 12:12:43 +0100 Subject: [PATCH 155/300] fix(rfc): Update with feedback --- proposals/0030-core-image-story.md | 41 ++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 0bb64d5f..6e582e06 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -47,7 +47,7 @@ import myImage from "../assets/my_image.png"; // Image is 1600x900 ```astro --- -import { Image } from "astro:image"; +import { Image } from "astro:assets"; // Image is 1920x1080 --- @@ -95,7 +95,7 @@ import { Image } from "astro:image"; /> ``` -Remote images are handled as in vanilla Markdown and have no special behaviour. +Remote images are handled as in vanilla Markdown and have no special behavior. ## JavaScript API for using optimized images @@ -132,12 +132,12 @@ import { defineConfig } from "astro/config"; // https://astro.build/config export default defineConfig({ image: { - service: "your-entrypoint", // 'astro/image/services/squoosh' | astro/image/services/sharp | string + service: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string }, }); ``` -### Export for local services +### Local services API ```ts import type { LocalImageService } from "astro"; @@ -164,7 +164,7 @@ const service: LocalImageService = { export default service; ``` -### Export for external services +### External services API ```ts import type { ExternalImageService } from "astro"; @@ -224,6 +224,8 @@ In this RFC, we'd like to outline a plan / API for a core story for images. The - Background generation - Picture component - Optimizing & resizing remote images +- Ability to choose a different image service per image +- Remote patterns for limiting the usage of remote images to specific domains - `.svg` support To be clear, we hope to tackle many of those points in the future, in separate, more precise RFCs. Images are hard. @@ -344,7 +346,7 @@ Keeping in line with how you can extend Astro in various ways (remark, rehype, i Two types of services exists: Local and External. -- Local services handle the image transformation directly at build in SSG / runtime in dev / SSR. You can think of those as wrapper around librairies like Sharp, ImageMagick or Squoosh. +- Local services handle the image transformation directly at build in SSG / runtime in dev / SSR. You can think of those as wrapper around libraries like Sharp, ImageMagick or Squoosh. - External services point to URLs and can be used for adding support for services such as Cloudinary, Vercel or any RIAPI-compliant server. Services definitions take the shape of an exported default object with various methods ("hooks") used to create all the required properties. The major difference, API-wise, between Local and External services is the presence of a `transform` method doing the actual transformation. @@ -372,7 +374,30 @@ Ultimately, it is up to the local endpoint (that `getURL` points to) to call bot - `getHTMLAttributes(options: ImageTransform): Record<string, any>` - Return all additional attributes needed to render the image in HTML. For instance, you might want to return a specific `class` or `style`, or `width` and `height`. -Overall, it's important to remember that 99% of users won't create services, especially local ones. Adapters (think: Vercel's Image Optimization) and integrations (ex: A Cloudinary integration) will supply and configure services for the users. +### User configuration + +User can choose the image service to use through their `astro.config.mjs` file. The config takes the following form: + +```ts +import { defineConfig } from "astro/config"; + +// https://astro.build/config +export default defineConfig({ + image: { + service: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string + }, +}); +``` + +At this time, it is not possible to override this on a per-image basis (see [Non goals of this RFC](#non-goals-of-this-rfc)). + +The `image.service` shape was chosen on purpose, for the future situation where multiple settings will be available under `image`. + +### Note + +Overall, it's important to remember that 99% of users won't create services, especially local ones. In addition to the services directly provided with Astro, third party packages can supply services for the users. + +It's easy to imagine, for example, a `cloudinary-astro` package exposing a service. Or, the `@astrojs/vercel` adapter exposing a service using Vercel's Image Optimization API that user could use through `service: '@astrojs/vercel/image`. # Testing Strategy @@ -389,7 +414,7 @@ Overall, I do not expect this feature to be particularly hard, as the `@astrojs/ # Drawbacks - Images are complicated! We've had many attempts in the community to build a perfect image components, but none of the current offering solved everyone's use case. Some people are also (understandably) cautious of using a third-party integration for this. -- Part of this is breaking! We'll offer flags and write migration guides. However, it's still possible that we'll get reports about behaviour changing unexpectedly / the experience being worse because it needs to be opt-in. +- Part of this is breaking! We'll offer flags and write migration guides. However, it's still possible that we'll get reports about behavior changing unexpectedly / the experience being worse because it needs to be opt-in. # Alternatives From c7113d5b567c71783d9d394c3add650a8a92e6a6 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Wed, 1 Mar 2023 13:02:32 +0100 Subject: [PATCH 156/300] fix(rfc): Adjust to feedback --- proposals/0030-core-image-story.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 6e582e06..6639d3cd 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -26,6 +26,7 @@ import { Image } from 'astro:assets'; import myImage from "../assets/my_image.png"; // Image is 1600x900 --- +<!-- `alt` is mandatory on the Image component --> <Image src={myImage} alt="..." /> ``` @@ -340,16 +341,16 @@ interface getImageResult { ## Image Services -By default, Astro will ship with two services can users can choose to transform their images: Squoosh, and Sharp. Both services more or less offer the same results and mostly differ in where they can run. Squoosh will be the default, because albeit slower than Sharp, it supports more platforms. +By default, Astro will ship with two services that users can choose from to transform their images. Those two services are powered by [Squoosh](https://github.com/GoogleChromeLabs/squoosh) and [Sharp](https://sharp.pixelplumbing.com/) respectively. Both more or less offer the same results and mostly differ in performance and where they can run. Squoosh will be the default, because albeit slower than Sharp, it supports more platforms due to its WASM nature. -Keeping in line with how you can extend Astro in various ways (remark, rehype, integrations etc), it's possible for users to create their own services. +Keeping in line with how you can extend Astro in various ways (remark and rehype plugins, integrations etc), it's possible for users to create their own services. Two types of services exists: Local and External. - Local services handle the image transformation directly at build in SSG / runtime in dev / SSR. You can think of those as wrapper around libraries like Sharp, ImageMagick or Squoosh. - External services point to URLs and can be used for adding support for services such as Cloudinary, Vercel or any RIAPI-compliant server. -Services definitions take the shape of an exported default object with various methods ("hooks") used to create all the required properties. The major difference, API-wise, between Local and External services is the presence of a `transform` method doing the actual transformation. +Services definitions take the shape of an exported default object with various required methods ("hooks") used to create all the required properties. The major difference, API-wise, between Local and External services is the presence of a `transform` method doing the actual transformation. The different methods available are the following: @@ -365,9 +366,9 @@ The different methods available are the following: - `parseURL(url: URL): ImageTransform` - For SSR and dev, parses the generated URLs by `getURL` back into an ImageTransform to be used by `transform`. - `transform(buffer: Buffer, options: ImageTransform): { data: Buffer, format: OutputFormat }` - - Transform and return the image. It is necessary to return a `format` to ensure that the proper MIME type is served to users. + - Transform and return the image. It is necessary to return a `format` to ensure that the proper MIME type is served to users in development and SSR. -Ultimately, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform`. Those two methods are exposed as convention and used by the two base services, but your endpoint is free to achieve this in a different way if needed. +Ultimately, in development and SSR, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform` if wanted. `transform` however, is called during the build to create the final assets files. **Optional** From d625260ce2fc40a68841555a061f518d16101a7d Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Thu, 2 Mar 2023 13:13:27 +0100 Subject: [PATCH 157/300] fix(rfc): Update with feedback --- proposals/0030-core-image-story.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 6639d3cd..1521a83a 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -390,9 +390,11 @@ export default defineConfig({ }); ``` -At this time, it is not possible to override this on a per-image basis (see [Non goals of this RFC](#non-goals-of-this-rfc)). +#### Facts -The `image.service` shape was chosen on purpose, for the future situation where multiple settings will be available under `image`. +- At this time, it is not possible to override this on a per-image basis (see [Non goals of this RFC](#non-goals-of-this-rfc)). +- The `image.service` shape was chosen on purpose, for the future situation where multiple settings will be available under `image`. +- A different image service can be set depending on the current mode (dev or build) using traditional techniques such as `process.env.MODE` ### Note From b2730b98100469cfb04b288568979ef82bdbe18e Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Fri, 3 Mar 2023 14:16:03 +0100 Subject: [PATCH 158/300] chore: add flag name --- proposals/0030-core-image-story.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 1521a83a..f429a20e 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -427,4 +427,4 @@ While I believe there may be alternatives in the technical sense, I think it is For the `Image` component, the JS API and the Content Collection integration, adoption is completely opt-in. You don't use it, it's not included and your image are treated normally. -The ESM shape change and Markdown integration are both breaking, as such both will have to be under a flag until Astro 3.0. The types will be configured automatically for the user similar to how Content Collections update your `env.d.ts` to avoid typing issues in the editor. +The ESM shape change and Markdown integration are both breaking, as such both will have to be under a flag (`experimental.assets`) until Astro 3.0. The types will be configured automatically for the user similar to how Content Collections update your `env.d.ts` to avoid typing issues in the editor. From 5e0157784e76064113bea40e4ceb132802346175 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Fri, 3 Mar 2023 15:47:41 +0000 Subject: [PATCH 159/300] Apply suggestions from code review Co-authored-by: Erika <3019731+Princesseuh@users.noreply.github.com> Co-authored-by: Matthew Phillips <matthew@skypack.dev> --- proposals/0031-astro-watch-check.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/0031-astro-watch-check.md b/proposals/0031-astro-watch-check.md index 2c36748e..d2dae5b7 100644 --- a/proposals/0031-astro-watch-check.md +++ b/proposals/0031-astro-watch-check.md @@ -16,7 +16,7 @@ # Summary -Add a `--watch` flag to `astro check` command. +Add a `--watch` flag to the `astro check` command. # Background & Motivation @@ -66,13 +66,13 @@ This will allow to us inspect the output of the process and test with the correc Although, these kinds of tests are very brittle and unstable, because they require listening to the output of a command, and this command might take a different amount of time based on the OS. -In absence of "stable" unit tests, I only see manual testing as an alternative. +In addition to unit tests, the feature will be manually tested with demo projects. # Drawbacks -- this feature could be implemented in user-land, but it wouldn't offer the same DX to +- This feature could be implemented in userland, but it wouldn't offer the same DX to developers; -- the command is a long-lived process, which means the user needs to kill the command +- The command is a long-lived process, which means the user needs to terminate the command manually; # Alternatives @@ -89,7 +89,8 @@ This proposal will offer a new tool! # Adoption strategy -- offer a new preview release +- Create a preview release for user feedback. +- Release the feature in as a `minor` without experimental flags since it is a small improvement without many questions. # Unresolved Questions From ec93ab0835a5445760f3d9b0cf35292191a7414a Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Fri, 3 Mar 2023 15:56:18 +0000 Subject: [PATCH 160/300] address feedback --- proposals/0031-astro-watch-check.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/proposals/0031-astro-watch-check.md b/proposals/0031-astro-watch-check.md index d2dae5b7..dc3f27b7 100644 --- a/proposals/0031-astro-watch-check.md +++ b/proposals/0031-astro-watch-check.md @@ -37,11 +37,10 @@ From the relevant issue: # Detailed Design -The new feature will use the existing tools that `astro` has. `vite` exposes a [`watcher` option](https://vitejs.dev/config/server-options.html#server-watch) -that allows users to watch files and react accordingly. +The suggested solution will use [`chokidar`](https://www.npmjs.com/package/chokidar), a battle tested +file watcher for Node.js. -The suggested solution would be to be spawn a `vite` server instance, watch `.astro` files changes and -check for diagnostics. +The file watcher will ignore files inside `node_modules` by default and listen to changes to `.astro` files. The output emitted will be the same, compared to `astro check`. @@ -59,9 +58,8 @@ The rest of the output will be the same as the command `astro check`. # Testing Strategy -This feature can be unit tested. The plan is to launch a process and listen to its `stdout` and `stderr`. - -This will allow to us inspect the output of the process and test with the correct assertions. +Override the default logger with one that is possible to inspect, listen to the stream of messages emitted by the watcher +and make sure that the messages received are correct. Although, these kinds of tests are very brittle and unstable, because they require listening to the output of a command, and this command might take a different amount of time based on the OS. @@ -82,9 +80,6 @@ that the user wouldn't benefit of future improvements around `astro check --watc Not having such a feature will impact the overall DX of users. -Astro already has its own LSP and VSCode extension, but there might be users who prefer other IDEs, -but this means that users need to rely on IDEs and their LSP protocol. - This proposal will offer a new tool! # Adoption strategy From 521247bec6cc219b690eddc53f8b08c84bfa1073 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Mon, 6 Mar 2023 09:29:15 +0000 Subject: [PATCH 161/300] address feedback --- proposals/0031-astro-watch-check.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/proposals/0031-astro-watch-check.md b/proposals/0031-astro-watch-check.md index dc3f27b7..03756ec2 100644 --- a/proposals/0031-astro-watch-check.md +++ b/proposals/0031-astro-watch-check.md @@ -61,9 +61,6 @@ The rest of the output will be the same as the command `astro check`. Override the default logger with one that is possible to inspect, listen to the stream of messages emitted by the watcher and make sure that the messages received are correct. -Although, these kinds of tests are very brittle and unstable, because they require -listening to the output of a command, and this command might take a different amount of time based on the OS. - In addition to unit tests, the feature will be manually tested with demo projects. # Drawbacks From 497ac30054fef2d1e4ef53d53fb81aa94e83aa27 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Mon, 6 Mar 2023 11:22:45 +0100 Subject: [PATCH 162/300] fix(rfc): Update with feedback, no more automatic refine, asset() -> image() --- proposals/0030-core-image-story.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index f429a20e..bca7bbae 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -256,17 +256,30 @@ It is fairly common for one of the property of a Markdown piece of content to ne In tandem with the `src/assets` folder, we'd like to introduce a way for users to specify that a specific property needs to refer to a valid asset from the `src/assets` folder: ```ts -import { asset, defineCollection, z } from "astro:content"; +import { image, defineCollection, z } from "astro:content"; const blogCollection = defineCollection({ schema: z.object({ title: z.string(), - image: asset({ width: 1120, height: 1120 }), + image: image(), }), }); ``` -Additionally, assets referred this way will be transformed automatically to the same shape as if the image was imported (see section below.), so they're ready to be used optimally. However, optimization / resizing is left to the user to do using the JavaScript API if they desire to. +Image assets referred this way will be transformed automatically to the same shape as if the image was imported (see section below), as such they can be checked using Zod's [`refine`](https://zod.dev/?id=refine) or [`superRefine`](https://zod.dev/?id=superrefine) methods, for example: + +```ts +import { image, defineCollection, z } from "astro:content"; + +const blogCollection = defineCollection({ + schema: z.object({ + title: z.string(), + image: image().refine((img) => img.width === 1080, { + message: "Image must be 1080px wide", + }), + }), +}); +``` ## New ESM shape for images imports From 27eabe8b959e555143d2200c6cffcd68f6728f81 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Mon, 6 Mar 2023 16:36:09 +0000 Subject: [PATCH 163/300] change number --- .../{0031-astro-watch-check.md => 0030-astro-watch-check.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{0031-astro-watch-check.md => 0030-astro-watch-check.md} (100%) diff --git a/proposals/0031-astro-watch-check.md b/proposals/0030-astro-watch-check.md similarity index 100% rename from proposals/0031-astro-watch-check.md rename to proposals/0030-astro-watch-check.md From a7652670d567d421c1e0ae103e625686695891aa Mon Sep 17 00:00:00 2001 From: wulinsheng123 <409187100@qq.com> Date: Thu, 16 Mar 2023 19:52:33 +0800 Subject: [PATCH 164/300] spell out the full name (#524) Co-authored-by: wuls <linsheng.wu@beantechs.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5d535c2f..05b082b7 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ ## Stage 2: Accepted Proposal -**Goal:** Confirm proposal feasibility with Astro Maintainers and TSC. +**Goal:** Confirm proposal feasibility with Astro Maintainers and the [Technical Steering Committee (TSC)](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#technical-steering-committee-tsc). **Requirements:** An existing proposal (Stage 1). In addition, a proposal is more likely to be accepted if it is detailed and well thought-out, can demonstrate community interest, has at least one champion volunteer, and has buy-in/interest from Astro maintainer(s). From 51d7fd2fa42f7e23b11f904e03b63b5efc7b2770 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Tue, 28 Mar 2023 15:59:28 -0400 Subject: [PATCH 165/300] Scoping RFC --- proposals/style-scoping.md | 73 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 proposals/style-scoping.md diff --git a/proposals/style-scoping.md b/proposals/style-scoping.md new file mode 100644 index 00000000..bc1fed0d --- /dev/null +++ b/proposals/style-scoping.md @@ -0,0 +1,73 @@ +- Start Date: 2023-03-28 +- Reference Issues: https://github.com/withastro/roadmap/issues/540 +- Implementation PR: <!-- leave empty --> + +# Summary + +Make the scoping strategy be configurable via a new config option, and add a strategy that uses class names in the selector, giving all Astro styles a +1 in specificity. + +# Example + +A new configuration option available: + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + scopedStyleStrategy: 'class' +}); +``` + +The valid options for this config are: + +* __where__: The current default which uses `:where(.hash)` in selectors, preventing a specificity bump. +* __class__: A new strategy which uses a a bare class name in the selectors, giving all styles a +1 specificity bump. + +# Background & Motivation + +Before 1.0 Astro used the `class` strategy. In the [scoped CSS with preserved specificity RFC](https://github.com/withastro/roadmap/blob/main/proposals/0012-scoped-css-with-preserved-specificity.md) we changed this to instead use the `where` strategy. We felt that this better reflected the styles *as written*. This strategy prevent the unknown/hidden specificity bump. + +However we have come to find that this creates many bugs where a user's global styles will some times *override* Astro styles that use the same selectors. The expected solution to this problem is to control style ordering. Styles ordered last are given priority by browsers. + +However, you cannot effectively control ordering in Astro due to bundling. Bundling makes ordering non-deterministic so there's no way for the user to explicitly declare their Astro styles as having a higher specificity. + +# Goals + +- Fix the immediate problem by allowing users to specify the `class` strategy. +- Set us up for 3.0 so that we can switch the default. +- Preserve the `where` strategy for advanced users. An advanced user can avoid the pitfalls of ordering non-determinism by, for example, using `@layer` to put global styles at a lower layer (lowering its specificity). + +# Non-Goals + +- Changing our bundling strategy. This would be another solution, for example if we either a) did not bundle at all or b) did not code-split CSS (putting each pages CSS into a single bundle) that could *potentially* solve this problem. But that would be a bigger change that would have other negative side-effects. + +# Detailed Design + +The scoping of CSS happens within the compiler. It happens [here](https://github.com/withastro/compiler/blob/0a9b30310bd5aea5ad3762da1ade614e9fbb533e/lib/esbuild/css_printer/astro_features.go#L10-L13) during Astro transformation. + +The compiler would be updated to also have a `options.ScopeStrategy` configuration value based on the values `where` and `class`. It would use this option when printing the selectors. + +The user would control the strategy via a top-level configuration value. The schema for this config is defined [here](https://github.com/withastro/astro/blob/239b9a2fb864fa785e4150cd8aa833de72dd3517/packages/astro/src/core/config/schema.ts#L17). + +The rest of the implementation is wiring up this configuration value to be passed down into the compiler. No other changes are necessary + +# Testing Strategy + +Our usual testing strategy for style things would hold here. There is a `0-css.test.js` test that tests many scenarios, this new option could also be tested there. In 3.0 if we are to change the default, this test file would need to be updated to reflect the new defaults. + +# Drawbacks + +- Having multiple ways to do the same thing is often not a great idea. In this case I think it is justified because there are pros and cons to both `where` and `class` approach. `where` gives you the maximum flexibility but comes with sharp corners. `class` is a more expected default and probably what most users are expecting when they write their styles, but it becomes harder to override component styles with globals. +- There's some maintenance burden to having any new option. In this case the affected areas are small (just a config passed to the compiler), so I don't worry about this being a maintenance issue. + +# Alternatives + +- A way to configure certain files as being "global" and having the compiler lower their specificity. Possibly using `@layer` to do so. +- Another way would be to use `@layer` but make global CSS have a lower layer than Astro's. + +# Adoption strategy + +- Add the configuration option in a minor release. +- Document this option. +- Update the example projects to use it by default, so that `npm init astro@latest` projects get the `class` strategy. +- Switch the default in 3.0. \ No newline at end of file From 7dd302e0134813f1de67637792883680de40d60d Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Thu, 30 Mar 2023 23:08:13 +0800 Subject: [PATCH 166/300] Create cdn-support.md --- proposals/cdn-support.md | 114 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 proposals/cdn-support.md diff --git a/proposals/cdn-support.md b/proposals/cdn-support.md new file mode 100644 index 00000000..344d5615 --- /dev/null +++ b/proposals/cdn-support.md @@ -0,0 +1,114 @@ +- Start Date: 2023-03-30 +- Reference Issues: https://github.com/withastro/roadmap/issues/534 +- Implementation PR: https://github.com/withastro/astro/pull/6714 + +# Summary + +Provide a `build.assetsPrefix` option to specify a CDN URL for static assets to serve from in production. + +# Example + +```js +// astro.config.mjs +import { defineConfig } from 'astro' + +export default defineConfig({ + build: { + assetsPrefix: 'http://cdn.example.com' + } +}) +``` + +# Background & Motivation + +Large traffic sites often have CDN servers that are optimized to serve assets only. For example, a site served from `https://astro.build` would reference all it's assets from `https://cdn.astro.build`. + +A CDN URL would also allow assets from multiple sites to be deployed to the same CDN server to share the performance, where they can be separated by URL subpaths. + +There are also prior art from other frameworks that users want in Astro too: + +- Nextjs: https://nextjs.org/docs/api-reference/next.config.js/cdn-support-with-asset-prefix +- Nuxt: https://nuxt.com/docs/api/configuration/nuxt-config#cdnurl +- SvelteKit: https://kit.svelte.dev/docs/configuration#paths + +# Goals + +- Provide an option to prepend generated asset links with the CDN URL. +- Provide an env variable to access the CDN URL (`import.meta.env.*`) to prepend links manually. +- Works with static and server output. +- Works with existing Astro features, like content collections, `astro:assets` images, and `@astrojs/image`. + +# Non-Goals + +- Auto-prefixing CDN URLs to user-created `<link href>` or `<img src>` etc. +- Allow changing the CDN URL in runtime. +- Handling CDN URLs for files in the `public` directory. + - Users have to preprend the URL themselves if needed. +- The Astro CDN service. + +# Detailed Design + +CDN support is added through the `build.assetsPrefix` config. For all generated asset links in the Astro codebase, before we only consider `base`: + +```js +const assetLink = base + href +``` + +Now we also need to consider `assetsPrefix`: + +```js +const assetLink = (assetsPrefix || base) + href +``` + +This needs to be applied everywhere, and maybe a utility might help with handling this. + +`import.meta.env.ASSETS_PREFIX` is also added for end-users to manually concat strings for assets not controlled by Astro. + +Other notes: + +1. `assetsPrefix` takes precedence over `base` because `base` applies to the user-facing site only, not the CDN domain: + - For example, the user visits `https://example.com/kaboom/` and it fetches assets from `https://cdn.example.com/_astro/explosion.123456.png` +2. `assetsPrefix` is _the_ prefix for the [`assets` option](https://docs.astro.build/en/reference/configuration-reference/#buildassets), which is `_astro` by default. + ``` + https://cdn.example.com/_astro/explosion.123456.png + |---------------------||-----||-------------------| + assetsPrefix assets assets href + ``` + +# Testing Strategy + +We should have these test cases: + +- CSS `<link>` +- Hydration scripts in `<astro-island>` +- Image `src` with both experimental assets feature and `@astrojs/image` +- Markdown asset links +- Content collections asset links + +Make sure all links are prefixed with `assetsPrefix` in both static and server builds. A simple integration test to build and check the generated/rendered HTML should be enough. + +# Drawbacks + +1. This touches a lot of code everywhere. +2. We need to be concious of `assetsPrefix` if `base` had not already caused us a lot of issues before 🥲 +3. Could be confusing with the `build.assets` option. + - "Why is there both `assets` and `assetsPrefix` options?" + +# Alternatives + +No CDN support. That would block sites that need it. + +# Adoption strategy + +This should be transparent to all users. Only when `build.assetsPrefix` is used where the feature kicks in. + +For third-party Astro libraries that contribute assets externally, they would need to consider `build.assetsPrefix` too. But I'm not aware of any libraries that does this (that's outside of Astro's asset pipeline). + +# Unresolved Questions + +1. Is the `build.assetsPrefix` name fine? I picked it to complement the existing `build.assets` option. `build.cdnUrl` feels "sticking out". + + + +Optional, but suggested for first drafts. +What parts of the design are still to be determined? From 0262dc9756768951695aa98ff7e59e1453326947 Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" <fkschott@gmail.com> Date: Fri, 31 Mar 2023 15:44:31 -0700 Subject: [PATCH 167/300] Update accepted-proposal.yml --- .github/ISSUE_TEMPLATE/accepted-proposal.yml | 37 +++++++++----------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/accepted-proposal.yml b/.github/ISSUE_TEMPLATE/accepted-proposal.yml index 5f217a7e..c2f8102e 100644 --- a/.github/ISSUE_TEMPLATE/accepted-proposal.yml +++ b/.github/ISSUE_TEMPLATE/accepted-proposal.yml @@ -12,7 +12,7 @@ body: - type: textarea id: main attributes: - label: Body + label: Details value: | - Accepted Date: <!-- today's date, YYYY-MM-DD --> - Reference Issues/Discussions: <!-- related issues, otherwise leave empty --> @@ -24,34 +24,29 @@ body: # Background & Motivation - Include any useful background detail that that explains why this RFC is important. - What are the problems that this RFC sets out to solve? Why now? Be brief! - - It can be useful to illustrate your RFC as a user problem in this section. - (ex: "Users have reported that it is difficult to do X in Astro today.") + Include any useful detail that that explains the background of this problem + at Astro and why this problem is worth solving. Why solve it now? Be brief! # Goals - A **concise, bulleted-list** outlining the intended goals of this RFC. + A **concise, bulleted-list** outlining the intended goals. - - What are the exact problems that you are trying to solve with this RFC? - - Separate these goals from the solution that you will outline below. - - If this RFC isn't approved, these goals can be used to develop alternative solutions. + - What are the exact problems that you are trying to solve? + - Avoid leading with a specific solution. Example: "Solve X" vs. "Solve X by doing Y" + - Well-defined goals can lead to many alternative solutions! # Non-Goals - A **concise, bulleted-list** outlining anything intentionally left out of this RFC: + A **concise, bulleted-list** outlining anything intentionally left out as a goal: - - Non-goal: A goal that is intentionally not addressed in this RFC. - - Out-of-scope: A goal that is related, but intentionally avoided here. + - Non-goal: A goal that is related, but intentionally avoided. + - Out-of-scope: A goal that is related, but intentionally avoided due to scope. - Future: A goal that is related, but left to be addressed in the future. - - This gives the reader the correct context on what is intentionally left out of scope. - It is okay to leave this section empty. - - # Example - - If the proposal involves a new or changed API, then include a basic code example. - Otherwise, omit this section if it's not applicable. + - It is okay to leave this section empty or N/A. validations: required: true + - type: markdown + attributes: + value: | + > **Reminder: Stage 1 & 2 Proposals are just for the problem to solve.** + > Want to discuss a specific solution? Comment after posting to kick off the convo! From a9abc324ba197a5229b760f15c0572955a195cdf Mon Sep 17 00:00:00 2001 From: "Fred K. Schott" <fkschott@gmail.com> Date: Fri, 31 Mar 2023 23:11:38 +0000 Subject: [PATCH 168/300] update proposal --- .github/ISSUE_TEMPLATE/accepted-proposal.md | 52 ++++++++++++++++++++ .github/ISSUE_TEMPLATE/accepted-proposal.yml | 52 -------------------- stage-2--issue-template.md | 45 ++++++++--------- 3 files changed, 73 insertions(+), 76 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/accepted-proposal.md delete mode 100644 .github/ISSUE_TEMPLATE/accepted-proposal.yml diff --git a/.github/ISSUE_TEMPLATE/accepted-proposal.md b/.github/ISSUE_TEMPLATE/accepted-proposal.md new file mode 100644 index 00000000..58e58a70 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/accepted-proposal.md @@ -0,0 +1,52 @@ +--- +name: '[Maintainers Only] Stage 2 - Create an Accepted Proposal' +about: 'This template is reserved for Astro maintainers only.' +title: '' +labels: '' +assignees: '' +--- + +<!-- + **This template is reserved for Astro maintainers!** + Any non-maintainer issues on this repo will be closed automatically. + + Instead, start a new discussion: https://github.com/withastro/roadmap/discussions/new + See README for more information: https://github.com/withastro/roadmap +--> + +- Accepted Date: <!-- today's date, YYYY-MM-DD --> +- Reference Issues/Discussions: <!-- related issues, otherwise leave empty --> +- Author: <!-- @mention the author (probably you!) --> +- Champion(s): <!-- @mention any proposal champions (probably you!) --> +- Implementation PR: <!-- leave empty --> + +# Summary + +A brief, one or two sentence explanation of the proposal. + +# Background & Motivation + +Include any useful detail that that explains the background of this problem +at Astro and why this problem is worth solving. Why solve it now? Be brief! + +# Goals + +A **concise, bulleted-list** outlining the intended goals. + +- What are the exact problems that you are trying to solve? +- Avoid leading with a specific solution. Example: "Solve X" vs. "Solve X by doing Y" +- Well-defined goals can lead to many alternative solutions! + +# Non-Goals + +A **concise, bulleted-list** outlining anything intentionally left out as a goal: + +- Non-goal: A goal that is related, but intentionally avoided. +- Out-of-scope: A goal that is related, but intentionally avoided due to scope. +- Future: A goal that is related, but left to be addressed in the future. +- It is okay to leave this section empty or N/A. + +<!-- + **Remember!** Stage 1 & 2 Proposals don't include any specific solution details. + Want to discuss solutions or share example code? Comment after posting to kick off the convo! +--> \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/accepted-proposal.yml b/.github/ISSUE_TEMPLATE/accepted-proposal.yml deleted file mode 100644 index c2f8102e..00000000 --- a/.github/ISSUE_TEMPLATE/accepted-proposal.yml +++ /dev/null @@ -1,52 +0,0 @@ -name: "[Maintainers Only] Stage 2 - Create an Accepted Proposal" -description: "This template is reserved for Astro maintainers only." -body: - - type: markdown - attributes: - value: | - > **This template is reserved for Astro maintainers to officially accept existing discussions.** - > You are probably looking to [**start a new discussion**](https://github.com/withastro/roadmap/discussions/new)! - - > To propose a new feature idea or enhancement, you should begin by creating a Discussion. - > See the repo [`README.md`](https://github.com/withastro/roadmap#readme) for more info. - - type: textarea - id: main - attributes: - label: Details - value: | - - Accepted Date: <!-- today's date, YYYY-MM-DD --> - - Reference Issues/Discussions: <!-- related issues, otherwise leave empty --> - - Implementation PR: <!-- leave empty --> - - # Summary - - A brief, one or two sentence explanation of the proposal. - - # Background & Motivation - - Include any useful detail that that explains the background of this problem - at Astro and why this problem is worth solving. Why solve it now? Be brief! - - # Goals - - A **concise, bulleted-list** outlining the intended goals. - - - What are the exact problems that you are trying to solve? - - Avoid leading with a specific solution. Example: "Solve X" vs. "Solve X by doing Y" - - Well-defined goals can lead to many alternative solutions! - - # Non-Goals - - A **concise, bulleted-list** outlining anything intentionally left out as a goal: - - - Non-goal: A goal that is related, but intentionally avoided. - - Out-of-scope: A goal that is related, but intentionally avoided due to scope. - - Future: A goal that is related, but left to be addressed in the future. - - It is okay to leave this section empty or N/A. - validations: - required: true - - type: markdown - attributes: - value: | - > **Reminder: Stage 1 & 2 Proposals are just for the problem to solve.** - > Want to discuss a specific solution? Comment after posting to kick off the convo! diff --git a/stage-2--issue-template.md b/stage-2--issue-template.md index 1ec8d9f7..b95bbda5 100644 --- a/stage-2--issue-template.md +++ b/stage-2--issue-template.md @@ -1,13 +1,15 @@ <!-- - Note: You are probably looking for `stage-1--discussion-template.md`! - This template is reserved for approved proposals and Astro maintainers only. - - Community members who would like to propose an idea or feature should begin - by creating a GitHub Discussion. See the repo README.md for more info. + **This template is reserved for Astro maintainers!** + Any non-maintainer issues on this repo will be closed automatically. + + Instead, start a new discussion: https://github.com/withastro/roadmap/discussions/new + See README for more information: https://github.com/withastro/roadmap --> - Accepted Date: <!-- today's date, YYYY-MM-DD --> - Reference Issues/Discussions: <!-- related issues, otherwise leave empty --> +- Author: <!-- @mention the author (probably you!) --> +- Champion(s): <!-- @mention any proposal champions (probably you!) --> - Implementation PR: <!-- leave empty --> # Summary @@ -16,32 +18,27 @@ A brief, one or two sentence explanation of the proposal. # Background & Motivation -Include any useful background detail that that explains why this RFC is important. -What are the problems that this RFC sets out to solve? Why now? Be brief! - -It can be useful to illustrate your RFC as a user problem in this section. -(ex: "Users have reported that it is difficult to do X in Astro today.") +Include any useful detail that that explains the background of this problem +at Astro and why this problem is worth solving. Why solve it now? Be brief! # Goals -A **concise, bulleted-list** outlining the intended goals of this RFC. +A **concise, bulleted-list** outlining the intended goals. -- What are the exact problems that you are trying to solve with this RFC? -- Separate these goals from the solution that you will outline below. -- If this RFC isn't approved, these goals can be used to develop alternative solutions. +- What are the exact problems that you are trying to solve? +- Avoid leading with a specific solution. Example: "Solve X" vs. "Solve X by doing Y" +- Well-defined goals can lead to many alternative solutions! # Non-Goals -A **concise, bulleted-list** outlining anything intentionally left out of this RFC: +A **concise, bulleted-list** outlining anything intentionally left out as a goal: -- Non-goal: A goal that is intentionally not addressed in this RFC. -- Out-of-scope: A goal that is related, but intentionally avoided here. +- Non-goal: A goal that is related, but intentionally avoided. +- Out-of-scope: A goal that is related, but intentionally avoided due to scope. - Future: A goal that is related, but left to be addressed in the future. +- It is okay to leave this section empty or N/A. -This gives the reader the correct context on what is intentionally left out of scope. -It is okay to leave this section empty. - -# Example - -If the proposal involves a new or changed API, then include a basic code example. -Otherwise, omit this section if it's not applicable. \ No newline at end of file +<!-- + **Remember!** Stage 1 & 2 Proposals don't include any specific solution details. + Want to discuss solutions or share example code? Comment after posting to kick off the convo! +--> \ No newline at end of file From 20f66ac2803c06f7719f95d641d4ec578628ee0c Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Thu, 6 Apr 2023 15:31:14 -0400 Subject: [PATCH 169/300] Update cdn support name --- proposals/{cdn-support.md => 0031-cdn-support.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{cdn-support.md => 0031-cdn-support.md} (100%) diff --git a/proposals/cdn-support.md b/proposals/0031-cdn-support.md similarity index 100% rename from proposals/cdn-support.md rename to proposals/0031-cdn-support.md From f73d51e07b6a98bdf4c36bf0127d826ffc370f40 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Thu, 13 Apr 2023 16:47:46 +0200 Subject: [PATCH 170/300] fix: update with current statae --- proposals/0030-core-image-story.md | 44 ++++++++++++++++-------------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index bca7bbae..55db40a3 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -204,7 +204,7 @@ The currently available `@astrojs/image` is great, however, users ultimately fin - Missing documentation - Due to not being a core integration, it requires manual setup - Partly due to the previous points, but also for other reasons, it wasn't always clear for users how the integration behaved. -- Hard to use in Markdown / MDX. +- Hard to use in Markdown / MDX / Markdoc. In this RFC, we'd like to outline a plan / API for a core story for images. The main motivation is to create a story that feels fitting to the Astro core, in that it's easy to understand, intuitive to use, and extendable while also being kind and respectful of the user. @@ -217,9 +217,9 @@ In this RFC, we'd like to outline a plan / API for a core story for images. The # Non-Goals of this RFC -- Advanced usage in Markdown (ability to set width, height, quality etc) +- Advanced usage when using the standard Markdown syntax for images (ability to set width, height, quality etc) - Using optimized images inside client-side framework components -- Automatic generation of `srcset` in the integrated services +- Automatic generation of `srcset` and `sizes` in the integrated services - Automatic `loading="eager"` for above the fold images - Placeholders generation - Background generation @@ -227,7 +227,7 @@ In this RFC, we'd like to outline a plan / API for a core story for images. The - Optimizing & resizing remote images - Ability to choose a different image service per image - Remote patterns for limiting the usage of remote images to specific domains -- `.svg` support +- Optimizing `.svg` files To be clear, we hope to tackle many of those points in the future, in separate, more precise RFCs. Images are hard. @@ -245,7 +245,7 @@ We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive ## `src/assets` -A new reserved folders for assets. Assets do not have to live in this folder, however putting them in this folder will unlock several benefits. Notably, we consider this folder to be a safe place for git-based CMSes to put any uploaded assets in. +A new reserved folders for assets. Assets do not have to live in this folder, but it'll be our recommended folder for assets from now on. Notably, we consider this folder to be a safe place for git-based CMSes to put any uploaded assets in. Through an included alias `~/assets` pointing to `src/assets`, it'll be easy for users to refer to any uploaded assets there. @@ -253,31 +253,33 @@ Through an included alias `~/assets` pointing to `src/assets`, it'll be easy for It is fairly common for one of the property of a Markdown piece of content to need to be a reference to an asset (think, cover image for an article, picture for an author etc). -In tandem with the `src/assets` folder, we'd like to introduce a way for users to specify that a specific property needs to refer to a valid asset from the `src/assets` folder: +We'd like to introduce a way for users to specify that a specific property needs to refer to a valid asset: ```ts -import { image, defineCollection, z } from "astro:content"; +import { defineCollection, z } from "astro:content"; const blogCollection = defineCollection({ - schema: z.object({ - title: z.string(), - image: image(), - }), + schema: ({ image }) => + z.object({ + title: z.string(), + image: image(), + }), }); ``` Image assets referred this way will be transformed automatically to the same shape as if the image was imported (see section below), as such they can be checked using Zod's [`refine`](https://zod.dev/?id=refine) or [`superRefine`](https://zod.dev/?id=superrefine) methods, for example: ```ts -import { image, defineCollection, z } from "astro:content"; +import { defineCollection, z } from "astro:content"; const blogCollection = defineCollection({ - schema: z.object({ - title: z.string(), - image: image().refine((img) => img.width === 1080, { - message: "Image must be 1080px wide", + schema: ({ image }) => + z.object({ + title: z.string(), + image: image().refine((img) => img.width === 1080, { + message: "Image must be 1080px wide", + }), }), - }), }); ``` @@ -334,13 +336,13 @@ Default is left to the services to choose. But for the services exposed by Astro ## JavaScript API -A `getImage` function taking the same parameters as the image component is also supported, for use cases inside the frontmatter or outside `img` tag (ex: `background-image`) +A `getImage` function taking the same parameters as the image component is also supported, for use cases such as inside the frontmatter or outside `img` tag (ex: `background-image`) This function returns an object with all the properties needed to use / show an image. ```ts // Example interface describing the content of `myImage` -interface getImageResult { +interface GetImageResult { // Contain the original options passed to `getImage` options: Record<string, any>; // Contain a path you can use to render the image @@ -381,10 +383,12 @@ The different methods available are the following: - `transform(buffer: Buffer, options: ImageTransform): { data: Buffer, format: OutputFormat }` - Transform and return the image. It is necessary to return a `format` to ensure that the proper MIME type is served to users in development and SSR. -Ultimately, in development and SSR, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform` if wanted. `transform` however, is called during the build to create the final assets files. +Ultimately, in development and SSR, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform` if wanted. `transform` however, is called automatically during the build in SSG and for pre-rendered pages to create the final assets files. **Optional** +- `validateOptions(options: ImageTransform): ImageTransform` + - Allows you to validate and augment the options passed by the user. This is useful for setting default options, or telling the user that a parameter is required. - `getHTMLAttributes(options: ImageTransform): Record<string, any>` - Return all additional attributes needed to render the image in HTML. For instance, you might want to return a specific `class` or `style`, or `width` and `height`. From 0ba226d018de5548533388a0469eb26926a2457b Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Thu, 13 Apr 2023 16:53:36 +0100 Subject: [PATCH 171/300] chore: first draft --- proposals/0032-middleware-api.md | 347 +++++++++++++++++++++++++++++++ 1 file changed, 347 insertions(+) create mode 100644 proposals/0032-middleware-api.md diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md new file mode 100644 index 00000000..18c338cb --- /dev/null +++ b/proposals/0032-middleware-api.md @@ -0,0 +1,347 @@ +<!-- + Note: You are probably looking for `stage-1--discussion-template.md`! + This template is reserved for anyone championing an already-approved proposal. + + Community members who would like to propose an idea or feature should begin + by creating a GitHub Discussion. See the repo README.md for more info. + + To use this template: create a new, empty file in the repo under `proposals/${ID}.md`. + Replace `${ID}` with the official accepted proposal ID, found in the GitHub Issue + of the accepted proposal. +--> + +- Start Date: +- Reference Issues: + - https://github.com/withastro/roadmap/issues/531 +- Implementation PR: https://github.com/withastro/astro/pull/6721 + +# Summary + +Introduce a middleware to Astro, where you can define code that runs on every request. +This API should work regardless of the rendering mode (SSG or SSR) or adapter used. +Also introduces a simple way to share request-specific data between proposed middleware, +API routes, and `.astro` routes. + +# Example + +The middleware pattern is useful to read `Request` and manipulate the `Response`, other than +set and share information across endpoints and pages. + + +For example, here's an example of how a user can share some information across routes: +```js +export const onRequest = (context, next) => { + if (!context.request.url.endsWith("/")) { + context.locals.isIndex = false; + } else { + context.locals.isIndex = true; + } +} +``` + +Or, set some logic to make redirects: +```js +const redirects = new Map([ + ["/old_1", "/new_1"], + ["/old_1", "/new_1"] +]) + +export const onRequest = (context, next) => { + for (const [oldRoute, newRoute] of redirects) { + if (context.request.url.endsWith(oldRoute)) { + return context.redirect(newRoute, 302); + } + } +} +``` + +# Background & Motivation + +Middleware has been one of the most heavily requested feature in Astro. +It's useful for handling common tasks like auth guards and setting cache headers. + +For me, it would make handling authentication much easier. + + +# Goals + +- Provide a way intercept requests and responses, allowing users to set cookies and headers +- Works both in SSR and SSG mode +- Allow users to use community-created middlewares (libraries) + - Make available via integrations API. +- Provide an API for request-specific data +- Non-Node runtimes specific APIs. ie. Cloudflare Durable Objects. + - Add middleware from adapter. + +# Non-Goals + +- Route specific middleware, middlewares that are run **only on specific** routes + +# Detailed Design + +To define a middleware, a user would need to create a physical file under the `src/` folder, called `middleware.js`. + +The resolution of the file follow the ECMA standards, which means that the following +alternatives are all valid in Astro: +- `src/middleware.js` +- `src/middleware.ts` +- `src/middleware/index.js` +- `src/middleware/index.ts` + +The file **must export** a function called `onRequest`. The exported function +_must not be a **default** export_. + +> **Note**: this part of the proposal differs from the [Stage 2](https://github.com/withastro/roadmap/issues/531) proposal. Read the +> [drawback section](#default-export) to understand why. + +Eventually, the file system of the `src/` folder will look like this: + +``` +src +├── env.d.ts +├── middleware +│ └── index.ts +└── pages + ├── first.astro + ├── index.astro + └── second.astro +``` + +Every time a page or endpoint is about to be rendered, the middleware is called. + +A middleware will look like this, in TypeScript: + +```ts +import { MiddlewareRequestHandler, APIContext, MiddlewareNextResponse } from "astro" + +const onRequest: MiddlewareRequestHandler = async (context: APIContext, next: MiddlewareNextResponse) => { + const { locals, request } = context; + // access to the request + if (request.url.endsWith("/old-url")) { + return new Response("body", { + status: 200 + }) + } + locals.user = {}; +} + +export { onRequest } +``` + +The `locals` object is a new API introduced by this RFC. The `locals` object is a new +global object that can be manipulated inside the middleware, and then it can be +accessed inside any `.astro` file: + +```md +--- +const user = Astro.locals.user; +--- + +<div> + <p>{user.handle}</p> +</div> + +``` + +The RFC provides a way to make `locals` typed. The implementation will leverage the existing +mechanism in place to type `Props`. The user will change the file `env.d.ts` and add the +following code: + +```ts +declare module "astro" { + interface Locals { + user: { + handle: string + } + } +} +``` + +Doing so, the user will be able to leverage the type-checking and auto-completion of TypeScript inside a file +called `middleware.ts` or `middleware.js` using JSDoc. + + +The `locals` object has the following restrictions: +1. it can store only serializable information; +2. it can't be overridden by other values that are different from objects; + +## `locals` needs to be serializable + +The reason why the information must be serializable is that it's not safe to store +information that can be evaluated at runtime. If, for example, we were able to store +a JavaScript function, an attacker would be able to exploit the victim website +and execute some unsafe code. + +In order avoid so, the new code will do a sanity check **in development mode**. +Some code like this: + +```js +export const onRequest = (contex, next) => { + context.locals.someInfo = { + f() { + alert("Hello!!") + } + } +} +``` + +Storing unsafe information will result in an Astro error: + +> The information stored in Astro.locals are not serializable when visiting "/index" path. +Make sure you store only data that are serializable. + +> **Note**: The content of the error is not final. The docs team will review it. + + +## `locals` can't be overridden + +The value of `locals` needs to be an object, and it can't be overridden at runtime. Doing +so would risk to wipe out all the information stored by the user. + +So, if there's some code like this: + +```js +export const onRequest = (contex, next) => { + context.locals = 111; +} +``` + +Astro will emit an error like this: + +> The locals can only be assigned to an object. Other values like numbers, strings, etc. are not accepted. + +> **Note**: The content of the error is not final. The docs team will review it. + +### `context` and `next` + +When defining a middleware, the function accepts two arguments: `context` and `next` + +The `next` function is a widely used function inside the middleware pattern. With the `next` +function, a user can retrieve the `Response` of a request. + +This is very useful in case, for example, a user needs to modify the HTML (the body) of the +response. + +Another usage of the `next` function, is to call the "next" middleware. + +A user is not forced to call the `next` function, and there are various reasons to not to. +For example, a user might not need it, or they might want to stop the chain of middlewares +in case some validation fails. + +The next section will explain more in detail how `next` function can be used and how +a user can have multiple middleware. + +## Multiple middlewares + +The RFC proposes a new API to combine multiple middlewares into one. The new API +is exposed via the new `astro/middleware` module. The new API is called `sequence`. + +Following an example of how a user can combine more than one middleware: + +```js +import {sequence} from "astro/middleware"; + +function validation() {} +function auth() {} + +export const onRequest = sequence(validation, auth); +``` +When working with many middlewares, it's important to understand the order of +how the code is executed. + +Let's take the following code: + +```js +import {sequence} from "astro/middleware"; + +async function validation(_, next) { + console.log("validation request"); + const response = await next(); + console.log("validation response"); + return response; +} +async function auth(_, next) { + console.log("auth request"); + const response = await next(); + console.log("auth response"); + return response; + +} +async function greeting(_, next) { + console.log("greeting request"); + const response = await next(); + console.log("greeting response"); + return response; + +} + +export const onRequest = sequence(validation, auth, greeting); +``` +Will result in the following console order: + +``` +validation request +auth request +greeting request +greeting response +auth response +validation response +``` + +When working with multiple middlewares, a middleware will always get the context of the previous +middleware, from left to right. When the response has been resolved, then +this response will travel from the right to left. + +```block + Context/Request Context/Request +validation --------------------> auth --------------------> greeting + +Then + Response Response +validation <-------------------- auth <-------------------- greeting +``` + +Eventually, `valiation` will be the **last** middleware to get the `Response` before being +handled to Astro and rendered by the browser. + +## Middleware workflow and examples + + + +# Testing Strategy + +How will this feature's implementation be tested? Explain if this can be tested with +unit tests or integration tests or something else. If relevant, explain the test +cases that will be added to cover all of the ways this feature might be used. + +# Drawbacks + +## Default export + +Why should we _not_ do this? Please consider: + +- Implementation cost, both in term of code size and complexity. +- Whether the proposed feature can be implemented in user space. +- Impact on teaching people Astro. +- Integration of this feature with other existing and planned features +- Cost of migrating existing Astro applications (_is it a breaking change?_) + +There are tradeoffs to choosing any path. Attempt to identify them here. + +# Alternatives + +What other designs have been considered? What is the impact of not doing this? + +# Adoption strategy + +Please consider: + +- If we implement this proposal, how will existing Astro developers adopt it? +- Is this a breaking change? Can we write a codemod? +- Can we provide a runtime adapter library for the original API it replaces? +- How will this affect other projects in the Astro ecosystem? + +# Unresolved Questions + +Optional, but suggested for first drafts. +What parts of the design are still to be determined? From 9395cb97c4ae133d21c9db1da27772b2c8987614 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Fri, 14 Apr 2023 15:15:18 +0100 Subject: [PATCH 172/300] Middleware API --- proposals/0032-middleware-api.md | 248 +++++++++++++++++++++++-------- 1 file changed, 185 insertions(+), 63 deletions(-) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index 18c338cb..b8db7e2a 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -42,8 +42,8 @@ export const onRequest = (context, next) => { Or, set some logic to make redirects: ```js const redirects = new Map([ - ["/old_1", "/new_1"], - ["/old_1", "/new_1"] + ["/old-1", "/new-1"], + ["/old-1", "/new-1"] ]) export const onRequest = (context, next) => { @@ -79,9 +79,23 @@ For me, it would make handling authentication much easier. # Detailed Design -To define a middleware, a user would need to create a physical file under the `src/` folder, called `middleware.js`. +## Changes from the Stage 2 proposal -The resolution of the file follow the ECMA standards, which means that the following +- `resolve` has been renamed to `next`; +- `next` doesn't accept an `APIContext` to work; +- `locals` values need to be serializable to avoid the introduction of + non-user code from third-party libraries that can run scripts; +- `middleware` export function has been renamed `onRequest`. This name change + has two benefits: + 1. It shows intent and explains when this function is called; + 2. It allows adding more functions with the `on*` prefix in the future, which could show the intent + of when the function is called in the Astro route life cycle; + +## Implementation instructions + +To define a middleware, a user must create a physical file under the `src/` folder, called `middleware.js`. + +The resolution of the file follows the ECMA standards, which means that the following alternatives are all valid in Astro: - `src/middleware.js` - `src/middleware.ts` @@ -89,10 +103,7 @@ alternatives are all valid in Astro: - `src/middleware/index.ts` The file **must export** a function called `onRequest`. The exported function -_must not be a **default** export_. - -> **Note**: this part of the proposal differs from the [Stage 2](https://github.com/withastro/roadmap/issues/531) proposal. Read the -> [drawback section](#default-export) to understand why. +_must not be a **default** export_. Eventually, the file system of the `src/` folder will look like this: @@ -109,7 +120,7 @@ src Every time a page or endpoint is about to be rendered, the middleware is called. -A middleware will look like this, in TypeScript: +A middleware will look like this in TypeScript: ```ts import { MiddlewareRequestHandler, APIContext, MiddlewareNextResponse } from "astro" @@ -129,7 +140,7 @@ export { onRequest } ``` The `locals` object is a new API introduced by this RFC. The `locals` object is a new -global object that can be manipulated inside the middleware, and then it can be +Astro global object that can be manipulated inside the middleware, and then it can be accessed inside any `.astro` file: ```md @@ -144,35 +155,35 @@ const user = Astro.locals.user; ``` The RFC provides a way to make `locals` typed. The implementation will leverage the existing -mechanism in place to type `Props`. The user will change the file `env.d.ts` and add the +mechanism to type `Props`. The user will change the file `env.d.ts` and add the following code: ```ts -declare module "astro" { - interface Locals { +/// <reference types="astro/client" /> + +interface Locals { user: { handle: string } - } } ``` -Doing so, the user will be able to leverage the type-checking and auto-completion of TypeScript inside a file +By doing so, the user can leverage the type-checking and auto-completion of TypeScript inside a file called `middleware.ts` or `middleware.js` using JSDoc. The `locals` object has the following restrictions: -1. it can store only serializable information; +1. it can store only serializable information; 2. it can't be overridden by other values that are different from objects; ## `locals` needs to be serializable -The reason why the information must be serializable is that it's not safe to store -information that can be evaluated at runtime. If, for example, we were able to store -a JavaScript function, an attacker would be able to exploit the victim website +The information must be serializable because storing +information that evaluates at runtime is unsafe. If, for example, we were able to store +With a JavaScript function, an attacker could exploit the victim's website and execute some unsafe code. -In order avoid so, the new code will do a sanity check **in development mode**. +Astro will do a sanity check **in development mode**. Some code like this: ```js @@ -188,15 +199,15 @@ export const onRequest = (contex, next) => { Storing unsafe information will result in an Astro error: > The information stored in Astro.locals are not serializable when visiting "/index" path. -Make sure you store only data that are serializable. +Make sure you store only serializable data. -> **Note**: The content of the error is not final. The docs team will review it. +> **Note**: The content of the error is not final. The docs team will review it. ## `locals` can't be overridden The value of `locals` needs to be an object, and it can't be overridden at runtime. Doing -so would risk to wipe out all the information stored by the user. +so would risk wiping out all the information stored by the user. So, if there's some code like this: @@ -212,31 +223,31 @@ Astro will emit an error like this: > **Note**: The content of the error is not final. The docs team will review it. -### `context` and `next` +### `context` and `next` When defining a middleware, the function accepts two arguments: `context` and `next` -The `next` function is a widely used function inside the middleware pattern. With the `next` +The `next` function is widely used in the middleware pattern. With the `next` function, a user can retrieve the `Response` of a request. -This is very useful in case, for example, a user needs to modify the HTML (the body) of the +Reading a response is very useful in case; for example, a user needs to modify the HTML (the body) of the response. -Another usage of the `next` function, is to call the "next" middleware. +Another usage of the `next` function is to call the "next" middleware. -A user is not forced to call the `next` function, and there are various reasons to not to. -For example, a user might not need it, or they might want to stop the chain of middlewares +Calling the `next` function is not mandatory, and there are various reasons not to. +For example, a user might not need it, or they might want to stop the chain of middleware in case some validation fails. -The next section will explain more in detail how `next` function can be used and how -a user can have multiple middleware. +The next section will explain in more detail how `next` function can be used and how +a user can have multiple middlewares. ## Multiple middlewares -The RFC proposes a new API to combine multiple middlewares into one. The new API -is exposed via the new `astro/middleware` module. The new API is called `sequence`. +The RFC proposes a new API to combine multiple middleware into one. The new API +is available via the new `astro/middleware` module. The new API is called `sequence`. -Following an example of how a user can combine more than one middleware: +Following is an example of how a user can combine more than one middleware: ```js import {sequence} from "astro/middleware"; @@ -246,8 +257,7 @@ function auth() {} export const onRequest = sequence(validation, auth); ``` -When working with many middlewares, it's important to understand the order of -how the code is executed. +When working with many middlewares, it's important to understand the execution order. Let's take the following code: @@ -277,7 +287,7 @@ async function greeting(_, next) { export const onRequest = sequence(validation, auth, greeting); ``` -Will result in the following console order: +This will result in the following console order: ``` validation request @@ -288,60 +298,172 @@ auth response validation response ``` -When working with multiple middlewares, a middleware will always get the context of the previous -middleware, from left to right. When the response has been resolved, then -this response will travel from the right to left. +When working with multiple middleware, a middleware will always get the context of the previous +middleware, from left to right. When the response resolves, then +this response will travel from right to left. ```block Context/Request Context/Request validation --------------------> auth --------------------> greeting -Then + + ===> Then the response is created <=== + + Response Response -validation <-------------------- auth <-------------------- greeting +greeting ----------------------> auth --------------------> validation ``` -Eventually, `valiation` will be the **last** middleware to get the `Response` before being +Eventually, `validation` will be the **last** middleware to get the `Response` before being handled to Astro and rendered by the browser. ## Middleware workflow and examples +Following some use cases with some examples to understand how the middleware work and +the expectation from the user's point of view. + +### Redirects + +It's an example provided before, but it's worth showing it again: + +```js +const redirects = new Map([ + ["/old-1", "/new-1"], + ["/old-2", "/new-2"] +]) + +export const onRequest = (context, next) => { + for (const [oldRoute, newRoute] of redirects) { + if (context.request.url.endsWith(oldRoute)) { + return context.redirect(newRoute); + } + } +} +``` + +### HTML manipulation + +An example, could the to **redact** some sensible information from the HTML being emitted: + +```js +export const onRequest = async (context, next) => { + const response = await next(); + const html = response.text(); + const redactedHtml = html.replace("PRIVATE INFO", "REDACTED"); + + return new Response(redactedHtml, { + status: 200, + headers: response.headers + }); +} +``` + +### Validation and authentication + +```ts +import { sequence } from "astro/middleware"; +import type { + MiddlewareResponseHandler, + MiddlewareNextResponse, + APIContext +} from "astro"; +import { doAuth } from "some-auth-library"; + + +const validation: MiddlewareResponseHandler = async ({ request, locals }: APIContext, next: MiddlewareNextResponse) => { + const formData = await request.formData(); + const userName = formData.get("username"); + const password = formData.get("password"); + // important information exist, let's continue to auth + if (typeof userName !== "undefined" && typeof userName !== "undefined") { + return await next(); + } else { + // We don't call `next`. Doing the `auth` function is not executed. + // We store some information in `locals` so the UI can show an error message. + locals.validationMessage = "Important information are missing"; + } +} + +const auth: MiddlewareResponseHandler = async ({ request, redirect }: APIContext, next: MiddlewareNextResponse) => { + // The user expectation is that `validation` was already executed (check `sequence`). + // This means we don't need to check if `userName` or `password` exit. + // If they don't exist, it's an user error. + const formData = await request.formData(); + const userName = formData.get("username"); + const password = formData.get("password"); + + // We run the authentication using a third-party service + const result = await doAuth({ userName, password }); + if (result.status === "SUCCESS") { + return redirect("/secure-area"); + } else { + locals.validationMessage = "User name and/or password are invalid"; + } +} +export const onRequest = sequence(validation, auth); +``` + +It's important to note that `locals` is an object that **lives and dies within a single Astro route**; +when your route page is rendered, `locals` won't exist anymore and a new one +will be created. + +If a user needs to persist some information that lives among multiple pages +requests, they will need to store that information somewhere else. + # Testing Strategy -How will this feature's implementation be tested? Explain if this can be tested with -unit tests or integration tests or something else. If relevant, explain the test -cases that will be added to cover all of the ways this feature might be used. +This feature requires integration tests. We need to have tests for +multiple scenarios: +- development server; +- static build; +- SSR build; +- adapters (Node.js, deno, Cloudflare, etc.); # Drawbacks -## Default export +The RFC doesn't break any existing code. It's an additional feature that should +not break the existing behaviour of an Astro application. + +Even though the middleware pattern is widely spread among backend frameworks, it's not +always easy to explain how the pattern works and the user expectations. -Why should we _not_ do this? Please consider: +I would expect some reports about "why my code doesn't work" and find out that +the issue was around the user code. -- Implementation cost, both in term of code size and complexity. -- Whether the proposed feature can be implemented in user space. -- Impact on teaching people Astro. -- Integration of this feature with other existing and planned features -- Cost of migrating existing Astro applications (_is it a breaking change?_) +This feature will unlock new patterns inside an Astro application, and I would expect more work from the Documentation Team +to frame the "most-used recipes" around the usage of middleware. -There are tradeoffs to choosing any path. Attempt to identify them here. +Due to Astro code base architecture, the implementation of the feature will have to happen in +three different places, risking having duplicated code or missing logic. # Alternatives -What other designs have been considered? What is the impact of not doing this? +I have considered implementing middleware using the configuration API. + +While this strategy is well-established in the Astro ecosystem, it could slow down +the implementation of middleware logic in user-land because the user would be forced +to create some boilerplate just to use the new logic. + +While Astro could have provided some API to reduce the boilerplate, this felt tedious. +Plus, this would have forced us to **crate another configuration field** where the user +could specify the order of the middleware. + # Adoption strategy -Please consider: +Considering how big the feature is, the API will be released under an experimental flag. -- If we implement this proposal, how will existing Astro developers adopt it? -- Is this a breaking change? Can we write a codemod? -- Can we provide a runtime adapter library for the original API it replaces? -- How will this affect other projects in the Astro ecosystem? +Users can opt in via new flag: -# Unresolved Questions +```js +export default defineConfig({ + experimental: { + middleware: true + } +}) +``` -Optional, but suggested for first drafts. -What parts of the design are still to be determined? +The team will seek feedback from the community and fix bugs if they arise. +After an arbitrary about of time, the experimental flag will be removed. From 664000fe53eb7e6b2cce2eb3d55f4d14fef55b5b Mon Sep 17 00:00:00 2001 From: Arsh <69170106+lilnasy@users.noreply.github.com> Date: Fri, 14 Apr 2023 20:13:18 +0000 Subject: [PATCH 173/300] New RFC: Inline StyleSheets --- proposals/0032-inline-stylesheets.md | 89 ++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 proposals/0032-inline-stylesheets.md diff --git a/proposals/0032-inline-stylesheets.md b/proposals/0032-inline-stylesheets.md new file mode 100644 index 00000000..5eea9f85 --- /dev/null +++ b/proposals/0032-inline-stylesheets.md @@ -0,0 +1,89 @@ + +- Start Date: 2023-04-15 +- Reference Issues: [#556](https://github.com/withastro/roadmap/issues/556) +- Implementation PR: [withastro/astro#6659](https://github.com/withastro/astro/pull/6659) + +# Summary + +Provide a configuration to control inlining behavior of styles authored or imported in astro modules. + +# Example + +```ts +export default defineConfig({ + build: { + // all styles necessary are sent in external stylsheets, default; maintains current behavior + inlineStylesheets: "never" + } +}) +``` +```ts +export default defineConfig({ + build: { + // stylesheets smaller than `ViteConfig.build.assetsInlineLimit` (default: 4kb) are inlined + inlineStylesheets: "auto" + } +}) +``` +```ts +export default defineConfig({ + build: { + // all styles necessary are inlined into <style> tags + inlineStylesheets: "always" + } +}) +``` + +# Background & Motivation + +There has been a constant interest in inlining styles while still taking advantage of scoping and other processing steps since before 1.0 (see: withastro/astro#918), with many users incorrectly assuming that `is:inline` directive is the solution (see: withastro/astro#6388). + +Simple one-page websites do not benefit from an external stylesheet, since there is no other page that could use the cached stylesheet. On the other hand, large websites are overoptimized for cacheability, since our chunking splits the styles too granularly. Case in point, Astro docs homepage has 20 linked stylesheets, 14 of them are less than 1kb (see: withastro/astro#6528). + +So far we have not provided a way to allow inlining stylesheets, prompting workarounds. However, coming from other frameworks, users might expect to be able to configure this behavior. +- SvelteKit allows configuring a threshold under which CSS files will be inlined: https://kit.svelte.dev/docs/configuration#inlinestylethreshold +- Fresh lets plugins inject inline styles: https://fresh.deno.dev/docs/concepts/plugins#hook-render +- Nuxt has a global configuration: https://nuxt.com/docs/api/configuration/nuxt-config#inlinessrstyles + +# Goals + +- Provide a way to reduce the number of HTTP requests for stylesheets. +- Maintain current behavior when not explicitly configured. +- Works with both `server` and `static` outputs. + +# Non-Goals + +- Identify "critical" CSS rules. +- Enable a single external stylesheet. +- Preloading. +- Inlining of scripts. + +# Detailed Design + +The decision to inline a stylesheet should be made at build time (`astro build`), based on the `build.inlineStylsheets` and `vite.build.assetsInlineLimit` configuration options. We already use `assetsInlineLimit` elsewhere to decide if a script should be inlined. `build.inlineStylsheets` option could be set to either `"always"`, `"never"`, or `"auto"`. To stay consistent with the current behavior, the default value should be `"never"`. When `build.inlineStylsheets` is set to `"auto"`, `vite.build.assetsInlineLimit` should be respected. If `assetsInlineLimit` is undefined, a default threshold of 4kb should be used. + +If the stylesheet is to be inlined, its contents should be added to `PageBuildData` of each page that it is used in. The asset should also be removed from vite's bundle to avoid unnecessary files in the output. + +Astro has run into CSS ordering issues and currently ensures that the `<link>` tags are in a specific order. Currently, this order can't be guaranteed when some stylesheets are to be inlined as `<style>` tags: the rendering pipeline receives the two separtely and inserts all the `<link>` tags first. Therefore, an implementation must pass both inline and external stylesheets in the same array or set that preserves the order, and the rendering pipeline should make sure each `SSRElement` gets serialized with the appropriate tag. + +# Testing Strategy + +An implementation can be tested adequately with fixture-based tests, ensuring the DOM from SSR responses and generated `.html` files has the expected count of style and link tags and that they are placed in the expected order. + +# Drawbacks + +- Might lead to unexpected results with our current CSS chunking strategy - it tends to create a lot of tiny stylesheets, and each of them might be under the limit, leading to more CSS being inlined than the user might expect. + +# Alternatives + +- `astro-critters` integration: it's viable only for static builds of small sites, as it slows to a crawl trying to match every CSS rule against every DOM element on every page. +- `<style critical> ... </style>`: implementation cost, mental overhead for developers, lack of evidence that the granularity will be worth it. + +# Adoption strategy + +For now, inlining should be opt-in. In the future, we can consider making "auto" the default. + +# Unresolved Questions + +- Should the configuration be `build.inlineStylsheets: "never" | "always" | "auto"`? An alternative I've considered is `build.stylesheets: "external" | "inline" | "auto"`. +- Should the user need to set two configuration to control the cutoff point? I doubt many users will need to change the default, but they might have already set `vite.build.assetsInlineLimit` explicitly for a different use-case. From 063d3e0b3f411463a451e0f738d60791e0984e9b Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Tue, 18 Apr 2023 15:06:19 +0200 Subject: [PATCH 174/300] code review --- proposals/0032-middleware-api.md | 33 +++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index b8db7e2a..f609e20b 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -68,15 +68,23 @@ For me, it would make handling authentication much easier. - Provide a way intercept requests and responses, allowing users to set cookies and headers - Works both in SSR and SSG mode - Allow users to use community-created middlewares (libraries) - - Make available via integrations API. -- Provide an API for request-specific data + + +# Out of Scope + +This is a list of requirements that won't be implemented as part of this RFC but +will be implemented in the second iteration of the middleware project: + - Non-Node runtimes specific APIs. ie. Cloudflare Durable Objects. - - Add middleware from adapter. +- Add middleware from adapter. +- Type-safe payload. Being able infer types from the previous middleware. # Non-Goals - Route specific middleware, middlewares that are run **only on specific** routes + + # Detailed Design ## Changes from the Stage 2 proposal @@ -93,7 +101,8 @@ For me, it would make handling authentication much easier. ## Implementation instructions -To define a middleware, a user must create a physical file under the `src/` folder, called `middleware.js`. +To define a middleware, a user must create a physical file under the [`config.srcDir`](https://docs.astro.build/en/reference/configuration-reference/#srcdir) +folder, called `middleware.js`. The resolution of the file follows the ECMA standards, which means that the following alternatives are all valid in Astro: @@ -139,11 +148,22 @@ const onRequest: MiddlewareRequestHandler = async (context: APIContext, next: Mi export { onRequest } ``` +Alternatively, a user can use an utility API to type the middleware: + +```ts +import {defineMiddleware} from "astro/middleware"; + + +const onRequest = defineMiddlware(async (context, next) => { + +}); +``` + The `locals` object is a new API introduced by this RFC. The `locals` object is a new Astro global object that can be manipulated inside the middleware, and then it can be accessed inside any `.astro` file: -```md +```astro --- const user = Astro.locals.user; --- @@ -465,5 +485,8 @@ export default defineConfig({ }) ``` +Plus, a user can enable this experimental feature via CLI using a +new argument called `--experimental-middleware`. + The team will seek feedback from the community and fix bugs if they arise. After an arbitrary about of time, the experimental flag will be removed. From 95e051010cfb1a4d35c8e2166db26756d2df1661 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Tue, 25 Apr 2023 16:09:09 +0100 Subject: [PATCH 175/300] add expectations --- proposals/0032-middleware-api.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index f609e20b..da6e45c7 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -148,7 +148,7 @@ const onRequest: MiddlewareRequestHandler = async (context: APIContext, next: Mi export { onRequest } ``` -Alternatively, a user can use an utility API to type the middleware: +Alternatively, a user can use the utility API to type the middleware: ```ts import {defineMiddleware} from "astro/middleware"; @@ -432,6 +432,16 @@ will be created. If a user needs to persist some information that lives among multiple pages requests, they will need to store that information somewhere else. +## Restrictions and expectations + +In order to set user expectations, the Astro middleware have the following restrictions: +- a middleware needs to return a `Response`; +- a middleware needs to call `next`; + +If the user doesn't do any of these two, Astro will throw an error. Plus, +the user is required to return exactly a `Response`. Failing to do so will result in +Astro throwing an error. + # Testing Strategy This feature requires integration tests. We need to have tests for From 55dafd27091f303d64197821728ab2a0469f1309 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Tue, 25 Apr 2023 13:19:09 -0400 Subject: [PATCH 176/300] Move style scoping proposal --- proposals/{style-scoping.md => 0032-style-scoping.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{style-scoping.md => 0032-style-scoping.md} (100%) diff --git a/proposals/style-scoping.md b/proposals/0032-style-scoping.md similarity index 100% rename from proposals/style-scoping.md rename to proposals/0032-style-scoping.md From 213cf99cd9ded5ac03b300366e79da21bb5f4332 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Thu, 27 Apr 2023 12:53:51 -0400 Subject: [PATCH 177/300] Add the hybrid rendering proposal --- proposals/hybrid-rendering.md | 106 ++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 proposals/hybrid-rendering.md diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md new file mode 100644 index 00000000..ccc5392f --- /dev/null +++ b/proposals/hybrid-rendering.md @@ -0,0 +1,106 @@ +- Start Date: 2023-04-27 +- Reference Issues: https://github.com/withastro/roadmap/issues/539 +- Implementation PR: <!-- leave empty --> + +# Summary + +Provide a new `output` option `'hybrid'` that treats all pages as prerendered by default, allowing an opt-out through `export const prerender = false`. + +# Example + +An existing static site can be changed to hybrid rendering, allowing some specific routes to not be prerendered. + +__astro.config.mjs__ + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + output: 'hybrid' +}); +``` + +__pages/api/form.ts__ + +```ts +export const prerender = false; + +export function get({ request }) { + // ... +} +``` + +# Background & Motivation + +In 2.0 Astro introduced prerendering as an option when using `output: 'server'`. Prerendering allows certain pages to be prerendered to HTML during the build. This means that a dynamic app can have certain pages, like a landing page be served faster through a CDN, while preserving dynamic pages to SSR. + +An immediate point of feedback from the community was that it felt odd that this worked in server output, but not in static output given Astro's roots as a static site generator. This choice was made to prevent maintenance burden of having a 3rd way that the build works. A build where some routes are not prerendered is more like the `'server'` output than like the `'static'`. + +However, there are use-cases and reasons for wanting to have some dynamic pages in a static site. + +## Use-cases + +A few of the use-cases collected when talking to users who have requested this feature: + +- A marketing site for an agency that contains a contact form. Most of the site can be static and served via CDN, but the endpoint to serve the contact form needs to be dynamic so that it can store the contact information in a database and alert the admin. +- A SaaS product where the API is the main product. Dynamic parts of pages are built with client components, API endpoints are the only server routes needed. +- A content site such as a recipe site that allows users to mark their favorite recipes. API routes would be dynamic, as would any pages that display the dynamic list sof favorites. + +# Goals + +- Allow default-static apps to have some pages that are dynamic. +- Align with the current implementation and prevent an extra code-path that will be difficult to maintain. +- Provide a better path when a site goes from static to server-rendered. Hybrid is a nice middleground. + +# Non-Goals + +- Any extra features outside of marking certain pages to not be prerendered. + +# Detailed Design + +The Astro config definition and types will need to be updated to allow the `'hybrid'` value for `output`. + +In `packages/astro/src/core/routing/manifest/create.ts` each route is set up to be `prerender: false` by default. This should be changed to be based on the `output` config option. If `'server'` then it should remain false, if `'hybrid'` it should be interpreted as true. + +In `packages/astro/src/vite-plugin-scanner/scanner.ts` it currently throws for falsey values. Since in hybrid rendering users will set `export const prerender = false`, this code will need to be updated to allow the false value when the `output: 'hybrid'`. + +Additionally there are a few places in the codebase that assume if `output !== 'static'` that it is server mode. Those might need to be changed, depending on what they are doing with that information. For example, in static mode you cannot access the `Astro.url.searchParams`. However, because this happens before we load a page component we cannot know at the time if the route is prerendered or not, so we cannot continue to enforce this restriction in hybrid rendering. For this reason, and for simplicity, in hybrid rendering these restrictions are lifted for all routes. + +# Testing Strategy + +Prerendering is currently tested via fixture testing, due to the fact that the build artifacts is what changes. This same strategy will be used to test hybrid rendering as well, only testing for the opposite effect. + +Likely we can use the same test fixtures, but only swap out the `output` when testing `'hybrid'`, which eliminates the need for new fixtures. + +# Drawbacks + +- Having a 3rd mode is a little confusing. Given that `'server'` output also has prerendering support, one might think that it also works in `'static'` mode, but just with the opposite default. +- Some integrations probably treat anything where `output !== 'static'` to mean it is server-rendering, and they might make wrong expectations based on that. + +# Alternatives + +The other design considered was to allow `export const prerender = false` in `output: 'static'`. There are a couple of downsides to this approach: + +- Astro's build uses an extra plugin in `'server'` mode which sets up the SSR and connects to the adapter. To support this alternative approach we'd need to include this plugin always and somehow adjust on-the-fly to how and where the build is output. +- Currently we restrict access to certain APIs, such as search params in the URL when using `output: 'static'`. Because this occurs before we know what route to use, we'd not be able to have that restriction any more. + +This is perhaps a better long-term way but would require significant refactor to align the implementations closer together. + +# Adoption strategy + +First step will be to release as an experimental feature: + +```js +export default defineConfig({ + output: 'hybrid', + experimental: { + hybridOutput: true + } +}) +``` + +Once users have a chance to provide feedback and the feature stabilizes we could unflag in a minor release. + +# Unresolved Questions + +It's unclear how much disruption this will cause to the ecosystem given that many might assume only 2 output modes. The experimental phase will help resolve that. \ No newline at end of file From 45a03ced0720f376b269543f9d6df9ebb2a9c872 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Fri, 28 Apr 2023 13:17:54 +0100 Subject: [PATCH 178/300] chore: add interface expansion --- proposals/0032-middleware-api.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index da6e45c7..ed16c50a 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -243,6 +243,25 @@ Astro will emit an error like this: > **Note**: The content of the error is not final. The docs team will review it. +`locals` can be typed using the `env.d.ts` file. In order to do so, this RFC will +expose a new `namespace` called `App`. This will be the first `namespace` exposed +by Astro: + +```ts +/// <reference types="astro/client" /> +declare global { + namespace App { + interface Locals { + user: { + name: string + } + } + } +} + +export {} +``` + ### `context` and `next` When defining a middleware, the function accepts two arguments: `context` and `next` From 79dca0febba24b8efbecb81bb62ae40ef533df71 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Fri, 28 Apr 2023 08:37:54 -0400 Subject: [PATCH 179/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index ccc5392f..1969949e 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -62,7 +62,7 @@ The Astro config definition and types will need to be updated to allow the `'hyb In `packages/astro/src/core/routing/manifest/create.ts` each route is set up to be `prerender: false` by default. This should be changed to be based on the `output` config option. If `'server'` then it should remain false, if `'hybrid'` it should be interpreted as true. -In `packages/astro/src/vite-plugin-scanner/scanner.ts` it currently throws for falsey values. Since in hybrid rendering users will set `export const prerender = false`, this code will need to be updated to allow the false value when the `output: 'hybrid'`. +In `packages/astro/src/vite-plugin-scanner/scan.ts` it currently throws for falsey values. Since in hybrid rendering users will set `export const prerender = false`, this code will need to be updated to allow the false value when the `output: 'hybrid'`. Additionally there are a few places in the codebase that assume if `output !== 'static'` that it is server mode. Those might need to be changed, depending on what they are doing with that information. For example, in static mode you cannot access the `Astro.url.searchParams`. However, because this happens before we load a page component we cannot know at the time if the route is prerendered or not, so we cannot continue to enforce this restriction in hybrid rendering. For this reason, and for simplicity, in hybrid rendering these restrictions are lifted for all routes. From bda6cc0e02ce608dea9e3d2f7c59a499e2040224 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Fri, 28 Apr 2023 08:38:16 -0400 Subject: [PATCH 180/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index 1969949e..906e5cda 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -44,7 +44,7 @@ A few of the use-cases collected when talking to users who have requested this f - A marketing site for an agency that contains a contact form. Most of the site can be static and served via CDN, but the endpoint to serve the contact form needs to be dynamic so that it can store the contact information in a database and alert the admin. - A SaaS product where the API is the main product. Dynamic parts of pages are built with client components, API endpoints are the only server routes needed. -- A content site such as a recipe site that allows users to mark their favorite recipes. API routes would be dynamic, as would any pages that display the dynamic list sof favorites. +- A content site such as a recipe site that allows users to mark their favorite recipes. API routes would be dynamic, as would any pages that display the dynamic list of favorites. # Goals From e7201722024da23461611a23def6db63b388168d Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Fri, 28 Apr 2023 08:38:24 -0400 Subject: [PATCH 181/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index 906e5cda..ac5b4d55 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -60,7 +60,7 @@ A few of the use-cases collected when talking to users who have requested this f The Astro config definition and types will need to be updated to allow the `'hybrid'` value for `output`. -In `packages/astro/src/core/routing/manifest/create.ts` each route is set up to be `prerender: false` by default. This should be changed to be based on the `output` config option. If `'server'` then it should remain false, if `'hybrid'` it should be interpreted as true. +In `packages/astro/src/core/routing/manifest/create.ts` each route is set up to be `prerender: false` by default. This should be changed to be based on the `output` config option. If `'hybrid'` it should be interpreted as true, otherwise it should remain false In `packages/astro/src/vite-plugin-scanner/scan.ts` it currently throws for falsey values. Since in hybrid rendering users will set `export const prerender = false`, this code will need to be updated to allow the false value when the `output: 'hybrid'`. From 362eebd0521667f3ee629d0d811f3962861715b3 Mon Sep 17 00:00:00 2001 From: Arsh <69170106+lilnasy@users.noreply.github.com> Date: Fri, 28 Apr 2023 19:30:15 +0530 Subject: [PATCH 182/300] "default: 4kb" -> "or 4kb if not defined by the user" --- proposals/0032-inline-stylesheets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0032-inline-stylesheets.md b/proposals/0032-inline-stylesheets.md index 5eea9f85..3c6bb418 100644 --- a/proposals/0032-inline-stylesheets.md +++ b/proposals/0032-inline-stylesheets.md @@ -20,7 +20,7 @@ export default defineConfig({ ```ts export default defineConfig({ build: { - // stylesheets smaller than `ViteConfig.build.assetsInlineLimit` (default: 4kb) are inlined + // stylesheets smaller than `ViteConfig.build.assetsInlineLimit` (or 4kb if not defined by the user) are inlined inlineStylesheets: "auto" } }) From 5509f09fb320fee79a0b03eae8d62a760c7b4b20 Mon Sep 17 00:00:00 2001 From: Princesseuh <princssdev@gmail.com> Date: Fri, 28 Apr 2023 18:42:14 +0200 Subject: [PATCH 183/300] fix: update to current version --- proposals/0030-core-image-story.md | 62 +++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 19 deletions(-) diff --git a/proposals/0030-core-image-story.md b/proposals/0030-core-image-story.md index 55db40a3..8274b552 100644 --- a/proposals/0030-core-image-story.md +++ b/proposals/0030-core-image-story.md @@ -133,7 +133,9 @@ import { defineConfig } from "astro/config"; // https://astro.build/config export default defineConfig({ image: { - service: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string + service: { + entrypoint: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string + }, }, }); ``` @@ -144,19 +146,23 @@ export default defineConfig({ import type { LocalImageService } from "astro"; const service: LocalImageService = { - getURL(options: ImageTransform) { + getURL(options: ImageTransform, serviceOptions: Record<string, any>) { return `/my_super_endpoint_that_transforms_images?w=${options.width}`; }, - parseURL(url: URL) { + parseURL(url: URL, serviceOptions: Record<string, any>) { return { width: url.searchParams.get("w"), }; }, - transform(options: ImageTransform) { - const { buffer } = mySuperLibraryThatEncodesImages(options); + transform( + buffer: Buffer, + options: ImageTransform, + serviceOptions: Record<string, any> + ) { + const { result } = mySuperLibraryThatEncodesImages(buffer, options); return { - data: buffer, + data: result, format: options.format, }; }, @@ -171,7 +177,7 @@ export default service; import type { ExternalImageService } from "astro"; const service: ExternalImageService = { - getURL(options: ImageTransform) { + getURL(options: ImageTransform, serviceOptions: Record<string, any>) { return `https://mysupercdn.com/${options.src}?q=${options.quality}`; }, }; @@ -181,10 +187,10 @@ export default service; The kind of service exposed is completely invisible to users, the user API is always the same: the `Image` component and `getImage`. Refer to previous examples for usage. -### Additional method +### Additional methods ```ts -getHTMLAttributes(options: ImageTransform) { +getHTMLAttributes(options: ImageTransform, serviceOptions: Record<string, any>) { return { width: options.width, height: options.height @@ -193,7 +199,17 @@ getHTMLAttributes(options: ImageTransform) { } ``` -This method is available on both local and external services. +```ts +validateOptions(options: ImageTransform, serviceOptions: Record<string, any>) { + if (!options.width) { + throw new Error('Width is required!') + } + + return options; +} +``` + +These methods are available on both local and external services. # Background & Motivation @@ -212,6 +228,7 @@ In this RFC, we'd like to outline a plan / API for a core story for images. The - Make an image component that is easy to use and intuitive to understand. - Make it easy to use optimized images in Markdown, MDX and future formats. +- Make it easy to extend - Good, core-worthy, DX with awesome error messages, good types, good documentation - No more integration to install! @@ -239,7 +256,8 @@ A new virtual module will be exported from a Vite plugin (similar to `astro:cont - `Image` (see [Image Component](#image-component-1)) - `getImage` (see [JavaScript API](#javascript-api)) -- `getImageService` (see [Image Services](#image-services)) +- `getConfiguredImageService` (see [Image Services](#image-services)) +- `imageServiceConfig` (see [Image Services](#image-services)) We choose `astro:assets` over `astro:image` on purpose, as to make it intuitive that more things might get exposed from there over time. @@ -371,25 +389,25 @@ The different methods available are the following: **Required methods** -- `getURL(options: ImageTransform): string` +- `getURL(options: ImageTransform, serviceOptions: Record<string, any>): string` - For local services, return the URL of the endpoint managing your transformation (in SSR and dev). - For external services, return the final URL of the image. - `options` contain the parameters passed by the user **Required for Local services only** -- `parseURL(url: URL): ImageTransform` +- `parseURL(url: URL, serviceOptions: Record<string, any>): ImageTransform` - For SSR and dev, parses the generated URLs by `getURL` back into an ImageTransform to be used by `transform`. -- `transform(buffer: Buffer, options: ImageTransform): { data: Buffer, format: OutputFormat }` +- `transform(buffer: Buffer, options: ImageTransform, serviceOptions: Record<string, any>): { data: Buffer, format: OutputFormat }` - Transform and return the image. It is necessary to return a `format` to ensure that the proper MIME type is served to users in development and SSR. Ultimately, in development and SSR, it is up to the local endpoint (that `getURL` points to) to call both `parseURL` and `transform` if wanted. `transform` however, is called automatically during the build in SSG and for pre-rendered pages to create the final assets files. **Optional** -- `validateOptions(options: ImageTransform): ImageTransform` +- `validateOptions(options: ImageTransform, serviceOptions: Record<string, any>): ImageTransform` - Allows you to validate and augment the options passed by the user. This is useful for setting default options, or telling the user that a parameter is required. -- `getHTMLAttributes(options: ImageTransform): Record<string, any>` +- `getHTMLAttributes(options: ImageTransform, serviceOptions: Record<string, any>): Record<string, any>` - Return all additional attributes needed to render the image in HTML. For instance, you might want to return a specific `class` or `style`, or `width` and `height`. ### User configuration @@ -402,22 +420,28 @@ import { defineConfig } from "astro/config"; // https://astro.build/config export default defineConfig({ image: { - service: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string + service: { + entrypoint: "your-entrypoint", // 'astro/image/services/squoosh' | 'astro/image/services/sharp' | string + config: {}, + }, }, }); ``` +A config can additionally be passed to the service and will be passed to every hooks available. This can be useful if you want to allow the user to set a list of possible widths or only allow images from certain remote sources. + #### Facts - At this time, it is not possible to override this on a per-image basis (see [Non goals of this RFC](#non-goals-of-this-rfc)). - The `image.service` shape was chosen on purpose, for the future situation where multiple settings will be available under `image`. - A different image service can be set depending on the current mode (dev or build) using traditional techniques such as `process.env.MODE` +- Integrations can set the image service for the user ### Note Overall, it's important to remember that 99% of users won't create services, especially local ones. In addition to the services directly provided with Astro, third party packages can supply services for the users. -It's easy to imagine, for example, a `cloudinary-astro` package exposing a service. Or, the `@astrojs/vercel` adapter exposing a service using Vercel's Image Optimization API that user could use through `service: '@astrojs/vercel/image`. +It's easy to imagine, for example, a `cloudinary-astro` package exposing a service. Or, the `@astrojs/vercel` adapter exposing a service using Vercel's Image Optimization API that user could use through a `service: vercelImageService()` function. # Testing Strategy @@ -429,7 +453,7 @@ Much like the current `@astrojs/image` integration, this can be tested in many w Certain parts can be hard to fully E2E tests, such as "Was this image really generated with a quality of 47?", nonetheless, we can test that we at least provided the correct parameters down the chain. -Overall, I do not expect this feature to be particularly hard, as the `@astrojs/image` testing suite has already proven to work correctly. +Overall, I do not expect this feature to be particularly hard to test, as the `@astrojs/image` testing suite has already proven to work correctly. # Drawbacks From 593908b0ede44c4018ba14195688962b75434ec9 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 1 May 2023 15:12:26 -0400 Subject: [PATCH 184/300] new: summary, example, background, goals --- proposals/0032-data-collections.md | 114 +++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 proposals/0032-data-collections.md diff --git a/proposals/0032-data-collections.md b/proposals/0032-data-collections.md new file mode 100644 index 00000000..a281aa27 --- /dev/null +++ b/proposals/0032-data-collections.md @@ -0,0 +1,114 @@ +- Start Date: 2023-05-01 +- Reference Issues: https://github.com/withastro/roadmap/issues/530 +- Implementation PR: https://github.com/withastro/astro/pull/6850 + +# Summary + +Introduce a standard to store data separately from your content (ex. JSON files). + +# Example + +Like content collections, data collections are created in the `src/content/` directory. These collections should include only JSON files: + +``` +src/content/ + authors/ + ben.json + tony.json +``` + +These collections are configured using the same `defineCollection()` utility in your `content/config.ts` with `type: 'data'` specified: + +```ts +// src/content/config.ts +import { defineCollection, z } from 'astro:content'; + +const authors = defineCollection({ + type: 'data', + schema: z.object({ + name: z.string(), + twitter: z.string().url(), + }) +}); + +export const collections = { authors }; +``` + +These collections can also be queried using the `getCollection()` and `getEntry()` utilities: + +```astro +--- +import { getCollection, getEntry } from 'astro:content'; + +const authors = await getCollection('authors'); +const ben = await getEntry('authors', 'ben'); +--- + +<p> + Authors: {authors.map(author => author.data.name).join(', ')} +</p> +<p> + The coolest author: <a href={ben.data.twitter}>{ben.data.name}</a> +</p> +``` + +# Background & Motivation + +Content collections are restricted to supporting `.md`, `.mdx`, and `.mdoc` files. This is limiting for other forms of data you may need to store, namely raw data formats like JSON. + +Taking a blog post as the example, there will likely be author information thats reused across multiple blog posts. To standardize updates when, say, updating an author's profile picture, it's best to store authors in a standalone data entry. + +The content collections API was built generically to support this future, choosing format-agnostic naming like `data` instead of `frontmatter` and `body` instead of `rawContent`. Because of this, expanding support to new data formats without API changes is a natural progression. + +# Goals + +- **Introduce JSON collection support,** configurable and queryable with similar APIs to content collections. +- **Determine where data collections are stored.** We may allow data collections within `src/content/`, or introduce a new reserved directory. + +# Non-Goals + +- **Separate RFC:** Referencing data collection entries from existing content collections. This unlocks referencing, say, an author from your blog post frontmatter. See the related RFC for details: [TODO: link] + +# Detailed Design + +This is the bulk of the RFC. Explain the design in enough detail for somebody +familiar with Astro to understand, and for somebody familiar with the +implementation to implement. This should get into specifics and corner-cases, +and include examples of how the feature is used. Any new terminology should be +defined here. + +# Testing Strategy + +How will this feature's implementation be tested? Explain if this can be tested with +unit tests or integration tests or something else. If relevant, explain the test +cases that will be added to cover all of the ways this feature might be used. + +# Drawbacks + +Why should we _not_ do this? Please consider: + +- Implementation cost, both in term of code size and complexity. +- Whether the proposed feature can be implemented in user space. +- Impact on teaching people Astro. +- Integration of this feature with other existing and planned features +- Cost of migrating existing Astro applications (_is it a breaking change?_) + +There are tradeoffs to choosing any path. Attempt to identify them here. + +# Alternatives + +What other designs have been considered? What is the impact of not doing this? + +# Adoption strategy + +Please consider: + +- If we implement this proposal, how will existing Astro developers adopt it? +- Is this a breaking change? Can we write a codemod? +- Can we provide a runtime adapter library for the original API it replaces? +- How will this affect other projects in the Astro ecosystem? + +# Unresolved Questions + +Optional, but suggested for first drafts. +What parts of the design are still to be determined? From 49d3a0ba48e9d722019aa26d0305211698353b4f Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 1 May 2023 16:39:46 -0400 Subject: [PATCH 185/300] chore: gitignore --- .gitignore | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 298d2b7d..ded0ce30 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,7 @@ .vscode/settings.json # ignore IDEA folders -.idea \ No newline at end of file +.idea + +# MacOS +.DS_Store \ No newline at end of file From a8dae6d5855557d7bec9a7da904de8c2069e8dc7 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 1 May 2023 16:40:08 -0400 Subject: [PATCH 186/300] new: detailed design, drawbacks, alternatives, adoption --- proposals/0032-data-collections.md | 114 -------------- proposals/0033-data-collections.md | 244 +++++++++++++++++++++++++++++ 2 files changed, 244 insertions(+), 114 deletions(-) delete mode 100644 proposals/0032-data-collections.md create mode 100644 proposals/0033-data-collections.md diff --git a/proposals/0032-data-collections.md b/proposals/0032-data-collections.md deleted file mode 100644 index a281aa27..00000000 --- a/proposals/0032-data-collections.md +++ /dev/null @@ -1,114 +0,0 @@ -- Start Date: 2023-05-01 -- Reference Issues: https://github.com/withastro/roadmap/issues/530 -- Implementation PR: https://github.com/withastro/astro/pull/6850 - -# Summary - -Introduce a standard to store data separately from your content (ex. JSON files). - -# Example - -Like content collections, data collections are created in the `src/content/` directory. These collections should include only JSON files: - -``` -src/content/ - authors/ - ben.json - tony.json -``` - -These collections are configured using the same `defineCollection()` utility in your `content/config.ts` with `type: 'data'` specified: - -```ts -// src/content/config.ts -import { defineCollection, z } from 'astro:content'; - -const authors = defineCollection({ - type: 'data', - schema: z.object({ - name: z.string(), - twitter: z.string().url(), - }) -}); - -export const collections = { authors }; -``` - -These collections can also be queried using the `getCollection()` and `getEntry()` utilities: - -```astro ---- -import { getCollection, getEntry } from 'astro:content'; - -const authors = await getCollection('authors'); -const ben = await getEntry('authors', 'ben'); ---- - -<p> - Authors: {authors.map(author => author.data.name).join(', ')} -</p> -<p> - The coolest author: <a href={ben.data.twitter}>{ben.data.name}</a> -</p> -``` - -# Background & Motivation - -Content collections are restricted to supporting `.md`, `.mdx`, and `.mdoc` files. This is limiting for other forms of data you may need to store, namely raw data formats like JSON. - -Taking a blog post as the example, there will likely be author information thats reused across multiple blog posts. To standardize updates when, say, updating an author's profile picture, it's best to store authors in a standalone data entry. - -The content collections API was built generically to support this future, choosing format-agnostic naming like `data` instead of `frontmatter` and `body` instead of `rawContent`. Because of this, expanding support to new data formats without API changes is a natural progression. - -# Goals - -- **Introduce JSON collection support,** configurable and queryable with similar APIs to content collections. -- **Determine where data collections are stored.** We may allow data collections within `src/content/`, or introduce a new reserved directory. - -# Non-Goals - -- **Separate RFC:** Referencing data collection entries from existing content collections. This unlocks referencing, say, an author from your blog post frontmatter. See the related RFC for details: [TODO: link] - -# Detailed Design - -This is the bulk of the RFC. Explain the design in enough detail for somebody -familiar with Astro to understand, and for somebody familiar with the -implementation to implement. This should get into specifics and corner-cases, -and include examples of how the feature is used. Any new terminology should be -defined here. - -# Testing Strategy - -How will this feature's implementation be tested? Explain if this can be tested with -unit tests or integration tests or something else. If relevant, explain the test -cases that will be added to cover all of the ways this feature might be used. - -# Drawbacks - -Why should we _not_ do this? Please consider: - -- Implementation cost, both in term of code size and complexity. -- Whether the proposed feature can be implemented in user space. -- Impact on teaching people Astro. -- Integration of this feature with other existing and planned features -- Cost of migrating existing Astro applications (_is it a breaking change?_) - -There are tradeoffs to choosing any path. Attempt to identify them here. - -# Alternatives - -What other designs have been considered? What is the impact of not doing this? - -# Adoption strategy - -Please consider: - -- If we implement this proposal, how will existing Astro developers adopt it? -- Is this a breaking change? Can we write a codemod? -- Can we provide a runtime adapter library for the original API it replaces? -- How will this affect other projects in the Astro ecosystem? - -# Unresolved Questions - -Optional, but suggested for first drafts. -What parts of the design are still to be determined? diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md new file mode 100644 index 00000000..f4f247c1 --- /dev/null +++ b/proposals/0033-data-collections.md @@ -0,0 +1,244 @@ +- Start Date: 2023-05-01 +- Reference Issues: https://github.com/withastro/roadmap/issues/530 +- Implementation PR: https://github.com/withastro/astro/pull/6850 + +# Summary + +Introduce a standard to store data separately from your content (ex. JSON files). + +# Example + +Like content collections, data collections are created in the `src/content/` directory. These collections should include only JSON files: + +``` +src/content/ + authors/ + ben.json + tony.json +``` + +These collections are configured using the same `defineCollection()` utility in your `content/config.ts` with `type: 'data'` specified: + +```ts +// src/content/config.ts +import { defineCollection, z } from 'astro:content'; + +const authors = defineCollection({ + type: 'data', + schema: z.object({ + name: z.string(), + twitter: z.string().url(), + }) +}); + +export const collections = { authors }; +``` + +These collections can also be queried using the `getCollection()` and `getEntry()` utilities: + +```astro +--- +import { getCollection, getEntry } from 'astro:content'; + +const authors = await getCollection('authors'); +const ben = await getEntry('authors', 'ben'); +--- + +<p> + Authors: {authors.map(author => author.data.name).join(', ')} +</p> +<p> + The coolest author: <a href={ben.data.twitter}>{ben.data.name}</a> +</p> +``` + +# Background & Motivation + +Content collections are restricted to supporting `.md`, `.mdx`, and `.mdoc` files. This is limiting for other forms of data you may need to store, namely raw data formats like JSON. + +Taking a blog post as the example, there will likely be author information thats reused across multiple blog posts. To standardize updates when, say, updating an author's profile picture, it's best to store authors in a standalone data entry. + +The content collections API was built generically to support this future, choosing format-agnostic naming like `data` instead of `frontmatter` and `body` instead of `rawContent`. Because of this, expanding support to new data formats without API changes is a natural progression. + +# Goals + +- **Introduce JSON collection support,** configurable and queryable with similar APIs to content collections. +- **Determine where data collections are stored.** We may allow data collections within `src/content/`, or introduce a new reserved directory. + +# Non-Goals + +- **Separate RFC:** Referencing data collection entries from existing content collections. This unlocks referencing, say, an author from your blog post frontmatter. See the related RFC for details: [TODO: link] + +# Detailed Design + +Data collections are created with the `defineCollection()` utility, and **must** include `type: 'data'` to store JSON files. This means "mixed" collections containing both content and data entries are not supported, and should raise a helpful error to correct. + +```ts +// src/content/config.ts +import { defineCollection, z } from 'astro:content'; + +const authors = defineCollection({ + type: 'data', + schema: z.object({ + name: z.string(), + twitter: z.string().url(), + }) +}); + +const blog = defineCollection({ + // `type` can be omitted for content collections + // to make this change non-breaking. + // `type: 'content'` can also be used for completeness. + schema: z.object({...}), +}) + +export const collections = { authors }; +``` + +## Return type + +Data collection entries include the same `id`, `data`, and `collection` properties as content collections: + +- `id (string)` - The entry file name with the extension omitted. Spaces and capitalization are preserved. +- `collection (string)` - The collection name +- `data (object)` - The entry data as a JS object, parsed by the configured collection schema (if any). + +This also means content-specific fields like `slug`, `body`, and `render()` properties are **not** included. This is due to the following: + +- `render()`: this function is used by content collections to parse the post body into a usable Content component. Data collections do not have HTML to render, so the function is removed. +- `slug`: This is provided by content collections as a URL-friendly version of the file `id` for use as pages. Since data collections are not meant to be used as pages, this is omitted. +- `body`: Unlike content collections, which feature a post body separate from frontmatter, data collections are just... data. This field could be returned as the raw JSON body, though this would give `body` a double meaning depending on the context: non-data information for content collections, and the "raw" data itself for data collections. We avoid returning the body to avoid this confusion. + +## Querying + +Today's `getCollection()` utility can be used to fetch both content or data collections: + +```astro +--- +import { getCollection } from 'astro:content'; + +const authors = await getCollection('authors'); +--- + +<h1>Our Authors</h1> +<ul> + {authors.map(author => ( + <li><a href={author.data.twitter}>{author.data.name}</a></li> + ))} +</ul> +``` + +To retrieve individual entries, `getEntry()` can be used. This receives both the collection name and the entry `id` as described in the [Return type](#return-type). These can be passed as separate arguments or as object keys. + +```astro +--- +// src/pages/en/index.astro + +// Ex. retrieve a translations document stored in `src/content/i18n/en.json` +// Option 1: separate args +const english = await getEntry('i18n', 'en'); +// Option 2: object keys +const english = await getEntry({ collection: 'i18n', id: 'en' }); +--- + +<h1>{english.data.homePage.title}</h1> +<p>{english.data.homePage.tagline}</p> +``` + +> Note: the object keys approach is primarily meant for resolving references. See the [collection references RFC](TODO: link) for more. + +Thanks to the generic function name, this can be used as a replacement for `getEntryBySlug()` as well. When querying a content collection, `getEntry()` uses `slug` as the identifier: + + +```astro +--- +// Option 1: separate args +const welcomePost = await getEntry('blog', 'welcome'); +// Option 2: object keys +const welcomePost = await getEntry({ collection: 'blog', slug: 'welcome' }); +const Content = await welcomePost.render(); +--- + +<h1>{welcomePost.data.title}</h1> +<Content /> +``` + +### `id` vs `slug` + +You may have noticed two competing identifiers depending on the collection type: `id` for data, and `slug` for content. This is an inconsistency we'd like to address in a future RFC, with `id` becoming the new standard for identifying collection entries. For now, `slug` will remain on content collections to make the introduction of data collections non-breaking. + +**Full background:** `slug` was originally introduced alongside the content `id` to have a URL-friendly version of the file path, which can be passed to `getStaticPaths()` for route generation. Data collections are not intended to be used as routes, so we don't want to perpetuate this pattern. `slug` also "slugifies" the file path by removing capitalization and replacing spaces with dashes. If we added this processing to data collection IDs, [collection references](TODO: link) will be less intuitive to use (i.e. "do I include spaces in the referenced ID here?"). + +## Implementation + +Data collections should following the API design of content collections with a stripped-down featureset. To wire up data collections, we will introduce an internal utility that mirrors our `addContentEntryType()` integration function. This example registers a new `dataEntryType` for `.json` files, with necessary logic to parse data as a JS object: + +```ts +addDataEntryType({ + extensions: ['.json'], + getEntryInfo({ contents, fileUrl }) { + // Handle empty JSON files, which cause `JSON.parse` to throw + if (contents === undefined || contents === '') return { data: {} }; + + const data = JSON.parse(contents); + + if (data == null || typeof data !== 'object') + throw new Error(`JSON collection entry ${fileUrl.pathname} must be an object.`); + + return { data }; + }, +}); +``` + +Then, we will update our type generator to recognize these data-specific file extensions. This should also raise errors when collections are misconfigured (i.e. `type: 'data'` is missing from the config file) and when a mix of content and data in the same collection is detected. + +The `astro:content` runtime module should also be updated to glob these file extensions, and respect the entry ID from the new `getEntry()` utility function. + +# Testing Strategy + +- Fixture tests defining data collections, ensuring schemas are respected and query APIs (`getCollection()` and `getEntry()`) return entries of the correct type. +- Test error states: mixed data / content collections are not allowed, `type: 'data'` is enforced + +# Drawbacks + +- JSON could be considered added complexity, when users can create `.md` files storing all data via frontmatter instead. Though true for trivial content, this is restrictive for use cases like i18n lookup files (which are typically JSON). + +# Alternatives + +## Using multi-file vs. single-file + +Early proposals supported single0file data collections. This allows storing _all_ data collection entries as an array in one `.json` file, instead of splitting up entries per-file as we do content collections today. For example, a single `src/content/authors.json` file instead of a few `src/content/authors/[name].json` files. + +We want to stick with a single API design for an experimental release. Unlike multi-file, there are some prohibitive reasons against single-file collections: + +- **Big arrays don't scale for complex data entries**, like i18n translation files. Users will likely want to split up by file here, especially where the status quo is `en.json | fr.json | jp.json ...` for this use case. +- **We'd need to parse the whole array of entries** to determine entry IDs. This could be a performance bottleneck vs. pulling IDs from file names. +- **It would be different from content collections,** which means a learning curve. + +Due to these, we decided against single-file for now. Though we do recognize the convenience of colocation that can be explored in the future. + +## Using a reserved `src/data/` directory + +Early proposals considered moving data collections to a separate directory from content called `src/data/`. We weighed pros and cons for this approach, and ultimately decided `src/contenet/` had a better set of tradeoffs: + +### In favor of `src/data/` + +- Follows patterns for storing arbitrary JSON or YAML in 11ty [(see the `_data/` convention).](https://www.11ty.dev/docs/data-global/) +- Clearly defines how data and content are distinct concepts that return different information (ex. content includes a `body` and a `render()` utility, while data does not). This makes data-specific APIs like `getDataEntryById()` easier to conceptualize. +- Avoids confusion on whether mixing content and data in the same collection is supported; Collections should be distinctly content _or_ data. We can add appropriate error states for `src/content/`, but using the directory to define the collection type creates a pit of success. + +### In favor of `src/content/` + +- [Follows Nuxt content's pattern](https://content.nuxtjs.org/guide/writing/json#json) for storing data in a `content/` directory +- Avoids a new reserved directory. This could mean a simpler learning curve, i.e. I already know collections live in `src/content/`, so I'll add my new "data" collection in this directory too. From user testing with the core team, this expectation arose a few times. +- Allows switching from JSON to MD and back again more easily, without moving the directory. Example: you find a need for landing pages and bios for your `authors` data collection, so you move `json -> md` while retaining your schema. +- Avoids the need for [moving the collection config to your project root](https://github.com/withastro/roadmap/discussions/551). With `src/data/`, requiring the `config` to live in a `src/content/config.ts` is confusing. This is amplified when you do not have any content collections. + +# Adoption strategy + +- Introduce behind an experimental flag through a minor release. Data collections can be introduced non-breaking. +- Document data collections alongside content collections for discoverability. + +# Unresolved Questions + +N/A From d89e2a4c28379108501aa6bf40d2f8d93d81ad02 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 1 May 2023 19:14:17 -0700 Subject: [PATCH 187/300] new: collection references draft --- proposals/0034-collection-references.md | 214 ++++++++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644 proposals/0034-collection-references.md diff --git a/proposals/0034-collection-references.md b/proposals/0034-collection-references.md new file mode 100644 index 00000000..75a3e433 --- /dev/null +++ b/proposals/0034-collection-references.md @@ -0,0 +1,214 @@ +- Start Date: 2023-05-01 +- Reference Issues: https://github.com/withastro/roadmap/issues/530 +- Implementation PR: https://github.com/withastro/astro/pull/6850 + +# Summary + +Introduce a standard to reference collection entries from other collections by ID. + +# Example + +References are defined in the content config using the `reference()` utility. This receives the collection name as the parameter: + +```ts +// src/content/config.ts +import { reference, defineCollection, z } from 'astro:content'; + +const blog = defineCollection({ + schema: z.object({ + title: z.string(), + author: reference('authors'), + }) +}); + +const authors = defineCollection({ + type: 'data', + schema: z.object({ + name: z.string(), + }) +}) + +export const collections = { authors, blog }; +``` + +Now, blog entries can include `author: [author-id]` in their frontmatter, where the `[author-id]` is a valid entry ID for that collection. This example references a member of the `authors` collection, `src/content/authors/ben-holmes.json`: + +```yaml +# src/content/blog/welcome.md +--- +title: "Welcome to references!" +author: ben-holmes +--- +``` + +This will validate the ID and return a reference object on `data.author`. To parse entry data, you can pass this reference to the `getEntry()` utility: + +```astro +--- +import { getEntry, getCollection } from 'astro:content'; + +export async function getStaticPaths() { + const posts = await getCollection('blog'); + return posts.map(p => ({ + params: { slug: p.slug }, + props: post, + })) +} + +const blogPost = Astro.props; +const author = await getEntry(blogPost.data.author); +const { Content } = await blogPost.render(); +--- + +<h1>{blogPost.data.title}</h1> +<p>Author: {author.data.name}</p> +<Content /> +``` + +# Background & Motivation + +Content collections were designed to be configured and queried individually. But as users have started adapting collections to more complex patterns, there are compelling use cases to "reference" one collection entry from another: + +- Reference "related posts" within a collection of blog posts. +- Create a collection of authors, and reference those authors from a collection of documentation pages. +- Create a collection of images with reusable alt text, and reference those images for article banners. +- Create a collection of tags with display text or icons, and reference those tags from a collection of blog posts. + +These use cases span data collection -> content collection references, content -> content references, and even references within the same collection. These make a "reference" primitive compelling as collections types grow. + +# Goals + +- Introduce an API to reference collection entries from another collection, regardless of the collection type (content or data). +- Support references to other entries within the same collection. +- Consider Both one-to-one and one-to-many relationships between content and data (ex. allow passing a list of author IDs in your frontmatter). + + +# Non-Goals + +- **First-class helpers for many-to-many references.** In other words, if a blog post has authors, Astro will not help retrieve blog posts by author. This will require manual querying for all blog posts, and filtering to find blog posts containing a particular author. + +# Detailed Design + +The `reference()` utility receives the collection name as a string, and validates this string at build-time using a Zod transform. This transform does _not_ attempt to import the referenced object directly, instead returning the referenced identifier and collection name after validating. + +## Configuration + +This example configures a `blog` collection with a few properties that use references: +- `banner` - entry in the `banners` data collection, containing image asset srcs and reusable alt text +- `relatedPosts` - list of entries in the current `blog` collection +- `authors` - list of entries in the `authors` data collection, containing author information + +```ts +import { defineCollection, reference, z } from 'astro:content'; + +const blog = defineCollection({ + schema: z.object({ + title: z.string(), + banner: reference('banners'), + authors: z.array(reference('authors')), + relatedPosts: z.array(reference('blog')), + }) +}); + +const banners = defineCollection({ + type: 'data', + schema: ({ image }) => z.object({ + src: image(), + alt: z.string(), + }) +}); + +const authors = defineCollection({ + type: 'data', + schema: z.object({ + name: z.string(), + }) +}); + +export const collections = { blog, banners, authors }; +``` + +## Entry references + +Entries are referenced by their `id` property for data collections, or by their `slug` property for content collections. These fit the recommended identifiers for each type. + +This is an example blog post using the configuration above: + +```yaml +# src/content/blog/welcome.md +--- +title: "Welcome to references!" +banner: welcome # references `src/content/banners/welcome.json` +authors: +- ben-holmes # references `src/content/authors/ben-holmes.json` +- tony-sull # references `src/content/authors/tony-sull.json` +relatedPosts: +- getting-started # references `src/content/blog/getting-started.md` +``` + +## Querying + +References return the `id` and `collection` (and `slug` for content collection entries) as an object. This is the example output when fetching the `welcome.md` file above using the new `getEntry()` helper: + +```astro +--- +import { getEntry, getCollection } from 'astro:content'; +import { Image } from 'astro:asset'; + +const { data } = await getEntry('blog', 'welcome'); +// data.banners -> [{ id: 'welcome.json', collection: 'banners' }] +// data.authors -> [{ id: 'ben-holmes.json', collection: 'authors' }, { id: 'tony-sull.json', collection: 'authors' }] +// data.relatedPosts -> [{ slug: 'getting-started', collection: 'blog' }] +--- +``` + +These entries are intentionally unresolved for a few reasons: +- To avoid unnecessary work resolving entry data until you're ready to _use_ that data. This is similar to the `.render()` extension function for retrieving the `Content` component. +- To prevent circular type dependencies in your Zod schemas. This is especially true for self references like `relatedPosts`, which causes TypeScript issues when using tools like Zod for type inference. +- To prevent infinite resolution loops for nested references. Again, this is a risk for self references. + +To retrieve entry data, pass a given reference to the `getEntry()` helper. This returns a type-safe result based on the id and collection name: + +```astro +--- +import { getEntry, getCollection } from 'astro:content'; +import { Image } from 'astro:asset'; + +const { data } = await getEntry('blog', 'welcome'); + +const banner = await getEntry(data.banner); +const authors = await getEntry(data.authors); +const relatedPosts = await getEntry(data.relatedPosts); +const { Content } = await blogPost.render(); +--- + +<Image src={banner.data.src} alt={banner.data.alt} /> +<h1>{blogPost.data.title}</h1> +<p>Authors: {authors.map(a => a.data.name).join(', ')}</p> +<Content /> + +<h2>You might also like</h2> +{relatedPosts.map(p => <Card {...p} />)} +``` + +# Testing Strategy + +- Integration tests for each combination of reference: content -> data, data -> content, content -> content, data -> data +- Unit tests for the `getEntry()` utility when resolving references +- Integration tests for self references + +# Drawbacks + +TODO + +# Alternatives + +TODO + +# Adoption strategy + +TODO + +# Unresolved Questions + +TODO \ No newline at end of file From 01cd403a030f45e4676fe8267af5b8d09a47dfb6 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 1 May 2023 19:20:04 -0700 Subject: [PATCH 188/300] chore: link --- proposals/0033-data-collections.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md index f4f247c1..a82ae6c0 100644 --- a/proposals/0033-data-collections.md +++ b/proposals/0033-data-collections.md @@ -67,7 +67,7 @@ The content collections API was built generically to support this future, choosi # Non-Goals -- **Separate RFC:** Referencing data collection entries from existing content collections. This unlocks referencing, say, an author from your blog post frontmatter. See the related RFC for details: [TODO: link] +- **Separate RFC:** Referencing data collection entries from existing content collections. This unlocks referencing, say, an author from your blog post frontmatter. See the [related collection references RFC](https://github.com/withastro/roadmap/blob/d89e2a4c28379108501aa6bf40d2f8d93d81ad02/proposals/0034-collection-references.md) for details. # Detailed Design @@ -145,7 +145,7 @@ const english = await getEntry({ collection: 'i18n', id: 'en' }); <p>{english.data.homePage.tagline}</p> ``` -> Note: the object keys approach is primarily meant for resolving references. See the [collection references RFC](TODO: link) for more. +> Note: the object keys approach is primarily meant for resolving references. See the [collection references RFC](https://github.com/withastro/roadmap/blob/d89e2a4c28379108501aa6bf40d2f8d93d81ad02/proposals/0034-collection-references.md) for more. Thanks to the generic function name, this can be used as a replacement for `getEntryBySlug()` as well. When querying a content collection, `getEntry()` uses `slug` as the identifier: @@ -167,7 +167,7 @@ const Content = await welcomePost.render(); You may have noticed two competing identifiers depending on the collection type: `id` for data, and `slug` for content. This is an inconsistency we'd like to address in a future RFC, with `id` becoming the new standard for identifying collection entries. For now, `slug` will remain on content collections to make the introduction of data collections non-breaking. -**Full background:** `slug` was originally introduced alongside the content `id` to have a URL-friendly version of the file path, which can be passed to `getStaticPaths()` for route generation. Data collections are not intended to be used as routes, so we don't want to perpetuate this pattern. `slug` also "slugifies" the file path by removing capitalization and replacing spaces with dashes. If we added this processing to data collection IDs, [collection references](TODO: link) will be less intuitive to use (i.e. "do I include spaces in the referenced ID here?"). +**Full background:** `slug` was originally introduced alongside the content `id` to have a URL-friendly version of the file path, which can be passed to `getStaticPaths()` for route generation. Data collections are not intended to be used as routes, so we don't want to perpetuate this pattern. `slug` also "slugifies" the file path by removing capitalization and replacing spaces with dashes. If we added this processing to data collection IDs, [collection references](https://github.com/withastro/roadmap/blob/d89e2a4c28379108501aa6bf40d2f8d93d81ad02/proposals/0034-collection-references.md) will be less intuitive to use (i.e. "do I include spaces in the referenced ID here?"). ## Implementation From 3c17eff9a1e4d7743857f115ecd6f3552471f65e Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 10:25:58 -0400 Subject: [PATCH 189/300] edit: allow `reference()` on schema function --- proposals/0034-collection-references.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/proposals/0034-collection-references.md b/proposals/0034-collection-references.md index 75a3e433..6759a24a 100644 --- a/proposals/0034-collection-references.md +++ b/proposals/0034-collection-references.md @@ -91,6 +91,29 @@ These use cases span data collection -> content collection references, content - The `reference()` utility receives the collection name as a string, and validates this string at build-time using a Zod transform. This transform does _not_ attempt to import the referenced object directly, instead returning the referenced identifier and collection name after validating. +This utility can be imported either as a top-level import, or as a function parameter on the collection schema. The latter mirrors our existing `image()` helper. Allowing both should make the experience simpler for users reliant on experimental assets, and familiar to those that prefer top-level imports instead. + +```ts +// Option 1: top-level import +import { reference } from 'astro:content'; + +const blog = defineCollection({ + schema: z.object({ + title: z.string(), + authors: z.array(reference('authors')), + }) +}); + +// Option 2: schema function parameter +const banners = defineCollection({ + type: 'data', + schema: ({ image, reference }) => z.object({ + src: image(), + artist: reference('artists'), + }) +}); +``` + ## Configuration This example configures a `blog` collection with a few properties that use references: From 04a9e33d6c7dba44ccac0a3e566b8daafe346b52 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 10:46:10 -0400 Subject: [PATCH 190/300] edit: update to JSON or YAML --- proposals/0033-data-collections.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md index a82ae6c0..7d353555 100644 --- a/proposals/0033-data-collections.md +++ b/proposals/0033-data-collections.md @@ -8,7 +8,7 @@ Introduce a standard to store data separately from your content (ex. JSON files) # Example -Like content collections, data collections are created in the `src/content/` directory. These collections should include only JSON files: +Like content collections, data collections are created in the `src/content/` directory. These collections should include only JSON or YAML files: ``` src/content/ @@ -54,7 +54,7 @@ const ben = await getEntry('authors', 'ben'); # Background & Motivation -Content collections are restricted to supporting `.md`, `.mdx`, and `.mdoc` files. This is limiting for other forms of data you may need to store, namely raw data formats like JSON. +Content collections are restricted to supporting `.md`, `.mdx`, and `.mdoc` files. This is limiting for other forms of data you may need to store, namely raw data formats like JSON or YAML. Taking a blog post as the example, there will likely be author information thats reused across multiple blog posts. To standardize updates when, say, updating an author's profile picture, it's best to store authors in a standalone data entry. @@ -62,7 +62,7 @@ The content collections API was built generically to support this future, choosi # Goals -- **Introduce JSON collection support,** configurable and queryable with similar APIs to content collections. +- **Introduce JSON and YAML collection support,** configurable and queryable with similar APIs to content collections. - **Determine where data collections are stored.** We may allow data collections within `src/content/`, or introduce a new reserved directory. # Non-Goals @@ -71,7 +71,7 @@ The content collections API was built generically to support this future, choosi # Detailed Design -Data collections are created with the `defineCollection()` utility, and **must** include `type: 'data'` to store JSON files. This means "mixed" collections containing both content and data entries are not supported, and should raise a helpful error to correct. +Data collections are created with the `defineCollection()` utility, and **must** include `type: 'data'` to store JSON or YAML files. This means "mixed" collections containing both content and data entries are not supported, and should raise a helpful error to correct. ```ts // src/content/config.ts @@ -105,7 +105,7 @@ Data collection entries include the same `id`, `data`, and `collection` properti This also means content-specific fields like `slug`, `body`, and `render()` properties are **not** included. This is due to the following: -- `render()`: this function is used by content collections to parse the post body into a usable Content component. Data collections do not have HTML to render, so the function is removed. +- `render()`: This function is used by content collections to parse the post body into a usable Content component. Data collections do not have HTML to render, so the function is removed. - `slug`: This is provided by content collections as a URL-friendly version of the file `id` for use as pages. Since data collections are not meant to be used as pages, this is omitted. - `body`: Unlike content collections, which feature a post body separate from frontmatter, data collections are just... data. This field could be returned as the raw JSON body, though this would give `body` a double meaning depending on the context: non-data information for content collections, and the "raw" data itself for data collections. We avoid returning the body to avoid this confusion. @@ -169,9 +169,9 @@ You may have noticed two competing identifiers depending on the collection type: **Full background:** `slug` was originally introduced alongside the content `id` to have a URL-friendly version of the file path, which can be passed to `getStaticPaths()` for route generation. Data collections are not intended to be used as routes, so we don't want to perpetuate this pattern. `slug` also "slugifies" the file path by removing capitalization and replacing spaces with dashes. If we added this processing to data collection IDs, [collection references](https://github.com/withastro/roadmap/blob/d89e2a4c28379108501aa6bf40d2f8d93d81ad02/proposals/0034-collection-references.md) will be less intuitive to use (i.e. "do I include spaces in the referenced ID here?"). -## Implementation +## Internals -Data collections should following the API design of content collections with a stripped-down featureset. To wire up data collections, we will introduce an internal utility that mirrors our `addContentEntryType()` integration function. This example registers a new `dataEntryType` for `.json` files, with necessary logic to parse data as a JS object: +Data collections should following the API design of content collections with a stripped-down featureset. To wire up data collections, we will introduce an internal utility that mirrors our `addContentEntryType()` integration function. JSON and YAML will be preconfigured by Astro, as a way to dogfood a generic API for wiring up any data collection type in the future. This example shows an internal implementation for `.json` files: ```ts addDataEntryType({ @@ -201,7 +201,7 @@ The `astro:content` runtime module should also be updated to glob these file ext # Drawbacks -- JSON could be considered added complexity, when users can create `.md` files storing all data via frontmatter instead. Though true for trivial content, this is restrictive for use cases like i18n lookup files (which are typically JSON). +- Data collections could be considered added complexity, when users can create `.md` files storing all data via frontmatter instead. Though true for trivial content, this is restrictive for use cases like i18n lookup files (which are typically JSON). # Alternatives @@ -231,7 +231,7 @@ Early proposals considered moving data collections to a separate directory from - [Follows Nuxt content's pattern](https://content.nuxtjs.org/guide/writing/json#json) for storing data in a `content/` directory - Avoids a new reserved directory. This could mean a simpler learning curve, i.e. I already know collections live in `src/content/`, so I'll add my new "data" collection in this directory too. From user testing with the core team, this expectation arose a few times. -- Allows switching from JSON to MD and back again more easily, without moving the directory. Example: you find a need for landing pages and bios for your `authors` data collection, so you move `json -> md` while retaining your schema. +- Allows switching from data to Markdown and back again more easily, without moving the directory. Example: you find a need for landing pages and bios for your `authors` data collection, so you move `json -> md` while retaining your schema. - Avoids the need for [moving the collection config to your project root](https://github.com/withastro/roadmap/discussions/551). With `src/data/`, requiring the `config` to live in a `src/content/config.ts` is confusing. This is amplified when you do not have any content collections. # Adoption strategy From c5363bab0388e1538f1824c1e7d6c9039be43d71 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 10:52:47 -0400 Subject: [PATCH 191/300] edit: add `getEntries()` to references --- proposals/0034-collection-references.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/0034-collection-references.md b/proposals/0034-collection-references.md index 6759a24a..d47e48d2 100644 --- a/proposals/0034-collection-references.md +++ b/proposals/0034-collection-references.md @@ -190,18 +190,21 @@ These entries are intentionally unresolved for a few reasons: - To prevent circular type dependencies in your Zod schemas. This is especially true for self references like `relatedPosts`, which causes TypeScript issues when using tools like Zod for type inference. - To prevent infinite resolution loops for nested references. Again, this is a risk for self references. -To retrieve entry data, pass a given reference to the `getEntry()` helper. This returns a type-safe result based on the id and collection name: +To retrieve entry data, pass a given reference to the new `getEntry()` and `getEntries()` helpers. The former is used for resolving a single reference, and the latter is used for arrays of references belonging to the same collection. These return a type-safe result based on the id and collection name. Some example cases are shown here: ```astro --- -import { getEntry, getCollection } from 'astro:content'; +import { getEntry, getEntries, getCollection } from 'astro:content'; import { Image } from 'astro:asset'; +// Get a blog post using positional arguments const { data } = await getEntry('blog', 'welcome'); +// Resolve singular reference const banner = await getEntry(data.banner); -const authors = await getEntry(data.authors); -const relatedPosts = await getEntry(data.relatedPosts); +// Resolve arrays of references +const authors = await getEntries(data.authors); +const relatedPosts = await getEntries(data.relatedPosts); const { Content } = await blogPost.render(); --- From 3d1114362d88cbdbcb2ab7f94f4ec9e96558e711 Mon Sep 17 00:00:00 2001 From: Ben Holmes <hey@bholmes.dev> Date: Mon, 8 May 2023 12:36:51 -0400 Subject: [PATCH 192/300] fix: typo "single0file" Co-authored-by: Emanuele Stoppa <my.burning@gmail.com> --- proposals/0033-data-collections.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md index 7d353555..f377612f 100644 --- a/proposals/0033-data-collections.md +++ b/proposals/0033-data-collections.md @@ -207,7 +207,7 @@ The `astro:content` runtime module should also be updated to glob these file ext ## Using multi-file vs. single-file -Early proposals supported single0file data collections. This allows storing _all_ data collection entries as an array in one `.json` file, instead of splitting up entries per-file as we do content collections today. For example, a single `src/content/authors.json` file instead of a few `src/content/authors/[name].json` files. +Early proposals supported single file data collections. This allows storing _all_ data collection entries as an array in one `.json` file, instead of splitting up entries per-file as we do content collections today. For example, a single `src/content/authors.json` file instead of a few `src/content/authors/[name].json` files. We want to stick with a single API design for an experimental release. Unlike multi-file, there are some prohibitive reasons against single-file collections: From fa80178362386418fcb73967d6ff27798847eea6 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 8 May 2023 15:49:06 -0400 Subject: [PATCH 193/300] HTML Minification RFC --- proposals/html-minification.md | 78 ++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 proposals/html-minification.md diff --git a/proposals/html-minification.md b/proposals/html-minification.md new file mode 100644 index 00000000..d8c01421 --- /dev/null +++ b/proposals/html-minification.md @@ -0,0 +1,78 @@ +- Start Date: 2023-05-08 +- Reference Issues: https://github.com/withastro/roadmap/issues/537 +- Implementation PR: https://github.com/withastro/astro/pull/6706 + +# Summary + +Provide a config value to enable HTML minification. + +# Example + +In the Astro config: + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + compressHTML: true +}) +``` + +This will enable HTML minfication for all pages. + +# Background & Motivation + +This is an often requested feature as minified HTML is a performance improvement some site measuring tools recommend. + +However there are some difficulties that have prevented it from being implemented sooner: + +- HTML minification can cause server/client hydration mismatches +- Streaming uses newlines to delineate when to flush buffered text in a response +- Frameworks can use HTML comments as expression markers, which can cause issues if these are removed by a minifier + +HTML minification is enabled in the compiler via the `compress` option, so not a lot of work is needed to enable this feature. + +# Goals + +- Safely (non-destructively) collapse whitespace during `.astro` file compilation. +- Allow users to opt-in to minification, keeping it disabled to prevent unexpected output. + +# Non-Goals + +- Minifying non-`.astro` components. This can be done with middleware. +- Minifying the full page. This would prevent streaming. + +# Detailed Design + +In `@astrojs/compiler` 1.0 support for the `compress` option was added. When enabled this option will remove whitespace when a `.astro` component is compiled to JavaScript. + +In Astro we'll add a `compressHTML` option that defaults to `false`. This option is passed directly to the `compress` option in the compiler. + +This is a top-level configuration value so that you get the same result in development and production modes. + +# Testing Strategy + +Fixture based tests are best here since this feature touches the config through rendering. Tests will check the output and ensure that they are + +How will this feature's implementation be tested? Explain if this can be tested with +unit tests or integration tests or something else. If relevant, explain the test +cases that will be added to cover all of the ways this feature might be used. + +# Drawbacks + +- This approach does not compress non-`.astro` component usage. +- There is some overlap with middleware. A middleware would be able to compress each chunk. The downside is that a user would need to import and use this middleware. + +# Alternatives + +- This could be implemented as a middleware function that a user could import and use. + - **Downside**: Middleware is still experimental. + - **Downside**: This is a lot of code just to enable a feature. User has to learn middleware where they otherwise might not be using it. + - **Downside**: Compression is already enabled in the compiler, so this is a small change to allow it to happen. + +# Adoption strategy + +- This is a small change so no experimental release is thought to be needed. +- Preview release will occur to allow user testing. +- Full release with a merged RFC in the next `minor`. +- Documented via the config reference page. \ No newline at end of file From 82fa78564671b48cd2dd6451057be16c70df8a8f Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 16:29:12 -0400 Subject: [PATCH 194/300] new: drawbacks, alts, adoption strat --- proposals/0034-collection-references.md | 32 +++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/proposals/0034-collection-references.md b/proposals/0034-collection-references.md index d47e48d2..33a37516 100644 --- a/proposals/0034-collection-references.md +++ b/proposals/0034-collection-references.md @@ -225,16 +225,40 @@ const { Content } = await blogPost.render(); # Drawbacks -TODO +- Adding references is a slippery slope down the "Astro is building its own ORM" rabbithole. Future features including many-to-many relations could bring complexity that hurts Astro's beginner friendliness. # Alternatives -TODO +## `collection.reference()` vs. `reference('collection-name')` + +An early implementation made references an extension function on each collection, rather than an imported function that accepts collection names as strings. This avoids confusion about where `reference()` is imported from. It also discourages infinite reference loops by relying on the ordering of variables in your config. An example implementation: + +```ts +import { defineCollection } from 'astro:content'; + +const authors = defineCollection({ + type: 'data', + schema: z.object({...}) +}); + +const blog = defineCollection({ + schema: z.object({ + authors: z.array(authors.reference()), + }) +}) +``` + +However, this has a few implementation drawbacks: +- **Self-references get more complicated.** Since a variable can't refer to itself, we'd need to [suggest `z.lazy()`](https://github.com/colinhacks/zod#recursive-types) for recursion. +- **Schemas do not have access to their collection.** Recall that collection names are defined not on the `defineCollection()` call, but the `export const collections = {...}` object. This requires code generation to tie references back to their collection name. + +These ultimately complicate our docs and source code more than the suggested implementation, so we've decided against this design. # Adoption strategy -TODO +- Release behind the proposed data collections experimental flag +- Document references alongside our existing content collections documentation # Unresolved Questions -TODO \ No newline at end of file +N/A \ No newline at end of file From 819260ff8966a2aa17462499b57eb5436225a64d Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 16:30:25 -0400 Subject: [PATCH 195/300] chore: remove now resolved question --- proposals/0033-data-collections.md | 1 - 1 file changed, 1 deletion(-) diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md index f377612f..b7000876 100644 --- a/proposals/0033-data-collections.md +++ b/proposals/0033-data-collections.md @@ -63,7 +63,6 @@ The content collections API was built generically to support this future, choosi # Goals - **Introduce JSON and YAML collection support,** configurable and queryable with similar APIs to content collections. -- **Determine where data collections are stored.** We may allow data collections within `src/content/`, or introduce a new reserved directory. # Non-Goals From 045bd4958f1aa28228ad0294aa43f3a3922414cd Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Mon, 8 May 2023 16:31:39 -0400 Subject: [PATCH 196/300] edit: clarify ids with the same filename are not allowed --- proposals/0033-data-collections.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0033-data-collections.md b/proposals/0033-data-collections.md index b7000876..b94b955c 100644 --- a/proposals/0033-data-collections.md +++ b/proposals/0033-data-collections.md @@ -98,7 +98,7 @@ export const collections = { authors }; Data collection entries include the same `id`, `data`, and `collection` properties as content collections: -- `id (string)` - The entry file name with the extension omitted. Spaces and capitalization are preserved. +- `id (string)` - The entry file name with the extension omitted. Spaces and capitalization are preserved. This means data collection files of the same name but different extensions ('ben.json' and 'ben.yaml') **should raise an error.** - `collection (string)` - The collection name - `data (object)` - The entry data as a JS object, parsed by the configured collection schema (if any). From c011c30187f3633beb671f60c93f7d14882a4b14 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Tue, 9 May 2023 15:56:27 -0400 Subject: [PATCH 197/300] Specify a final comment period --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 05b082b7..b498dcf3 100644 --- a/README.md +++ b/README.md @@ -54,9 +54,9 @@ The proposal champion can request feedback on their RFC at any point, either asy ## Stage 4: Ship it! -An RFC is ready to be approved and finalized once it's Pull Request is ready for its final review. RFC approval can happen asynchronously, or in-person during one of our weekly community calls. +An RFC is ready to be approved and finalized once it's Pull Request is ready for its final review. RFC approval can be happened through a "call for consensus". When a champion thinks the RFC is ready he can ask for a call for consensus. -Final RFC approval happens by vote, following our existing [RFC Proposal](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#voting-rfc-proposals) voting process. +Some member of the core team will motion for a final comment period (FCP). This follows our existing [RFC Proposal](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#voting-rfc-proposals) voting process. Once the final comment period has elapsed the RFC will be merged. --- From 58ffcf8cef95cdf906e4539fc972e320540c841b Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Tue, 9 May 2023 15:58:49 -0400 Subject: [PATCH 198/300] Fix language --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b498dcf3..a92bd242 100644 --- a/README.md +++ b/README.md @@ -54,9 +54,9 @@ The proposal champion can request feedback on their RFC at any point, either asy ## Stage 4: Ship it! -An RFC is ready to be approved and finalized once it's Pull Request is ready for its final review. RFC approval can be happened through a "call for consensus". When a champion thinks the RFC is ready he can ask for a call for consensus. +An RFC is ready to be approved and finalized once it's Pull Request is ready for its final review. When a champion thinks the RFC is ready he can ask for a call for consensus. -Some member of the core team will motion for a final comment period (FCP). This follows our existing [RFC Proposal](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#voting-rfc-proposals) voting process. Once the final comment period has elapsed the RFC will be merged. +At this time, some member of the core team will motion for a final comment period (FCP). This follows our existing [RFC Proposal](https://github.com/withastro/.github/blob/main/GOVERNANCE.md#voting-rfc-proposals) voting process. Once the final comment period has elapsed the RFC will be merged if there are no objections. --- From bfb1be422edcc1b553feeebb5adff65123fbe987 Mon Sep 17 00:00:00 2001 From: bholmesdev <hey@bholmes.dev> Date: Wed, 10 May 2023 13:09:57 -0400 Subject: [PATCH 199/300] chore: remove second reference() import option --- proposals/0034-collection-references.md | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/proposals/0034-collection-references.md b/proposals/0034-collection-references.md index 33a37516..2a2b574f 100644 --- a/proposals/0034-collection-references.md +++ b/proposals/0034-collection-references.md @@ -89,12 +89,9 @@ These use cases span data collection -> content collection references, content - # Detailed Design -The `reference()` utility receives the collection name as a string, and validates this string at build-time using a Zod transform. This transform does _not_ attempt to import the referenced object directly, instead returning the referenced identifier and collection name after validating. - -This utility can be imported either as a top-level import, or as a function parameter on the collection schema. The latter mirrors our existing `image()` helper. Allowing both should make the experience simpler for users reliant on experimental assets, and familiar to those that prefer top-level imports instead. +The `reference()` utility receives the collection name as a string, and validates this string at build-time using a Zod transform. This transform does _not_ attempt to import the referenced object directly, instead returning the referenced identifier and collection name after validating. This utility can be imported as a top-level import like so: ```ts -// Option 1: top-level import import { reference } from 'astro:content'; const blog = defineCollection({ @@ -103,15 +100,6 @@ const blog = defineCollection({ authors: z.array(reference('authors')), }) }); - -// Option 2: schema function parameter -const banners = defineCollection({ - type: 'data', - schema: ({ image, reference }) => z.object({ - src: image(), - artist: reference('artists'), - }) -}); ``` ## Configuration From 12404dc2e4dc7e37a8637643864c7b872f56c666 Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Fri, 12 May 2023 21:49:18 +0800 Subject: [PATCH 200/300] Create custom-client-directives.md --- proposals/custom-client-directives.md | 111 ++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 proposals/custom-client-directives.md diff --git a/proposals/custom-client-directives.md b/proposals/custom-client-directives.md new file mode 100644 index 00000000..b3fd81ef --- /dev/null +++ b/proposals/custom-client-directives.md @@ -0,0 +1,111 @@ +- Start Date: 2023-05-12 +- Reference Issues/Discussions: + - https://github.com/withastro/roadmap/discussions/272 + - Legacy https://github.com/withastro/roadmap/pull/212 +- Implementation PR: https://github.com/withastro/astro/pull/7074 + +# Summary + +Provide an API for integrations to implement custom `client:` directives to provide greater control for when client-side JS is loaded and executed. + +# Example + +```js +import { defineConfig } from 'astro/config'; +import onClickDirective from '@matthewp/astro-click-directive'; + +export default defineConfig({ + integrations: [onClickDirective()] +}); +``` + +```js +export default function onClickDirective() { + return { + hooks: { + 'astro:config:setup': ({ addClientDirective }) => { + addClientDirective({ + name: 'click', + entrypoint: fileUrlToPath(new URL('./click.js', import.meta.url)) + }); + }, + } + } +} +``` + +# Background & Motivation + +The last client directive added to core was the `client:only` directive in [August 2021](https://github.com/withastro/astro/issues/751). Since that time the core team has been hesitant to add new client directives despite the community asking about them. + +Allowing custom client directives would both: + +- Allow the community to experiment with different approaches to lazy-loading client JavaScript. +- Provide evidence, through telemetry data, on which directives are most used. This data could be used to determine if a directive should be brought into core. + +Some examples of custom directives that people have wanted in the past: + +- Loading JavaScript on client interactive, such as mouseover or click. +- Loading JavaScript when an element is visible, as opposed to within the viewport as `client:visible` currently does. +- The [Idle Until Urgent](https://philipwalton.com/articles/idle-until-urgent/) pattern which loads on either idle or interaction, whichever comes first. + +# Goals + +- Provide a way to customize loading of client components. +- Allow integrations to add their own directives. +- Allow integrations to provide type definitions for their new directives. + +# Non-Goals + +- Allowing overriding builtin directives. +- Allowing for additional customization via new types of directives outside of `client:`. +- Allowing multiple directives to run at the same time. + +Previously goals in Stage 2: +- Refactor the implementation of `client:` loading to get rid of the precompile step (this is a core repo refactor / improvement). + +(Moved as non-goal as it's more performant to precompile the builtin directives still) + +# Detailed Design + +When loading the Astro config and running the integrations, those that add new client directives are kept in a `Map` together with Astro's default set of client directives. + +Each client directive entrypoint will be bundled with esbuild before starting the dev server or build. This method is chosen as: + +1. Client directives should be small and simple, so we don't need the entire Vite toolchain to build (it's also complex to rely on the existing Vite build). +2. It's easier to handle the builds upfront so the consumer can render HTML synchronously. + +Once we have a `Map` of client directive names to compiled code, it's a matter of passing this down to `SSRResult` so the runtime renderer can pick the right compiled code to inline. + +For typings, libraries can define this in their `.d.ts` file (module): + +```ts +declare module 'astro' { + interface AstroClientDirectives { + 'client:click'?: boolean + } +} +``` + +# Testing Strategy + +An e2e test will be setup to make sure the client directive API works and loaded. Typings are a bit hard to test, so I'm doing it manually for now. + +# Drawbacks + +- Larger API surface area +- Opens up partial hydration code pattern +- Future builtin client directives are breaking changes +- Third-party Astro libraries could rely on non-standard client directives + +# Alternatives + +Don't do this. We add new client directives ourselves as we go. + +# Adoption strategy + +This won't be a breaking change. The user will only use this feature if they add a client directive through an integration. + +# Unresolved Questions + +1. Is the typings pattern good? `declare module` and `AstroClientDirectives`. It deviates from Astro middleware `namespace App` pattern since I can't seem to get `astro-jsx.d.ts` to reference `App`. From 447a3b85b9fb64f8002e225f0ec147761fdfe2c7 Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Fri, 12 May 2023 22:34:05 +0800 Subject: [PATCH 201/300] Update custom-client-directives.md --- proposals/custom-client-directives.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/custom-client-directives.md b/proposals/custom-client-directives.md index b3fd81ef..96b49255 100644 --- a/proposals/custom-client-directives.md +++ b/proposals/custom-client-directives.md @@ -34,6 +34,19 @@ export default function onClickDirective() { } ``` +```ts +import type { ClientDirective } from 'astro' + +const clickDirective: ClientDirective = (load, opts, el) => { + window.addEventListener('click', async () => { + const hydrate = await load() + await hydrate() + }, { once: true }) +} + +export default clickDirective +``` + # Background & Motivation The last client directive added to core was the `client:only` directive in [August 2021](https://github.com/withastro/astro/issues/751). Since that time the core team has been hesitant to add new client directives despite the community asking about them. From b378e8908aa31c8c1276e1dae673f5fb588452f3 Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Fri, 12 May 2023 22:54:29 +0800 Subject: [PATCH 202/300] Update custom-client-directives.md --- proposals/custom-client-directives.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/proposals/custom-client-directives.md b/proposals/custom-client-directives.md index 96b49255..f52b792b 100644 --- a/proposals/custom-client-directives.md +++ b/proposals/custom-client-directives.md @@ -15,7 +15,10 @@ import { defineConfig } from 'astro/config'; import onClickDirective from '@matthewp/astro-click-directive'; export default defineConfig({ - integrations: [onClickDirective()] + integrations: [onClickDirective()], + experimental: { + customClientDirectives: true + } }); ``` @@ -26,7 +29,7 @@ export default function onClickDirective() { 'astro:config:setup': ({ addClientDirective }) => { addClientDirective({ name: 'click', - entrypoint: fileUrlToPath(new URL('./click.js', import.meta.url)) + entrypoint: 'astro-click-directive/click.js' }); }, } From 41bb1160ca505799dbd815e2f71a92a3c965364a Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Fri, 12 May 2023 23:29:56 +0800 Subject: [PATCH 203/300] Update custom-client-directives.md --- proposals/custom-client-directives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/custom-client-directives.md b/proposals/custom-client-directives.md index f52b792b..76768758 100644 --- a/proposals/custom-client-directives.md +++ b/proposals/custom-client-directives.md @@ -84,7 +84,7 @@ Previously goals in Stage 2: # Detailed Design -When loading the Astro config and running the integrations, those that add new client directives are kept in a `Map` together with Astro's default set of client directives. +When loading the Astro config and running the integrations, added new client directives are kept in a `Map` together with Astro's default set of client directives. Each client directive entrypoint will be bundled with esbuild before starting the dev server or build. This method is chosen as: From 2ad887acee892c1488eca6c8a69a2f9fc618848d Mon Sep 17 00:00:00 2001 From: Bjorn Lu <bjornlu.dev@gmail.com> Date: Sun, 14 May 2023 00:39:24 +0800 Subject: [PATCH 204/300] Update custom-client-directives.md --- proposals/custom-client-directives.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/custom-client-directives.md b/proposals/custom-client-directives.md index 76768758..b69bde42 100644 --- a/proposals/custom-client-directives.md +++ b/proposals/custom-client-directives.md @@ -76,6 +76,7 @@ Some examples of custom directives that people have wanted in the past: - Allowing overriding builtin directives. - Allowing for additional customization via new types of directives outside of `client:`. - Allowing multiple directives to run at the same time. +- Replay interaction on hydrate, e.g. if a `client:click` directive hydrates on clicking the button, Astro doesn't replay the click event to trigger some reaction after it hydrates. The user has to click the (now hydrated) button again to trigger a reaction. Previously goals in Stage 2: - Refactor the implementation of `client:` loading to get rid of the precompile step (this is a core repo refactor / improvement). @@ -113,6 +114,7 @@ An e2e test will be setup to make sure the client directive API works and loaded - Opens up partial hydration code pattern - Future builtin client directives are breaking changes - Third-party Astro libraries could rely on non-standard client directives +- Users could bring in large dependencies, causing big file sizes for a client directive # Alternatives From c5caacee4709add7aa5e406b7e500d9e09b7fa98 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 15 May 2023 08:52:42 -0400 Subject: [PATCH 205/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index ac5b4d55..2ddcd999 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -1,6 +1,6 @@ - Start Date: 2023-04-27 - Reference Issues: https://github.com/withastro/roadmap/issues/539 -- Implementation PR: <!-- leave empty --> +- Implementation PR: https://github.com/withastro/astro/pull/6991 # Summary From c71ad1f374986f5a87445b35c043a0ea3a9decaf Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 15 May 2023 13:02:59 -0400 Subject: [PATCH 206/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index 2ddcd999..6985867b 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -12,13 +12,15 @@ An existing static site can be changed to hybrid rendering, allowing some specif __astro.config.mjs__ -```js +```diff import { defineConfig } from 'astro/config'; +import vercel from "@astrojs/vercel" export default defineConfig({ - output: 'hybrid' +- output: 'static', ++ output: 'hybrid', ++ adapter: vercel() }); -``` __pages/api/form.ts__ From 759c3c3eeeaef68347d884aa2dac71efff02227c Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa <my.burning@gmail.com> Date: Wed, 17 May 2023 14:58:04 +0100 Subject: [PATCH 207/300] chore: update restrictions --- proposals/0032-middleware-api.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index ed16c50a..4e97509c 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -453,13 +453,9 @@ requests, they will need to store that information somewhere else. ## Restrictions and expectations -In order to set user expectations, the Astro middleware have the following restrictions: -- a middleware needs to return a `Response`; -- a middleware needs to call `next`; +In order to set user expectations, the Astro middleware **must** return a `Response`. -If the user doesn't do any of these two, Astro will throw an error. Plus, -the user is required to return exactly a `Response`. Failing to do so will result in -Astro throwing an error. +If the user doesn't do that, Astro will throw an error. # Testing Strategy From 44eea8b567394f9960dacff4670b325af9d8b7a8 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 22 May 2023 08:19:00 -0400 Subject: [PATCH 208/300] Update proposals/hybrid-rendering.md Co-authored-by: Happydev <81974850+MoustaphaDev@users.noreply.github.com> --- proposals/hybrid-rendering.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/hybrid-rendering.md b/proposals/hybrid-rendering.md index 6985867b..b864ece5 100644 --- a/proposals/hybrid-rendering.md +++ b/proposals/hybrid-rendering.md @@ -72,7 +72,7 @@ Additionally there are a few places in the codebase that assume if `output !== ' Prerendering is currently tested via fixture testing, due to the fact that the build artifacts is what changes. This same strategy will be used to test hybrid rendering as well, only testing for the opposite effect. -Likely we can use the same test fixtures, but only swap out the `output` when testing `'hybrid'`, which eliminates the need for new fixtures. +Likely we can use most of the prerendering test fixtures, and only swap out the `output` when testing `'hybrid'`, which reduces the need for new fixtures. # Drawbacks From e3370436c8125fb748ad6b2c3fcdc562973d2298 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 22 May 2023 09:37:47 -0400 Subject: [PATCH 209/300] Move HTML minification RFC --- proposals/{html-minification.md => 0035-html-minification.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{html-minification.md => 0035-html-minification.md} (100%) diff --git a/proposals/html-minification.md b/proposals/0035-html-minification.md similarity index 100% rename from proposals/html-minification.md rename to proposals/0035-html-minification.md From 34f91fcf75ba909df07a77dd3140ca7b03fda0d5 Mon Sep 17 00:00:00 2001 From: Matthew Phillips <matthew@skypack.dev> Date: Mon, 22 May 2023 10:07:19 -0400 Subject: [PATCH 210/300] Redirects RFC --- proposals/redirects.md | 116 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 proposals/redirects.md diff --git a/proposals/redirects.md b/proposals/redirects.md new file mode 100644 index 00000000..b79ab253 --- /dev/null +++ b/proposals/redirects.md @@ -0,0 +1,116 @@ +<!-- + Note: You are probably looking for `stage-1--discussion-template.md`! + This template is reserved for anyone championing an already-approved proposal. + + Community members who would like to propose an idea or feature should begin + by creating a GitHub Discussion. See the repo README.md for more info. + + To use this template: create a new, empty file in the repo under `proposals/${ID}.md`. + Replace `${ID}` with the official accepted proposal ID, found in the GitHub Issue + of the accepted proposal. +--> + +- Start Date: 2023-05-22 +- Reference Issues: https://github.com/withastro/roadmap/issues/466 +- Implementation PR: https://github.com/withastro/astro/pull/7067/ + +# Summary + +Add a `redirects` config option to the Astro config which allows you to define redirects in a central place. Allow integrations to read this information and apply it to it's own redirects configuration for deployment. + +# Example + +New config option used like so: + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + redirects: { + '/other': '/place' + } +}); +``` + +# Background & Motivation + +This was original proposed as a stage 1 discussion [here](https://github.com/withastro/roadmap/discussions/319) and was one of the top voted proposals. + +As websites age there are times where routes are rearchitectured. In order preserve existing links to content on the web it is common to set up redirects in your web server. + +Redirect configuration varies depending on what web server or host you use. Some times you might even want to change hosts, in which case your redirects need to be converted over to a new format. + +Having a redirects configuration within Astro itself allows a single place to define redirects that works everywhere. +# Goals + +- Works in development mode. +- Writes out `<meta http-equiv="refresh">` tags in static builds. +- Gives proper hooks to integrations so that they can write to their own configuration. + +# Non-Goals + +- Specifying temporary redirects. These are best done at runtime, so SSR is a more appropriate way to do those. + - In the future we could expand this API to give more control over what status code is used, once use-cases are known. + +# Detailed Design + +This will be implemented as a feature of the internal routing. The `RouteData` type will be extended to include: + +```js +export interface RouteData { + type: 'redirect'; + // ... + redirect?: string; +} +``` + +Our core rendering handles routing and will detect this type of route and return a `Response` with a status code of `301` with the `Location` header set to the value of the route's `redirect` property. + +## Static generation + +Currently the static generation code throws for any non-200 response. With this change it will now accept `301` as a valid response code. It will generate an HTML doc that looks like: + +```html +`<!doctype html> +<title>OLD_LOCATION + +``` + +## Adapters + +Adapters can integration with this new route type through the `astro:build:done` hook which includes the `routes` property. This is an array of route datas that were built. Redirects will be part of this array. + +# Testing Strategy + +- We have existing redirect tests for SSR. These will be updated to test SSG redirect behavior for: + - `redirects` config + - Redirects created via `Astro.redirect()` and `new Response` as those are now enabled by this change. + +# Drawbacks + +- Adds some new complexity to the config. +- New type of route in the RouteData. +- There could be some expectations that all adapters handle this perfectly. This is a good expectation! We should do our best to make sure as many adapters support this as possible. + - The HTML based fallback still does work. + +# Alternatives + +The other major design would be to allow defining redirects in the file where the page now lives. For example in a markdown page you could do: + +```md +--- +redirect_from: + - /one + - /two +--- +``` + +This method is nice because it is file-based and in the file where the redirect is ultimately going to go to. + +The major problem with this design is that we need to load and process every page before we know about these routes. So this would significantly slow down dev, where we don't need to know about all pages until they are requested. + +A secondary problem with this alternative is that it's not clear how we should define redirects in non-Markdown pages, for example in an API route how would you define them? Via special comment syntax? The `export const foo` pattern from prerender? + +# Adoption strategy + +- This is a small change and can go through in the next minor release. \ No newline at end of file From 7a84a7097bd0650a32c50ce74531990ce0d3c415 Mon Sep 17 00:00:00 2001 From: Emanuele Stoppa Date: Tue, 23 May 2023 13:44:04 +0100 Subject: [PATCH 211/300] chore: remove serialisation restriction --- proposals/0032-middleware-api.md | 34 +------------------------------- 1 file changed, 1 insertion(+), 33 deletions(-) diff --git a/proposals/0032-middleware-api.md b/proposals/0032-middleware-api.md index 4e97509c..43dfd9a3 100644 --- a/proposals/0032-middleware-api.md +++ b/proposals/0032-middleware-api.md @@ -91,8 +91,6 @@ will be implemented in the second iteration of the middleware project: - `resolve` has been renamed to `next`; - `next` doesn't accept an `APIContext` to work; -- `locals` values need to be serializable to avoid the introduction of - non-user code from third-party libraries that can run scripts; - `middleware` export function has been renamed `onRequest`. This name change has two benefits: 1. It shows intent and explains when this function is called; @@ -192,37 +190,7 @@ By doing so, the user can leverage the type-checking and auto-completion of Type called `middleware.ts` or `middleware.js` using JSDoc. -The `locals` object has the following restrictions: -1. it can store only serializable information; -2. it can't be overridden by other values that are different from objects; - -## `locals` needs to be serializable - -The information must be serializable because storing -information that evaluates at runtime is unsafe. If, for example, we were able to store -With a JavaScript function, an attacker could exploit the victim's website -and execute some unsafe code. - -Astro will do a sanity check **in development mode**. -Some code like this: - -```js -export const onRequest = (contex, next) => { - context.locals.someInfo = { - f() { - alert("Hello!!") - } - } -} -``` - -Storing unsafe information will result in an Astro error: - -> The information stored in Astro.locals are not serializable when visiting "/index" path. -Make sure you store only serializable data. - -> **Note**: The content of the error is not final. The docs team will review it. - +The `locals` object can't be overridden by other values that are different from objects; ## `locals` can't be overridden From 3aa1322696291d13f82bcb52d7f295d6b76252a1 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 23 May 2023 08:53:34 -0400 Subject: [PATCH 212/300] Updates: - Show object notation syntax. - Explain dynamic routes. - Specify the other status code. --- proposals/redirects.md | 50 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 5 deletions(-) diff --git a/proposals/redirects.md b/proposals/redirects.md index b79ab253..2c9e6af1 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -32,6 +32,21 @@ export default defineConfig({ }); ``` +You can also specify the status code by using an object notation: + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + redirects: { + '/other': { + status: 302, + destination: '/place' + } + } +}); +``` + # Background & Motivation This was original proposed as a stage 1 discussion [here](https://github.com/withastro/roadmap/discussions/319) and was one of the top voted proposals. @@ -49,26 +64,51 @@ Having a redirects configuration within Astro itself allows a single place to de # Non-Goals -- Specifying temporary redirects. These are best done at runtime, so SSR is a more appropriate way to do those. - - In the future we could expand this API to give more control over what status code is used, once use-cases are known. +- Dynamic behavior that goes beyond the capabilities of the file-based routing system. So nothing based on the properties of the request, user session, etc. Regular routes still should be used for this scenario. +- Redirects to pages that do not exist in the Astro project. +- External redirects. # Detailed Design This will be implemented as a feature of the internal routing. The `RouteData` type will be extended to include: ```js +interface RedirectConfig = string | { + status: 300 | 301 | 302 | 303 | 304 | 307 | 308; + destination: string; +} + export interface RouteData { type: 'redirect'; // ... - redirect?: string; + redirect?: RedirectConfig; + redirectRoute?: RouteData; } ``` -Our core rendering handles routing and will detect this type of route and return a `Response` with a status code of `301` with the `Location` header set to the value of the route's `redirect` property. +Our core rendering handles routing and will detect this type of route and return a `Response` with a status code in the 3xx range with the `Location` header set to the value of the route's `redirect` property. + +When using the object notation `{ destination: string; status: number; }` + +## Dynamic routes + +Dynamic routes are supported through the same syntax as in the file-based routing system. For example, if a site moved its blog it might set up a redirect like so: + +```js +import { defineConfig } from 'astro/config'; + +export default defineConfig({ + redirects: { + '/blog/[...slug]': '/team/articles/[..slug]' + } +}); +``` + +In SSG mode this will call the destinations `getStaticPaths` method to get valid static paths. Those paths will be used to generate the HTML files for the redirects. ## Static generation -Currently the static generation code throws for any non-200 response. With this change it will now accept `301` as a valid response code. It will generate an HTML doc that looks like: +Currently the static generation code throws for any non-200 response. With this change it will now accept any 3xx as a valid response codes. It will generate an HTML doc that looks like: ```html ` From c043f2484f0c00c78b72df85a981258156c83720 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 23 May 2023 08:55:10 -0400 Subject: [PATCH 213/300] Specify 308 for non-GET --- proposals/redirects.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/redirects.md b/proposals/redirects.md index 2c9e6af1..220095d2 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -88,7 +88,7 @@ export interface RouteData { Our core rendering handles routing and will detect this type of route and return a `Response` with a status code in the 3xx range with the `Location` header set to the value of the route's `redirect` property. -When using the object notation `{ destination: string; status: number; }` +When using the object notation `{ destination: string; status: number; }` the status code specified there will be used. Otherwise the default is `301` for `GET` requests and `308` for any other method. ## Dynamic routes From d85f7fb4b669be6c37119ea405035d0273f556ac Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 23 May 2023 10:52:45 -0400 Subject: [PATCH 214/300] Rename the inline stylesheets RFC --- .../{0032-inline-stylesheets.md => 0036-inline-stylesheets.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{0032-inline-stylesheets.md => 0036-inline-stylesheets.md} (100%) diff --git a/proposals/0032-inline-stylesheets.md b/proposals/0036-inline-stylesheets.md similarity index 100% rename from proposals/0032-inline-stylesheets.md rename to proposals/0036-inline-stylesheets.md From c16aa45e5203207f04a0af4b738bd15d4365ff9f Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Tue, 23 May 2023 13:28:09 -0500 Subject: [PATCH 215/300] Update proposals/redirects.md --- proposals/redirects.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/redirects.md b/proposals/redirects.md index 220095d2..46197488 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -99,7 +99,7 @@ import { defineConfig } from 'astro/config'; export default defineConfig({ redirects: { - '/blog/[...slug]': '/team/articles/[..slug]' + '/blog/[...slug]': '/team/articles/[...slug]' } }); ``` From 9e2d0eb9f2e38789cab0490fe56a18ee75034116 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Thu, 25 May 2023 10:02:48 -0400 Subject: [PATCH 216/300] Specify routing priority --- proposals/redirects.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/redirects.md b/proposals/redirects.md index 46197488..c8cdd66d 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -106,6 +106,10 @@ export default defineConfig({ In SSG mode this will call the destinations `getStaticPaths` method to get valid static paths. Those paths will be used to generate the HTML files for the redirects. +## Routing priority + +Redirects will be given the same priority as file-system routing. This means that if there are conflicts, the same rules applies as with the file-system routes. For example, you could have a file system route `/src/pages/blog/contributing.astro` and a redirect route `/blog/[...slug]`. If these were both filesystem routes you would expect `contributing.astro` to be prioritized over the spread route. This is the case with redirects a well. + ## Static generation Currently the static generation code throws for any non-200 response. With this change it will now accept any 3xx as a valid response codes. It will generate an HTML doc that looks like: From fb0a86dd9100fed2fef484bc77ec425942fd2546 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Thu, 25 May 2023 10:06:44 -0400 Subject: [PATCH 217/300] Update proposals/redirects.md Co-authored-by: Chris Swithinbank --- proposals/redirects.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/redirects.md b/proposals/redirects.md index c8cdd66d..fd4a71ba 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -122,7 +122,7 @@ Currently the static generation code throws for any non-200 response. With this ## Adapters -Adapters can integration with this new route type through the `astro:build:done` hook which includes the `routes` property. This is an array of route datas that were built. Redirects will be part of this array. +Adapters can integrate with this new route type through the `astro:build:done` hook which includes the `routes` property. This is an array of route datas that were built. Redirects will be part of this array. # Testing Strategy From afd858959c16eef0f8022f5d34287a847b3c996e Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 30 May 2023 08:31:37 -0400 Subject: [PATCH 218/300] Rename middleware --- proposals/{0032-middleware-api.md => 0037-middleware-api.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{0032-middleware-api.md => 0037-middleware-api.md} (100%) diff --git a/proposals/0032-middleware-api.md b/proposals/0037-middleware-api.md similarity index 100% rename from proposals/0032-middleware-api.md rename to proposals/0037-middleware-api.md From dfd349eefc2251a532b306415ccc320568076e9d Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 30 May 2023 14:14:10 -0400 Subject: [PATCH 219/300] Update text to reflect disabling of the HTML generation --- proposals/redirects.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/proposals/redirects.md b/proposals/redirects.md index fd4a71ba..6971bc7e 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -124,6 +124,24 @@ Currently the static generation code throws for any non-200 response. With this Adapters can integrate with this new route type through the `astro:build:done` hook which includes the `routes` property. This is an array of route datas that were built. Redirects will be part of this array. +Additionally adapters can disable the HTML generation via a new configuration value `build.redirects` which can be set to `false`. Users can also set this value, but it is more likely to come from an integration via the `astro:config:setup` hook: + +```js +export default { + hooks: { + 'astro:config:setup': ({ updateConfig }) => { + updateConfig({ + build: { + redirects: false + } + }); + } + } +} +``` + +Adapters who have their own configuration files, such as the Netlify and Cloudflare `_redirects` file, will disable HTML generation because the host serves HTML before it checks configuration. + # Testing Strategy - We have existing redirect tests for SSR. These will be updated to test SSG redirect behavior for: From fff55c37152737c4c9ca982740c6c276a989edd6 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 30 May 2023 18:16:37 -0400 Subject: [PATCH 220/300] Specify that FS routes override redirects --- proposals/redirects.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/redirects.md b/proposals/redirects.md index 6971bc7e..4797ba45 100644 --- a/proposals/redirects.md +++ b/proposals/redirects.md @@ -108,7 +108,9 @@ In SSG mode this will call the destinations `getStaticPaths` method to get valid ## Routing priority -Redirects will be given the same priority as file-system routing. This means that if there are conflicts, the same rules applies as with the file-system routes. For example, you could have a file system route `/src/pages/blog/contributing.astro` and a redirect route `/blog/[...slug]`. If these were both filesystem routes you would expect `contributing.astro` to be prioritized over the spread route. This is the case with redirects a well. +Redirects will use the priority assignment algorithm as file-system routing. For example, you could have a file system route `/src/pages/blog/contributing.astro` and a redirect route `/blog/[...slug]`. If these were both filesystem routes you would expect `contributing.astro` to be prioritized over the spread route. This is the case with redirects a well. + +In the case of exact matches, the file-system route will be prioritized (by being ordered first in the list). This is meant to match the behavior of host systems (such as Netlify). ## Static generation From eac9ced74101819b851f75fbed0b7e1f14dcda08 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Fri, 2 Jun 2023 07:59:27 -0400 Subject: [PATCH 221/300] Rename --- ...stom-client-directives.md => 0038-custom-client-directives.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{custom-client-directives.md => 0038-custom-client-directives.md} (100%) diff --git a/proposals/custom-client-directives.md b/proposals/0038-custom-client-directives.md similarity index 100% rename from proposals/custom-client-directives.md rename to proposals/0038-custom-client-directives.md From fcc34eeaab2042eb9c6156adb3d0aed2138f5e14 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Mon, 5 Jun 2023 09:35:00 -0400 Subject: [PATCH 222/300] Rename hybrid rendering --- proposals/{hybrid-rendering.md => 0039-hybrid-rendering.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{hybrid-rendering.md => 0039-hybrid-rendering.md} (100%) diff --git a/proposals/hybrid-rendering.md b/proposals/0039-hybrid-rendering.md similarity index 100% rename from proposals/hybrid-rendering.md rename to proposals/0039-hybrid-rendering.md From e9049a79753c219864ec1df0f275bc468265cc57 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Thu, 29 Jun 2023 15:28:28 -0400 Subject: [PATCH 223/300] View Transitions RFC --- proposals/0040-view-transitions.md | 279 +++++++++++++++++++++++++++++ 1 file changed, 279 insertions(+) create mode 100644 proposals/0040-view-transitions.md diff --git a/proposals/0040-view-transitions.md b/proposals/0040-view-transitions.md new file mode 100644 index 00000000..7dd9993f --- /dev/null +++ b/proposals/0040-view-transitions.md @@ -0,0 +1,279 @@ +- Start Date: 2023-06-29 +- Reference Issues: https://github.com/withastro/roadmap/issues/532 +- Implementation PR: https://github.com/withastro/astro/pull/7511 + +# Summary + +Introduce APIs to make using [View Transitions](https://developer.chrome.com/docs/web-platform/view-transitions/) as easy as possible in Astro. This proposal includes: + +- A component `` that adds a client-side router that uses View Transitions to update the page. +- A set of directives that allow specifying animations on specific elements. + +# Example + +## Enabling support + +A user can enable view-transitions one of two ways: + +```diff + + ++ + + + + + +``` + +Adding this meta tag to the head will enable the built-in support for MPA view transitions. *However*, this currently only works in Chrome Canary behind a flag. A more practical usage is to use our `` built-in component: + +```diff ++ --- ++ import { ViewTransitions } from 'astro/components'; ++ --- + + +- ++ + + + + + +``` + +Simply by doing this the site will do a cross-fade between pages (browser default). If that's all you want then there's nothing else to do. + +## Animations + +You can use our built-in animations by using the `transition:animate` directive like so: + +```astro +--- +import { ViewTransitions } from 'astro/components'; +--- + + + + + + + + +``` + +This will do an animation where the body slides in and out. On back navigation it has the opposite animation. + +# Background & Motivation + +View Transitions aligns very well with Astro's content site focus. We still believe that MPA is the right approach to building this type of site. With View Transitions there is the prospect of keeping multi-page architecture but enabling smooth transitions between pages and eliminating the "full page refresh" look that a lot of people dislike. + +However, currently View Transitions are a new API and there's a bit of work needed to use them. This proposal seeks to make it easier. + +## Animations + +By default a view transition uses a cross-fade animation. The old page fades out and the new page fades in. The default animation is fine, but some times you'll want to do more. You can do this yourself if you want, by using the various pseudo-selectors in CSS, like so: + +```css +@keyframes fadeIn { + from { opacity: 0; } +} + +@keyframes fadeOut { + to { opacity: 0; } +} + +@keyframes slideFromRight { + from { transform: translateX(100%); } +} + +@keyframes slideToLeft { + to { transform: translateX(-100%); } +} + +body { + view-transition-name: body; +} + +/* Old stuff going out */ +::view-transition-old(body) { + animation: 90ms cubic-bezier(0.4, 0, 1, 1) both fadeOut, + 300ms cubic-bezier(0.4, 0, 0.2, 1) both slideToLeft; +} + +/* New stuff coming in */ +::view-transition-new(body) { + animation: 210ms cubic-bezier(0, 0, 0.2, 1) 90ms both fadeIn, + 300ms cubic-bezier(0.4, 0, 0.2, 1) both slideFromRight; +} +``` + +However this amounts to a lot of code. Also you have to create a unique `view-transition-name` for each element that you want to animate. This can be tricky to do, especially if trying to animate a list; impossible if done dynamically. + +With this proposal Astro will auto-generate a `view-transition-name` for you that is distinct to just that element. + +# Goals + +- Provide a router that works with the SPA view transition API. +- Provide some built-in animations. +- Provide a way to persist some DOM between page navigations (the media player use-case). +- Have some fallback for browser that do not support view transitions. Ideally we would mimick the API as closely as possible; but it's more likely that we will only be able to support a subset of animations. + +# Non-Goals + +- App-like use-cases are still not a goal here. Many should be able to be used with these new APIs, but we'd still recommend using an island with a client-side router for full apps. +- This is not something you turn on to always get client-side routing; it's not a configuration toggle. Instead you control what pages use CSR by using the `` component or adding the built-in meta tag. + +# Out of scope + +- It's possible to do in-page animations using view transitions. This gives you the nice morphing effect. This is something we are set up to support, but do not currently have an API for and not the use-case this proposal is targeting. + +# Detailed Design + +There are 3 parts to this proposal: + +- A component that allows View Transitions to occur. +- Some directives to control which elements get special transitions and animations. +- A directive to persist an island between pages. + +## ViewTransitions component + +The `` component is the router for view transition support. It includes a script that will: + +- Intercept forward navigates within the site (same origin). +- Intercept back buttons. + +From there it acts as a router and: + +- Fetches the next page via `fetch()`. +- Tells the browser it is entering a view transition via the `document.startTransition()` API. +- Swaps the contents of the page to the next page. + +Animations are provided via CSS and the ViewTransitions component does not need to trigger them. `document.startTransition(cb)` takes a callback. Inside that callback the actual DOM manipulation occurs. The browser will: + +- Take a screenshot of the page between the callback. +- Take a screenshot after the callback. +- Use the CSS animations to transition to the next view. + +### Opt-in per route + +To enable CSR the `` must be on each page where it is wanted. Usually apps will have a layout component or a head component. Using ViewTransitions there will enable it on every page that uses that component. + +Once the ViewTransitions client-side script is installed it will persist between *all pages* until a MPA navigation occurs. That's because the browser does not unload scripts. To ensure that only pages that ask for CSR get it, this component will need to check for the presence of a special meta tag, `` which is added by the ViewTransitions component. If this tag does not exist then the component knows to allow MPA navigation to the next page. + +## Animation directives + +There are 2 directives used to control animations for specific elements. + +### transition:animate + +This is the directive you'll most often use. You can use it to set a specific animation on an element between pages: + +```astro + +
    + +``` + +With this the body will do a slide animation. However the header will not, it is specified to d a morph. The user will see the slide everywhere on the page *except* for the header. + +Here are the built-in animations: + +- __slide__: A slide in and out animation. The old page slides out to the left and the new page slides in from the right. On backwards navigation the opposite occurs; the old page slides out to the right and the new page slides in from the left. +- __fade__: This is a cross fade where the old page fades out to `opacity: 0` and the new page fades in. +- __morph__: This tells the browser to morph the element between pages. What this looks like is dependent on how the element is different between pages and is determined by the browser. If you have an image in both old and new pages but the elements are otherwise different, you'll see animation where the old element seems to "morph" into the new one. If the elements are completely different, however, you'll see a cross-fade. + +The algorithm for determining the `view-transition-name` is: + +1. Take the hash used for the component, which will be something like `abcde`. +2. Use a depth-first counter to assign an index for the component, for example `5`. +3. Hash these two values creating a new hash, for example `fghijkl`. +4. When rendering, keep a count of the number of `transition:animate` calls there are and increment a counter for each one. The final id becomes `fghijkl-5` and that is used as the `view-transition-name`. + +### transition:name + +When using a `transition:animate` Astro will automatically assign that element a `view-transition-name`. This is because in most cases the elements are roughly the same between pages. + +Some times you might want to morph two different elements that come from different components and live at different locations within the page. The auto-assigned names will not result in the morphing that you desire. In this case you can specify the `view-transition-name` yourself: + +__one.astro__ + +```astro +
  • +``` + +__two.astro__ + +```astro +
    +``` + +## Advanced animation API + +Animations can be customized by importing the animation from `astro:transitions`: + +```astro +--- +import { slide } from "astro:transitions"; +--- + + +``` + +This allows users to define their own animations. The API for what these functions returns is: + +```ts +interface TransitionAnimation { + name: string; // The name of the keyframe + delay?: number | string; + duration?: number | string; + easing?: string; +}; +``` + +## Persistent islands + +Some times you have elements which are exactly the same between pages, but you want to keep some state that exists. A common use-case for this is a media player. You have a song playing and want the song to continue playing on the next page. + +An island can be set to persist using the `transition:persist` directive: + +```astro + +``` + +Astro will give this island an id using the same algorithm used to calculate the `view-transition-name`. You can also specify a name like: `transition:persist="media"` for the case where the elements are in very different spots on the page. + +When the next page loads Astro will pull the island's root `` from the old page and have it replace the same element on the next page. + +# Testing Strategy + +This feature is mostly client-side so it will be tested via the Playwright e2e test suite. + +# Drawbacks + +- This feature is primarily about taking advantage of cutting edge features. Currently it is Chromium browsers only. There is some risk that other browsers will not adopt these APIs and we'll be left having to do a fallback for a long time. +- This approach is not the best for apps. We would probably need a whole separate API if we wanted to support that better. + +# Alternatives + +- SPA mode toggle was prototyped here: https://github.com/withastro/docs/issues/3314 This worked really well and the same technique is used by other frameworks. The major downside to this approach was that it was a boolean; either your entire site used CSR or none did. View transitions allowed a more granular approach. +- Persistent islands proposal is here: https://github.com/withastro/roadmap/discussions/307 The idea of keeping an island between navigation is now part of this proposal. + +# Adoption strategy + +## Release 1 +- Support for browsers with `document.startTransition` (Chromium browsers at the moment). +- Custom animations + +# Release 2 +- Fallback for Safari and Firefox. Likely this will be more limited in scope (only certain types of animations). + +# Release 3 +- Persistent islands + + +# Unresolved Questions + +Optional, but suggested for first drafts. +What parts of the design are still to be determined? From 396ddc0074ea23f2271c0d1ad8079cc5177a0015 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Thu, 13 Jul 2023 08:56:06 -0400 Subject: [PATCH 224/300] Update the RFC to explain the fallback behavior --- proposals/0040-view-transitions.md | 56 +++++++++++++++++++++++++++--- 1 file changed, 51 insertions(+), 5 deletions(-) diff --git a/proposals/0040-view-transitions.md b/proposals/0040-view-transitions.md index 7dd9993f..6d964dea 100644 --- a/proposals/0040-view-transitions.md +++ b/proposals/0040-view-transitions.md @@ -30,7 +30,7 @@ Adding this meta tag to the head will enable the built-in support for MPA view t ```diff + --- -+ import { ViewTransitions } from 'astro/components'; ++ import { ViewTransitions } from 'astro:transitions'; + --- @@ -51,7 +51,7 @@ You can use our built-in animations by using the `transition:animate` directive ```astro --- -import { ViewTransitions } from 'astro/components'; +import { ViewTransitions } from 'astro:transitions'; --- @@ -224,14 +224,33 @@ import { slide } from "astro:transitions"; This allows users to define their own animations. The API for what these functions returns is: ```ts -interface TransitionAnimation { +export interface TransitionAnimation { name: string; // The name of the keyframe delay?: number | string; duration?: number | string; easing?: string; -}; + fillMode?: string; + direction?: string; +} + +export interface TransitionAnimationPair { + old: TransitionAnimation | TransitionAnimation[]; + new: TransitionAnimation | TransitionAnimation[]; +} + +export interface TransitionDirectionalAnimations { + forwards: TransitionAnimationPair; + backwards: TransitionAnimationPair; +} ``` +This defines: + +- `forwards` and `backwards` transitions to handle the case where you want the animation to go in the reverse direction when the user hits the Back button. +- `old` and `new` so that you can control the old and new pages separately. + +Note here that you still need to define a [keyframe](https://developer.mozilla.org/en-US/docs/Web/CSS/@keyframes) some where else, such as imported CSS. + ## Persistent islands Some times you have elements which are exactly the same between pages, but you want to keep some state that exists. A common use-case for this is a media player. You have a song playing and want the song to continue playing on the next page. @@ -246,6 +265,33 @@ Astro will give this island an id using the same algorithm used to calculate the When the next page loads Astro will pull the island's root `` from the old page and have it replace the same element on the next page. +## Fallback + +In order to support browsers that do not support native view transition APIs, Astro will simulate the behavior using regular CSS and DOM manipulation. On a transition Astro will: + +- Add the `data-astro-transition-fallback="old"` attribute to the outgoing page. +- Wait for animations to end. +- Add the `data-astro-transition-fallback="new"` to the incoming page. +- Replace the `document.documentElement` with the incoming page. +- Wait for animations to end. +- Remove the `data-astro-transition-fallback` attribute. + +Internally Astro will enable these animations to work in both environments by using selectors in the inserted CSS. A user can control fallback behavior with the `fallback` prop on the `ViewTransitions` component. + +```astro +--- +import { ViewTransitions } from 'astro:transitions'; +--- + + +``` + +The possible values are: + +- `animate`: The default, perform a fallback with simulated animations. +- `swap`: A fallback where the DOM is swapped without animations. +- `none`: Do not fallback for non-supporting browsers, allow MPA navigation. + # Testing Strategy This feature is mostly client-side so it will be tested via the Playwright e2e test suite. @@ -253,7 +299,7 @@ This feature is mostly client-side so it will be tested via the Playwright e2e t # Drawbacks - This feature is primarily about taking advantage of cutting edge features. Currently it is Chromium browsers only. There is some risk that other browsers will not adopt these APIs and we'll be left having to do a fallback for a long time. -- This approach is not the best for apps. We would probably need a whole separate API if we wanted to support that better. +- Full apps that never navigate to different pages are still likely better served by client-side routers. This router is targeting multi-page sites where you want to make transitions appear more smooth and integrated. # Alternatives From ae10c62e39bfad21fb770231f6c4319dead36a90 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Wed, 16 Aug 2023 09:58:42 -0400 Subject: [PATCH 225/300] Add new events --- proposals/0040-view-transitions.md | 37 ++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/proposals/0040-view-transitions.md b/proposals/0040-view-transitions.md index 6d964dea..43673a7c 100644 --- a/proposals/0040-view-transitions.md +++ b/proposals/0040-view-transitions.md @@ -292,6 +292,43 @@ The possible values are: - `swap`: A fallback where the DOM is swapped without animations. - `none`: Do not fallback for non-supporting browsers, allow MPA navigation. +## Events + +These are some initial events that are dispatched on the `document`: + +### `astro:afterswap` + +This event occurs during a transition, immediately after the new page has been swapped in for the old page. This gives you a chance to update the DOM before it is painted by the browser. + +A use-case is to restore dark mode: + +```html + +``` + +### `astro:navigationsuccess` + +This event occurs after a navigation has occured, the DOM is swapped, and all resources have been loaded. This event happens both on initial page load and on any transitions, so it is a good place to do any sort of page setup logic: + +```html + +``` + # Testing Strategy This feature is mostly client-side so it will be tested via the Playwright e2e test suite. From b41a562b00e598e28f028c1998aa51cda1534be8 Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Mon, 21 Aug 2023 11:44:40 -0400 Subject: [PATCH 226/300] Update to not block on bikeshedding of event names. --- proposals/0040-view-transitions.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/proposals/0040-view-transitions.md b/proposals/0040-view-transitions.md index 43673a7c..8b2e4cc5 100644 --- a/proposals/0040-view-transitions.md +++ b/proposals/0040-view-transitions.md @@ -296,7 +296,9 @@ The possible values are: These are some initial events that are dispatched on the `document`: -### `astro:afterswap` +### After swap + +> Tentatively shown as `astro:afterswap` here, but the name is subject to bikeshedding before released. This event occurs during a transition, immediately after the new page has been swapped in for the old page. This gives you a chance to update the DOM before it is painted by the browser. @@ -315,7 +317,9 @@ A use-case is to restore dark mode: ``` -### `astro:navigationsuccess` +### Page load + +> Tentatively shown as `astro:pageload` here, but the name is subject to bikeshedding before released. This event occurs after a navigation has occured, the DOM is swapped, and all resources have been loaded. This event happens both on initial page load and on any transitions, so it is a good place to do any sort of page setup logic: @@ -325,7 +329,7 @@ This event occurs after a navigation has occured, the DOM is swapped, and all re /** ... */ } - document.addEventListener('astro:navigationsuccess', setupPage); + document.addEventListener('astro:pageload', setupPage); ``` From 760afc724e2691754d75e769b5f333c93938e3b5 Mon Sep 17 00:00:00 2001 From: Princesseuh Date: Tue, 26 Sep 2023 19:20:10 +0200 Subject: [PATCH 227/300] feat: picture component RFC --- proposals/0041-picture-component.md | 273 ++++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 proposals/0041-picture-component.md diff --git a/proposals/0041-picture-component.md b/proposals/0041-picture-component.md new file mode 100644 index 00000000..1fc06d56 --- /dev/null +++ b/proposals/0041-picture-component.md @@ -0,0 +1,273 @@ +- Start Date: 2023-09-22 +- Reference Issues: N/A +- Implementation PR: https://github.com/withastro/astro/pull/8620 + +# Summary + +The current `` component and `getImage` APIs are great to show a single image, however it doesn't answer some of the common needs of the web, such as fallback formats and responsive images. This RFC hopes to define the addition of two features: + +- Possibility for image services to generate a complete `srcset` request +- A Picture component in Astro itself + +# Example + +### Image service integration + +```ts +const imageService = { + ..., + // New optional hook for image services + getSrcSet(options, imageConfig) { + const srcSets = options.widths.map((width) => { + return { + transform: { + ...options, + width: width + }, + // Descriptor to use for this dimension, follow HTML's descriptors. + descriptor: `${width}w`, + // Additional HTML attributes to be used when rendering this srcset + // This is notably helpful for `` + attributes: { + type: `image/${options.format}` + } + } + }) + + return srcSets; + } +} +``` + +This API powers both examples below. As always with image services, 99.9% of users will never need to do this, this is only for the people (mainly us at Astro) implementing image services. + +Read the [detailed design section](#detailed-design) for more information on how image services can interact with the user's props to generate a `srcset`. + +--- + +### `srcset` support + +**User code** + +```astro +--- +import { Image } from "astro:assets"; +import myImage from "../something.png"; +--- + + +My image available in 2x and 3x densities + + +My image with precise widths +``` + +**Result** + +```html + + + +``` + +### Picture component + +**User Code** + +```astro +--- +import { Picture } from "astro:assets"; +import myImage from "../something.png"; +--- + + + +``` + +**Result** + +```html + + + +``` + +**NOTE:** `Picture` can take all the arguments of `Image`. + +**NOTE:** This code represent only the values accepted by the base image services in Astro. A custom image service is free to use other attributes and completely different methods to generate a `srcset` value. + +# Background & Motivation + +The statement from the original RFC for `astro:assets` still stands: Images on the web are hard! Getting your image, at the perfect quality, in the perfect format, at the perfect size at the specific size you need is, a lot of work! + +Abstracting all that work is also a fairly consequential challenge API-design wise, as there's a lot of features to support and concessions to be made for the brevity of the public API while still keeping the underlying API flexible enough. + +The main motivation here is again, the same as `astro:assets` itself: Creating a image in multiple formats and multiple sizes in Astro should be as easy as possible, and the API flexible enough for user land to be able to create the missing pieces that the lean core experience cannot fulfil. + +# Goals + +- Add a hook to generate a `srcset` to image services +- Add support for generating a `srcset` value for `` +- Add a built-in `` component, allowing to use multiple formats and image sizes, powered by the image service's ability to generate a `srcset` + +# Non-Goals + +## For the built-in experience + +- Automatic generation or API to make writing `sizes` easier +- Complex art direction with the built-in `` component + - Art direction is very flexible and can do a lot of things. Supporting all the features while still keeping the API easy to use is fairly challenging. It's definitely possible that we would support it in the future, but currently we'd rather make the API flexible enough and let users do what they need by themselves. + +As always, those non-goals are only relevant to **this proposal**. It is possible that in the future, we would expand the work outlined here to include those features. + +# Detailed Design + +`@astrojs/image` included a Picture component that, outside of re-using `getImage`, was 100% isolated in how it handled `srcset`. For the effort in `astro:assets`, going the other way seems more optimal: Add `srcset` to `Image` first, and reuse the underlying API to implement the same feature for `Picture`. + +Outside of the obvious benefit of having support for `srcset` on both `Image` and `Picture`, this also has the benefit of requiring the creation of a inherently more flexible underlying API, able to support multiple use-cases. + +I propose for this to be powered by a new optional hook in image services, dedicated to generating a `srcset` value. This hook would have the following signature: + +```ts +type SrcSetValue = { + transform: ImageTransform; + descriptor?: string; + attributes?: Record; +}; + +generateSrcSets(options: ImageTransform, imageConfig: AstroConfig['image']): SrcSetValue[] +``` + +This hook would be used to return an additional value from `getImage`: + +```ts +interface SrcSetResult { + url: string; + descriptor?: string; + attributes?: Record; // Additional attributes needed to show this `srcset`. This is mostly relevant for ``, where a source could need additional attributes (such as `type` for the format). +} + + +const result = { + srcSets: { + values: SrcSetResult[], + attribute: string // The ... 200w, ... 400w etc string, automatically generated from the values. + } +} +``` + +On the user side of things, the way to interact with this new API would be the following: + +**NOTE:** As always, this is specific to the base services, a custom image service might not accept those attributes, or at least, do nothing with them. + +## `` + +Image accept two new optional properties: + +- `widths: (number | ${number})[]` + - Allow to generate a number of widths to generate. Typically its value will look like `[1280, 1980, 3840, ...]`. + - At this moment, `heights` cannot be specified and the behaviour here follows the one of `width` (without a `s`) which means images will always keep the same aspect-ratio. +- `densities: (number | "${number}x")` + - Allow to generate a number of pixel densities to generate. Typically, its value will look like `["2x", "3x", ...]`. + +Those properties are mutually exclusive, so only one can be specified at a time. Either of those attributes will be used to generate a `srcset` value. For instance, the following code: + +```astro +--- +import { Image } from "astro:assets"; +import image from "../my_image.png"; +--- + + +``` + +will generate the following HTML: + +```html + +``` + +Similarly, `widths` will generate a `srcset`, albeit with width descriptors instead of pixels ones. It should be noted that `` won't generate a `sizes` for you, as such, the inclusion of that attribute is required when using `widths`, much like it is on the native `img` element. + +## `` + +Picture accepts all the properties that `` does, including the two new attributes described previously. However, it also support three other specific attributes: + +- `formats: ImageOutputFormat[];` + - Allow the specification of multiple image formats to use, every entry will be added as `` elements in the order they were inputted. +- `fallbackFormat: ImageOutputFormat;` + - Allow the specification of the image format to use as a fallback value (this will be used to fill the `` fallback inside the generated ``) +- `pictureAttributes: HTMLAttributes<'picture'>;` + - Allow specifying a list of attributes to add to the generated `` element. Every other attributes, apart from the ones used for the image transformation, will be applied to the inner `` element. + +As an example, the following code: + +```astro +--- +import { Image } from "astro:assets"; +import image from "../my_image.png"; +--- + + +``` + +will generate the following HTML: + +```html + + + + + +``` + +# Testing Strategy + +This can be tested the same way `astro:assets` is, so through a mix of fixtures and unit tests. + +# Drawbacks + +- Picture component are complicated, we can't serve all the use cases so maybe we shouldn't do it at all + - People are asking for it massively, and rightfully so. It's impossible to do optimized images in SSG (80%+ of Astro users) without ``, so +- People will be able to easily generate a lot of images now, so performance issues in image generation might shine through a bit more now + +# Alternatives + +I experimented a bit with a more composition-based API, but found it mostly to be annoying for users due to needing two imports. There was ways to work around that, but everything seemed cumbersome and/or didn't help users much. + +# Adoption strategy + +A new feature, so current users are not impacted at all! We'll document this alongside the current `` component From f485b5fbca5973185aeaeb3a4ec1cab2066b80e4 Mon Sep 17 00:00:00 2001 From: Princesseuh Date: Tue, 26 Sep 2023 19:24:04 +0200 Subject: [PATCH 228/300] fix: example --- proposals/0041-picture-component.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/proposals/0041-picture-component.md b/proposals/0041-picture-component.md index 1fc06d56..d616ff83 100644 --- a/proposals/0041-picture-component.md +++ b/proposals/0041-picture-component.md @@ -88,7 +88,17 @@ import myImage from "../something.png"; ```html - + + + My image available in 3 formats and 3 densities ``` From 837e16373ffb9f6f9a841cfd3fab331f6537740c Mon Sep 17 00:00:00 2001 From: Matthew Phillips Date: Tue, 3 Oct 2023 16:40:45 -0400 Subject: [PATCH 229/300] Fragments RFC --- proposals/fragments.md | 103 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 proposals/fragments.md diff --git a/proposals/fragments.md b/proposals/fragments.md new file mode 100644 index 00000000..9ebdd788 --- /dev/null +++ b/proposals/fragments.md @@ -0,0 +1,103 @@ +- Start Date: 2023-10-03 +- Reference Issues: https://github.com/withastro/roadmap/issues/697 +- Implementation PR: + +# Summary + +Provide a configuration flag for page components to opt-in to *fragment* behavior, preventing head injection of scripts and styles, and the doctype. + +# Example + +In any component inside of the __pages__ directory set the `fragment` option: + +```astro +--- +export const fragment = true; +--- + +
    This is a fragment!
    +``` + +# Background & Motivation + +Partials are a technique that has been used by web applications for decades, popularized in frameworks such as Ruby on Rails. Frontend oriented JavaScript frameworks have typically not used partials, but instead use JSON APIs and front-end templating in order to dynamically change parts of the page. Nevertheless, partials have remained a niche feature, and with Astro's backend focus we have had interest in support for a long time. + +Recently the popularity of projects like [htmx](https://htmx.org/) and [Unpoly](https://unpoly.com/) have revived interest in this technique. Since Astro treats each page request as a request for a full page it automatically attaches the doctype and head elements, making it difficult to simply inject into the page. + +# Goals + +- The ability to request a URL from Astro that excludes the usual page elements (doctype, head injection). +- The base feature should work the same in SSG and SSR apps. Partials are still output as `.html` files in a static build. + +# Non-Goals + +- This isn't an integration specifically for HTMX or any one library. It should work with any library or manual DOM manipulation that does `innerHTML`. +- No client-side scripts from Astro will be part of this change. +- Support for integrations or middleware at this time. It could be possible to allow middleware to communicate that a request is for a partial, but that is not part of this RFC. + +# Detailed Design + +Fragments are opted into on a per-page basis. Any page within the `pages` directory can become a fragment through this config: + +```astro +--- +export const fragment = true; +--- +``` + +This value must be either: + +- A literal boolean of `true` or `false` (there's no reason to every use false). +- A configuration value from `import.meta` such as: + + ```astro + --- + export const fragment = import.meta.env.USE_FRAGMENTS; + --- + ``` + +The value *must* be identified statically. This means that a page can't be both a full page and a fragment. If you want to share the same logic and template for a partial and fragment you can do so by putting the common code into a component. + +## Implementation + +This is a very small change. Internally Astro uses an object known as a `result` that stores intoformation about a request. That result object will add a `fragment` property: + +```ts +interface AstroResult { + fragment: boolean; +} +``` + +When rendering the fragment value is taken from the __component module__ and placed on the result object. + +This boolean is then used in two places: + +- In head injection it will be used to prevent head injection. +- In page rendering it will be used to prevent doctype prepending. + +These are the only changes needed. + +# Testing Strategy + +We want to verify that fragments do not include head content with tests for: + +- A component that contains `