Access documents of the current collection in transform

[wottpal]

Hey @sdorra, I would really appreciate if you could find a way to make this possible. Referring to #245.

My current workaround (probably interesting for you @Kuzry, too) is defining a dummy/helper collection first (with only very few schema attributes that all pages share like slug or title). And then I can iterate over those via context.documents(helpers).map(…) as usual.

Though, for performance & DRY reasons I would highly appreciate being able to access all pages in the transform function w/o that.

My suggestion: As I understand it's impossible to make them accessible post-transform, I would suggest having some kind of pre-transform document collections of the current collection being accessible during transform (hope that's understandable).

There are just so many use cases this would enable:

• Very helpful in multilingual setups where I want to determine the internationalized version of the current page.
• If I have some kind of embeddings and want to pre-compute "related pages".
• …

[sdorra]

I believe that it is already possible to access the documents of the collection in the transform function:

content-collections/packages/core/src/transformer.ts

Line 134 in 8f0de86

documents: (collection) => {

It is probably just a type problem.

content-collections/packages/core/src/config.ts

Line 50 in 8f0de86

documents<TCollection extends AnyCollection>(

At first glance it looks right. I'm on vacation for a week now, but I'll take a look at it when I get back.

[wottpal]

Wow this indeed seems to work, though I tried this before. 🤔 Amazing!

Now I only need some kind of equivalent for the GetTypeByName generic to get the "pre-transform" type of a collection document.

[sdorra]

Okay, I've looked into it, and I believe it is not possible to get the types right for the documents function of the Context. This is because the function requires the collection as input, and it uses the type of the collection to determine the type of the resulting documents. However, the type of the collection is based on the result of the transform function. Therefore, we cannot use the type in a function that is required to determine the type. Does this make sense?

We could introduce a new API to retrieve the documents of the current collection. The context already includes a collection. We could add a documents function without any parameters. An example usage could look like this:

import { defineCollection, defineConfig } from "@content-collections/core";

const posts = defineCollection({
  name: "posts",
  directory: "sources/posts",
  include: "*.md",
  schema: (z) => ({
    title: z.string(),
    description: z.string(),
  }),
  transform: async (post, { collection }) => {
    const docs = await collection.documents()
    const idx = docs.indexOf(post);
    const prev = docs[idx - 1];
    const next = docs[idx + 1];

    return {
      ...post,
      prev,
      next
    }
  }
});

export default defineConfig({
  collections: [posts],
});

But I'm not sure about the naming. It's clear that the ctx.documents function is for retrieving documents from other collections, while ctx.collection.documents is for documents from the current collection?

[sdorra]

@wottpal, could you please check if the functionality of #290 meets your requirements?

To install the preview packages, add the package you need as described in this comment:

#290 (comment)

The documentation for the new feature can be found here:

https://content-collections-otminv2j1-nitilon.vercel.app/docs/transform#access-sibling-documents

[wottpal]

Hey, @sdorra, yes #290 looks pretty useful indeed.

Though I was not able to test it, even though I've installed all the pkg.pr.new deps with bun. Context.collection still only has the name & directory attributes for me (also force-reinstalled, restarted VSCode, etc.)

[sdorra]

@wottpal you are right the github workflow has used the main branch instead of the pr head. Now the packages should contain the code from the pr. If your package manager does not refresh the packages with pr suffix (290) you could use 639c2ea as suffix e.g.:

    "@content-collections/core": "https://pkg.pr.new/sdorra/content-collections/@content-collections/core@639c2ea",
    "@content-collections/mdx": "https://pkg.pr.new/sdorra/content-collections/@content-collections/mdx@639c2ea",
    "@content-collections/next": "https://pkg.pr.new/sdorra/content-collections/@content-collections/next@639c2ea",

[wottpal]

Works & works 👌 ✅