Cannot generate embeddings

[antonioglass]

Bug report

antonio@Antonios-MacBook-Pro nextjs-openai-doc-search % pnpm run embeddings

> [email protected] embeddings /Users/antonio/Desktop/nextjs-openai-doc-search
> tsx lib/generate-embeddings.ts

Discovered 1 pages
Checking which pages are new or have changed
Page '/docs/openai_embeddings' or one/multiple of its page sections failed to store properly. Page has been marked with null checksum to indicate that it needs to be re-generated.
{
  code: 'PGRST116',
  details: 'The result contains 0 rows',
  hint: null,
  message: 'JSON object requested, multiple (or no) rows returned'
}
Embedding generation complete

Describe the bug

Cannot generate embeddings when running pnpm run embeddings.

To Reproduce

1. Set up a local project that is connected to a remote Supabase project.
2. Run pnpm run embeddings locally.

Expected behavior

Successful generation of embeddings.

System information

• OS: macOS
• Version of supabase-js: ^2.13.0
• Version of Node.js: 18.15.10

[thiemo--bleeken]

got the same issue

[thorwebdev]

Can you double check that your local .env file is configured with the correct keys and if yes, can you share the content of your nods_page and nods_page_section tables?

[jackf723]

I'm having the same issue. My keys are correct in the local env, and on the production environment Supabase and Vercel are integrated. But pnpm run embeddings fails consistently.

[wonkishtofu]

This seems to have resolved on my end after upgrading the supabase client package.
You can do so by running pnpm up @supabase/supabase-js

[danielnyan]

After updating the version of Supabase, we can generate embeddings now, but now one of the TypeScript files fail to compile after updating the version of Supabase.

I would post my fix, but I'm very much not a software developer and I'm pretty sure my solution violates at least 10 good practices of software engineering, so I'll let someone else propose a fix.

[jackf723]

After updating supabase, same error as before when attempting a search. Sad news, the search has failed! Please try again.

[jackf723]

And also getting the ts compilation errors on deployment.
Failed to compile.

./lib/generate-embeddings.ts:334:36
Type error: Conversion of type '{ id: any; path: any; }[]' to type 'undefined' may be a mistake because neither type sufficiently overlaps with the other. If this was intentional, convert the expression to 'unknown' first.

332 | // We use checksum to determine if this page & its sections need to be regenerated
333 | if (!shouldRefresh && existingPage?.checksum === checksum) {

334 | const existingParentPage = existingPage?.parentPage as Singular<
| ^
335 | typeof existingPage.parentPage
336 | >
337 |

[danielnyan]

As of 19 Sep 2023, 00:09 UTC, Vercel has identified an Increased middleware and edge function failures in pdx1 which causes the error of "Sad news, the search has failed!" to occur. This issue doesn't seem to lie with Supabase because I logged the queries and responses in a backend Supabase table, and it seems that the queries are being sent back and forth from OpenAI just fine. As of right now (04:03 UTC), my front-end is receiving the outputs from OpenAI successfully.
(Edit: nevermind, it's still a bit iffy. My request times out every now and then, even now)

I'm not entirely sure what lines 333 to 337 is doing either, but I used this quick fix in my project by replacing lines 333 to 337 with the following: .

if (!shouldRefresh && existingPage !== null && existingPage.checksum === checksum) {
        const existingParentPage = existingPage.parentPage?.length === 1 ? existingPage.parentPage[0] : undefined;

I'm not sure if this is the intended fix, but it should minimally get the project up and running in the meantime.

[r3fs]

@antonioglass Did you ever find a fix? upgrading the supabase-js package and got this error

{
  code: 'PGRST102',
  details: null,
  hint: null,
  message: 'Content-Type not acceptable: application/json, text/plain'
}

[r3fs]

@antonioglass Did you ever find a fix? upgrading the supabase-js package and got this error

{
  code: 'PGRST102',
  details: null,
  hint: null,
  message: 'Content-Type not acceptable: application/json, text/plain'
}

fix this issue by upgrading supabase and node js to v20

[wonkishtofu]

Apologies, took a break before coming back to this.
I made fixed this a bit ago by:

• swapping on line 331 the undefined type to unknown.
• modify the if statement on line 340, to assert a 'string' type on the path property in the existingParentPage object.

These changes were made on my own forked project, so I can't be sure if it would work on this one.
Perhaps someone could give it a spin? ;)

[jackf723]

Line numbers must be mismatched on my end. If you don't mind posting this generate-embeddings.ts code, or at least a side by side of the original lines vs the corrected ones, that would be appreciated.