TurboPack build fails: Cannot resolve 'fs' in pdf-parse next 15

[rossanodr]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import { WebPDFLoader } from "@langchain/community/document_loaders/web/pdf";
 await new WebPDFLoader(file, {
        splitPages: false,
        parsedItemSeparator: "",

      }).load();

or

import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";

await new PDFLoader(file, {
        splitPages: false,
        parsedItemSeparator: "",
      }).load();

Error Message and Stack Trace (if applicable)

× ./node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:17378:10
Module not found: Can't resolve 'fs'
17378 | var fs = require('fs');
| ^^^^^^^^^^^^^
17379 | var http = require('http');
17380 | var https = require('https');
17381 | var url = require('url');

Description

I am trying to build a Next.js project with TurboPack enabled and using WebPDFLoader or PDFLoader.
I expect the build to succeed, as it does when using Webpack with appropriate fallback settings.
Instead, the build fails due to a module resolution error for fs, http, and other Node-specific modules.
If I don't use the --turbo tag it works

System Info

 "next": "15.0.3-canary.2",

devDependencies: {
'@faker-js/faker': '^7.6.0',
'@jest/globals': '^29.5.0',
'@langchain/anthropic': '',
'@langchain/aws': '',
'@langchain/cerebras': '',
'@langchain/cohere': '',
'@langchain/core': 'workspace:',
'@langchain/google-genai': '',
'@langchain/google-vertexai': '',
'@langchain/google-vertexai-web': '',
'@langchain/groq': '',
'@langchain/mistralai': '',
'@langchain/ollama': '',
'@langchain/scripts': '>=0.1.0 <0.2.0',
'@swc/core': '^1.3.90',
'@swc/jest': '^0.2.29',
'@tsconfig/recommended': '^1.0.2',
'@types/handlebars': '^4.1.0',
'@types/html-to-text': '^9',
'@types/js-yaml': '^4',
'@types/jsdom': '^21.1.1',
'@types/uuid': '^9',
'@types/ws': '^8',
'@typescript-eslint/eslint-plugin': '^5.58.0',
'@typescript-eslint/parser': '^5.58.0',
axios: '^0.26.0',
cheerio: '1.0.0-rc.12',
dotenv: '^16.0.3',
dpdm: '^3.12.0',
eslint: '^8.33.0',
'eslint-config-airbnb-base': '^15.0.0',
'eslint-config-prettier': '^8.6.0',
'eslint-plugin-import': '^2.27.5',
'eslint-plugin-jest': '^27.6.0',
'eslint-plugin-no-instanceof': '^1.0.1',
'eslint-plugin-prettier': '^4.2.1',
handlebars: '^4.7.8',
jest: '^29.5.0',
'jest-environment-node': '^29.6.4',
openai: '^4.41.1',
peggy: '^3.0.2',
prettier: '^2.8.3',
'release-it': '^17.6.0',
rimraf: '^5.0.1',
rollup: '^3.19.1',
'ts-jest': '^29.1.0',
typeorm: '^0.3.20',
typescript: '~5.1.6',
wikipedia: '^2.1.2'
},
peerDependencies: {
'@langchain/anthropic': '',
'@langchain/aws': '',
'@langchain/cerebras': '',
'@langchain/cohere': '',
'@langchain/core': '>=0.2.21 <0.4.0',
'@langchain/google-genai': '',
'@langchain/google-vertexai': '',
'@langchain/google-vertexai-web': '',
'@langchain/groq': '',
'@langchain/mistralai': '',
'@langchain/ollama': '',
axios: '',
cheerio: '',
handlebars: '^4.7.8',
peggy: '^3.0.2',
typeorm: ''
}

[dosubot]

There are similar issues related to the "Module not found: Can't resolve 'fs'" error when using LangChain.js with Next.js. Here are some relevant threads and solutions:

1. Module not found: Error: Can't resolve 'fs' when using langchain document loader: This discussion suggests modifying the next.config.js file to disable the 'fs' module in the client-side code. You can do this by setting fs to false in the resolve.fallback section of the webpack configuration. Here's an example:

   /** @type {import('next').NextConfig} */
   const nextConfig = {
     webpack: (config, { isServer }) => {
       if (!isServer) {
         config.resolve.fallback = {
           ...config.resolve.fallback,
           fs: false,
           "node:fs/promises": false,
           module: false,
           perf_hooks: false,
         };
       }
       return config;
     },
   }
   
   module.exports = nextConfig;

2. Getting Module build failed: UnhandledSchemeError: Reading from "node:fs/promises" is not handled by plugins (Unhandled scheme). in building React app with Langchain JS: This issue also deals with similar module resolution problems and suggests creating a mock for the 'fs' module. You can create a mocks/fs.js file with module.exports = {}; and then alias it in your Next.js config:

   const path = require('path');
   module.exports = {
     webpack: (config, { isServer }) => {
       if (!isServer) {
         config.resolve.alias['fs'] = path.resolve(__dirname, './mocks/fs.js');
       }
       return config;
     },
   }

These solutions should help resolve the module resolution errors you're encountering when building your Next.js project with TurboPack.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[jacoblee93]

Easiest thing to do might be to enable node compatibility if possible - the web loader should work with turbopack but bundlers can be tricky :(

[rossanodr]

even with "moduleResolution": "node" I can't make it work :(

Easiest thing to do might be to enable node compatibility if possible - the web loader should work with turbopack but bundlers can be tricky :(