[BUG] Protocol error (Fetch.getResponseBody): Can only get response body on requests captured after headers received.

[sasa130]

Environment

• chromium Version: 122.0.0
• puppeteer / puppeteer-core Version: 22.4.1
• Node.js Version: 20.x
• Lambda / GCF Runtime: nodejs20.x

Expected Behavior

page.setContent executes successfully

Current Behavior

Protocol error (Fetch.getResponseBody): Can only get response body on requests captured after headers received.

is thrown by this line of code:

await page.setContent(htmlString, {
waitUntil: 'load',
});

Steps to Reproduce

The error occurs in the call to convertToPdf, specifically in page.setContent (since the last log we saw was 'interceptAndResizeImages: success'). The same set of code works for most of the emails we're converting to html string, but for some reason, fails for some emails.

import chromium from '@sparticuz/chromium';

let browser: Browser | undefined = undefined;

const converToPdf = async (htmlString: string): Promise<Buffer> => {
  const page = await (await init()).newPage();
  await interceptAndResizeImages(page);
  logger.debug('interceptAndResizeImages: success');
  await page.setContent(htmlString, {
    waitUntil: 'load',
  });
  logger.debug('page.setContent: success');
  await page.emulateMediaType('screen');
  await page.evaluateHandle('document.fonts.ready');
  await page.waitForSelector('*');
  const pdfBuffer = await page.pdf({
    format: 'a4',
    printBackground: true,
  });

  await page.close();

  return pdfBuffer;
};

export const init = async (): Promise<Browser> => {
  if (browser && browser.isConnected()) {
    return browser;
  }

  chromium.setGraphicsMode = false;
  const executablePath = await chromium.executablePath();
  browser = await launch({
    args: chromium.args,
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath(),
    headless: true,
    ignoreHTTPSErrors: true,
  });

  return browser;
};


const interceptAndResizeImages = async (page: Page): Promise<void> => {
  const client = await page.target().createCDPSession();
  client.on('Fetch.requestPaused', async (event: Protocol.Fetch.RequestPausedEvent) => {
    const response = await client.send('Fetch.getResponseBody', {
      requestId: event.requestId,
    });

    await client.send('Fetch.fulfillRequest', {
      requestId: event.requestId,
      responseCode: 200,
      body: await resizeImage(response.body),
    });
  });
  await client.send('Fetch.enable', {
    handleAuthRequests: false,
    patterns: [{ requestStage: 'Response', resourceType: 'Image' }],
  });
};

Possible Solution

[simonjsp]

Apparently it is related to the fact that the Node 18.X runtime (in my case) went from 18.26 to 18.28 and the latter breaks something in the library (at least in my case it worked perfectly until AWS started the function with 18.26). Unfortunately you cannot choose the minor version in the configuration of each lambda.

[alexander-dermicus]

Potentially related (with fix): #229 (comment)