Skip to content

Latest commit

 

History

History
11 lines (6 loc) · 375 Bytes

README.md

File metadata and controls

11 lines (6 loc) · 375 Bytes

GPT-Crawler-Config

Replace your config in Builder IO's GPT Cralwer with this one to set the config parameters via command line arguments.

GPT Cralwer: https://github.com/BuilderIO/gpt-crawler

Builder IO seems to have not wanted to bother with the CLI, and I don't want to edit the config every time.

Usage:

npm start url match maxPagesToCrawl outputFileName maxTokens