Skip to content

Commit

Permalink
README
Browse files Browse the repository at this point in the history
  • Loading branch information
MatthewZMSU committed Dec 20, 2024
1 parent a36c72f commit 3b65883
Showing 1 changed file with 14 additions and 2 deletions.
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,21 @@ DOWNLOADER_MIDDLEWARES = {
}

PUPPETEER_SERVICE_URL = "http://localhost:3000" # Not necessary in other execution methods
```
You may want not to use scrapy-puppeteer-service.
In this case aou are able to run locally with playwright and pyppeteer.

# To change the execution method, you must add the corresponding setting:
EXECUTION_METHOD = "Puppeteer"
```python
DOWNLOADER_MIDDLEWARES = {
'scrapypuppeteer.middleware.PuppeteerServiceDownloaderMiddleware': 1042
}

DOWNLOAD_HANDLERS = { # Add browser download handler
"http": "scrapypuppeteer.browser_managers.browser_downloader_handler.BrowserDownloaderHandler",
"https": "scrapypuppeteer.browser_managers.browser_downloader_handler.BrowserDownloaderHandler",
}
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor" # Need to install asyncio reactor
EXECUTION_METHOD = "playwright" # Choose execution method
```
Available methods: `Puppeteer`, `Pyppeteer`, `Playwright`

Expand Down

0 comments on commit 3b65883

Please sign in to comment.