You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MONGODB_PUBLIC_ADDRESS = '127.0.0.1:27017' # This will be shown on the web intee
rface, but won't be used for connecting to DB
MONGODB_URI = '127.0.0.1:27017' # Actual uri to connect to DB
MONGODB_USER = ''
MONGODB_PASSWORD = ''
MONGODB_SHARDED = False
MONGODB_BUFFER_DATA = 100
LINK_GENERATOR = 'http://127.0.0.1:6800' # Set your link generator worker addree
ss here
SCRAPERS = ['172.17.0.4:3800', '172.17.0.2:6800'] # Set your scraper worker addd
resses here
LINUX_USER_CREATION_ENABLED = False # Set this to True if you want a linux userr
account created during registration`
And This is what I get when I curl them from link generator -
`root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.4:3800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:31 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0
root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.2:6800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:50 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0`
So workers are up .. but somehow link generator cant connect to them.
The text was updated successfully, but these errors were encountered:
I started scrapyd , mongo and rabbitmq in worker threads still it shows worker status: unreachable , version: unknown
When I cal ping workers from Link generator curl working file
`SCHEDULER = ".rabbitmq.scheduler.Scheduler"
SCHEDULER_PERSIST = True
RABBITMQ_HOST = '127.0.0.1'
RABBITMQ_PORT = 5672
RABBITMQ_USERNAME = 'guest'
RABBITMQ_PASSWORD = 'guest'
MONGODB_PUBLIC_ADDRESS = '127.0.0.1:27017' # This will be shown on the web intee
rface, but won't be used for connecting to DB
MONGODB_URI = '127.0.0.1:27017' # Actual uri to connect to DB
MONGODB_USER = ''
MONGODB_PASSWORD = ''
MONGODB_SHARDED = False
MONGODB_BUFFER_DATA = 100
LINK_GENERATOR = 'http://127.0.0.1:6800' # Set your link generator worker addree
ss here
SCRAPERS = ['172.17.0.4:3800', '172.17.0.2:6800'] # Set your scraper worker addd
resses here
LINUX_USER_CREATION_ENABLED = False # Set this to True if you want a linux userr
account created during registration`
`root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.4:3800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:31 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0
root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.2:6800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:50 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0`
So workers are up .. but somehow link generator cant connect to them.
The text was updated successfully, but these errors were encountered: