You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I tried with this repository and followed all the steps. Still it is not generating any doc or index in ElasticSearch.
Here is the output (ES index name is nutch, cluster name is nutch)
...
LinkDb: merging with existing linkdb: TestCrawl2/linkdb
LinkDb: finished at 2014-04-14 18:46:47, elapsed: 00:00:02
Dedup on crawldb
Indexing 20140414184635 on ElasticSearch index -> nutch
Indexer: starting at 2014-04-14 18:46:52
Indexer: deleting gone documents: false
Indexer: URL filtering: false
Indexer: URL normalizing: false
Active IndexWriters :
ElasticIndexWriter
elastic.cluster : elastic prefix cluster
elastic.host : hostname
elastic.port : port
elastic.index : elastic index command
elastic.max.bulk.docs : elastic bulk index doc counts. (default 250)
elastic.max.bulk.size : elastic bulk index length. (default 2500500 ~2.5MB)
Indexer: finished at 2014-04-14 18:46:54, elapsed: 00:00:02
Mon Apr 14 18:46:55 PDT 2014 : Iteration 2 of 5
Generating a new segment
Generator: starting at 2014-04-14 18:46:56
Generator: Selecting best-scoring urls due for fetch.
Generator: filtering: false
Generator: normalizing: true
Generator: topN: 50000
-shouldFetch rejected 'https://nutch.apache.org/', fetchTime=1397611053313, curTime=1397526416071
Generator: 0 records selected for fetching, exiting ...
finally resolved the issue -- the elasticsearch port should be 9300 rather than 9200.
elasticsearch version should be 0.90.11 instead of 0.90.1 . You will have rebuild the nutch 1.8 after updating ivy.xml
The text was updated successfully, but these errors were encountered:
As far as I can see nothing is fetched that is why nothing gets indexed.
You can download the complete bundle with nutch 1.8, add your settings and all should work.
finally resolved the issue -- the elasticsearch port should be 9300 rather than 9200.
elasticsearch version should be 0.90.11 instead of 0.90.1 . You will have rebuild the nutch 1.8 after updating ivy.xml
I tried with this repository and followed all the steps. Still it is not generating any doc or index in ElasticSearch.
Here is the output (ES index name is nutch, cluster name is nutch)
...
LinkDb: merging with existing linkdb: TestCrawl2/linkdb
LinkDb: finished at 2014-04-14 18:46:47, elapsed: 00:00:02
Dedup on crawldb
Indexing 20140414184635 on ElasticSearch index -> nutch
Indexer: starting at 2014-04-14 18:46:52
Indexer: deleting gone documents: false
Indexer: URL filtering: false
Indexer: URL normalizing: false
Active IndexWriters :
ElasticIndexWriter
elastic.cluster : elastic prefix cluster
elastic.host : hostname
elastic.port : port
elastic.index : elastic index command
elastic.max.bulk.docs : elastic bulk index doc counts. (default 250)
elastic.max.bulk.size : elastic bulk index length. (default 2500500 ~2.5MB)
Indexer: finished at 2014-04-14 18:46:54, elapsed: 00:00:02
Mon Apr 14 18:46:55 PDT 2014 : Iteration 2 of 5
Generating a new segment
Generator: starting at 2014-04-14 18:46:56
Generator: Selecting best-scoring urls due for fetch.
Generator: filtering: false
Generator: normalizing: true
Generator: topN: 50000
-shouldFetch rejected 'https://nutch.apache.org/', fetchTime=1397611053313, curTime=1397526416071
Generator: 0 records selected for fetching, exiting ...
finally resolved the issue -- the elasticsearch port should be 9300 rather than 9200.
elasticsearch version should be 0.90.11 instead of 0.90.1 . You will have rebuild the nutch 1.8 after updating ivy.xml
The text was updated successfully, but these errors were encountered: