Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not generating elasticindex #2

Open
mhvenkat opened this issue Apr 15, 2014 · 2 comments
Open

Not generating elasticindex #2

mhvenkat opened this issue Apr 15, 2014 · 2 comments

Comments

@mhvenkat
Copy link

I tried with this repository and followed all the steps. Still it is not generating any doc or index in ElasticSearch.

Here is the output (ES index name is nutch, cluster name is nutch)
...

LinkDb: merging with existing linkdb: TestCrawl2/linkdb
LinkDb: finished at 2014-04-14 18:46:47, elapsed: 00:00:02
Dedup on crawldb
Indexing 20140414184635 on ElasticSearch index -> nutch
Indexer: starting at 2014-04-14 18:46:52
Indexer: deleting gone documents: false
Indexer: URL filtering: false
Indexer: URL normalizing: false
Active IndexWriters :
ElasticIndexWriter
elastic.cluster : elastic prefix cluster
elastic.host : hostname
elastic.port : port
elastic.index : elastic index command
elastic.max.bulk.docs : elastic bulk index doc counts. (default 250)
elastic.max.bulk.size : elastic bulk index length. (default 2500500 ~2.5MB)

Indexer: finished at 2014-04-14 18:46:54, elapsed: 00:00:02
Mon Apr 14 18:46:55 PDT 2014 : Iteration 2 of 5
Generating a new segment
Generator: starting at 2014-04-14 18:46:56
Generator: Selecting best-scoring urls due for fetch.
Generator: filtering: false
Generator: normalizing: true
Generator: topN: 50000
-shouldFetch rejected 'https://nutch.apache.org/', fetchTime=1397611053313, curTime=1397526416071
Generator: 0 records selected for fetching, exiting ...


finally resolved the issue -- the elasticsearch port should be 9300 rather than 9200.
elasticsearch version should be 0.90.11 instead of 0.90.1 . You will have rebuild the nutch 1.8 after updating ivy.xml

@andreivisan
Copy link
Owner

As far as I can see nothing is fetched that is why nothing gets indexed.
You can download the complete bundle with nutch 1.8, add your settings and all should work.

@mhvenkat
Copy link
Author

finally resolved the issue -- the elasticsearch port should be 9300 rather than 9200.
elasticsearch version should be 0.90.11 instead of 0.90.1 . You will have rebuild the nutch 1.8 after updating ivy.xml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants