Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There are some issues in article "Launch a benchmarking cluster" #139

Open
qianlong-zhang opened this issue Apr 21, 2014 · 0 comments
Open

Comments

@qianlong-zhang
Copy link

I want to run pagerank on Graphx, following the instructions(https://github.com/amplab/graphx/wiki/Launch-a-benchmarking-cluster), I encountered some problems.
First, the running command(~/graphx/run-example org.apache.spark.graph.Analytics spark://$MASTERS:7077 pagerank hdfs://$MASTERS:9000/soc-LiveJournal1.txt --numIter=20 --numEPart=128) is wrong, I changed it to "./bin/run-example org.apache.spark.graphx.lib.Analytics spark://XXX:7077 pagerank hdfs://XXX:8020/soc-LiveJournal1.txt". The parameter "--numIter" can not be found in running pagerank, I read the source code and find it is used in cc benchmark.

when I run the command above, spark throw warnings and errors:
14/04/21 10:24:59 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
……
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Spark cluster looks down

What I am sure is spark UI is ok, and other benchmarks can run rightly.

What I expect is, if you can give some hint in deploying Graphx on standlone cluster?
Thanks very very much, cause I have been blocked on this for 1 week. :)

ankurdave pushed a commit to ankurdave/graphx that referenced this issue Oct 12, 2016
Author: Holden Karau <[email protected]>

Closes amplab#139 from holdenk/add_aggregate_to_python_api and squashes the following commits:

0f39ae3 [Holden Karau] Merge in master
4879c75 [Holden Karau] CR feedback, fix issue with empty RDDs in aggregate
70b4724 [Holden Karau] Style fixes from code review
96b047b [Holden Karau] Add aggregate to python rdd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant