PySparkDockerExample

Example PySpark application using docker-compose.

To run:

Pull the repo and cd into the directory. Then build the images: docker-compose build

And then run the PySpark job: docker-compose run py-spark

Play around by changing entrypoint.py or add more workers to docker-compose.yml

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
Dockerfile		Dockerfile
README.md		README.md
arquivo-cod-001.txt		arquivo-cod-001.txt
arquivo-cod-002.txt		arquivo-cod-002.txt
arquivo-cod-003.txt		arquivo-cod-003.txt
arquivo-cod-004.txt		arquivo-cod-004.txt
arquivo-cod-005.txt		arquivo-cod-005.txt
arquivo-cod-006.txt		arquivo-cod-006.txt
docker-compose.yml		docker-compose.yml
entrypoint.py		entrypoint.py
requirements.txt		requirements.txt