Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deepfm -- fix the excessively high memory usage of data processing #141

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kaierlong
Copy link
Contributor

What type of PR is this?

/kind enhancement

What does this PR do / why do we need it:
fix the excessively high memory usage of data processing for deepfm demo

Which issue(s) this PR fixes:
fix the excessively high memory usage of data processing for deepfm demo

test install and run:

git clone https://github.com/kaierlong/tinyms.git && cd tinyms
git checkout deepfm

cat requirements.txt |sed "s#mindspore#mindspore-gpu#" > requirements_gpu.txt
cat setup.py |sed "s#mindspore#mindspore-gpu#" > setup_gpu.py

pip3 install -r requirements_gpu.txt
python3 setup_gpu.py install

mkdir ~/test
cp tests/st/deepfm.py ~/test/
cd ~/test

nohup python3 deepfm.py --device_target=GPU --epoch_size=5 --batch_size=16000 --checkpoint_dir=. &

@hellowaywewe
Copy link
Collaborator

@kaierlong Sorry, it is so late to reply. Our team has built a new environment in recent days, I will test your code in this new environment, thanks.

@hellowaywewe
Copy link
Collaborator

@kaierlong Thanks for your contribution. The test has been passed, can you resolve the conflict?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants