From fae361e0ac5b28c72c7c19b739c85a6c9b2bde35 Mon Sep 17 00:00:00 2001 From: qjing666 Date: Thu, 27 Feb 2020 14:09:04 +0800 Subject: [PATCH 1/3] add k8s deployment solution --- README.md | 5 +++++ README_cn.md | 8 ++++++++ 2 files changed, 13 insertions(+) diff --git a/README.md b/README.md index c0efffc3..d0bac36a 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,11 @@ In PaddleFL, components for defining a federated learning task and training a fe Please reference [Quick Start](https://paddlefl.readthedocs.io/en/latest/instruction.html) for installation and quick-start example. +## Easy deployment with kubernetes +```sh +kubectl apply -f ./paddle_fl/examples/k8s_deployment/master.yaml + +``` ## Benchmark task Gru4Rec [9] introduces recurrent neural network model in session-based recommendation. PaddlePaddle's Gru4Rec implementation is in https://github.com/PaddlePaddle/models/tree/develop/PaddleRec/gru4rec. An example is given in [Gru4Rec in Federated Learning](https://paddlefl.readthedocs.io/en/latest/examples/gru4rec_examples.html) diff --git a/README_cn.md b/README_cn.md index dc2733a8..aea34816 100644 --- a/README_cn.md +++ b/README_cn.md @@ -55,6 +55,14 @@ PaddleFL是一个基于PaddlePaddle的开源联邦学习框架。研究人员可 请参考[快速开始](https://paddlefl.readthedocs.io/en/latest/instruction.html)。 +## K8S 一键部署 + +```sh + +kubectl apply -f ./paddle_fl/examples/k8s_deployment/master.yaml + +``` + ## 性能测试 Gru4Rec [9] 在基于会话的推荐中引入了递归神经网络模型。PaddlePaddle的GRU4RC实现代码在 https://github.com/PaddlePaddle/models/tree/develop/PaddleRec/gru4rec. 一个基于联邦学习训练Gru4Rec模型的示例请参考[Gru4Rec in Federated Learning](https://paddlefl.readthedocs.io/en/latest/examples/gru4rec_examples.html) From c2eb45356bd802e0c615783f1708f9e61d5a4599 Mon Sep 17 00:00:00 2001 From: qjing666 Date: Thu, 27 Feb 2020 16:15:25 +0800 Subject: [PATCH 2/3] add k8s deployment solution --- README.md | 2 + README_cn.md | 4 +- paddle_fl/examples/k8s_deployment/README.md | 50 +++++++++++++++++++++ 3 files changed, 54 insertions(+), 2 deletions(-) create mode 100644 paddle_fl/examples/k8s_deployment/README.md diff --git a/README.md b/README.md index d0bac36a..c876cd6c 100644 --- a/README.md +++ b/README.md @@ -60,9 +60,11 @@ Please reference [Quick Start](https://paddlefl.readthedocs.io/en/latest/instruc ## Easy deployment with kubernetes ```sh + kubectl apply -f ./paddle_fl/examples/k8s_deployment/master.yaml ``` +Please refer [K8S deployment example](./paddle_fl/examples/k8s_deployment/README.md) for details ## Benchmark task Gru4Rec [9] introduces recurrent neural network model in session-based recommendation. PaddlePaddle's Gru4Rec implementation is in https://github.com/PaddlePaddle/models/tree/develop/PaddleRec/gru4rec. An example is given in [Gru4Rec in Federated Learning](https://paddlefl.readthedocs.io/en/latest/examples/gru4rec_examples.html) diff --git a/README_cn.md b/README_cn.md index aea34816..850e51be 100644 --- a/README_cn.md +++ b/README_cn.md @@ -55,14 +55,14 @@ PaddleFL是一个基于PaddlePaddle的开源联邦学习框架。研究人员可 请参考[快速开始](https://paddlefl.readthedocs.io/en/latest/instruction.html)。 -## K8S 一键部署 +## Kubernetes简单部署 ```sh kubectl apply -f ./paddle_fl/examples/k8s_deployment/master.yaml ``` - +请参考[K8S 部署实例](./paddle_fl/examples/k8s_deployment/README.md) ## 性能测试 Gru4Rec [9] 在基于会话的推荐中引入了递归神经网络模型。PaddlePaddle的GRU4RC实现代码在 https://github.com/PaddlePaddle/models/tree/develop/PaddleRec/gru4rec. 一个基于联邦学习训练Gru4Rec模型的示例请参考[Gru4Rec in Federated Learning](https://paddlefl.readthedocs.io/en/latest/examples/gru4rec_examples.html) diff --git a/paddle_fl/examples/k8s_deployment/README.md b/paddle_fl/examples/k8s_deployment/README.md new file mode 100644 index 00000000..73b1ae76 --- /dev/null +++ b/paddle_fl/examples/k8s_deployment/README.md @@ -0,0 +1,50 @@ +# PaddleFL deployment example with Kubernetes + +## compile time + +#### Master + +```sh + +#Define distributed training config for trainer and server +python fl_master.py --trainer_num 2 +tar -zcvf fl_job_config.tar.gz fl_job_config + +#Start HTTP server and wait download request from trainer and server +python -m SimpleHTTPServer 8000 + +``` + +## Run time + +#### Scheduler +```sh + +#Start a Scheduler +python fl_scheduler.py --trainer_num 2 + +``` + +#### Server +```sh + +#Download job config file from master +wget ${FL_MASTER_SERVICE_HOST}:${FL_MASTER_SERVICE_PORT_FL_MASTER}/fl_job_config.tar.gz +tar -xf fl_job_config.tar.gz + +#Start a Server +python -u fl_server.py > server.log 2>&1 + +``` + +###Trainer +```sh + +#Download job config file from master +wget ${FL_MASTER_SERVICE_HOST}:${FL_MASTER_SERVICE_PORT_FL_MASTER}/fl_job_config.tar.gz +tar -xf fl_job_config.tar.gz + +#Start the ith trainer +python -u fl_trainer.py 0 + +``` From 19ed9855b1cbbb5e2aac9eb76729c9396dff8f5d Mon Sep 17 00:00:00 2001 From: qjing666 Date: Thu, 27 Feb 2020 18:27:41 +0800 Subject: [PATCH 3/3] add k8s deployment solution --- paddle_fl/examples/k8s_deployment/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paddle_fl/examples/k8s_deployment/README.md b/paddle_fl/examples/k8s_deployment/README.md index 73b1ae76..320f7b2b 100644 --- a/paddle_fl/examples/k8s_deployment/README.md +++ b/paddle_fl/examples/k8s_deployment/README.md @@ -45,6 +45,6 @@ wget ${FL_MASTER_SERVICE_HOST}:${FL_MASTER_SERVICE_PORT_FL_MASTER}/fl_job_config tar -xf fl_job_config.tar.gz #Start the ith trainer -python -u fl_trainer.py 0 +python -u fl_trainer.py i ```