Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge develop-1.9.0 into master for release purpose (#738)
* fix issue, can not upload file on namenode and change fixed value to variable on istio gateway (#601) * fix issue, can not upload file on namenode Signed-off-by: stone-wlg <[email protected]> * fix issue, change fixed value to variable on istio gateway Signed-off-by: stone-wlg <[email protected]> * add default log level for fateflow service (#609) * Make the tutorial up to date with 1.7.2 on master branch (#570) * optimize the tutorial and make sure the example python scripts on notebook can run Signed-off-by: Chen Jing <[email protected]> * Change the wrong version in tutorial and remove a useless file Signed-off-by: Chen Jing <[email protected]> Co-authored-by: Chen Jing <[email protected]> Signed-off-by: Sun Xia <[email protected]> * Update contributing document (#575) * Update the contributing.md document for license and how to build, test. Signed-off-by: Layne Peng <[email protected]> * Add no-code way contributing part. Signed-off-by: Layne Peng <[email protected]> * Update the license up to 2022. Signed-off-by: Layne Peng <[email protected]> Signed-off-by: Sun Xia <[email protected]> * export variable cacheType Signed-off-by: Sun Xia <[email protected]> * fix bug Signed-off-by: Sun Xia <[email protected]> * add default log level for fateflow service Signed-off-by: Sun Xia <[email protected]> Co-authored-by: Chen <[email protected]> Co-authored-by: Chen Jing <[email protected]> Co-authored-by: Layne Peng <[email protected]> * fix issue, fateflow logging error (#610) Signed-off-by: stone-wlg <[email protected]> * Resolve the log flood issue of kubefate pod (#612) * Resolve the log flood issue of kubefate pod Signed-off-by: Chen Jing <[email protected]> * Use TCP for the kubefate probe Signed-off-by: Chen Jing <[email protected]> * Make it possible to upload dataset which is more than 100M in k8s spark deployment (#614) * Change hdfs datanode to statefulset and fix the rabbitmq example Signed-off-by: Chen Jing <[email protected]> * Optimize the name of the template Signed-off-by: Chen Jing <[email protected]> * Make the replicas of datanode configurable Signed-off-by: Chen Jing <[email protected]> * Make sure in docker compose mode, we can upload a dataset > 100M size (#617) * add the hdfs datanode number to 3 Signed-off-by: Chen Jing <[email protected]> * add more binary file types to avoid auto-convertion on a linux machine Signed-off-by: Chen Jing <[email protected]> * fix issues Signed-off-by: Chen Jing <[email protected]> * fix: Remove variable 'ingressHost' that is no longer in use (#620) Signed-off-by: magic-hya <[email protected]> * fix the condition: any issue of pulsar docker compose file (#627) Signed-off-by: Chen Jing <[email protected]> * add/bind federatedml dir/volumn in eggroll containers (#629) Signed-off-by: gxcuit <[email protected]> Co-authored-by: Chen Jing <[email protected]> * optmized the pr template (#630) * optmized the pr template Signed-off-by: Chen Jing <[email protected]> * fix typo Signed-off-by: Chen Jing <[email protected]> * optimize the docs: part I (#646) Signed-off-by: Chen Jing <[email protected]> * Feat: add docker build of fate_test (#637) * add docker build of fate_test Signed-off-by: Chenlong Ma <[email protected]> * update doc Signed-off-by: Chenlong Ma <[email protected]> * Fix fateflow's model_local_cache path (#639) * Fix fateflow's model_local_cache path Signed-off-by: Chenlong Ma <[email protected]> * fix fateflow's jobs cache path Signed-off-by: Chenlong Ma <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Update the docs of Slim FATE (#632) * Update the docs of Slim FATE Signed-off-by: Chenlong Ma <[email protected]> * Add comparison between different backends Signed-off-by: Chenlong Ma <[email protected]> * Update docker-deploy/README.md accurate expression Co-authored-by: Chen Jing <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> * Add English documentation for Introduction_to_Backend_Architecture Signed-off-by: Chenlong Ma <[email protected]> * Update docker-deploy/README.md Co-authored-by: Chen Jing <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> * Fix broken URLs Signed-off-by: Chenlong Ma <[email protected]> * resolve comments Signed-off-by: Chenlong Ma <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Do not let datanodes to share one pvc (#650) * Do not let datanodes to share one pvc Signed-off-by: Chen Jing <[email protected]> * fix the and grammar Signed-off-by: Chen Jing <[email protected]> * Fix an format issue Signed-off-by: Chen Jing <[email protected]> * Fix an un expected typo Signed-off-by: Chen Jing <[email protected]> * Support the validation for cluster.yaml (#636) * add ValuesTemplateExample to HelmChart Model Signed-off-by: hang lv <[email protected]> * add Vadd getValueTemplateExample api Signed-off-by: hang lv <[email protected]> * add Vvalidation implement and tests Signed-off-by: hang lv <[email protected]> * req and resp in cli side Signed-off-by: hang lv <[email protected]> * reqauto generated doc Signed-off-by: hang lv <[email protected]> * bug fix Signed-off-by: hang lv <[email protected]> * remove test case from local file Signed-off-by: hang lv <[email protected]> * bug fix Signed-off-by: hang lv <[email protected]> * bug fix Signed-off-by: hang lv <[email protected]> * Optimize the docs: part II (#654) * Optimize the docs: part II Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * fix change deployment to statefulset in nodemanager (#653) * fix change deployment to statefulset in nodemanager Signed-off-by: Chenlong Ma <[email protected]> * remove default value of nodeSelector tolerations and affinity Signed-off-by: Chenlong Ma <[email protected]> * optimize the docs: part III (#655) * optimize the docs: part III Signed-off-by: Chen Jing <[email protected]> * Fix a mistake Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * feat add ssh port configuration in docker deploy (#656) Signed-off-by: magic-hya <[email protected]> * Fix the wrong example link of Notebook (#659) Signed-off-by: Chenyang Gao <[email protected]> change the example link from http://party9999.fateboard.example.com to http://party9999.notebook.example.com Co-authored-by: Chen Jing <[email protected]> * Fix the typo of README.txt (#660) Signed-off-by: Chenyang Gao <[email protected]> fixed the mistake spell of "shown" Co-authored-by: Chen Jing <[email protected]> * modify eggroll manager ports (#661) Signed-off-by: hang lv <[email protected]> * Remove service url scheme hardcode in cli (#663) Signed-off-by: Chenyang Gao <[email protected]> There's hardcode scheme(http://) of KubeFATE service url in Kubefate cli, so that it can only support service without TLS (http & ws). It's neither graceful nor security. This commit removes the hardcode and add a new option in the config.yaml. (Modified the README accordingly) * fix configuration PATH of job_default_config.yaml (#667) Signed-off-by: Chenlong Ma <[email protected]> * Reduce the necessary size of the spark base image (#669) Signed-off-by: Chen Jing <[email protected]> * Signed-off-by: Chenyang Gao <[email protected]> (#670) Add doc about how to config TLS kubefate service Few days ago, I made a PR to remove the hardcode URL scheme in the cli. To give more help to the guys want to deploy the kubefate service with a TLS, I add a markdown doc as tutorial. It's not so detail. But may be a very first step for the ones may be interested in. * alert user if the modules not match backend (#674) * alert user if the modules not match backend Signed-off-by: hang lv <[email protected]> * bug fix for UT Signed-off-by: hang lv <[email protected]> * add supportbundle function to cli (#672) * add supportbundle function to cli Signed-off-by: hang lv <[email protected]> * split supportbundle module into 3 parts Signed-off-by: hang lv <[email protected]> * bug fix and more field filled Signed-off-by: hang lv <[email protected]> * correct spell of comment * detail ingress on rules, ports and address Signed-off-by: hang lv <[email protected]> * Merge branch 'fix-issue-597' of github.com:n063h/KubeFATE into fix-issue-597 Signed-off-by: hang lv <[email protected]> * Unit Test for k8sclient and supportbundle (#676) * add supportbundle function to cli Signed-off-by: hang lv <[email protected]> * split supportbundle module into 3 parts Signed-off-by: hang lv <[email protected]> * bug fix and more field filled Signed-off-by: hang lv <[email protected]> * correct spell of comment * detail ingress on rules, ports and address Signed-off-by: hang lv <[email protected]> * Merge branch 'fix-issue-597' of github.com:n063h/KubeFATE into fix-issue-597 Signed-off-by: hang lv <[email protected]> * unit test for k8sclient Signed-off-by: hang lv <[email protected]> * unit test for supportbundle Signed-off-by: hang lv <[email protected]> * Support defining resource requirements for rabbitmq, pulsar and nodemanager (#675) * Make resource customizable for eggroll, pulsar and rabbitmq Signed-off-by: Chen Jing <[email protected]> * Improvement Signed-off-by: Chen Jing <[email protected]> * fix typo Signed-off-by: Chen Jing <[email protected]> * remove warning on chartVersion (#678) Signed-off-by: hang lv <[email protected]> * Add the proposal for rolling upgrade (#679) Signed-off-by: Chen Jing <[email protected]> * Fix a typo everywhere (#680) Signed-off-by: Chen Jing <[email protected]> * Optimize the proposal (#683) Signed-off-by: Chen Jing <[email protected]> * Change the deployments to statefulSets for whoever has a PVC (#681) * Change the deployments to statefulSets for whoever has a PVC Signed-off-by: Chen Jing <[email protected]> * Fix issues Signed-off-by: Chen Jing <[email protected]> * Fix a mistake of the chart Signed-off-by: Chen Jing <[email protected]> * add service names for the yaml files of the sts Signed-off-by: Chen Jing <[email protected]> * set the default replicas for 2 for node manager, set default storage for pulsar Signed-off-by: Chen Jing <[email protected]> * fix typo Signed-off-by: Chen Jing <[email protected]> * add default replicas for hdfs datanode Signed-off-by: Chen Jing <[email protected]> * add default values for hdfs Signed-off-by: Chen Jing <[email protected]> * fix typo Signed-off-by: Chen Jing <[email protected]> * Part 1: Support getting the list of the scripts by the new/old versions (#686) * Support getting the list of the scripts by the versions Signed-off-by: Chen Jing <[email protected]> * fix issues Signed-off-by: Chen Jing <[email protected]> * address comments Signed-off-by: Chen Jing <[email protected]> * fix an error msg Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * chart name should be fate not FATE Signed-off-by: Chen Jing <[email protected]> * fix an issue Signed-off-by: Chen Jing <[email protected]> * Part 2: Implement the upgrade manager (#688) * add the docker file and the helm chart for fate upgrade manager Signed-off-by: Chen Jing <[email protected]> * add image pull policy for the template Signed-off-by: Chen Jing <[email protected]> * optimize the job name to make it include the versions Signed-off-by: Chen Jing <[email protected]> * make the default version a normal version string Signed-off-by: Chen Jing <[email protected]> * move the docker file stuff to fate builder Signed-off-by: Chen Jing <[email protected]> * Part 3: Fum shutdown flow (#690) * initial checkin Signed-off-by: Chen Jing <[email protected]> * update helm chart Signed-off-by: Chen Jing <[email protected]> * update helm chart Signed-off-by: Chen Jing <[email protected]> * update helm chart Signed-off-by: Chen Jing <[email protected]> * Change clusterrole to role Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * fix issues Signed-off-by: Chen Jing <[email protected]> * Kubefate support rolling upgrade (#691) * initial checkin Signed-off-by: Chen Jing <[email protected]> * initial checkin Signed-off-by: Chen Jing <[email protected]> * Implemetation, happy path test passed Signed-off-by: Chen Jing <[email protected]> * Fix self-review mistakes Signed-off-by: Chen Jing <[email protected]> * submit go mod and go sum Signed-off-by: Chen Jing <[email protected]> * fix UT Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * make the code beautiful (#694) Signed-off-by: Chen Jing <[email protected]> * Part 1 : fixed an issue which caused fate to not work with istio (#696) * fixed an issue which caused fate to not work with istio Signed-off-by: hang lv <[email protected]> * annotate rewriteAppHTTPProbers only if istio enabled Signed-off-by: hang lv <[email protected]> * Fix the issue found when try to uprade to v1.9.0 (#697) Signed-off-by: Chen Jing <[email protected]> * Fix the issue that we failed to stop upgrading from v1.7.1 (#698) Signed-off-by: Chen Jing <[email protected]> * fix time difference between host and container (#689) Signed-off-by: Zhang Hui <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Add the doc for upgrade (#700) * Add the doc for upgrade Signed-off-by: Chen Jing <[email protected]> * Polish the doc Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * clean up the build code in KubeFATE (#701) * clean up the build code in KubeFATE Signed-off-by: Chen Jing <[email protected]> * address comment Signed-off-by: Chen Jing <[email protected]> * Support TLS for eggroll: p2p mode (#703) Signed-off-by: Chen Jing <[email protected]> * [2/3][3/3] Support TLS for eggroll: exchange mode and docker-compose (#705) * Support TLS for eggroll: exchange mode Signed-off-by: Chen Jing <[email protected]> * fix comments Signed-off-by: Chen Jing <[email protected]> * set swagger default auth (#710) Signed-off-by: hang lv <[email protected]> * add tls support for gateways with istio (#709) * add tls support for gateways with istio Signed-off-by: hang lv <[email protected]> * add guide and fix credential name Signed-off-by: hang lv <[email protected]> * modified guid Signed-off-by: hang lv <[email protected]> * Update docs/Secure_Gateways_with_Istio Co-authored-by: Chen Jing <[email protected]> * Update Eggroll_with_TLS.md (#711) Fix some issue about the doc * Feat: add images select for docker-compose and chart (#706) * docker-compose supports selecting images according to configuration Signed-off-by: Chenlong Ma <[email protected]> * k8s support images select Signed-off-by: Chenlong Ma <[email protected]> * update docs Signed-off-by: Chenlong Ma <[email protected]> * update for comments Signed-off-by: Chenlong Ma <[email protected]> * fix serving-tag Signed-off-by: Chenlong Ma <[email protected]> * fix typo Signed-off-by: Chenlong Ma <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> * Create document of deploying FATE cluster with admin role in certain namespace (#718) * Add document on how to deploy and use kubefate in k8s mode when users only has admin roles in certain namespaces(#708) Signed-off-by: fwenqian <[email protected]> * Update some description in Deploy_FATE_Cluster_with_Admin_Role_in_Certain_Namespace.md Signed-off-by: fwenqian <[email protected]> * Update docs/Deploy_FATE_Cluster_with_Admin_Role_in_Certain_Namespace.md Signed-off-by: fwenqian <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Update error messages when listing namespaces or getting node ip fails with admin role in particular namespace. (#721) Update error messages when listing namespaces or getting node ip fails. Signed-off-by: fwenqian <[email protected]> * Feat: KubeFATE support FATE_Serving v2.1.6 (#722) * Feat: KubeFATE support FATE-Serving v2.1.6 Signed-off-by: Chenlong Ma <[email protected]> * Fix: Serving api return "serviceId is not bind model" after pod restarted Signed-off-by: Chenlong Ma <[email protected]> * fix volumes path Signed-off-by: Chenlong Ma <[email protected]> * fix: zookeeper cannot create directory '/bitnami/zookeeper/data': Permission denied Signed-off-by: Chenlong Ma <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> * Fix chart format (#720) * fix chart format to meet validation requirement Signed-off-by: hang lv <[email protected]> * add skippedkeys Signed-off-by: hang lv <[email protected]> Signed-off-by: hang lv <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Fix validate issue (#714) * error message if chart not found Signed-off-by: hang lv <[email protected]> * change pattern for validation Signed-off-by: hang lv <[email protected]> * check module&backend Signed-off-by: hang lv <[email protected]> * UT for modules&backend validation Signed-off-by: hang lv <[email protected]> Signed-off-by: hang lv <[email protected]> Co-authored-by: Chen Jing <[email protected]> * remove imageTag checks & leave it to default configured in values.yaml (#723) * remove imageTag checks & leave it to default configured in values.yaml Signed-off-by: fwenqian <[email protected]> * remove all imageTag related content Signed-off-by: fwenqian <[email protected]> Signed-off-by: fwenqian <[email protected]> Co-authored-by: Chen Jing <[email protected]> * Fix validation issue (#729) * bug fix Signed-off-by: hang lv <[email protected]> * log skippedError Signed-off-by: hang lv <[email protected]> * add more test and fix backend validation bug Signed-off-by: hang lv <[email protected]> Signed-off-by: hang lv <[email protected]> * Fix validate issue (#732) * yaml storage default setting corrected Signed-off-by: hang lv <[email protected]> * fix a bug which cause list obj returns no error Signed-off-by: hang lv <[email protected]> * allow empty value & add test Signed-off-by: hang lv <[email protected]> Signed-off-by: hang lv <[email protected]> * Support FATE v1.9.0 (#712) * Support FATE v1.9.0 Signed-off-by: Chen Jing <[email protected]> * Fix mistakes Signed-off-by: Chen Jing <[email protected]> * add a 1.9 config in the cm of python Signed-off-by: Chen Jing <[email protected]> * bump kubefate version Signed-off-by: Chen Jing <[email protected]> * support spark 3 Signed-off-by: Chen Jing <[email protected]> * change the port of probe of hadoop Signed-off-by: Chen Jing <[email protected]> * change the port of probe of datanode Signed-off-by: Chen Jing <[email protected]> * update pulsar version per Webank's request Signed-off-by: Chen Jing <[email protected]> * fix a bug of service_conf Signed-off-by: Chen Jing <[email protected]> * fix the bug of spark-local Signed-off-by: Chen Jing <[email protected]> * fix the bug of spark-local Signed-off-by: Chen Jing <[email protected]> * fix the bug of spark-local Signed-off-by: Chen Jing <[email protected]> * fix the bug of spark-local Signed-off-by: Chen Jing <[email protected]> * fix the docker_deploy file Signed-off-by: Chen Jing <[email protected]> * add 2 job configs for docker compose Signed-off-by: Chen Jing <[email protected]> * fix little issues Signed-off-by: Chen Jing <[email protected]> * address comments Signed-off-by: Chen Jing <[email protected]> * address comments Signed-off-by: Chen Jing <[email protected]> * typo Signed-off-by: Chen Jing <[email protected]> Signed-off-by: Chen Jing <[email protected]> * Fix issue 719 (#731) * allow users to customize pulsar env Signed-off-by: hang lv <[email protected]> * fix format error Signed-off-by: hang lv <[email protected]> * allow users to customize pulsar confs Signed-off-by: hang lv <[email protected]> * add/delete some conf in yaml Signed-off-by: hang lv <[email protected]> * update pulsa mappings to doc Signed-off-by: hang lv <[email protected]> * update doc Signed-off-by: hang lv <[email protected]> Signed-off-by: hang lv <[email protected]> * update docs for rm backend (#733) * update docs for rm backend Signed-off-by: Chenlong Ma <[email protected]> * Revert its Signed-off-by: Chenlong Ma <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> * Update the tutorial doc (#734) * Update the tutorial doc Signed-off-by: Chen Jing <[email protected]> * address comments Signed-off-by: Chen Jing <[email protected]> * modify ip address Signed-off-by: Chen Jing <[email protected]> Signed-off-by: Chen Jing <[email protected]> * fix a little typo (#737) Signed-off-by: Chen Jing <[email protected]> Signed-off-by: Chen Jing <[email protected]> Signed-off-by: stone-wlg <[email protected]> Signed-off-by: Chen Jing <[email protected]> Signed-off-by: magic-hya <[email protected]> Signed-off-by: Chenlong Ma <[email protected]> Signed-off-by: hang lv <[email protected]> Signed-off-by: fwenqian <[email protected]> Co-authored-by: stone.wlg <[email protected]> Co-authored-by: Sun Xia <[email protected]> Co-authored-by: Layne Peng <[email protected]> Co-authored-by: magic-hya <[email protected]> Co-authored-by: gxcuit <[email protected]> Co-authored-by: owlet42 <[email protected]> Co-authored-by: hang lv <[email protected]> Co-authored-by: Chenyang Gao <[email protected]> Co-authored-by: Zhang Hui <[email protected]> Co-authored-by: fwenqian <[email protected]>
- Loading branch information
11 people
authored
Aug 30, 2022
1 parent
d710a7d
commit a7d8839