Add UAI Service API & Update UAI Service Tool

1.Add uai_sdk/uai/api folder which includes python api for UAI Service Platform 2.Merge different deploy tools into uai_sdk/uai_tools/uai_tool.py 3.Add "uai_tool.py packdocker" command to help user pack local program into UCloud Docker Image. API docs: http://docs.ucloud.cn/ai/uai-service/use/oplist/packdata_docker
ucloud · Oct 19, 2017 · f8d024e · f8d024e
1 parent ff3542d
commit f8d024e
Show file tree

Hide file tree

Showing 98 changed files with 3,238 additions and 439 deletions.
diff --git a/README.md b/README.md
@@ -1,22 +1,20 @@
 # uai-sdk
-UCloud AI service SDK
-This SDK include basic APIs to run AI inference services on a http server.
-It currently only support tensorflow, mxnet, keras, caffe, we will continue to add more platform into it.
+## What is UCloud AI SDK
+
+1. This SDK include **pack tool** to pack local AI inference program into UCloud web service image which could run on UCloud **UAI Service** Platform.
+2. This SDK include whole **APIs** which could interactive with UCloud **UAI Service** Platform.
+3. This SDK include **pack tool** to help pack local AI trainning program into UCloud Docker image which could run on UCloud **UAI Train** Platform.
+4.  **APIs** for **UAI Train**  is coming.
 
-UCloud AI train SDK
-This SDK include tools for packing AI train codes into docker
-It currently only support tensorflow, we will continue to add more platform into it.
 
-## Supported AI Platform
-UaiService
-Tensorflow (0.11.0 tested)
-Tensorflow（1.1.0 tested）
-MXNet(0.9.5 tested)
-Keras(1.2.0 tested)
-Caffe(1.0.0 tested)
 
-UaiTrain
-Tensorflow（1.1.0 tested）
+## Supported AI Platform
+- Tensorflow (0.11.0 tested)
+- Tensorflow（1.1.0 tested）
+- Tensorflow（1.2.0 tested）
+- MXNet(0.9.5 tested)
+- Keras(1.2.0 tested)
+- Caffe(1.0.0 tested)
 
 ## How to install
 1. Install your deep learning python package, such as Tensorflow, MXNet, Keras, Caffe (tested version preferred)
@@ -37,26 +35,9 @@ Tensorflow（1.1.0 tested）
 
 5. Install all other lib your inference code required
 
-## How to setup your own http service
-1. Write your inference code according to uai API (Please refer to ./uai/README.md
-2. Put your code into ./http_server/
-3. Now run your http service, the default port is 8080
-
-## UCloud Http Server for UAI
-https://github.com/ucloud/uai-sdk-httpserver
-
-## How to interactive with Ucloud api
-1. Choose your deep learning python package, such as Tensorflow, MXNet, Keras, Caffe (tested version preferred)
-2. Find corresponding tool under uai_tools folder
-3. Use xx_deploy.py, i.e. for tensorflow
-        $ python tf_deploy.py --help
-   See help docs: https://docs.ucloud.cn/ai/uai-service/index
-4. Notes: ai_arch, language, os, os_deps，pip files should be under the same path as xx_deploy.py
-
-
-## How to pack AI train code
-1. Find corresponding tool under uaitrain_tools folder
-2. Use xx_deploy.py, i.e. for tensorflow
-        $ python tf_deploy.py --help
-   See help docs: https://docs.ucloud.cn/ai/uai-train
-4. Notes: when use pack tool, please ensure all your code is under the same path where the pack tool located
+
+## How to interactive with UCloud UAI Platform
+### UAI Service Docs
+https://docs.ucloud.cn/ai/uai-service/use
+### UAI Train Docs
+https://docs.ucloud.cn/ai/uai-train/use
diff --git a/setup.py b/setup.py
@@ -10,8 +10,7 @@
     setup = setuptools.setup
     find_packages = setuptools.find_packages
 except ImportError:
-    setuptools = None
-    from distutils.core import setup
+    raise RuntimeError("Please install setuptools")
 
 packages = find_packages()
 

diff --git a/uai/__init__.py b/uai/__init__.py
@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
-  
+
 """
 UCloud ai SDK for python
 """
-           
-__version__ = '1.0.0'
+
+__version__ = '1.0.0'
diff --git a/uai/api/__init__.py b/uai/api/__init__.py
diff --git a/uai/api/base_api.py b/uai/api/base_api.py
@@ -0,0 +1,71 @@
+import requests
+import json
+from uai.utils.utils import _verfy_ac
+from uai.utils.logger import uai_logger
+from uai.utils.retcode_checker import *
+
+class BaseUaiServiceApiOp(object):
+    """ The Base api for uai Service
+    """
+    UCLOUD_API_URL = 'http://api.ucloud.cn'
+    PARAMS_DEFAULT_REGION = "cn-bj2"
+    PARAMS_DEFAULT_ZONE = "cn-bj2-04"
+
+    # UCLOUD_API_URL = 'http://api.pre.ucloudadmin.com'
+    # PARAMS_DEFAULT_REGION = "pre"
+    # PARAMS_DEFAULT_ZONE = "pre"
+
+    PARAMS_DEFAULT_BUSINESSGROUP = "Default"
+    PACKAGE_TYPE = {'os': 'OS', 'language': 'Python', 'ai_arch': 'AIFrame', 'os_deps': 'AptGet', 'pip': 'Pip'}
+
+    def __init__(self, action, public_key, private_key, project_id='', region='', zone=''):
+        self.action = action
+        self.public_key = public_key
+        self.private_key = private_key
+        self.project_id = project_id
+        self.region = self.PARAMS_DEFAULT_REGION if region == '' else region
+        self.zone = self.PARAMS_DEFAULT_ZONE if zone == '' else zone
+
+        self.cmd_params = {}
+        self.cmd_params['Action'] = self.action
+        self.cmd_params['PublicKey'] = self.public_key
+        self.cmd_params['Region'] = self.region
+        self.cmd_params['Zone'] = self.zone
+        if self.project_id != '':
+            self.cmd_params['ProjectId'] = self.project_id
+        self.cmd_url = self.UCLOUD_API_URL
+        # add other params in subclasses#
+
+
+    def _check_args(self, params):
+        #add implements in subclasses#
+        return
+
+    def _cmd_common_request(self):
+        if 'Signature' in self.cmd_params:
+            self.cmd_params.pop('Signature')
+        self.cmd_params['Signature'] = _verfy_ac(self.private_key,
+                                                 self.cmd_params)
+        uai_logger.info("Call http request: {0} ".format(get_request(self.cmd_url, params=self.cmd_params)))
+        r = requests.get(self.cmd_url, params=self.cmd_params)
+        print(r.text)
+        self.rsp = json.loads(r.text, encoding="utf-8")
+        if self.rsp["RetCode"] != 0:
+            uai_logger.error("{0} Fail: [{1}]{2}".format(self.cmd_params["Action"], self.rsp["RetCode"],
+                                                         self.rsp["Message"].encode('utf-8')))
+            return False
+        else:
+            del self.rsp['Action']
+            uai_logger.info("{0} Success: {1}".format(self.cmd_params["Action"], get_response(self.rsp, 0)))
+            return True
+        # add other operations in subclasses#
+
+
+
+    def call_api(self):
+        if self._check_args(self.cmd_params) == False:
+            return False, None
+
+        succ = self._cmd_common_request()
+        return succ, self.rsp
+
diff --git a/uai/api/check_uai_base_img_exist.py b/uai/api/check_uai_base_img_exist.py
@@ -0,0 +1,37 @@
+from uai.api.base_api import BaseUaiServiceApiOp
+
+
+class CheckUAIBaseImgExistOp(BaseUaiServiceApiOp):
+
+    ACTION_NAME = "CheckUAIBaseImgExist"
+
+    def __init__(self, public_key, private_key, os_version, python_version, ai_frame_version, os_deps='', pip='', project_id='', region='', zone=''):
+        super(CheckUAIBaseImgExistOp, self).__init__(self.ACTION_NAME, public_key, private_key, project_id, region, zone)
+        self.cmd_params['Region'] = region if region != '' else super(CheckUAIBaseImgExistOp, self).PARAMS_DEFAULT_REGION
+        self.cmd_params['OSVersion'] = os_version
+        self.cmd_params['PythonVersion'] = python_version
+        self.cmd_params['AIFrameVersion'] = ai_frame_version
+
+        self.cmd_params['AptGetPKGID.0'] = ''
+        self.cmd_params['PipPKGID.0'] = ''
+        # add other params in subclasses#
+
+    def _check_args(self, params):
+        if params['OSVersion'] == '':
+            return False
+        if params['PythonVersion'] == '':
+            return False
+        if params['AIFrameVersion'] == '':
+            return False
+        return True
+
+    def call_api(self):
+        succ, self.rsp = super(CheckUAIBaseImgExistOp, self).call_api()
+        return succ, self.rsp
+
+
+
+
+
+
+
diff --git a/uai/api/check_uai_deploy_progress.py b/uai/api/check_uai_deploy_progress.py
@@ -0,0 +1,31 @@
+from uai.api.base_api import BaseUaiServiceApiOp
+
+
+class CheckUAIDeployProgressOp(BaseUaiServiceApiOp):
+
+    ACTION_NAME = "CheckUAIDeployProgress"
+
+    def __init__(self, public_key, private_key, service_id, srv_version, project_id='', region='', zone=''):
+        super(CheckUAIDeployProgressOp, self).__init__(self.ACTION_NAME, public_key, private_key, project_id, region, zone)
+        self.cmd_params['Region'] = region if region != '' else super(CheckUAIDeployProgressOp, self).PARAMS_DEFAULT_REGION
+        self.cmd_params['ServiceID'] = service_id
+        self.cmd_params['SrvVersion'] = srv_version
+        # add other params in subclasses#
+
+    def _check_args(self, params):
+        if params['ServiceID'] == '':
+            return False
+        if params['SrvVersion'] == '':
+            return False
+        return True
+
+    def call_api(self):
+        succ, self.rsp = super(CheckUAIDeployProgressOp, self).call_api()
+        return succ, self.rsp
+
+
+
+
+
+
+
diff --git a/uai/api/create_uai_service.py b/uai/api/create_uai_service.py
@@ -0,0 +1,45 @@
+from uai.api.base_api import BaseUaiServiceApiOp
+
+
+class CreateUAIServiceOp(BaseUaiServiceApiOp):
+
+    ACTION_NAME = "CreateUAIService"
+
+    def __init__(self, public_key, private_key, srv_name, cpu, memory, business_group='', project_id='', region='', zone=''):
+        super(CreateUAIServiceOp, self).__init__(self.ACTION_NAME, public_key, private_key, project_id, region, zone)
+        self.cmd_params['Region'] = region if region != '' else super(CreateUAIServiceOp, self).PARAMS_DEFAULT_REGION
+        self.cmd_params['SrvName'] = srv_name
+        self.cmd_params['CPU'] = cpu
+        self.cmd_params['Memory'] = memory
+
+        self.cmd_params['BusinessGroup'] = business_group if business_group != '' else self.PARAMS_DEFAULT_BUSINESSGROUP
+        # add other params in subclasses#
+
+    def _check_args(self, params):
+
+        if params['SrvName'] == '':
+            return False
+        if 'CPU' in params and params['CPU'] != '':
+            if int(params['CPU']) < 1 or int(params["CPU"]) > 8:
+                print('cpu should be int between 1 to 8, but now:{0}'.format(params['CPU']))
+                return False
+        if 'Memory' in params and params['Memory'] != '':
+            if int(params['Memory']) < 1 or int(params["Memory"]) > 8:
+                print ('memory should be int between 1 to 8, but now:{0}'.format(params['Memory']))
+                return False
+        if params['CPU'] != params['Memory']:
+            print ('the value of cpu should be equal to memory , but now:cpu is {0}, memory is {1}'.
+                   format(params['CPU'], params['Memory']))
+            return False
+
+        return True
+
+    def call_api(self):
+        succ, self.rsp = super(CreateUAIServiceOp, self).call_api()
+        return succ, self.rsp
+
+
+
+
+
+
diff --git a/uai/api/delete_uai_service.py b/uai/api/delete_uai_service.py
@@ -0,0 +1,29 @@
+from uai.api.base_api import BaseUaiServiceApiOp
+
+
+class DeleteUAIServiceOp(BaseUaiServiceApiOp):
+
+    ACTION_NAME = "DeleteUAIService"
+
+    def __init__(self, public_key, private_key, service_id, srv_paas_id='', srv_version='', project_id='', region='', zone=''):
+        super(DeleteUAIServiceOp, self).__init__(self.ACTION_NAME, public_key, private_key, project_id, region, zone)
+        self.cmd_params['Region'] = region if region != '' else super(DeleteUAIServiceOp, self).PARAMS_DEFAULT_REGION
+        self.cmd_params['ServiceID'] = service_id
+
+        self.cmd_params['SrvPaasID'] = srv_paas_id
+        self.cmd_params['SrvVersion'] = srv_version
+
+    def _check_args(self, params):
+        if params['ServiceID'] == '':
+            return False
+        return True
+
+    def call_api(self):
+        succ, self.rsp = super(DeleteUAIServiceOp, self).call_api()
+        return succ, self.rsp
+
+
+
+
+
+
diff --git a/uai/api/deploy_uai_service.py b/uai/api/deploy_uai_service.py
@@ -0,0 +1,62 @@
+from uai.api.base_api import BaseUaiServiceApiOp
+
+class DeployUAIService(BaseUaiServiceApiOp):
+
+    ACTION_NAME = "DeployUAIService"
+
+    def __init__(self, public_key, private_key, service_id, os_version, python_v, ai_frame_v, ufile_url,
+                 apt_list='', pip_list='', version_info='', deploy_weight='', project_id='', region='', zone=''):
+        super(DeployUAIService, self).__init__(self.ACTION_NAME, public_key, private_key, project_id, region, zone)
+        self.cmd_params['Region'] = region if region != '' else super(DeployUAIService, self).PARAMS_DEFAULT_REGION
+        self.cmd_params['ServiceID'] = service_id
+        self.cmd_params['OSVersion'] = os_version
+        self.cmd_params['PythonVersion'] = python_v
+        self.cmd_params['AIFrameVersion'] = ai_frame_v
+        self.cmd_params['UfileBucket'], self.cmd_params['UfileName'] = self.parse_ufile_url(
+            ufile_url)
+        self.cmd_params['UfileURL'] = ufile_url
+
+        self.cmd_params['DeployWeight'] = deploy_weight
+        self.cmd_params['SrvVerMemo'] = version_info
+
+        if len(apt_list) != 0:
+            i = 0
+            for aptgetid in apt_list:
+                self.cmd_params['AptGetPKGID.' + str(i) ] = aptgetid
+                i = i + 1
+        else:
+            self.cmd_params['AptGetPKGID.0'] = ''
+
+        if len(pip_list):
+            i = 0
+            for pipid in pip_list:
+                self.cmd_params['PipPKGID.' + str(i) ] = pipid
+                i = i + 1
+        else:
+            self.cmd_params['PipPKGID.0'] = ''
+
+    def parse_ufile_url(self, url):
+        bucket = url[url.find('://') + 3: url.find('.')]
+        parturl = url.split('.ufileos.com/')
+        if len(parturl) > 1:
+            region = parturl[0][parturl[0].find('.') + 1: ]
+        if url.find('?') > 0:
+            filename = url[url.rfind('/') + 1: url.find('?')]
+        else:
+            filename = url[url.rfind('/') + 1:]
+
+        return bucket, filename
+
+    def _check_args(self, params):
+        if params['ServiceID'] == '':
+            return False
+
+        if 'DeployWeight' in params and params['DeployWeight'] != '':
+            if int(params['DeployWeight']) < 1 or int(params["DeployWeight"]) > 100:
+                print ('deploy_weight should be int between 1 to 100, but now:{0}'.format(params['DeployWeight']))
+                return False
+        return True
+
+    def call_api(self):
+        succ, rsp = super(DeployUAIService, self).call_api()
+        return succ, rsp