add reasoning_visualization_experience.md doc

tinyms-ai · Dec 30, 2021 · e17c4c8 · e17c4c8
1 parent 13c9cf8
commit e17c4c8
Show file tree

Hide file tree

Showing 17 changed files with 123 additions and 7 deletions.
diff --git a/docs/en/source/_static/tinyms_visulization_origin.jpg b/docs/en/source/_static/tinyms_visulization_origin.jpg
diff --git a/docs/en/source/_static/tinyms_visulization_reasoning.jpg b/docs/en/source/_static/tinyms_visulization_reasoning.jpg
diff --git a/docs/en/source/_static/tinyms_web_index.jpg b/docs/en/source/_static/tinyms_web_index.jpg
diff --git a/docs/en/source/_static/tinyms_web_reasoning.jpg b/docs/en/source/_static/tinyms_web_reasoning.jpg
diff --git a/docs/en/source/design/concepts.md b/docs/en/source/design/concepts.md
@@ -268,7 +268,7 @@ detection_bbox_data = object_detection_predict(input, detector, is_training=Fals
 # 5.Draw the box for the input image and and view it using OpenCV.
 detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                               box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                              font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                              font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
 cv2.imshow('object detection image', cv2.resize(detection_image_np, (600, 1000)))
 cv2.waitKey(0)
 ```
@@ -301,7 +301,7 @@ while True:
     # 5.Draw the box for the input frame image and view it using OpenCV.
     detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                                   box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                                  font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                                  font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
     cv2.imshow('object detection camera', cv2.resize(detection_image_np, (800, 600)))
 
     if cv2.waitKey(25) & 0xFF == ord('q'):

diff --git a/docs/en/source/index.rst b/docs/en/source/index.rst
@@ -16,6 +16,7 @@ designed to providing quick-start guidelines for machine learning beginners.
    quickstart/overview
    quickstart/install
    quickstart/quickstart_in_one_minute.ipynb
+   quickstart/reasoning_visualization_experience
 
 .. toctree::
    :maxdepth: 2

diff --git a/docs/en/source/quickstart/reasoning_visualization_experience.md b/docs/en/source/quickstart/reasoning_visualization_experience.md
@@ -0,0 +1,57 @@
+# TinyMS Reasoning Visualization Experience
+
+Combined with OpenCV image vision library, TinyMS V0.3.0 focus visualization features. Through simple and intuitive image visualization, it helps users to understand the effect of model reasoning more quickly.
+
+For users who do not want to write code, TinyMS provides a visual interface of WEB UI. Users only need to upload the image to be inferred on the browser page to experience easily. Currently, it supports for `LeNet5`, `CycleGan` and `SSD300` models.
+
+### WEB UI reasoning visualization
+
+Users need to deploy the visual server first, details please see [TinyMS Nginx Verion Installation](https://tinyms.readthedocs.io/en/latest/quickstart/install.html). After the server is successfully deployed, the home page and reasoning effect page (taking `CycleGan` model as an example) presented by the browser are as follows:
+
+![Index Page](../_static/tinyms_web_index.jpg)
+
+![Predict Page](../_static/tinyms_web_reasoning.jpg)
+
+For users who want to run code, TinyMS provides model reasoning visualization module, which only needs `5` step code to experience quickly. Currently, it only supports `SSD300` object detection model.
+
+### Model reasoning visualization module application
+
+If users need to experience the model reasoning visualization module application for the first time, they can download code from [TinyMS Official Repo](https://github.com/tinyms-ai/tinyms), then do the following operations:
+
+* Static image detection
+
+Note: Please make sure your system has a visual desktop.
+
+```script
+# Download the TinyMS project
+git clone https://github.com/tinyms-ai/tinyms.git
+cd tinyms/tests/st/app/object_detection/
+# Run static image detection
+python opencv_image_app.py
+```
+
+The image to be detected and the image after inference are shown as follows:
+
+![Input Image](../_static/tinyms_visulization_origin.jpg)
+
+![Reasoning Image](../_static/tinyms_visulization_reasoning.jpg)
+
+* Real-time dynamic detection of video images collected by computer camera
+
+Note: In addition to ensuring that your system has a visual desktop, you also need to have a camera.
+
+If using the container to run code, the camera of the host must be mounted to the container. For example, the host is Ubuntu 18.04 with a camera device and a visual desktop. Perform from step 1 to the final.  If using the host server, perform from step 4 to the final.
+
+```script
+# 1.Install xServer on the host and set permissions
+apt install x11-xserver-utils
+# 2.Allow all users to access the display interface
+xhost +
+# 3.Run container
+docker run -it --rm --device=/dev/video0 -e DISPLAY=unix$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix tinyms/tinyms:0.3.0 /bin/bash
+# 4.Download the TinyMS project in the container
+git clone https://github.com/tinyms-ai/tinyms.git
+cd tinyms/tests/st/app/object_detection/
+# 5.Run dynamic video image detection collected by camera
+python opencv_camera_app.py
+```
diff --git a/docs/zh_CN/source/_static/tinyms_visulization_origin.jpg b/docs/zh_CN/source/_static/tinyms_visulization_origin.jpg
diff --git a/docs/zh_CN/source/_static/tinyms_visulization_reasoning.jpg b/docs/zh_CN/source/_static/tinyms_visulization_reasoning.jpg
diff --git a/docs/zh_CN/source/_static/tinyms_web_index.jpg b/docs/zh_CN/source/_static/tinyms_web_index.jpg
diff --git a/docs/zh_CN/source/_static/tinyms_web_reasoning.jpg b/docs/zh_CN/source/_static/tinyms_web_reasoning.jpg
diff --git a/docs/zh_CN/source/design/concepts.md b/docs/zh_CN/source/design/concepts.md
@@ -269,7 +269,7 @@ detection_bbox_data = object_detection_predict(input, detector, is_training=Fals
 # 5.调用OpenCV库为图像画检测框并显示检测图像检测效果图
 detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                               box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                              font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                              font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
 cv2.imshow('object detection image', cv2.resize(detection_image_np, (600, 1000)))
 cv2.waitKey(0)
 ```
@@ -302,12 +302,12 @@ while True:
     # 5.调用OpenCV库为每帧图像画检测框并显示动态视频图像检测效果图
     detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                                   box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                                  font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                                  font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
     cv2.imshow('object detection camera', cv2.resize(detection_image_np, (800, 600)))
 
     if cv2.waitKey(25) & 0xFF == ord('q'):
         break
 
 cap.release()
 cv2.destroyAllWindows()
-```
+```
diff --git a/docs/zh_CN/source/index.rst b/docs/zh_CN/source/index.rst
@@ -13,6 +13,7 @@ Welcome to TinyMS's documentation!
    quickstart/overview
    quickstart/install
    quickstart/quickstart_in_one_minute.ipynb
+   quickstart/reasoning_visualization_experience
 
 .. toctree::
    :maxdepth: 2

diff --git a/docs/zh_CN/source/quickstart/reasoning_visualization_experience.md b/docs/zh_CN/source/quickstart/reasoning_visualization_experience.md
@@ -0,0 +1,57 @@
+# TinyMS推理可视化体验
+
+结合OpenCV图像视觉库，TinyMS v0.3.0聚焦可视化特性。通过简单直观的图片可视化，帮助用户更快地了解模型推理的效果。
+
+针对不想编写代码的用户，TinyMS提供了WEB UI可视化界面，用户只需在浏览器页面上传待推理的图片即可轻松体验，当前提供了`LeNet5`、`CycleGan`和`SSD300`模型的支持。
+
+### WEB UI推理可视化
+
+用户需要先部署可视化服务器，详情请看[Nginx版本的TinyMS](https://tinyms.readthedocs.io/zh_CN/latest/quickstart/install.html)安装。服务器部署成功后，访问浏览器呈现的首页和推理效果页（以`CycleGan`模型为例）如下：
+
+![首页](../_static/tinyms_web_index.jpg)
+
+![推理页](../_static/tinyms_web_reasoning.jpg)
+
+针对想运行代码的用户，TinyMS提供了模型推理可视化模块，仅需`5`步骤代码即可快速体验，当前仅提供`SSD300`对象检测模型的支持。
+
+### 模型推理可视化模块应用
+
+如果您需要第一时间体验模型推理可视化模块应用，可下载[TinyMS官方仓项目](https://github.com/tinyms-ai/tinyms)代码，执行如下操作：
+
+* 静态图像检测
+
+注：请确保您的系统有可视化桌面
+
+```script
+# 下载tinyms项目
+git clone https://github.com/tinyms-ai/tinyms.git
+cd tinyms/tests/st/app/object_detection/
+# 运行静态图像检测
+python opencv_image_app.py
+```
+
+待检测图片和执行推理后的图片展示如下：
+
+![待检测图片](../_static/tinyms_visulization_origin.jpg)
+
+![推理效果图](../_static/tinyms_visulization_reasoning.jpg)
+
+* 摄像头采集的动态视频图像检测
+
+注：除确保您的系统有可视化桌面，还需带有摄像头。
+
+若使用容器访问，则容器需要挂载主机的摄像头，如: 宿主机是带有摄像头设备和可视化桌面的Ubuntu 18.04系统, 从步骤1开始执行；若直接在宿主机运行，则从步骤4开始执行：
+
+```script
+# 1.在宿主机内安装xserver，并设置权限
+apt install x11-xserver-utils
+# 2.允许所有用户访问显示接口
+xhost +
+# 3.运行容器
+docker run -it --rm --device=/dev/video0 -e DISPLAY=unix$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix tinyms/tinyms:0.3.0 /bin/bash
+# 4.在容器内下载tinyms项目
+git clone https://github.com/tinyms-ai/tinyms.git
+cd tinyms/tests/st/app/object_detection/
+# 5.运行摄像头采集的动态视频图像检测
+python opencv_camera_app.py
+```
diff --git a/tests/st/app/object_detection/opencv_camera_app.py b/tests/st/app/object_detection/opencv_camera_app.py
@@ -39,7 +39,7 @@
     # 5.Draw the box for the input frame image and view it using OpenCV.
     detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                                   box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                                  font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                                  font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
     cv2.imshow('object detection camera', cv2.resize(detection_image_np, (800, 600)))
 
     if cv2.waitKey(25) & 0xFF == ord('q'):

diff --git a/tests/st/app/object_detection/opencv_image_app.py b/tests/st/app/object_detection/opencv_image_app.py
@@ -53,7 +53,7 @@ def parse_args():
     # 5.Draw the box for the input image and view it using OpenCV.
     detection_image_np = visualize_boxes_on_image(image_np, detection_bbox_data, box_color=(0, 255, 0),
                                                   box_thickness=3, text_font=cv2.FONT_HERSHEY_PLAIN,
-                                                  font_scale=3, text_color=(0, 0, 255), font_size=3, show_scores=True)
+                                                  font_scale=2, text_color=(0, 0, 255), font_size=3, show_scores=True)
     cv2.imshow('object detection image', cv2.resize(detection_image_np, args_opt.window_size))
     cv2.waitKey(0)
 

diff --git a/tests/st/app/object_detection/pic/test.jpg b/tests/st/app/object_detection/pic/test.jpg