Merge branch 'develop' into ap-enhance-with-static-type

.github/workflows/build.yml

@@ -1,4 +1,4 @@
-name: myrobotlab ci/cd
+name: Java CI
 
 on: [push]
 
@@ -9,19 +9,11 @@ jobs:
     steps:
       - uses: actions/checkout@v3
 
-      - name: setup jdk 11
+      - name: Set up JDK 11
         uses: actions/setup-java@v3
 
         with:
           java-version: '11'
           distribution: 'adopt'
-      # - name: Test with Maven
-      #   run: mvn test -Dtest=!org.myrobotlab.opencv.OpenCVFilterDL4JTest
-      # - name: build with maven
-      #   run: mvn --batch-mode --update-snapshots -DskipTests package
-          run: mvn -DskipTests package
-      - name: build with maven
-        # run: mvn --version
-        # run: mvn -Dtest=Service* package
-        # run: mvn --batch-mode --update-snapshots package
-          run: echo "I wish I could test here"
+      - name: Build with Maven
+        run: mvn --batch-mode -Dtest=!**/OpenCV* test -X

.gitignore

@@ -1,9 +1,43 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Distribution / packaging
+dist/
+build/
+*.egg-info/
+*.egg
+
+# IDEs and editors
+.idea/
+.vscode/
+*.pyc
+
+# Local development settings
+.env
+.env.local
+.env.*.local
+
+# Logs and other generated files
+*.log
+*.out
+*.pid
+*.sock
+
+
 # TODO clean this up !
 /.project
 /.classpath
 /.settings
 /src/main/resources/resource/WebGui/react
 /src/main/resources/resource/Vertx/app
+
+# ignore all resources from seperate modular projects
+/src/main/resources/resource/ProgramAB
+/src/main/resources/resource/InMoov2
+/src/main/resources/resource/SpotMicro
+
 /data
 /resource
 /bin
@@ -82,3 +116,4 @@
 /lastRestart.py
 /.factorypath
 start.yml
+*.iml

.prettierrc

@@ -0,0 +1,4 @@
+{
+    "htmlWhitespaceSensitivity": "ignore",
+    "printWidth": 180
+}

README.md

@@ -86,7 +86,6 @@ If you want to compile and skip the tests, you can use the standard maven approa
 * [Title Caps for field names and elements](https://learn.microsoft.com/en-us/previous-versions/windows/desktop/bb246428(v=vs.85)?redirectedfrom=MSDN)
 * [No semi-colons for field names if labels exist](https://ux.stackexchange.com/questions/3611/should-label-and-field-be-separated-with-colon)
 
-
 ## Contributing
 
 All development is done on the `develop` branch.  To contribute code, the typical approach is to create an issue about the feature/bug you're working on.
@@ -99,3 +98,191 @@ When code is ready, submit a pull request to the develop branch!
 Enjoy the code review, address issues and concern in the code review
 Reviewer merges pull request to develop.
 Reviewer deletes branch.
+
+
+The following config should be useful to work directly on WebGui UI and
+InMoov2 UI if the repos are checked out at the same level
+```yml
+!!org.myrobotlab.service.config.WebGuiConfig
+autoStartBrowser: true
+enableMdns: false
+listeners: null
+peers: null
+port: 8888
+resources:
+  # these are the only two in usual runtime
+- ./resource/WebGui/app
+- ./resource
+  # the rest are useful when doing dev
+- ../InMoov2/resource/WebGui/app
+- ./src/main/resources/resource/WebGui/app
+- ./src/main/resources/resource/WebGui
+- ./src/main/resources/resource
+- ./src/main/resources
+type: WebGui
+```
+```yml
+!!org.myrobotlab.service.config.RuntimeConfig
+enableCli: true
+id: null
+listeners:
+locale: null
+logLevel: info
+peers: null
+registry:
+- runtime
+- security
+- webgui
+- python
+resource: src/main/resources/resource
+type: Runtime
+virtual: false
+```
+
+# Network Distributed Architecture
+
+## Websockets - Default Response for New Connection
+
+```mermaid
+sequenceDiagram
+    autonumber
+    box Process Id p1
+    participant runtime@p1   
+    end
+
+    box Process Id p2
+    participant webgui@p2
+    participant runtime@p2
+    end
+
+    Note right of runtime@p1: Client runtime@p1 opens a <br/>websocket to remote webgui
+
+    runtime@p1->>webgui@p2: Connect<br/>ws://localhost:8888/api/messages?user=root&pwd=pwd&session_id=2309adf3dlkdk&id=p1
+
+    Note left of webgui@p2: Remote webgui@p2 and runtime@p2 attempt to subscribe <br/> to the describe method of the runtime@p1
+    webgui@p2->>runtime@p1: addListener describe
+    Note left of webgui@p2: runtime@p2 sends a describe request to runtime@p1
+    webgui@p2->>runtime@p1: describe 
+    Note right of runtime@p1: runtime@p1 responds with a describe response
+    runtime@p1->>webgui@p2: onDescribe 
+    Note left of webgui@p2: Based on the results of the describe,<br/> more querying and subscriptions can be processed
+
+
+    %% opt Server Add Listener, Describe and Reserve
+    %% end
+
+
+```
+
+
+### Minimal Message API Definition
+
+```json
+{
+  "name": "runtime",
+  "method": "connect",
+  "data": [
+    "\"http://main.myrobotlab.org:8888\""
+  ],
+}
+
+```
+
+### Path API Definition
+```
+/{service-name}/{method-name}/{json-param1}/{json-param2}/{json-param3}...
+```
+The Path API definition is a simple way to define a RESTful API.  The path is parsed and the service name, method name, and parameters are extracted.  The parameters are json encoded and converted to the correct type when the method is invoked.  The response is returned as a JSON object. The REST and CLI both use this API definition.
+
+#### Examples
+```
+http://localhost:8888/runtime/getUptime
+http://localhost:8888/runtime/connect/"http://main.myrobotlab.org:8888"
+http://localhost:8888/arduino/connect/"COM3"
+```
+The exact same paths can be used in the CLI
+```
+/runtime/getUptime
+/runtime/connect/"http://main.myrobotlab.org:8888"
+/arduino/connect/"COM3"
+```
+
+
+### 1 Connection
+A connection with a websocket starts with an HTTP GET 
+```
+ws://localhost:8888/api/messages?user=root&pwd=pwd&session_id=2309adf3dlkdk&id=p1
+```
+### 2 Subscribe to Runtime Describe
+When a connection is established between two different myrobotlab instances p1 and p2,
+the following messages are sent.  The first message is adding a subscription to the runtime of the
+other process.  The subscription is for the function "describe".  The second message will be a describe request.
+```json
+{
+  "msgId": 1690145377501,
+  "name": "runtime",
+  "sender": "webgui@p2",
+  "sendingMethod": "",
+  "historyList": [],
+  "properties": null,
+  "status": null,
+  "encoding": "json",
+  "method": "addListener",
+  "data": [
+    "{\"topicMethod\":\"describe\",\"callbackName\":\"runtime@caring-hector\",\"callbackMethod\":\"onDescribe\",\"class\":\"org.myrobotlab.framework.MRLListener\"}"
+  ],
+  "class": "org.myrobotlab.framework.Message"
+}
+```
+### 3 Send Describe
+
+```json
+{
+  "msgId": 1690145377501,
+  "name": "runtime",
+  "sender": "webgui@p2",
+  "sendingMethod": "",
+  "historyList": [],
+  "properties": null,
+  "status": null,
+  "encoding": "json",
+  "method": "describe",
+  "data": [
+    "\"fill-uuid\"",
+    "{\"id\":\"caring-hector\",\"uuid\":\"383b4070-2848-4c3d-85f4-e7f6e081d18e\",\"platform\":{\"os\":\"linux\",\"arch\":\"x86\",\"osBitness\":64,\"jvmBitness\":64,\"lang\":\"java\",\"vmName\":\"OpenJDK 64-Bit Server VM\",\"vmVersion\":\"11\",\"mrlVersion\":\"unknownVersion\",\"isVirtual\":false,\"id\":\"caring-hector\",\"branch\":\"develop\",\"pid\":\"1500044\",\"hostname\":\"t14-gperry\",\"commit\":\"55d0163663825dd0aaa10568bc01e035c7f21532\",\"build\":null,\"motd\":\"resistance is futile, we have cookies and robots ...\",\"startTime\":1690135873670,\"manifest\":{\"git.branch\":\"develop\",\"git.build.host\":\"t14-gperry\",\"git.build.time\":\"2023-07-23T10:38:46-0700\",\"git.build.user.email\":\"[email protected]\",\"git.build.user.name\":\"grog\",\"git.build.version\":\"0.0.1-SNAPSHOT\",\"git.closest.tag.commit.count\":\"13447\",\"git.closest.tag.name\":\"1.0.119\",\"git.commit.author.time\":\"2023-07-22T20:15:51-0700\",\"git.commit.committer.time\":\"2023-07-22T20:15:51-0700\",\"git.commit.id\":\"55d0163663825dd0aaa10568bc01e035c7f21532\",\"git.commit.id.abbrev\":\"55d0163\",\"git.commit.id.describe\":\"1.0.119-13447-g55d0163\",\"git.commit.id.describe-short\":\"1.0.119-13447\",\"git.commit.message.full\":\"Cron enhanced 2 (#1318)\\n\\n* Improved Cron and Cron history\\r\\n\\r\\n* forgot one\\r\\n\\r\\n* Teamwork fix of Hd44780\\r\\n\\r\\n* updated from review\",\"git.commit.message.short\":\"Cron enhanced 2 (#1318)\",\"git.commit.time\":\"2023-07-22T20:15:51-0700\",\"git.commit.user.email\":\"[email protected]\",\"git.commit.user.name\":\"GroG\",\"git.dirty\":\"false\",\"git.local.branch.ahead\":\"0\",\"git.local.branch.behind\":\"0\",\"git.remote.origin.url\":\"[email protected]:MyRobotLab/myrobotlab.git\",\"git.tags\":\"1.1.1194\",\"git.total.commit.count\":\"14104\"},\"shortCommit\":\"55d0163\",\"class\":\"org.myrobotlab.framework.Platform\"},\"class\":\"org.myrobotlab.framework.DescribeQuery\"}"
+  ],
+  "class": "org.myrobotlab.framework.Message"
+}
+
+```
+
+The describe message sends instance and platform information in return, the return data from describe, will return with an onDescribe message that contains all the service information from the remote (p2) process
+
+### 4 Process onDescribe Response
+```json
+{
+   "msgId":1692478237121,
+   "name":"runtime@webgui-client",
+   "sender":"runtime@unhealthy-giddy",
+   "sendingMethod":"describe",
+   "historyList":[
+      "unhealthy-giddy"
+   ],
+   "properties":null,
+   "status":null,
+   "encoding":"json",
+   "method":"onDescribe",
+   "data":[
+      "{\"id\":\"unhealthy-giddy\",\"uuid\":null,\"request\":null,\"platform\":{\"os\":\"linux\",\"arch\":\"x86\",\"osBitness\":64,\"jvmBitness\":64,\"lang\":\"java\",\"vmName\":\"OpenJDK 64-Bit Server VM\",\"vmVersion\":\"11\",\"mrlVersion\":\"unknownVersion\",\"isVirtual\":false,\"id\":\"unhealthy-giddy\",\"branch\":\"grog\",\"pid\":\"2462761\",\"hostname\":\"t14-gperry\",\"commit\":\"5610a6602bef704a84160d7d067af3b417be1998\",\"build\":null,\"motd\":\"resistance is futile, we have cookies and robots ...\",\"startTime\":1692478154179,\"manifest\":{\"git.branch\":\"grog\",\"git.build.host\":\"t14-gperry\",\"git.build.time\":\"2023-08-19T09:36:36-0700\",\"git.build.user.email\":\"[email protected]\",\"git.build.user.name\":\"grog\",\"git.build.version\":\"0.0.1-SNAPSHOT\",\"git.closest.tag.commit.count\":\"13778\",\"git.closest.tag.name\":\"1.0.119\",\"git.commit.author.time\":\"2023-08-19T09:21:48-0700\",\"git.commit.committer.time\":\"2023-08-19T09:21:48-0700\",\"git.commit.\"class\":\"org.myrobotlab.framework.DescribeResults\"} ....   WAY WAY TOO MUCH DATA !!!!  ..."
+   ],
+   "class":"org.myrobotlab.framework.Message"
+}
+```
+
+The response back will include serialized services which should be refactor to be more minimal, and more describe parameters which can return interfaces or methods
+Services returned from the Describe request have been registered.  By default they are only the currently "local" registered services.  
+
+The response should be refactored so that the material returned is related to criterial requested. 
+For example, if only a list of services and their names are needed, that is all that is returned.  If a specific service's interfaces are requested, then that is only returned.
+Fill-UUID should be refactored out.
+

doc/service-life-cycle.md

@@ -0,0 +1,52 @@
+# Service Life Cycle
+```mermaid
+stateDiagram
+
+    start: start(name, type)
+    load: load(name, type)
+    loadService: loadService(plan, name, type, true, 0)
+
+    [*] --> start
+
+    start --> load
+    load --> loadService
+    loadService --> getDefault
+    getDefault --> readServiceConfig
+    readServiceConfig --> loadService
+    loadService --> createServicesFromPlan
+    createServicesFromPlan --> createService
+    createService --> setConfig
+    setConfig --> apply
+    apply --> startService
+    startService --> stopService
+    stopService --> releaseService
+    releaseService --> release
+    release --> [*]
+
+    [*] --> create
+    create --> load
+
+```
+
+### start(name, type)
+Creates and starts a service with the given name and type
+
+### load
+Starts loading the hierarchy of configuration
+FIXME - this should not be the memory plan, but should exist on the filesystem
+Default config is used if no config found ~~in memory~~ on filesystem
+
+### loadService
+Recursively loads a service config into a plan
+
+### createServicesFromPlan(plan, createdServices, name)
+Loops through all "loaded" services in the plan and creates them all
+
+### createService
+Instantiates instance of service
+
+### setConfig
+Sets the config of the service
+
+### apply
+Applies the config to the service

myrobotlab.sh

@@ -2,8 +2,7 @@
 
 REPO_FILE=libraries/repo.json
 
-# fancy way to get real cwd ?
-APPDIR="$(dirname -- "$(readlink -f -- "${0}")" )"
+APPDIR="$(dirname -- ${0})"
 
 echo APPDIR=${APPDIR}