LiteAgent core Dart Server

English · 中文

LLM AI Agent multi session HTTP/WebSocket service

Feature

Support pure text agent without JSON Spec.
Support OpenAPI/OpenRPC/OpenModbus/OpenTool JSON Spec.
Support LLM Function calling to HTTP API/json-rpc 2.0 over HTTP/Modbus and more custom tools.
HTTP Server wrapper Lite Agent core Dart
Base on Lite Agent Core AgentService(DTO included), add Controller、Router, wrapper to HTTP/WS API.

Usage

1. Prepare

Some OpenSpec json file, according to /example/json/open*/*.json, which is callable.
Run your tool server, which is described in json file.

Add .env file in the example folder, and add below content in the .env file：

baseUrl = https://xxx.xxx.com         # LLM API BaseURL
apiKey = sk-xxxxxxxxxxxxxxxxxxxx      # LLM API ApiKey

2. Develop run server

debug or run mode run /bin/server.dart file main()

3.1 HTTP API

session control command, include：
- /version：get version number, to confirm server running
- /init: initial new session, server return session id
- /history: get session messages not be cleared
- /stop: stop session, when current message done, will not run next message
- /clear: clear session context messages, and close websocket connection

BaseURL

http://127.0.0.1:9527/api

[GET] /version

Feature：get version number, to confirm server running
Request params: empty
Response body sample：
```
{
    "version": "0.3.0"
}
```

[POST] /init

Feature: initial new session agent

Request body：

LLM config: baseUrl, apiKey, model
System Prompt: Agent character, ToDo/NotToDo description
Tools Description: (Optional) openapi, openrpc, openmodbus Spec. According to third APIs in Spec to set apiKey or not
Session List: (Optional) for multi agents supported. Init other agents, and add their sessionIds in this field
Timeout：3600 seconds in default. When agent stopped, massages context will be clear
Sample:

{
  "llmConfig": {
      "baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
      "apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
      "model": "<LLM API model name, e.g. gpt-3.5-turbo. And temperature、maxTokens、topP can be changed below >",
      "temperature": 0,
      "maxTokens": 4096,
      "topP": 1
  },
  "systemPrompt": "<System Prompt. LLM character, capabilities, need to help user fixed what problems>",
  "openSpecList": [
      {
          "openSpec": "<(Optional) tool spec json string, support openapi、openmodbus、openrpc>",
          "apiKey": {
              "type": "<basic or bearer>",
              "apiKey": "<Third APIs apiKey>"
          },
          "protocol": "Support openapi, openmodbus, jsonrpcHttp"
      },
      {
          "openSpec": "<(Optional) Another spec json string, can be another protocol>",
          "protocol": "Support openapi, openmodbus, jsonrpcHttp"
      }
  ],
  "sessionList": [
      {
          "id": "<Sub Agent sessionId 1>"
      },
      {
          "id": "<Sub Agent sessionId 2>"
      }
  ],
  "timeoutSeconds": 3600
}

Response body：
- sessionId, will be used as session websocket subscribe, stop and clear operations.
- Sample:
```
{
    "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
}
```

[GET] /history?id=<sessionId>

Response body:

agent messages context as list
Sample:

[
{
  "sessionId": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30",
  "from": "system | user | agent | llm | tool",
  "to": "user | agent | llm | tool | client",
  "type": "text | imageUrl | functionCallList | toolReturn | contentList",
  "message": "<need to parse according type>",
  "completions": {
      "tokenUsage": {
        "promptTokens": 100,
        "completionTokens": 522,
        "totalTokens": 622
      },
      "id": "chatcmpl-9bgYkOjpdtLV0o0JugSmnNzGrRFMG",
      "model": "gpt-3.5-turbo"
   },
  "createTime": "2023-06-18T15:45:30.000+0800"
}
]

[GET] /stop?id=<sessionId>

Response body:
- sessionId, to confirm the operation of the session
- Sample:
```
{
    "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
}
```

[GET] /clear?id=<sessionId>

Response body:
- sessionId, to confirm the operation of the session
- Sample:
```
{
    "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
}
```

3.2 WebSocket API

Send and subscribe session AgentMessage

Endpoint

ws://127.0.0.1:9527/api/chat?id=<sessionId>

3.2.1 alive

client(ping) -> server: send "ping" to server
client <- server(pong): respond "pong" to client

3.2.2 client send UserMessageDto List to server

client([UserMessageDto]) -> server ：Wrap and send server
Sample:

[
  {
    "type": "text",
    "message": "Get some tool status"
  }
]

{
  "taskId": "Optional. For identify which task AgentMessage from. If NULL, server will create one.",
  "contentList": [
    {
      "type": "text",
      "message": "Get some tool status"
    }
  ]
}

3.2.3 server feedback AgentMessage to client

client <- server(AgentMessage) ：server will keep sending AgentMessage to client
Sample:

{
  "sessionId": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30",
  "taskId": "0b127f1d-4667-4a52-bbcb-0b636f9a471a",
  "from": "system | user | agent | llm | tool",
  "to": "user | agent | llm | tool | client",
  "type": "text | imageUrl | functionCallList | toolReturn | contentList",
  "message": "<need to parse according type>",
  "completions": {
    "tokenUsage": {
      "promptTokens": 100,
      "completionTokens": 522,
      "totalTokens": 622
    },
    "id": "chatcmpl-9bgYkOjpdtLV0o0JugSmnNzGrRFMG",
    "model": "gpt-3.5-turbo"
  },
  "createTime": "2023-06-18T15:45:30.000+0800"
}

According type to parse message

text、imageUrl：
- String
- Sample: "Tool result: PASS"

functionCallList：

Struct:

[
    {
        "id":"<LLM respond id in function call>",
        "name":"<function name>",
        "parameters": "<LLM respond parameters in map>"
    }
]

Sample:

[
  {
    "id":"call_z5FK2dAfU8TXzn61IJXzRl5I",
    "name":"SomeFunction",
    "parameters": {
      "operation":"result"
    }
  }
]

toolReturn：

Struct:

{
  "id":"<LLM respond id in function call>",
  "result": "<JSON Map, different tools in defferent result>"
}

Sample:

{
  "id":"call_z5FK2dAfU8TXzn61IJXzRl5I",
  "result": {
    "statusCode":200,
    "body":"{\"code\":200,\"message\":\"PASS\"}"
  }
}

contentList:

Struct:

[
  {
    "type":"text | imageUrl",
    "message":"String"
  }
]

Sample:

[
  {
    "type":"text",
    "message":"What’s in this image?"
  },
  {
    "type":"imageUrl",
    "message":"https://www.xxx.com/xxx.jpg"
  }
]

When to=Client, message in below status:
- "[TASK_START]"：agent receive user messages, and ready to run task
- "[TOOLS_START]": ready to call Tools
- "[TOOLS_DONE]": Tools return finished
- "[TASK_STOP]"：agent receive stop or clear command, stop task
- "[TASK_DONE]"：agent run task finished

3.3 Typical Interaction

[/init request] {llmConfig: ..., systemPrompt:..., openSpecList: [...]}
[/init response SessionId] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a}
[After /chat connect ws, send userTaskDto] {taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a, contentList: [{type: text, message: Get some tool status}]}
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔗CLIENT: [text] [TASK_START]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 👤USER -> 🤖AGENT: [text] Get some tool status
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 💡LLM: [text] Get some tool status
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 💡LLM -> 🤖AGENT: [functionCallList] [{"id":"call_73xLVZDe70QgLHsURgY5BNT0","name":"SomeFunction","parameters":{"operation":"result"}}]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔧TOOL: [functionCallList] [{"id":"call_73xLVZDe70QgLHsURgY5BNT0","name":"SomeFunction","parameters":{"operation":"result"}}]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔗CLIENT: [text] [TOOLS_START]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🔧TOOL -> 🤖AGENT: [toolReturn] {"id":"call_73xLVZDe70QgLHsURgY5BNT0","result":{"statusCode":200,"body":"{\"code\":200,\"message\":\"FAIL\"}"}}
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 💡LLM: [toolReturn] {"id":"call_73xLVZDe70QgLHsURgY5BNT0","result":{"statusCode":200,"body":"{\"code\":200,\"message\":\"FAIL\"}"}}
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🔧TOOL -> 🤖AGENT: [text] [TOOLS_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔗CLIENT: [text] [TOOLS_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 💡LLM -> 🤖AGENT: [text] Tool status: FAIL.
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 👤USER: [text] Tool status: FAIL.
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔗CLIENT: [text] [TASK_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842, taskId: 0b127f1d-4667-4a52-bbcb-0b636f9a471a# 🤖AGENT -> 🔗CLIENT: [text] [TASK_STOP]
[/stop request] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842}
[/clear request] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842}
[ws close] WebSocket connection closed

Build and Run

Build in shell script:

dart compile exe bin/server.dart -o build/lite_agent_core_dart_server

Then the lite_agent_core_dart_server file will be in build folder
Copy config(within config.json) folder to lite_agent_core_dart_server same folder
Run in shell script:
```
./lite_agent_core_dart_server
```

Terminal will show:

INFO: 2024-06-24 14:48:05.862057: PID 34567: [HTTP] Start Server - http://0.0.0.0:9527/api

After server running, will create log folder and agent.log file in the folder, to record server running logs.

Docker

Build image, cdto project root directory, then run:
```
 docker build -t lite_agent_core_dart_server .
```

Run container:

docker run -d -p 9527:9527 lite_agent_core_dart_server

Or mount config and log folder to update config and get log info

docker run -d -p 9527:9527 -v ./log:/app/log -v ./config:/app/config  lite_agent_core_dart_server

Name		Name	Last commit message	Last commit date
Latest commit History 42 Commits
bin		bin
config		config
example		example
lib/src		lib/src
.dockerignore		.dockerignore
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README-zh_CN.md		README-zh_CN.md
README.md		README.md
pubspec.yaml		pubspec.yaml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

LiteAgent core Dart Server

Feature

Usage

1. Prepare

2. Develop run server

3. HTTP/WS API

3.1 HTTP API

BaseURL

[GET] /version

[POST] /init

[GET] /history?id=<sessionId>

[GET] /stop?id=<sessionId>

[GET] /clear?id=<sessionId>

3.2 WebSocket API

Endpoint

3.2.1 alive

3.2.2 client send UserMessageDto List to server

3.2.3 server feedback AgentMessage to client

3.3 Typical Interaction

Build and Run

Docker

About

Releases 4

Packages

Languages

License

LiteVar/lite_agent_core_dart_server

Folders and files

Latest commit

History

Repository files navigation

LiteAgent core Dart Server

Feature

Usage

1. Prepare

2. Develop run server

3. HTTP/WS API

3.1 HTTP API

BaseURL

[GET] /version

[POST] /init

[GET] /history?id=<sessionId>

[GET] /stop?id=<sessionId>

[GET] /clear?id=<sessionId>

3.2 WebSocket API

Endpoint

3.2.1 alive

3.2.2 client send UserMessageDto List to server

3.2.3 server feedback AgentMessage to client

3.3 Typical Interaction

Build and Run

Docker

About

Resources

License

Stars

Watchers

Forks

Releases 4

Packages 0

Languages

Packages