Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: add server with git history. #322

Merged
merged 29 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
b667c16
initial commit
jkhsjdhjs Mar 30, 2024
65b0796
add readme
jkhsjdhjs Mar 30, 2024
38303ef
move nginx config to subdirectory
jkhsjdhjs Mar 30, 2024
73bf9e8
`docker build` is deprecated
jkhsjdhjs Mar 30, 2024
46545f6
Dockerfile: update a comment
jkhsjdhjs Mar 30, 2024
e9ee8b0
README.md: Add link to API specification
s-heppner Apr 1, 2024
9f4b9ca
readme: add note about windows paths
jkhsjdhjs Jun 13, 2024
7fe62b4
readme: move a link to the bottom
jkhsjdhjs Jun 13, 2024
dd92330
nginx: add CORS header
jkhsjdhjs Jun 15, 2024
1f62df9
app: support AASX files
jkhsjdhjs Jun 15, 2024
31f9ea3
app: support JSON and XML files
zrgt Aug 21, 2024
b32c97c
Revert "app: support JSON and XML files "
zrgt Aug 21, 2024
73569bf
app: Add support of JSON and XML files
zrgt Aug 21, 2024
f8c81ba
app: lower the suffixes
zrgt Sep 2, 2024
6ad831f
Remove comments
zrgt Sep 3, 2024
bfbf677
Dockerfile: Update branch
s-heppner Sep 25, 2024
edd21b1
Merge pull request #5 from rwth-iat/Fix/UpdateBranch
zrgt Sep 26, 2024
32c35b9
Add docker compose file (#6)
otto-ifak Oct 4, 2024
1e6e4e4
app/main.py: Fix default storage_path (#7)
Frosty2500 Oct 14, 2024
4078a0a
Dockerfile: Change the path to the main Eclipse BaSyx Python SDK repo…
s-heppner Oct 21, 2024
2ff0234
Merge pull request #10 from rwth-iat/Dockerfile/SDKSource
zrgt Oct 21, 2024
4da7f14
Adapt Dockerfile to new BaSyx-Python SDK repository structure
s-heppner Nov 4, 2024
d8c5ef6
Merge pull request #13 from rwth-iat/Fix/Dockerfile/InstallPath
Ornella33 Nov 4, 2024
7cfe4de
Refactor towards a Dockerfile from Python-Alpine (#11)
Frosty2500 Nov 6, 2024
ef826c6
Merge remote-tracking branch 'server/main' into add_server_to_basyx
Frosty2500 Nov 11, 2024
e39a39e
implement requested changes
Frosty2500 Nov 15, 2024
947d3ba
implement requested changes
Frosty2500 Nov 15, 2024
317ac0b
remove unfinished ci changes
Frosty2500 Nov 15, 2024
898c1dd
add docker-compose documentation
Frosty2500 Nov 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions server/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Ignore the default AAS storage directory
storage/

# Ignore PyCharm IDE configuration
.idea
Frosty2500 marked this conversation as resolved.
Show resolved Hide resolved
45 changes: 45 additions & 0 deletions server/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
FROM python:3.11-alpine

LABEL org.label-schema.name="Eclipse BaSyx" \
org.label-schema.version="1.0" \
org.label-schema.description="Docker image for the basyx-python-sdk server application" \
org.label-schema.maintainer="Eclipse BaSyx"

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# If we have more dependencies for the server it would make sense
# to refactor uswgi to the pyproject.toml
RUN apk update && \
apk add --no-cache nginx supervisor gcc musl-dev linux-headers python3-dev git bash && \
pip install uwsgi && \
pip install --no-cache-dir git+https://github.com/eclipse-basyx/basyx-python-sdk@main#subdirectory=sdk && \
apk del git bash


COPY uwsgi.ini /etc/uwsgi/
COPY supervisord.ini /etc/supervisor/conf.d/supervisord.ini
COPY stop-supervisor.sh /etc/supervisor/stop-supervisor.sh
RUN chmod +x /etc/supervisor/stop-supervisor.sh

# Makes it possible to use a different configuration
ENV UWSGI_INI=/etc/uwsgi/uwsgi.ini
# object stores aren't thread-safe yet
# https://github.com/eclipse-basyx/basyx-python-sdk/issues/205
ENV UWSGI_CHEAPER=0
ENV UWSGI_PROCESSES=1
ENV NGINX_MAX_UPLOAD=1M
ENV NGINX_WORKER_PROCESSES=1
ENV LISTEN_PORT=80
ENV CLIENT_BODY_BUFFER_SIZE=1M

# Copy the entrypoint that will generate Nginx additional configs
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

COPY ./app /app
WORKDIR /app

CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.ini"]
74 changes: 74 additions & 0 deletions server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Eclipse BaSyx Python SDK - HTTP Server

This repository contains a Dockerfile to spin up an exemplary HTTP/REST server following the [Specification of the AAS Part 2 API][6] with ease.
Frosty2500 marked this conversation as resolved.
Show resolved Hide resolved
The server currently implements the following interfaces:

- [Asset Administration Shell Repository Service][4]
- [Submodel Repository Service][5]

It uses the [HTTP API][1] and the [AASX][7], [JSON][8], and [XML][9] Adapters of the [BaSyx Python SDK][3], to serve regarding files from a given directory.
The files are only read, chages won't persist.

Alternatively, the container can also be told to use the [Local-File Backend][2] instead, which stores AAS and Submodels as individual JSON files and allows for persistent changes (except supplementary files, i.e. files referenced by `File` submodel elements).
See [below](#options) on how to configure this.

## Building
The container image can be built via:
```
$ docker buildx build -t basyx-python-sdk-http-server .
```

## Running

### Storage
The container needs to be provided with the directory `/storage` to store AAS and Submodel files: AASX, JSON, XML or JSON files of Local-File Backend.

This directory can be mapped via the `-v` option from another image or a local directory.
To map the directory `storage` inside the container, `-v ./storage:/storage` can be used.
The directory `storage` will be created in the current working directory, if it doesn't already exist.

### Port
The HTTP server inside the container listens on port 80 by default.
To expose it on the host on port 8080, use the option `-p 8080:80` when running it.

### Options
The container can be configured via environment variables:
- `API_BASE_PATH` determines the base path under which all other API paths are made available.
Default: `/api/v3.0`
- `STORAGE_TYPE` can be one of `LOCAL_FILE_READ_ONLY` or `LOCAL_FILE_BACKEND`:
- When set to `LOCAL_FILE_READ_ONLY` (the default), the server will read and serve AASX, JSON, XML files from the storage directory.
The files are not modified, all changes done via the API are only stored in memory.
- When instead set to `LOCAL_FILE`, the server makes use of the [LocalFileBackend][2], where AAS and Submodels are persistently stored as JSON files.
Supplementary files, i.e. files referenced by `File` submodel elements, are not stored in this case.
- `STORAGE_PATH` sets the directory to read the files from *within the container*. If you bind your files to a directory different from the default `/storage`, you can use this variable to adjust the server accordingly.

### Running Examples

Putting it all together, the container can be started via the following command:
```
$ docker run -p 8080:80 -v ./storage:/storage basyx-python-sdk-http-server
```

Since Windows uses backslashes instead of forward slashes in paths, you'll have to adjust the path to the storage directory there:
```
> docker run -p 8080:80 -v .\storage:/storage basyx-python-sdk-http-server
```

Per default, the server will use the `LOCAL_FILE_READ_ONLY` storage type and serve the API under `/api/v3.0` and read files from `/storage`. If you want to change this, you can do so like this:
```
$ docker run -p 8080:80 -v ./storage2:/storage2 -e API_BASE_PATH=/api/v3.1 -e STORAGE_TYPE=LOCAL_FILE_BACKEND -e STORAGE_PATH=/storage2 basyx-python-sdk-http-server
```
## Acknowledgments
Frosty2500 marked this conversation as resolved.
Show resolved Hide resolved

This Dockerfile is inspired by the [tiangolo/uwsgi-nginx-docker][10] repository.

[1]: https://github.com/eclipse-basyx/basyx-python-sdk/pull/238
[2]: https://basyx-python-sdk.readthedocs.io/en/latest/backend/local_file.html
[3]: https://github.com/eclipse-basyx/basyx-python-sdk
[4]: https://app.swaggerhub.com/apis/Plattform_i40/AssetAdministrationShellRepositoryServiceSpecification/V3.0.1_SSP-001
[5]: https://app.swaggerhub.com/apis/Plattform_i40/SubmodelRepositoryServiceSpecification/V3.0.1_SSP-001
[6]: https://industrialdigitaltwin.org/content-hub/aasspecifications/idta_01002-3-0_application_programming_interfaces
[7]: https://basyx-python-sdk.readthedocs.io/en/latest/adapter/aasx.html#adapter-aasx
[8]: https://basyx-python-sdk.readthedocs.io/en/latest/adapter/json.html
[9]: https://basyx-python-sdk.readthedocs.io/en/latest/adapter/xml.html
[10]: https://github.com/tiangolo/uwsgi-nginx-docker
46 changes: 46 additions & 0 deletions server/app/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import os
import pathlib
import sys

from basyx.aas import model, adapter
from basyx.aas.adapter import aasx

from basyx.aas.backend.local_file import LocalFileObjectStore
from basyx.aas.adapter.http import WSGIApp

storage_path = os.getenv("STORAGE_PATH", "/storage")
storage_type = os.getenv("STORAGE_TYPE", "LOCAL_FILE_READ_ONLY")
base_path = os.getenv("API_BASE_PATH")

wsgi_optparams = {}

if base_path is not None:
wsgi_optparams["base_path"] = base_path

if storage_type == "LOCAL_FILE_BACKEND":
application = WSGIApp(LocalFileObjectStore(storage_path), aasx.DictSupplementaryFileContainer(), **wsgi_optparams)

elif storage_type in "LOCAL_FILE_READ_ONLY":
object_store: model.DictObjectStore = model.DictObjectStore()
file_store: aasx.DictSupplementaryFileContainer = aasx.DictSupplementaryFileContainer()

for file in pathlib.Path(storage_path).iterdir():
if not file.is_file():
continue
print(f"Loading {file}")

if file.suffix.lower() == ".json":
with open(file) as f:
adapter.json.read_aas_json_file_into(object_store, f)
elif file.suffix.lower() == ".xml":
with open(file) as f:
adapter.xml.read_aas_xml_file_into(object_store, file)
elif file.suffix.lower() == ".aasx":
with aasx.AASXReader(file) as reader:
reader.read_into(object_store=object_store, file_store=file_store)

application = WSGIApp(object_store, file_store, **wsgi_optparams)

else:
print(f"STORAGE_TYPE must be either LOCAL_FILE or LOCAL_FILE_READ_ONLY! Current value: {storage_type}",
file=sys.stderr)
7 changes: 7 additions & 0 deletions server/compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
services:
app:
build: .
ports:
- "8080:80"
volumes:
- ./storage:/storage
71 changes: 71 additions & 0 deletions server/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env sh
set -e

# Get the maximum upload file size for Nginx, default to 0: unlimited
USE_NGINX_MAX_UPLOAD=${NGINX_MAX_UPLOAD:-0}

# Get the number of workers for Nginx, default to 1
USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}

# Set the max number of connections per worker for Nginx, if requested
# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
NGINX_WORKER_CONNECTIONS=${NGINX_WORKER_CONNECTIONS:-1024}

# Get the listen port for Nginx, default to 80
USE_LISTEN_PORT=${LISTEN_PORT:-80}

# Get the client_body_buffer_size for Nginx, default to 1M
USE_CLIENT_BODY_BUFFER_SIZE=${CLIENT_BODY_BUFFER_SIZE:-1M}

# Create the conf.d directory if it doesn't exist
if [ ! -d /etc/nginx/conf.d ]; then
mkdir -p /etc/nginx/conf.d
fi

if [ -f /app/nginx.conf ]; then
cp /app/nginx.conf /etc/nginx/nginx.conf
else
content='user nginx;\n'
# Set the number of worker processes in Nginx
content=$content"worker_processes ${USE_NGINX_WORKER_PROCESSES};\n"
content=$content'error_log /var/log/nginx/error.log warn;\n'
content=$content'pid /var/run/nginx.pid;\n'
content=$content'events {\n'
content=$content" worker_connections ${NGINX_WORKER_CONNECTIONS};\n"
content=$content'}\n'
content=$content'http {\n'
content=$content' include /etc/nginx/mime.types;\n'
content=$content' default_type application/octet-stream;\n'
content=$content' log_format main '"'\$remote_addr - \$remote_user [\$time_local] \"\$request\" '\n"
content=$content' '"'\$status \$body_bytes_sent \"\$http_referer\" '\n"
content=$content' '"'\"\$http_user_agent\" \"\$http_x_forwarded_for\"';\n"
content=$content' access_log /var/log/nginx/access.log main;\n'
content=$content' sendfile on;\n'
content=$content' keepalive_timeout 65;\n'
content=$content' include /etc/nginx/conf.d/*.conf;\n'
content=$content'}\n'
content=$content'daemon off;\n'
# Set the max number of open file descriptors for Nginx workers, if requested
if [ -n "${NGINX_WORKER_OPEN_FILES}" ] ; then
content=$content"worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};\n"
fi
# Save generated /etc/nginx/nginx.conf
printf "$content" > /etc/nginx/nginx.conf

content_server='server {\n'
content_server=$content_server" listen ${USE_LISTEN_PORT};\n"
content_server=$content_server' location / {\n'
content_server=$content_server' include uwsgi_params;\n'
content_server=$content_server' uwsgi_pass unix:///tmp/uwsgi.sock;\n'
content_server=$content_server' }\n'
content_server=$content_server'}\n'
# Save generated server /etc/nginx/conf.d/nginx.conf
printf "$content_server" > /etc/nginx/conf.d/nginx.conf

# # Generate additional configuration
printf "client_max_body_size $USE_NGINX_MAX_UPLOAD;\n" > /etc/nginx/conf.d/upload.conf
printf "client_body_buffer_size $USE_CLIENT_BODY_BUFFER_SIZE;\n" > /etc/nginx/conf.d/body-buffer-size.conf
printf "add_header Access-Control-Allow-Origin *;\n" > /etc/nginx/conf.d/cors-header.conf
fi

exec "$@"
8 changes: 8 additions & 0 deletions server/stop-supervisor.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env sh

printf "READY\n"

while read line; do
echo "Processing Event: $line" >&2
kill $PPID
done < /dev/stdin
27 changes: 27 additions & 0 deletions server/supervisord.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[supervisord]
nodaemon=true

[program:uwsgi]
command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
startsecs = 0
autorestart=false
# may make sense to have autorestart enabled in production

[program:nginx]
command=/usr/sbin/nginx
stdout_logfile=/var/log/nginx.out.log
stdout_logfile_maxbytes=0
stderr_logfile=/var/log/nginx.err.log
stderr_logfile_maxbytes=0
stopsignal=QUIT
startsecs = 0
autorestart=false
# may make sense to have autorestart enabled in production

[eventlistener:quit_on_failure]
events=PROCESS_STATE_STOPPED,PROCESS_STATE_EXITED,PROCESS_STATE_FATAL
command=/etc/supervisor/stop-supervisor.sh
9 changes: 9 additions & 0 deletions server/uwsgi.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[uwsgi]
wsgi-file = /app/main.py
socket = /tmp/uwsgi.sock
chown-socket = nginx:nginx
chmod-socket = 664
hook-master-start = unix_signal:15 gracefully_kill_them_all
need-app = true
die-on-term = true
show-config = false
Loading