Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System tracing provider for tracing via native probes (BPF) #1288

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from
Draft

System tracing provider for tracing via native probes (BPF) #1288

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e5b6c29
Add system tracing provider for runtime tracing via BPF/Systemtap
lc525 Jul 6, 2023
5d38039
Update license generation to include poetry extras
lc525 Jul 6, 2023
543a0f0
Update Dockerfile for building mlserver with tracepoints
lc525 Jul 6, 2023
362286f
Initialize system tracing for MLServer and workers
lc525 Jul 7, 2023
6101a3f
Fire tracepoints on model lifecycle events
lc525 Jul 7, 2023
091d08a
Add tracepoints dependencies to poetry lock
lc525 Jul 7, 2023
1c42607
Fix testdata for tracepoint settings
lc525 Jul 7, 2023
eb5c346
Fix some linting warnings
lc525 Jul 7, 2023
0a4f7f6
Update Dockerfile to install all extras unconditionally
lc525 Jul 12, 2023
189bf7c
Simplify tracepoint settings
lc525 Jul 12, 2023
552e754
Fix linting & support for python 3.8
lc525 Jul 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update Dockerfile for building mlserver with tracepoints
@lc525
lc525 committed Jul 12, 2023
commit 543a0f096cd6cea39feafdb9282583971d47f81b
71 changes: 69 additions & 2 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
# MLSERVER_SYS_TRACING may be set to:
# - "sdt-sys-tracing" build SDT (static-defined tracing) native dependencies
# - "without-sys-tracing" no system tracing dependencies
ARG OPT_MLSERVER_SYS_TRACING="sdt-sys-tracing"
ARG LIBSTAPSDT_VERSION="0.1.1"


FROM python:3.10-slim AS wheel-builder
SHELL ["/bin/bash", "-l", "-c"]

ARG POETRY_VERSION="1.4.2"
ARG OPT_MLSERVER_SYS_TRACING

ENV OPT_MLSERVER_SYS_TRACING=${OPT_MLSERVER_SYS_TRACING:-without-sys-tracing}

COPY ./hack/build-wheels.sh ./hack/build-wheels.sh
COPY ./mlserver ./mlserver
@@ -18,19 +28,57 @@ COPY \
# https://github.com/python-poetry/poetry-plugin-export/issues/210
RUN pip install poetry==$POETRY_VERSION && \
./hack/build-wheels.sh /opt/mlserver/dist && \
_extras=() && \
if [[ "${OPT_MLSERVER_SYS_TRACING}" == "sdt-sys-tracing" ]]; then \
_extras+=("-E tracepoints"); \
fi && \
poetry export --with all-runtimes \
--without-hashes \
"${_extras[@]}" \
--format constraints.txt \
-o /opt/mlserver/dist/constraints.txt && \
sed -i 's/\[.*\]//g' /opt/mlserver/dist/constraints.txt


# Build native dependencies for tracepoints;
# Almalinux is binary-compatible with rhel ubi images but contains repositories
# with additional devel packages (elfutils-libelf-devel needed here)
FROM almalinux/9-minimal AS libstapsdt-builder
SHELL ["/bin/bash", "-c"]

ARG LIBSTAPSDT_VERSION

# Install libstapsdt dev dependencies
RUN microdnf update -y && \
microdnf install -y \
wget \
tar \
gzip \
gcc \
make \
findutils \
elfutils-libelf-devel

# Get libstapsdt sources, compile and install into separate tree
# We also need to patch the resulting library symlink to be relative so that
# we may copy the resulting files in a different container directly
RUN wget "https://github.com/linux-usdt/libstapsdt/archive/refs/tags/v${LIBSTAPSDT_VERSION}.tar.gz" && \
tar -xzf v${LIBSTAPSDT_VERSION}.tar.gz && \
cd libstapsdt-${LIBSTAPSDT_VERSION} && \
make && \
make install DESTDIR=/libstapsdt-install && \
cd /libstapsdt-install/usr/lib && \
readlink libstapsdt.so | sed s+/libstapsdt-install/usr/lib/++ | xargs -I % ln -fs % libstapsdt.so


FROM registry.access.redhat.com/ubi9/ubi-minimal
SHELL ["/bin/bash", "-c"]

ARG PYTHON_VERSION=3.10.11
ARG CONDA_VERSION=23.1.0
ARG MINIFORGE_VERSION=${CONDA_VERSION}-1
ARG RUNTIMES="all"
ARG OPT_MLSERVER_SYS_TRACING

# Set a few default environment variables, including `LD_LIBRARY_PATH`
# (required to use GKE's injected CUDA libraries).
@@ -39,6 +87,7 @@ ARG RUNTIMES="all"
ENV MLSERVER_MODELS_DIR=/mnt/models \
MLSERVER_ENV_TARBALL=/mnt/models/environment.tar.gz \
MLSERVER_PATH=/opt/mlserver \
OPT_MLSERVER_SYS_TRACING=${OPT_MLSERVER_SYS_TRACING:-without-sys-tracing} \
CONDA_PATH=/opt/conda \
PATH=/opt/mlserver/.local/bin:/opt/conda/bin:$PATH \
LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/opt/conda/lib/python3.10/site-packages/nvidia/cuda_runtime/lib:$LD_LIBRARY_PATH \
@@ -53,7 +102,16 @@ RUN microdnf update -y && \
libgomp \
mesa-libGL \
glib2-devel \
shadow-utils
shadow-utils && \
if [[ "${OPT_MLSERVER_SYS_TRACING}" == "sdt-sys-tracing" ]]; then \
microdnf install -y \
elfutils-libelf; \
fi

# Install libstapsdt
COPY --from=libstapsdt-builder /libstapsdt-install /
# Update symlinks & ldconfig cache
RUN ldconfig

# Install Conda, Python 3.10 and FFmpeg
RUN microdnf install -y wget && \
@@ -91,6 +149,10 @@ COPY --from=wheel-builder /opt/mlserver/dist ./dist
# NOTE: Removing explicitly requirements.txt file from spaCy's test
# dependencies causing false positives in Snyk.
RUN . $CONDA_PATH/etc/profile.d/conda.sh && \
_extras=() && \
if [[ "${OPT_MLSERVER_SYS_TRACING}" == "sdt-sys-tracing" ]]; then \
_extras+=( "tracepoints" ); \
fi && \
pip install --upgrade pip wheel setuptools && \
if [[ $RUNTIMES == "all" ]]; then \
for _wheel in "./dist/mlserver_"*.whl; do \
@@ -107,7 +169,12 @@ RUN . $CONDA_PATH/etc/profile.d/conda.sh && \
pip install $_wheel --constraint ./dist/constraints.txt; \
done \
fi && \
pip install $(ls "./dist/mlserver-"*.whl) --constraint ./dist/constraints.txt && \
if [[ ${#_extras[@]} -gt 0 ]]; then \
extras_list=$(IFS=, ; echo "${_extras[@]}") && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the same reasoning as the other comment, unless we have some reason to avoid it I'd install all the extras in the Dockerfile - that way the seldonio/mlserver image is always ready as-is to enable tracepoints.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to do that if the decision (given above comments) is to install all extras. I don't know if pip has a simple way of doing that here, I might still have to build a list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also see if Poetry supports an all extras target? Otherwise we can always add one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and / or just hardcode a pip install mlserver[extras1, extras2, etc.])

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Poetry currently supports --all-extras on install but not export (this is tracked in python-poetry/poetry-plugin-export#45). I've added an everything extra. I've intentionally avoided the name all as that might be reserved in the future by poetry/pip, but we can move to it once there is enough support.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I would just use all as it's more standard. If poetry decides to always add one automatically we can just remove it and use Poetry's built-in target.

pip install $(ls "./dist/mlserver-"*.whl)[${extras_list}] --constraint ./dist/constraints.txt; \
else \
pip install $(ls "./dist/mlserver-"*.whl) --constraint ./dist/constraints.txt; \
fi && \
rm -f /opt/conda/lib/python3.10/site-packages/spacy/tests/package/requirements.txt && \
rm -rf /root/.cache/pip