Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major Update: Replace Docker with virtual environments and update versions #117

Merged
merged 51 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
15811a9
DH version to 0.33.3
chipkent Apr 25, 2024
780185e
Updated the ib api version to 10.19.04
chipkent Apr 25, 2024
65b9ea9
Streamline and simplify the IB artifact building.
chipkent Apr 29, 2024
4d8f657
Added a script to build deephaven-ib virtual environments.
chipkent Apr 30, 2024
482a2bb
Made IB wheel building fail on errors.
chipkent Apr 30, 2024
527e3d7
Updated to server-ui deephaven images.
chipkent Apr 30, 2024
bdc638d
Debug CI
chipkent Apr 30, 2024
6a8781d
Debug CI
chipkent Apr 30, 2024
91972ea
Debug CI
chipkent Apr 30, 2024
5d1525d
Debug CI
chipkent Apr 30, 2024
f906127
Removed the server-ui change since it requires 0.34
chipkent Apr 30, 2024
ba169a3
Made the versions easier to update.
chipkent Apr 30, 2024
9be1e6c
Remove legacy docker
chipkent Apr 30, 2024
b82bb72
Updated documentation.
chipkent Apr 30, 2024
01e0ab5
Updated documentation.
chipkent Apr 30, 2024
eef03c0
Made ib version handling robust.
chipkent May 1, 2024
122a7c7
Removed all docker requirements.
chipkent May 1, 2024
70c5cb0
Release the hard version constraint on deephaven.
chipkent May 1, 2024
0b51312
Updated the minimum python version to avoid core dump in jpy.
chipkent May 1, 2024
b44eab1
Fix broken dependency version handling.
chipkent May 1, 2024
7aa5b8c
Fix broken short rate downloads.
chipkent May 1, 2024
2905ca6
Fix DH queries that are broken.
chipkent May 1, 2024
26fe2d5
Fix DH examples.
chipkent May 1, 2024
018d8f0
Add an option to create a venv for use in pycharm.
chipkent May 1, 2024
fc54e17
Upgrade to Deephaven 0.34.1
chipkent May 7, 2024
05fcdc5
Addressing the review.
chipkent May 7, 2024
a0bf0b1
Addressing the review.
chipkent May 7, 2024
d87b35e
Support using existing venv or giving a venv a custom name.
chipkent May 7, 2024
8e60095
Support using system python instead of just venvs.
chipkent May 7, 2024
0cf8414
Fix actions to build IB wheels
chipkent May 7, 2024
bc67759
Fix actions to build IB wheels
chipkent May 7, 2024
6dade6e
Fix actions to build IB wheels
chipkent May 7, 2024
0719fab
Fix actions to build IB wheels
chipkent May 7, 2024
d031a83
Fix actions to build IB wheels
chipkent May 7, 2024
d95aac9
Fix actions to build IB wheels
chipkent May 7, 2024
d0df8e6
Fix actions to build IB wheels
chipkent May 7, 2024
a559857
Fix actions to build IB wheels
chipkent May 7, 2024
37e43a0
Fix actions to build IB wheels
chipkent May 7, 2024
853ae75
Fix venv-less install.
chipkent May 7, 2024
ca1568a
Improved readme.
chipkent May 8, 2024
0b613d7
Improved readme.
chipkent May 8, 2024
f57ed16
Improved readme.
chipkent May 8, 2024
59a70e3
Improved readme.
chipkent May 8, 2024
53923a9
Build both the ib wheel and dhib wheel
chipkent May 8, 2024
c274896
Updated the readme to use the `deephaven` command
chipkent May 8, 2024
f73871d
Update README.md
chipkent May 8, 2024
f9b1c25
Trying to fix sphinx build
chipkent May 8, 2024
3f8070f
Trying to fix sphinx build
chipkent May 8, 2024
8cd1055
Trying to fix sphinx build
chipkent May 8, 2024
7e634ea
Trying to fix sphinx build
chipkent May 8, 2024
2e3b2f4
Trying to fix sphinx build
chipkent May 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/Dockerfile.pip
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
FROM ubuntu:22.04

RUN apt update && \
apt install -y openjdk-11-jdk python3-pip python3-venv curl zip && \
apt install -y openjdk-17-jdk python3-pip python3-venv curl zip && \
pip3 install --upgrade pip setuptools wheel

ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64/

COPY ./wheels /wheels
COPY ./ib-wheels /ib-wheels
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/Dockerfile.server
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ARG DH_VERSION

FROM ghcr.io/deephaven/server:${DH_VERSION}
FROM ghcr.io/deephaven/server-ui:${DH_VERSION}

RUN apt update && \
apt install -y python3-pip python3-venv curl zip && \
Expand Down
47 changes: 20 additions & 27 deletions .github/workflows/build-and-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,41 +12,34 @@ on:
- released

env:
IB_VERSION: 10.19.01
DH_VERSION: 0.28.1
IB_VERSION: 10.19.04
DH_VERSION: 0.34.1

jobs:
build-ib-whl:
name: Build IB WHL
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
steps:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools wheel build twine
- name: Checkout
uses: actions/checkout@v1
- name: Build
run: |
IB_VERSION_DOWNLOAD=$(echo ${IB_VERSION} | sed 's/[.]//')
echo "Downloading IB API version ${IB_VERSION_DOWNLOAD}"
curl -o ./api.zip "https://interactivebrokers.github.io/downloads/twsapi_macunix.${IB_VERSION_DOWNLOAD}.zip"
unzip api.zip
cd ./IBJts/source/pythonclient
python -m build
python3 --version
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements_dhib_env.txt
python3 dhib_env.py ib-wheel --ib_version ${{ env.IB_VERSION }}
find . -name \*.whl
- name: Archive build artifacts
uses: actions/upload-artifact@v2
with:
name: ib-wheels
path: |
./IBJts/source/pythonclient/dist/*
dist/ib/*


build-whl:
name: Build WHL
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@master
- name: Setup Python
Expand Down Expand Up @@ -88,7 +81,7 @@ jobs:

publish-whl:
name: Publish WHL
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build-whl]
if: ${{ github.event_name == 'release' && github.event.action == 'released' }}
steps:
Expand All @@ -107,16 +100,16 @@ jobs:

build-sphinx:
name: Build Sphinx
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build-ib-whl, build-whl]
steps:
- uses: actions/checkout@v1
- name: Apt installs
run: |
sudo apt update
sudo apt install -y openjdk-11-jdk
sudo apt install -y openjdk-17-jdk
- name: Pip installs
run: pip3 install --upgrade sphinx==4.2.0 sphinx-autodoc-typehints furo==2021.10.9
run: pip3 install --upgrade sphinx~=7.3.0 sphinx-autodoc-typehints furo==2024.5.6
- name: Download IB wheels
uses: actions/download-artifact@v3
with:
Expand All @@ -132,7 +125,7 @@ jobs:
- name: Run Sphinx
working-directory: ./sphinx
env:
JAVA_HOME: /usr/lib/jvm/java-11-openjdk-amd64
JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64
run: |
make html
touch build/html/.nojekyll
Expand All @@ -152,7 +145,7 @@ jobs:

publish-sphinx:
name: Publish Sphinx
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build-sphinx]
if: ${{ github.event_name == 'release' && github.event.action == 'released' }}
steps:
Expand All @@ -178,7 +171,7 @@ jobs:

docker-pip:
name: Build and Publish Docker (pip-installed Deephaven)
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build-ib-whl, build-whl]
permissions:
contents: read
Expand Down Expand Up @@ -236,7 +229,7 @@ jobs:

docker-dhserver:
name: Build and Publish Docker (Deephaven server image)
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build-ib-whl, build-whl]
permissions:
contents: read
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
.idea
venv
venv-*
build
dist
src/deephaven_ib.egg-info
docker/data
Expand Down
273 changes: 143 additions & 130 deletions README.md
chipkent marked this conversation as resolved.
Show resolved Hide resolved

Large diffs are not rendered by default.

Loading
Loading