Skip to content

Commit

Permalink
chore: merge just commands (#982)
Browse files Browse the repository at this point in the history
  • Loading branch information
grieve54706 authored Dec 13, 2024
1 parent dd88b77 commit 83a6050
Show file tree
Hide file tree
Showing 7 changed files with 37 additions and 21 deletions.
2 changes: 0 additions & 2 deletions .github/workflows/ibis-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@ jobs:
- name: Install dependencies
run: |
just install
just install-core
just update-core
- name: Run tests
env:
WREN_ENGINE_ENDPOINT: http://localhost:8080
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/stable-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
context: ./ibis-server
build-args: |
RUST_PROFILE=--release
ENV=prod
build-contexts: |
wren-core-py=./wren-core-py
wren-core=./wren-core
Expand Down
7 changes: 5 additions & 2 deletions ibis-server/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
FROM python:3.11-buster AS builder
ARG RUST_PROFILE

ARG ENV
ENV ENV=$ENV


# libpq-dev is required for psycopg2
RUN apt-get update && apt-get -y install libpq-dev

Expand Down Expand Up @@ -30,7 +34,6 @@ COPY --from=wren-core . /wren-core
WORKDIR /app
COPY . .
RUN just install --without dev
RUN just install-core $RUST_PROFILE


FROM python:3.11-slim-buster AS runtime
Expand Down
2 changes: 1 addition & 1 deletion ibis-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ vim .env
```
Install the dependencies
```bash
just install && just install-core
just install
```
Run the server
```bash
Expand Down
26 changes: 20 additions & 6 deletions ibis-server/docs/development.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Development
This document describes the process for running this application on your local computer.

This document describes the process for running this application on your local computer.

## Getting started

This application is powered by Python and Rust! :sparkles: :snake: :gear: :sparkles:

It runs on macOS, Windows, and Linux environments.
Expand All @@ -12,63 +13,76 @@ You'll need `Python 3.11` to run the application. To install Python, [download t
You'll also need `Rust` and `Cargo`. To install them, follow the instructions on the [Rust installation page](https://www.rust-lang.org/tools/install).

Next, install the following tools:

- [poetry](https://github.com/python-poetry/poetry)
- [casey/just](https://github.com/casey/just)
- [pre-commit](https://pre-commit.com)
- [taplo](https://github.com/tamasfe/taplo)

After installing these tools, run:

```shell
just pre-commit-install
```
This installs the pre-commit hooks.

This installs the pre-commit hooks.

## Start the server

To get the application running:

1. Execute `just install` to install the dependencies
2. Execute `just install-core` to Install the core. If you modify the core, you can update it by `just update-core`.
2. If you modified the core, you can install it into python venv by `just install-core`.
3. Create a `.env` file and fill in the required environment variables (see [Environment Variables](#Environment-Variables))

To start the server:

- Execute `just run` to start the server
- Execute `just dev` to start the server in development mode (auto-reloads on code changes)
- The default port is `8000`. You can change it by running `just port=8001 run` or `just port=8001 dev`

### Docker

- Build the image: `just docker-build`
- Run the container: `just docker-run`

### Run the testing

- Prepare the Java Engine server (see [Java Engine Example](../../example/README.md))
- Use the `.env` file to set the `WREN_ENGINE_ENDPOINT` environment variable to change the endpoint of the Java Engine server.
```
WREN_ENGINE_ENDPOINT=http://localhost:8080
```
More information about the environment variables can be found in the [Environment Variables](#Environment-Variables) section.
- Run specific data source test using [pytest marker](https://docs.pytest.org/en/stable/example/markers.html). There are some markers for different data sources. See the list in [pyproject.toml](../pyproject.toml).
- Run specific data source test using [pytest marker](https://docs.pytest.org/en/stable/example/markers.html). There are some markers for different data sources. See the list
in [pyproject.toml](../pyproject.toml).
```
just test postgres
```

### Environment Variables
- `WREN_ENGINE_ENDPOINT`: The endpoint of the Wren Java engine

- `WREN_ENGINE_ENDPOINT`: The endpoint of the Wren Java engine

## How to add new data source
Please see [How to Add a New Data Source](how-to-add-data-source.md) for more information.

Please see [How to Add a New Data Source](how-to-add-data-source.md) for more information.

## Troubleshooting

### No driver for MS SQL Server

If you want to run tests related to MS SQL Server, you may need to install the `Microsoft ODBC 18 driver`. \
You can follow the instructions to install the driver:

- [Microsoft ODBC driver for SQL Server (Linux)](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server).
- [Microsoft ODBC driver for SQL Server (macOS)](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos).
- [Microsoft ODBC driver for SQL Server (Windows)](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server).

### [ODBC Driver 18 for SQL Server]SSL Provider: [error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:self signed certificate]

If you encounter this error, you can add the `TrustServerCertificate` parameter to the connection string.

```json
{
"connectionInfo": {
Expand Down
12 changes: 5 additions & 7 deletions ibis-server/justfile
Original file line number Diff line number Diff line change
@@ -1,21 +1,19 @@
default:
@just --list --unsorted

build-core *args:
cd ../wren-core-py && just install && just build {{ args }}
build-core:
cd ../wren-core-py && just install && just build

core-wheel-path := "../wren-core-py/target/wheels/wren_core_py-*.whl"

install-core *args:
just build-core {{ args }}
install-core:
just build-core
# Using pip install to avoid adding wheel to pyproject.toml
poetry run pip install {{ core-wheel-path }}

update-core: build-core
poetry run pip install --force-reinstall {{ core-wheel-path }}

install *args:
poetry install {{ args }}
just install-core

pre-commit-install:
poetry run pre-commit install
Expand Down
7 changes: 5 additions & 2 deletions wren-core-py/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@ default:
install:
poetry install --no-root

build *args:
poetry run maturin build {{ args }}
env := env("ENV", "dev")
release_flag := if env == "prod" { "--release" } else { "" }

build:
poetry run maturin build {{ release_flag }}

develop:
poetry run maturin develop
Expand Down

0 comments on commit 83a6050

Please sign in to comment.