Skip to content

Commit

Permalink
fix(interactive): Fix interactive doc and image building (alibaba#3767)
Browse files Browse the repository at this point in the history
  • Loading branch information
zhanglei1949 authored May 9, 2024
1 parent 5307437 commit 8b8b4bb
Show file tree
Hide file tree
Showing 8 changed files with 111 additions and 28 deletions.
5 changes: 1 addition & 4 deletions docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,7 @@ interactive-sdk:
@cd ${WORKING_DIR}/../flex/interactive/sdk/ && ./generate_sdk.sh -g java \
&& ./generate_sdk.sh -g python

# copy the generated files to ${WORKING_DIR}/flex/interactive/development/
@cp -r ${WORKING_DIR}/../flex/interactive/sdk/java/docs/* ${WORKING_DIR}/flex/interactive/development/java/
@cp -r ${WORKING_DIR}/../flex/interactive/sdk/python/docs/* ${WORKING_DIR}/flex/interactive/development/python/

@cd ${WORKING_DIR}/../flex/interactive/sdk/ && bash generate_sdk_doc.sh -o ${WORKING_DIR}/flex/interactive/development/

doxygen: interactive-sdk
@mkdir -p _build
Expand Down
2 changes: 1 addition & 1 deletion docs/flex/interactive/deployment.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GraphScope Interactive Deployment
# Deployment

## Deploy with docker

Expand Down
34 changes: 17 additions & 17 deletions docs/flex/interactive/development/restful_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ This API lists all graphs currently managed by the Interactive service, providin

#### Curl Command Example
```bash
curl -X GET -H "Content-Type: application/json" "http://[host]/v1/graph"
curl -X GET -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph"
```

#### Expected Response
Expand Down Expand Up @@ -312,7 +312,7 @@ This API create a new graph according to the specified schema in request body.

#### Curl Command Example
```bash
curl -X POST -H "Content-Type: application/json" -d @path/to/yourfile.json "http://[host]/v1/graph"
curl -X POST -H "Content-Type: application/json" -d @path/to/yourfile.json "http://{INTERACTIVE_ENDPOINT}/v1/graph"
```

#### Expected Response
Expand Down Expand Up @@ -343,7 +343,7 @@ Delete a graph by name, including schema, indices and stored procedures.

#### Curl Command Example
```bash
curl -X DELETE -H "Content-Type: application/json" "http://[host]/v1/graph/{graph_id}"
curl -X DELETE -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}"
```

#### Expected Response
Expand Down Expand Up @@ -371,7 +371,7 @@ Get the schema for the specified graph.

#### Curl Command Example
```bash
curl -X GET -H "Content-Type: application/json" "http://[host]/v1/graph/{graph_id}"
curl -X GET -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}"
```


Expand Down Expand Up @@ -695,7 +695,7 @@ Client can use the returned job_id to [query the status of job](#getjobbyid-jobm

#### Curl Command Example
```bash
curl -X POST -H "Content-Type: application/json" -d @path/to/json "http://[host]/v1/graph/{graph_id}/dataloading"
curl -X POST -H "Content-Type: application/json" -d @path/to/json "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}/dataloading"
```

#### Expected Response
Expand Down Expand Up @@ -737,7 +737,7 @@ Create a new stored procedure.

#### Curl Command Example
```bash
curl -X POST -H "Content-Type: application/json" -d @/path/to/json "http://[host]/v1/graph/{graph_id}/procedure"
curl -X POST -H "Content-Type: application/json" -d @/path/to/json "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}/procedure"
```


Expand Down Expand Up @@ -768,7 +768,7 @@ List all procedures bound to a graph.

#### Curl Command Example
```bash
curl -X GET -H "Content-Type: application/json" "http://[host]/v1/graph/{graph_id}/procedure"
curl -X GET -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}/procedure"
```

#### Expected Response
Expand Down Expand Up @@ -826,7 +826,7 @@ Get a single procedure's metadata.

#### Curl Command Example
```bash
curl -X GET -H "Content-Type: application/json" "http://[host]/v1/graph/{graph_id}/procedure/{procedure_id}"
curl -X GET -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}/procedure/{procedure_id}"
```


Expand Down Expand Up @@ -891,7 +891,7 @@ Update a procedure's metadata, enable/disable status, description. The procedure

#### Curl Command Example
```bash
curl -X PUT -H "Content-Type: application/json" -d @/path/to/json "http://[host]//v1/graph/{graph_id}/procedure/{procedure_id}"
curl -X PUT -H "Content-Type: application/json" -d @/path/to/json "http://{INTERACTIVE_ENDPOINT}//v1/graph/{graph_id}/procedure/{procedure_id}"
```

#### Expected Response
Expand Down Expand Up @@ -948,7 +948,7 @@ Delete a procedure bound to the graph.

#### Curl Command Example
```bash
curl -X DELETE -H "Content-Type: application/json" "http://[host]/v1/graph/{graph_id}/procedure/{procedure_id}"
curl -X DELETE -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/graph/{graph_id}/procedure/{procedure_id}"
```

#### Expected Response
Expand Down Expand Up @@ -997,7 +997,7 @@ The response of the http request will be like

#### Curl Command Example
```bash
curl -X POST -H "Content-Type: application/json" "http://[host]/v1/service/start"
curl -X POST -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/service/start"
```

#### Expected Response
Expand Down Expand Up @@ -1028,7 +1028,7 @@ Restart the graph query service on current running graph.

#### Curl Command Example
```bash
curl -X POST "http://[host]/v1/service/restart"
curl -X POST "http://{INTERACTIVE_ENDPOINT}/v1/service/restart"
```

#### Expected Response
Expand Down Expand Up @@ -1064,7 +1064,7 @@ but you will receive the following error message to each request:

#### Curl Command Example
```bash
curl -X POST "http://[host]/v1/service/stop"
curl -X POST "http://{INTERACTIVE_ENDPOINT}/v1/service/stop"
```

#### Expected Response
Expand Down Expand Up @@ -1117,7 +1117,7 @@ Get node status.

#### Curl Command Example
```bash
curl -X GET -H "Content-Type: application/json" "http://[host]/v1/node/status"
curl -X GET -H "Content-Type: application/json" "http://{INTERACTIVE_ENDPOINT}/v1/node/status"
```

#### Expected Response
Expand Down Expand Up @@ -1150,7 +1150,7 @@ The job id is received when you [launch a bulk loading job](#importgraph-graphma

#### Curl Command Example
```bash
curl -X GET "http://[host]/v1/job/{job_id}"
curl -X GET "http://{INTERACTIVE_ENDPOINT}/v1/job/{job_id}"
```

#### Expected Response
Expand Down Expand Up @@ -1189,7 +1189,7 @@ Get the metadata of all running/cancelled/success/failed jobs.

#### Curl Command Example
```bash
curl -X GET "http://[host]/v1/job/"
curl -X GET "http://{INTERACTIVE_ENDPOINT}/v1/job/"
```

#### Expected Response
Expand Down Expand Up @@ -1229,7 +1229,7 @@ Cancel a job according to the give job_id.

#### Curl Command Example
```bash
curl -X DELETE "http://[host]/v1/job/{job_id}"
curl -X DELETE "http://{INTERACTIVE_ENDPOINT}/v1/job/{job_id}"
```

#### Expected Response
Expand Down
1 change: 0 additions & 1 deletion docs/flex/interactive/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ To start the Interactive service, provide the raw graph data and meta info. For
bin/gs_interactive service start
```
You should be able to see a message telling you what the `Gremlin/Cypher` endpoint that you should connect to.

### Stop the Service
To stop the Interactive service, simple type in the following command:

Expand Down
2 changes: 1 addition & 1 deletion docs/flex/interactive_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ maxdepth: 2
---
interactive/overview
interactive/installation
interactive/deployment
interactive/getting_started
interactive/custom_graph_data
interactive/stored_procedures
Expand All @@ -19,6 +20,5 @@ interactive/data_model
interactive/data_import
interactive/dev_guide
interactive/glossary
interactive/deployment
interactive/tools
```
2 changes: 1 addition & 1 deletion flex/interactive/docker/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ while [[ $# -gt 0 ]]; do
exit 0
;;
*) # unknown option
err "unknown option $1"
echo "unknown option $1"
usage
exit 1
;;
Expand Down
9 changes: 6 additions & 3 deletions flex/interactive/docker/interactive-runtime.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ARG ARCH=x86_64
FROM registry.cn-hongkong.aliyuncs.com/graphscope/interactive-base:latest AS builder
FROM registry.cn-hongkong.aliyuncs.com/graphscope/interactive-base:v0.0.4 AS builder

ARG ARCH
ARG ENABLE_COORDINATOR="false"
Expand All @@ -10,7 +10,7 @@ COPY --chown=graphscope:graphscope . /home/graphscope/GraphScope
SHELL ["/bin/bash", "-c"]

# install arrow
RUN cd /tmp && sudo apt-get update && sudo apt-get install -y -V ca-certificates lsb-release wget && \
RUN cd /tmp && sudo apt-get update && sudo apt-get install -y -V ca-certificates lsb-release wget libcurl4-openssl-dev && \
curl -o apache-arrow-apt-source-latest.deb https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
sudo apt-get install -y ./apache-arrow-apt-source-latest.deb && \
sudo apt-get update && sudo apt-get install -y libarrow-dev=8.0.0-1
Expand All @@ -23,7 +23,7 @@ RUN curl -sf -L https://static.rust-lang.org/rustup.sh | \
cargo --version

# install opentelemetry
RUN cd /tmp && git clone https://github.com/open-telemetry/opentelemetry-cpp && cd opentelemetry-cpp && \
RUN cd /tmp && git clone -b v1.14.2 --single-branch https://github.com/open-telemetry/opentelemetry-cpp && cd opentelemetry-cpp && \
cmake . -DCMAKE_INSTALL_PREFIX=/opt/flex -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD=17 \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON -DBUILD_SHARED_LIBS=ON \
-DWITH_OTLP_HTTP=ON -DWITH_OTLP_GRPC=OFF \
Expand Down Expand Up @@ -81,6 +81,7 @@ RUN mkdir -p /opt/flex/share/gs_interactive_default_graph/
COPY --from=builder /home/graphscope/GraphScope/flex/interactive/examples/modern_graph/* /opt/flex/share/gs_interactive_default_graph/
COPY --from=builder /home/graphscope/GraphScope/flex/tests/hqps/engine_config_test.yaml /opt/flex/share/engine_config.yaml
COPY --from=builder /home/graphscope/GraphScope/flex/interactive/docker/entrypoint.sh /opt/flex/bin/entrypoint.sh
COPY --from=builder /home/graphscope/GraphScope/flex/third_party/nlohmann-json/single_include/* /opt/flex/include/
RUN sed -i 's/name: modern_graph/name: gs_interactive_default_graph/g' /opt/flex/share/gs_interactive_default_graph/graph.yaml
# change the default graph name.
RUN sed -i 's/default_graph: ldbc/default_graph: gs_interactive_default_graph/g' /opt/flex/share/engine_config.yaml
Expand All @@ -90,6 +91,7 @@ RUN rm -rf /opt/flex/bin/run_app

COPY --from=builder /usr/lib/$ARCH-linux-gnu/libsnappy*.so* /usr/lib/$ARCH-linux-gnu/
COPY --from=builder /usr/include/arrow /usr/include/arrow
COPY --from=builder /usr/include/yaml-cpp /usr/include/yaml-cpp
COPY --from=builder /usr/lib/$ARCH-linux-gnu/libgflags*.so* /usr/lib/$ARCH-linux-gnu/
COPY --from=builder /usr/lib/$ARCH-linux-gnu/libglog*.so* /usr/lib/$ARCH-linux-gnu/
COPY --from=builder /usr/lib/$ARCH-linux-gnu/libyaml-cpp*.so* /usr/lib/$ARCH-linux-gnu/
Expand All @@ -112,6 +114,7 @@ COPY --from=builder /usr/lib/$ARCH-linux-gnu/libre2*.so* /usr/lib/$ARCH-linux-gn
COPY --from=builder /usr/include/glog /usr/include/glog
COPY --from=builder /usr/include/gflags /usr/include/gflags
COPY --from=builder /usr/lib/$ARCH-linux-gnu/libprotobuf* /usr/lib/$ARCH-linux-gnu/
COPY --from=builder /usr/lib/$ARCH-linux-gnu/libfmt*.so* /usr/lib/$ARCH-linux-gnu/

COPY --from=builder /usr/lib/$ARCH-linux-gnu/openmpi/include/ /opt/flex/include
COPY --from=builder /usr/include/boost /usr/include/boost
Expand Down
84 changes: 84 additions & 0 deletions flex/interactive/sdk/generate_sdk_doc.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/bin/bash
# Copyright 2020 Alibaba Group Holding Limited.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This script is used to generate the Java SDK from the Flex Interactive API
# It uses the Swagger Codegen tool to generate the SDK


GENERATED_ENDPOINT="https://virtserver.swaggerhub.com/GRAPHSCOPE/InteractiveAPI/1.0.0"
ENDPOINT_PLACE_HOLDER="{INTERACTIVE_ENDPOINT}"

function usage() {
echo "Usage: $0 [options]"
echo "Options:"
echo " -h, --help Show this help message and exit"
echo " -o, --output-dir Generate the SDK for the specified language"
}

function do_gen() {
echo "Generating SDK documentation"
OUTPUT_PATH="$1"
echo "Output path: ${OUTPUT_PATH}"
# First check whether java/docs and python/docs exist
if [ ! -d "./java/docs" ]; then
echo "java/docs not found, please generate the SDK first"
exit 1
fi
if [ ! -d "./python/docs" ]; then
echo "python/docs not found, please generate the SDK first"
exit 1
fi
# post process the generated docs inplace.
# replace all occurrence of $GENERATED_ENDPOINT with $ENDPOINT_PLACE_HOLDER
# in the generated docs, for all files under ./java/docs and ./python/docs
cmd="sed -i 's|${GENERATED_ENDPOINT}|${ENDPOINT_PLACE_HOLDER}|g' ./java/docs/*"
echo "Running command: ${cmd}"
eval $cmd
cmd="sed -i 's|${GENERATED_ENDPOINT}|${ENDPOINT_PLACE_HOLDER}|g' ./python/docs/*"
echo "Running command: ${cmd}"
eval $cmd

# Copy the generated docs to the output path
cmd="cp -r ./java/docs/* ${OUTPUT_PATH}/java/"
echo "Running command: ${cmd}"
eval $cmd
cmd="cp -r ./python/docs/* ${OUTPUT_PATH}/python/"
echo "Running command: ${cmd}"
eval $cmd

echo "SDK documentation generated successfully."
}


while [[ $# -gt 0 ]]; do
key="$1"

case $key in
-h | --help)
usage
exit
;;
-o | --output-dir)
shift
do_gen "$@"
exit 0
;;
*) # unknown option
err "unknown option $1"
usage
exit 1
;;
esac
done

0 comments on commit 8b8b4bb

Please sign in to comment.