Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAS protobuffs #710

Merged
merged 32 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
91feb28
Rough draft for DAS protobuffs.
cody-littley Aug 19, 2024
fe9abf1
Fix whitespace.
cody-littley Aug 19, 2024
4f5aaf4
Removed unecessary RPC.
cody-littley Aug 22, 2024
3fd77c7
Make suggested changes.
cody-littley Aug 27, 2024
4a1a1ac
Revert some changes.
cody-littley Aug 27, 2024
135bae7
Minor changes.
cody-littley Aug 27, 2024
b8795c0
Move things to original location.
cody-littley Aug 27, 2024
31610fb
Revert whitespace changes.
cody-littley Aug 27, 2024
dce9903
Fix retriever protos.
cody-littley Aug 27, 2024
808311c
Remove unused messages.
cody-littley Aug 27, 2024
204949a
Made small changes.
cody-littley Aug 28, 2024
0dad952
Made suggested changes.
cody-littley Aug 29, 2024
45bb359
Update protobufs.
cody-littley Aug 29, 2024
d8e15e8
Add placeholder implementations.
cody-littley Aug 29, 2024
cc291e4
Iterating on protos.
cody-littley Aug 29, 2024
60c0390
Fix proto versions.
cody-littley Aug 29, 2024
0a29fb2
Docker image for building protobufs.
cody-littley Aug 29, 2024
06005c7
Made suggested changes.
cody-littley Aug 30, 2024
0d88c86
Revert changes to dependencies.
cody-littley Aug 30, 2024
3659a68
Fix build issue.
cody-littley Aug 30, 2024
28f3e20
Make requested changes.
cody-littley Sep 4, 2024
bcf4718
Made suggested changes.
cody-littley Sep 6, 2024
a82c869
Merge branch 'master' into das-protobuffs
cody-littley Sep 6, 2024
267b0c6
Rebuilt protos.
cody-littley Sep 6, 2024
e8fec17
Made suggested changes.
cody-littley Sep 6, 2024
ad6c587
Make suggested changes.
cody-littley Sep 9, 2024
a524c0b
Fix whitespace.
cody-littley Sep 9, 2024
d3a9be4
Made suggested changes.
cody-littley Sep 16, 2024
ad0a710
Moved getChunk to relay
cody-littley Sep 17, 2024
9f44421
Made suggested change.
cody-littley Sep 17, 2024
9ac3965
Made suggested change, removed RPC that won't be used in v1
cody-littley Sep 18, 2024
4be81a3
Compile protobufs.
cody-littley Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions api/builder/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
FROM debian

# Install core dependencies
RUN apt update
RUN apt install -y wget unzip bash build-essential

# Set up user
RUN useradd -m -s /bin/bash user
USER user
WORKDIR /home/user
# Remove default crud
RUN rm .bashrc
RUN rm .bash_logout
RUN rm .profile

# Install golang
RUN wget https://go.dev/dl/go1.21.12.linux-arm64.tar.gz
RUN tar -xf ./*.tar.gz
RUN rm ./*.tar.gz
RUN mkdir -p ~/.local/share
RUN mv go ~/.local/share/go

# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v23.4/protoc-23.4-linux-aarch_64.zip
RUN mkdir protoc
RUN cd protoc && unzip ../*.zip
RUN rm ./*.zip
RUN mv protoc ~/.local/share/protoc

# Setup PATH
RUN touch ~/.bashrc
RUN echo 'export GOPATH=~/.local/share/go' >> ~/.bashrc
RUN echo 'export PATH=~/.local/share/go/bin:$PATH' >> ~/.bashrc
RUN echo 'export PATH=~/.local/share/protoc/bin:$PATH' >> ~/.bashrc

# Install go protobuf extensions
RUN bash -c 'source ~/.bashrc && go install google.golang.org/protobuf/cmd/[email protected]'
RUN bash -c 'source ~/.bashrc && go install google.golang.org/grpc/cmd/[email protected]'
12 changes: 12 additions & 0 deletions api/builder/build-docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env bash

# The location where this script can be found.
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

# Add the --no-cache flag to force a rebuild.
# Add the --progress=plain flag to show verbose output during the build.

docker build \
-f "${SCRIPT_DIR}/Dockerfile" \
--tag pbuf-compiler:latest \
.
10 changes: 10 additions & 0 deletions api/builder/build-protobufs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash

# The location where this script can be found.
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
ROOT="${SCRIPT_DIR}/../.."

docker container run \
--rm \
--mount "type=bind,source=${ROOT},target=/home/user/eigenda" \
pbuf-compiler bash -c 'source .bashrc && cd eigenda && make protoc'
14 changes: 14 additions & 0 deletions api/builder/debug.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env bash

# This is a handy little script for debugging the docker container. Attaches a bash shell to the container.

# The location where this script can be found.
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
ROOT="${SCRIPT_DIR}/../.."

docker container run \
--rm \
--mount "type=bind,source=${ROOT},target=/home/user/eigenda" \
-it \
pbuf-compiler bash

Loading
Loading