Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: replace build-artifact-s3 with new workflow, add local tpch benches #3864

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

kevinzwang
Copy link
Member

@kevinzwang kevinzwang commented Feb 26, 2025

This PR adds a few things:

  • publish-dev-s3.yml workflow to replace build-artifact-s3.yml with one that uses the unified build process in build-wheel.yml as well as serve a Python simplified repository API through Cloudfront
  • benchmark-local-tpch.yml which runs benchmarks for local TPC-H at 100SF. It will automatically build and publish if necessary, and is run both manually and as a nightly test
  • fix index URLs in nightlies-tests.yml and add local TPC-H benchmarks

Eventually I plan on moving all of our current nightly benchmarks to our main repo, and get them publishing to Google sheets. Then we can deprecate our postgres/looker setup as well as our K8s ray cluster

@github-actions github-actions bot added the ci label Feb 26, 2025
@kevinzwang kevinzwang requested a review from jaychia February 26, 2025 20:05
@kevinzwang kevinzwang marked this pull request as draft February 26, 2025 20:07
@kevinzwang kevinzwang requested a review from samster25 February 26, 2025 20:15
@kevinzwang kevinzwang marked this pull request as ready for review February 26, 2025 20:25
Copy link

codecov bot commented Feb 26, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.35%. Comparing base (c3591a2) to head (ec0406e).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3864      +/-   ##
==========================================
+ Coverage   76.77%   78.35%   +1.58%     
==========================================
  Files         762      762              
  Lines       98616    96327    -2289     
==========================================
- Hits        75708    75480     -228     
+ Misses      22908    20847    -2061     

see 24 files with indirect coverage changes

@kevinzwang kevinzwang requested a review from raunakab February 27, 2025 22:44
@raunakab raunakab removed their request for review February 27, 2025 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant