-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: add performance script to PR jobs #3274
base: main
Are you sure you want to change the base?
Changes from all commits
1a6c0e6
f433b57
0eeab9b
1d06cee
2ef91de
fbad4e2
713cd81
07095f9
b7022d2
da89299
6c33b43
001eda5
6f52bc9
4388003
38ab11b
2789afd
f29a751
e8b09b8
143ab10
81a1a12
ea8e812
c7587e7
036b023
f27d966
a6efa1a
896ffd9
0b834fe
55bb709
432c443
87f74fe
19d5135
da88616
aa99873
ca57b9d
fc23590
3ce7591
d051dd6
1d0b50b
b8e2316
8a358db
a7ce6c9
3d7f998
e498465
747aa83
c5b279d
4a923db
27d9d81
a829619
f17acd8
cd83210
128160c
248ee43
b910e1a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -159,6 +159,135 @@ jobs: | |
- name: Run tests | ||
run: pnpm --filter integration run-tests | ||
|
||
performance-test: | ||
runs-on: ubuntu-22.04 | ||
needs: prerequisite | ||
timeout-minutes: 5 | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup environment | ||
uses: ./.github/workflows/rafiki/env-setup | ||
|
||
- name: Set up k6 | ||
uses: grafana/setup-k6-action@v1 | ||
|
||
- name: Setup hosts | ||
run: | | ||
echo "127.0.0.1 cloud-nine-wallet-test-backend" | sudo tee -a /etc/hosts | ||
echo "127.0.0.1 cloud-nine-wallet-test-auth" | sudo tee -a /etc/hosts | ||
echo "127.0.0.1 happy-life-bank-test-backend" | sudo tee -a /etc/hosts | ||
echo "127.0.0.1 happy-life-bank-test-auth" | sudo tee -a /etc/hosts | ||
|
||
- name: Build dependencies | ||
run: pnpm --filter performance build:deps | ||
|
||
- name: Bundle | ||
run: pnpm --filter performance bundle | ||
|
||
- name: Start test environment | ||
run: pnpm --filter performance testenv:compose up --wait --build -d | ||
|
||
- name: Start Mock Account Entity Services | ||
run: nohup pnpm --filter performance start-mases > mases.log 2>&1 & | ||
|
||
- name: Run performance tests | ||
id: perf_test | ||
run: | | ||
pnpm --filter performance run-tests:testenv -k --vus 4 --duration 1m | tee performance.log | ||
continue-on-error: true # Don't fail here, allows showing logs | ||
|
||
- name: Print MASE logs | ||
if: always() | ||
run: cat mases.log | ||
|
||
- name: Post/Update Performance Test Results as PR Comment | ||
Comment on lines
+195
to
+205
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is a grafana/k6 action that runs the tests and exports to grafana cloud and leaves a comment here with a link: I opted for doing it more manually as a start - was simpler to get going. And even with that action we might need to manual write/edit a comment if we want the actual metrics in the PR comment (looks like the comment just leaves a link). But that could probably be mitigated by setting good thresholds (fail if iterations < x etc. ) and just use the job for pass/fail and venture into grafana when it fails for more details. It does appear like the free tier is fairly generous and it probably gives us a lot of stuff for free like a centralized view of the history of runs. Open to reworking if we want that but I think this gives us what we're looking for for now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the comment generated: #3274 (comment) It just upserts with the latest results and test logs. |
||
if: always() | ||
uses: actions/github-script@v7 | ||
with: | ||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||
script: | | ||
Comment on lines
+205
to
+210
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am in-lining the javascript here because I found it easier than passing everything from the actions context in and it was just easier for development. We can move this to a file and just run it with node if we prefer though. |
||
const fs = require('fs'); | ||
const summaryPath = './test/performance/k6-test-summary.txt'; | ||
const logPath = './performance.log'; | ||
let summaryContent = ''; | ||
let logContent = ''; | ||
|
||
// Read the k6 summary file | ||
if (fs.existsSync(summaryPath)) { | ||
summaryContent = fs.readFileSync(summaryPath, 'utf8'); | ||
} else { | ||
summaryContent = 'Performance test summary not found.'; | ||
} | ||
|
||
// Read the full logs | ||
if (fs.existsSync(logPath)) { | ||
logContent = fs.readFileSync(logPath, 'utf8'); | ||
} else { | ||
logContent = 'Performance log not found.'; | ||
} | ||
|
||
// Limit in case large file size to avoid error posting comment (65kb limit) | ||
// https://github.com/orgs/community/discussions/41331 | ||
const maxLogSize = 60000; | ||
const truncatedLogs = logContent.length > maxLogSize ? `...(truncated)...\n${logContent.slice(-maxLogSize)}` : logContent; | ||
|
||
|
||
const commentBody = ` | ||
### 🚀 Performance Test Results | ||
|
||
${summaryContent} | ||
|
||
<details> | ||
<summary>📜 Logs</summary> | ||
|
||
\`\`\` | ||
${truncatedLogs} | ||
\`\`\` | ||
|
||
</details> | ||
`; | ||
|
||
const { owner, repo } = context.repo; | ||
const prNumber = context.payload.pull_request?.number; | ||
|
||
if (prNumber) { | ||
const comments = await github.rest.issues.listComments({ | ||
owner, | ||
repo, | ||
issue_number: prNumber | ||
}); | ||
|
||
// Identify existing comment via title | ||
const existingComment = comments.data.find(comment => comment.body.includes('🚀 Performance Test Results')); | ||
|
||
if (existingComment) { // upsert comment | ||
await github.rest.issues.updateComment({ | ||
owner, | ||
repo, | ||
comment_id: existingComment.id, | ||
body: commentBody | ||
}); | ||
} else { | ||
await github.rest.issues.createComment({ | ||
owner, | ||
repo, | ||
issue_number: prNumber, | ||
body: commentBody | ||
}); | ||
} | ||
} else { | ||
console.log("No PR detected, skipping comment."); | ||
} | ||
|
||
- name: Fail if performance tests failed | ||
if: steps.perf_test.outcome == 'failure' | ||
run: exit 1 | ||
|
||
|
||
|
||
|
||
node-build: | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 5 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do we want the test config to be?
I opted for 4VUs and 1M after testing some different values. I think we want to:
I think the hardware is determined by
runs-on: ubuntu-22.04
and should always be the same across runs.