-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor ping pong example #977
Conversation
|
Report | Sun, July 28, 2024 at 20:39:20 UTC |
Project | Stratum v2 (SRI) |
Branch | refactor-ping-pong |
Testbed | sv2 |
🚨 10 ALERTS: Threshold Boundary Limits exceeded!
Benchmark | Measure (units) | View | Value | Lower Boundary | Upper Boundary |
---|---|---|---|---|---|
client_sv2_mining_message_submit_standard | L2 Accesses (accesses) | 🚨 (view plot | view alert) | 26.00 (+47.32%) | 23.31 (111.55%) | |
client_sv2_mining_message_submit_standard_serialize | L2 Accesses (accesses) | 🚨 (view plot | view alert) | 55.00 (+15.97%) | 52.78 (104.21%) | |
client_sv2_mining_message_submit_standard_serialize_deserialize | Instructions (instructions) | 🚨 (view plot | view alert) | 10,585.00 (+0.40%) | 10,567.30 (100.17%) | |
client_sv2_mining_message_submit_standard_serialize_deserialize | L1 Accesses (accesses) | 🚨 (view plot | view alert) | 15,399.00 (+0.38%) | 15,373.74 (100.16%) | |
client_sv2_open_channel_serialize_deserialize | Instructions (instructions) | 🚨 (view plot | view alert) | 8,027.00 (+0.53%) | 8,009.69 (100.22%) | |
client_sv2_open_channel_serialize_deserialize | L1 Accesses (accesses) | 🚨 (view plot | view alert) | 11,671.00 (+0.48%) | 11,648.39 (100.19%) | |
client_sv2_open_channel_serialize_deserialize | L2 Accesses (accesses) | 🚨 (view plot | view alert) | 84.00 (+14.29%) | 82.57 (101.73%) | |
client_sv2_setup_connection | L2 Accesses (accesses) | 🚨 (view plot | view alert) | 15.00 (+63.31%) | 13.96 (107.42%) | |
client_sv2_setup_connection_serialize_deserialize | Instructions (instructions) | 🚨 (view plot | view alert) | 14,855.00 (+0.29%) | 14,837.01 (100.12%) | |
client_sv2_setup_connection_serialize_deserialize | L1 Accesses (accesses) | 🚨 (view plot | view alert) | 21,810.00 (+0.27%) | 21,785.19 (100.11%) |
Click to view all benchmark results
Benchmark | Estimated Cycles | Estimated Cycles Results estimated cycles | (Δ%) | Estimated Cycles Upper Boundary estimated cycles | (%) | Instructions | Instructions Results instructions | (Δ%) | Instructions Upper Boundary instructions | (%) | L1 Accesses | L1 Accesses Results accesses | (Δ%) | L1 Accesses Upper Boundary accesses | (%) | L2 Accesses | L2 Accesses Results accesses | (Δ%) | L2 Accesses Upper Boundary accesses | (%) | RAM Accesses | RAM Accesses Results accesses | (Δ%) | RAM Accesses Upper Boundary accesses | (%) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
client_sv2_handle_message_common | ✅ (view plot) | 2,111.00 (+2.95%) | 2,129.48 (99.13%) | ✅ (view plot) | 473.00 (+0.49%) | 486.40 (97.25%) | ✅ (view plot) | 731.00 (-0.16%) | 755.22 (96.79%) | ✅ (view plot) | 10.00 (+42.86%) | 10.89 (91.86%) | ✅ (view plot) | 38.00 (+3.64%) | 38.61 (98.42%) |
client_sv2_handle_message_mining | ✅ (view plot) | 8,219.00 (+0.27%) | 8,336.65 (98.59%) | ✅ (view plot) | 2,137.00 (+0.46%) | 2,171.37 (98.42%) | ✅ (view plot) | 3,159.00 (+0.46%) | 3,216.13 (98.22%) | ✅ (view plot) | 39.00 (+1.20%) | 43.61 (89.43%) | ✅ (view plot) | 139.00 (+0.11%) | 141.98 (97.90%) |
client_sv2_mining_message_submit_standard | ✅ (view plot) | 6,316.00 (+0.64%) | 6,389.65 (98.85%) | ✅ (view plot) | 1,750.00 (+0.03%) | 1,763.19 (99.25%) | ✅ (view plot) | 2,546.00 (-0.30%) | 2,575.92 (98.84%) | 🚨 (view plot | view alert) | 26.00 (+47.32%) | 23.31 (111.55%) | ✅ (view plot) | 104.00 (+0.16%) | 106.91 (97.28%) |
client_sv2_mining_message_submit_standard_serialize | ✅ (view plot) | 14,691.00 (-0.56%) | 15,033.69 (97.72%) | ✅ (view plot) | 4,694.00 (+0.01%) | 4,707.19 (99.72%) | ✅ (view plot) | 6,751.00 (-0.05%) | 6,775.26 (99.64%) | 🚨 (view plot | view alert) | 55.00 (+15.97%) | 52.78 (104.21%) | ✅ (view plot) | 219.00 (-1.51%) | 229.98 (95.23%) |
client_sv2_mining_message_submit_standard_serialize_deserialize | ✅ (view plot) | 27,499.00 (+0.08%) | 27,844.68 (98.76%) | 🚨 (view plot | view alert) | 10,585.00 (+0.40%) | 10,567.30 (100.17%) | 🚨 (view plot | view alert) | 15,399.00 (+0.38%) | 15,373.74 (100.16%) | ✅ (view plot) | 89.00 (+6.19%) | 89.63 (99.30%) | ✅ (view plot) | 333.00 (-0.54%) | 345.30 (96.44%) |
client_sv2_open_channel | ✅ (view plot) | 4,451.00 (-0.90%) | 4,610.65 (96.54%) | ✅ (view plot) | 1,461.00 (+0.06%) | 1,474.45 (99.09%) | ✅ (view plot) | 2,156.00 (+0.15%) | 2,173.00 (99.22%) | ✅ (view plot) | 11.00 (-8.05%) | 15.23 (72.23%) | ✅ (view plot) | 64.00 (-1.71%) | 68.16 (93.89%) |
client_sv2_open_channel_serialize | ✅ (view plot) | 14,040.00 (-1.23%) | 14,463.18 (97.07%) | ✅ (view plot) | 5,064.00 (+0.02%) | 5,077.45 (99.74%) | ✅ (view plot) | 7,320.00 (+0.03%) | 7,339.32 (99.74%) | ✅ (view plot) | 42.00 (+12.44%) | 42.27 (99.37%) | ✅ (view plot) | 186.00 (-2.98%) | 198.98 (93.48%) |
client_sv2_open_channel_serialize_deserialize | ✅ (view plot) | 22,591.00 (-0.21%) | 23,023.86 (98.12%) | 🚨 (view plot | view alert) | 8,027.00 (+0.53%) | 8,009.69 (100.22%) | 🚨 (view plot | view alert) | 11,671.00 (+0.48%) | 11,648.39 (100.19%) | 🚨 (view plot | view alert) | 84.00 (+14.29%) | 82.57 (101.73%) | ✅ (view plot) | 300.00 (-1.47%) | 315.00 (95.24%) |
client_sv2_setup_connection | ✅ (view plot) | 4,693.00 (-0.11%) | 4,764.75 (98.49%) | ✅ (view plot) | 1,502.00 (+0.05%) | 1,515.45 (99.11%) | ✅ (view plot) | 2,273.00 (-0.17%) | 2,299.69 (98.84%) | 🚨 (view plot | view alert) | 15.00 (+63.31%) | 13.96 (107.42%) | ✅ (view plot) | 67.00 (-1.28%) | 69.64 (96.21%) |
client_sv2_setup_connection_serialize | ✅ (view plot) | 15,976.00 (-1.75%) | 16,501.05 (96.82%) | ✅ (view plot) | 5,963.00 (+0.01%) | 5,976.45 (99.77%) | ✅ (view plot) | 8,661.00 (+0.07%) | 8,677.76 (99.81%) | ✅ (view plot) | 49.00 (+9.61%) | 49.21 (99.57%) | ✅ (view plot) | 202.00 (-4.21%) | 217.87 (92.72%) |
client_sv2_setup_connection_serialize_deserialize | ✅ (view plot) | 35,450.00 (-0.23%) | 35,747.32 (99.17%) | 🚨 (view plot | view alert) | 14,855.00 (+0.29%) | 14,837.01 (100.12%) | 🚨 (view plot | view alert) | 21,810.00 (+0.27%) | 21,785.19 (100.11%) | ✅ (view plot) | 110.00 (+10.20%) | 113.21 (97.16%) | ✅ (view plot) | 374.00 (-1.44%) | 384.75 (97.21%) |
Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help
|
Report | Sun, July 28, 2024 at 20:39:22 UTC |
Project | Stratum v2 (SRI) |
Branch | 977/merge |
Testbed | sv1 |
🚨 1 ALERT: Threshold Boundary Limit exceeded!
Benchmark | Measure (units) | View | Value | Lower Boundary | Upper Boundary |
---|---|---|---|---|---|
client-sv1-authorize-serialize/client-sv1-authorize-serialize | Latency (nanoseconds (ns)) | 🚨 (view plot | view alert) | 257.97 (+3.85%) | 256.27 (100.66%) |
Click to view all benchmark results
Benchmark | Latency | Latency Results nanoseconds (ns) | (Δ%) | Latency Upper Boundary nanoseconds (ns) | (%) |
---|---|---|---|
client-submit-serialize | ✅ (view plot) | 6,326.30 (-8.63%) | 7,370.57 (85.83%) |
client-submit-serialize-deserialize | ✅ (view plot) | 6,972.80 (-11.02%) | 8,390.59 (83.10%) |
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle | ✅ (view plot) | 7,662.40 (-8.85%) | 8,889.49 (86.20%) |
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle | ✅ (view plot) | 856.71 (-4.53%) | 929.10 (92.21%) |
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize | ✅ (view plot) | 672.38 (-3.44%) | 718.92 (93.53%) |
client-sv1-authorize-serialize/client-sv1-authorize-serialize | 🚨 (view plot | view alert) | 257.97 (+3.85%) | 256.27 (100.66%) |
client-sv1-get-authorize/client-sv1-get-authorize | ✅ (view plot) | 145.23 (-7.39%) | 163.57 (88.79%) |
client-sv1-get-submit | ✅ (view plot) | 5,951.00 (-10.97%) | 7,183.90 (82.84%) |
client-sv1-get-subscribe/client-sv1-get-subscribe | ✅ (view plot) | 275.35 (-1.14%) | 291.21 (94.55%) |
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle | ✅ (view plot) | 714.02 (-4.29%) | 777.28 (91.86%) |
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize | ✅ (view plot) | 573.88 (-6.41%) | 640.51 (89.60%) |
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize | ✅ (view plot) | 200.93 (-2.73%) | 220.02 (91.32%) |
Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help
|
Report | Sun, July 28, 2024 at 20:39:21 UTC |
Project | Stratum v2 (SRI) |
Branch | refactor-ping-pong |
Testbed | sv2 |
Click to view all benchmark results
Benchmark | Latency | Latency Results nanoseconds (ns) | (Δ%) | Latency Upper Boundary nanoseconds (ns) | (%) |
---|---|---|---|
client_sv2_handle_message_common | ✅ (view plot) | 44.68 (+0.26%) | 45.24 (98.75%) |
client_sv2_handle_message_mining | ✅ (view plot) | 72.71 (-0.17%) | 80.35 (90.49%) |
client_sv2_mining_message_submit_standard | ✅ (view plot) | 14.66 (+0.08%) | 14.69 (99.81%) |
client_sv2_mining_message_submit_standard_serialize | ✅ (view plot) | 264.18 (-0.13%) | 284.23 (92.94%) |
client_sv2_mining_message_submit_standard_serialize_deserialize | ✅ (view plot) | 596.86 (+0.42%) | 627.90 (95.06%) |
client_sv2_open_channel | ✅ (view plot) | 163.53 (-1.44%) | 173.45 (94.28%) |
client_sv2_open_channel_serialize | ✅ (view plot) | 273.33 (-3.40%) | 293.51 (93.12%) |
client_sv2_open_channel_serialize_deserialize | ✅ (view plot) | 373.03 (-1.39%) | 423.85 (88.01%) |
client_sv2_setup_connection | ✅ (view plot) | 163.74 (-0.25%) | 174.88 (93.63%) |
client_sv2_setup_connection_serialize | ✅ (view plot) | 446.36 (-5.79%) | 507.01 (88.04%) |
client_sv2_setup_connection_serialize_deserialize | ✅ (view plot) | 971.67 (+0.05%) | 1,039.08 (93.51%) |
Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help
|
Report | Sun, July 28, 2024 at 20:39:22 UTC |
Project | Stratum v2 (SRI) |
Branch | refactor-ping-pong |
Testbed | sv1 |
🚨 2 ALERTS: Threshold Boundary Limits exceeded!
Benchmark | Measure (units) | View | Value | Lower Boundary | Upper Boundary |
---|---|---|---|---|---|
serialize_deserialize_authorize | RAM Accesses (accesses) | 🚨 (view plot | view alert) | 299.00 (+1.39%) | 297.96 (100.35%) | |
serialize_deserialize_handle_authorize | RAM Accesses (accesses) | 🚨 (view plot | view alert) | 369.00 (+1.40%) | 367.32 (100.46%) |
Click to view all benchmark results
Benchmark | Estimated Cycles | Estimated Cycles Results estimated cycles | (Δ%) | Estimated Cycles Upper Boundary estimated cycles | (%) | Instructions | Instructions Results instructions | (Δ%) | Instructions Upper Boundary instructions | (%) | L1 Accesses | L1 Accesses Results accesses | (Δ%) | L1 Accesses Upper Boundary accesses | (%) | L2 Accesses | L2 Accesses Results accesses | (Δ%) | L2 Accesses Upper Boundary accesses | (%) | RAM Accesses | RAM Accesses Results accesses | (Δ%) | RAM Accesses Upper Boundary accesses | (%) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
get_authorize | ✅ (view plot) | 8,538.00 (+1.20%) | 8,726.94 (97.83%) | ✅ (view plot) | 3,746.00 (+0.15%) | 3,855.71 (97.15%) | ✅ (view plot) | 5,248.00 (+0.06%) | 5,402.10 (97.15%) | ✅ (view plot) | 7.00 (-11.90%) | 10.40 (67.31%) | ✅ (view plot) | 93.00 (+3.27%) | 94.10 (98.84%) |
get_submit | ✅ (view plot) | 95,509.00 (-0.05%) | 96,143.87 (99.34%) | ✅ (view plot) | 59,439.00 (-0.05%) | 59,779.65 (99.43%) | ✅ (view plot) | 85,359.00 (-0.05%) | 85,838.01 (99.44%) | ✅ (view plot) | 49.00 (-11.09%) | 62.58 (78.30%) | ✅ (view plot) | 283.00 (+0.32%) | 287.74 (98.35%) |
get_subscribe | ✅ (view plot) | 7,991.00 (+0.17%) | 8,271.92 (96.60%) | ✅ (view plot) | 2,841.00 (+0.39%) | 2,940.12 (96.63%) | ✅ (view plot) | 3,971.00 (+0.44%) | 4,099.95 (96.85%) | ✅ (view plot) | 13.00 (-19.66%) | 19.82 (65.60%) | ✅ (view plot) | 113.00 (+0.31%) | 116.96 (96.61%) |
serialize_authorize | ✅ (view plot) | 12,291.00 (+0.69%) | 12,505.17 (98.29%) | ✅ (view plot) | 5,317.00 (+0.11%) | 5,426.71 (97.98%) | ✅ (view plot) | 7,411.00 (+0.05%) | 7,565.44 (97.96%) | ✅ (view plot) | 10.00 (-7.41%) | 13.38 (74.73%) | ✅ (view plot) | 138.00 (+1.78%) | 140.29 (98.37%) |
serialize_deserialize_authorize | ✅ (view plot) | 24,588.00 (+0.48%) | 24,711.24 (99.50%) | ✅ (view plot) | 9,898.00 (-0.02%) | 10,026.68 (98.72%) | ✅ (view plot) | 13,958.00 (-0.06%) | 14,151.75 (98.63%) | ✅ (view plot) | 33.00 (-9.79%) | 41.62 (79.29%) | 🚨 (view plot | view alert) | 299.00 (+1.39%) | 297.96 (100.35%) |
serialize_deserialize_handle_authorize | ✅ (view plot) | 30,331.00 (+0.61%) | 30,348.55 (99.94%) | ✅ (view plot) | 12,101.00 (+0.05%) | 12,210.71 (99.10%) | ✅ (view plot) | 17,116.00 (-0.01%) | 17,280.54 (99.05%) | ✅ (view plot) | 60.00 (+2.04%) | 64.73 (92.69%) | 🚨 (view plot | view alert) | 369.00 (+1.40%) | 367.32 (100.46%) |
serialize_deserialize_handle_submit | ✅ (view plot) | 126,406.00 (-0.00%) | 127,033.73 (99.51%) | ✅ (view plot) | 73,224.00 (-0.04%) | 73,621.57 (99.46%) | ✅ (view plot) | 104,946.00 (-0.04%) | 105,513.86 (99.46%) | ✅ (view plot) | 120.00 (-0.42%) | 131.31 (91.38%) | ✅ (view plot) | 596.00 (+0.19%) | 599.24 (99.46%) |
serialize_deserialize_handle_subscribe | ✅ (view plot) | 27,529.00 (+0.27%) | 27,604.46 (99.73%) | ✅ (view plot) | 9,643.00 (+0.12%) | 9,742.12 (98.98%) | ✅ (view plot) | 13,639.00 (+0.12%) | 13,775.90 (99.01%) | ✅ (view plot) | 62.00 (-5.91%) | 73.82 (83.98%) | ✅ (view plot) | 388.00 (+0.57%) | 388.66 (99.83%) |
serialize_deserialize_submit | ✅ (view plot) | 114,999.00 (-0.07%) | 115,655.20 (99.43%) | ✅ (view plot) | 68,001.00 (-0.08%) | 68,404.14 (99.41%) | ✅ (view plot) | 97,559.00 (-0.09%) | 98,152.61 (99.40%) | ✅ (view plot) | 65.00 (-6.39%) | 75.30 (86.32%) | ✅ (view plot) | 489.00 (+0.18%) | 492.39 (99.31%) |
serialize_deserialize_subscribe | ✅ (view plot) | 22,922.00 (+0.20%) | 23,112.96 (99.17%) | ✅ (view plot) | 8,195.00 (+0.11%) | 8,297.86 (98.76%) | ✅ (view plot) | 11,542.00 (+0.11%) | 11,683.29 (98.79%) | ✅ (view plot) | 36.00 (-8.63%) | 44.09 (81.65%) | ✅ (view plot) | 320.00 (+0.46%) | 321.54 (99.52%) |
serialize_submit | ✅ (view plot) | 99,826.00 (-0.07%) | 100,472.00 (99.36%) | ✅ (view plot) | 61,483.00 (-0.05%) | 61,829.06 (99.44%) | ✅ (view plot) | 88,206.00 (-0.05%) | 88,691.56 (99.45%) | ✅ (view plot) | 49.00 (-12.04%) | 62.30 (78.65%) | ✅ (view plot) | 325.00 (+0.10%) | 329.21 (98.72%) |
serialize_subscribe | ✅ (view plot) | 11,328.00 (+0.01%) | 11,607.70 (97.59%) | ✅ (view plot) | 4,188.00 (+0.27%) | 4,287.12 (97.69%) | ✅ (view plot) | 5,828.00 (+0.28%) | 5,959.18 (97.80%) | ✅ (view plot) | 15.00 (-8.33%) | 18.98 (79.05%) | ✅ (view plot) | 155.00 (-0.15%) | 159.78 (97.01%) |
Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help
@jbesraa @Shourya742 @pythcoiner since you're all interested in learning more about the low-level crates, how about we do a "PR Review Club"-style call? We could follow this agenda:
We could record it and share on SRI YouTube to spread the knowledge to the community. Also, this could be a good opportunity to brainstorm design decisions on the |
sure! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tACK
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Added one small question.
Could you please add a gh workflow to run this example instead of the removed one?
// initialize decoder | ||
let mut decoder = StandardDecoder::<Ping>::new(); | ||
|
||
// right now, the decoder buffer can only read a frame header |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, I am not getting why the first read fails, could you pleas elaborate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like we're only reading the first 6 bytes of this specific message type and not reaching the total byte length, which causes an error signaling that more bytes are needed to match the payload size. But I'm wondering, why are we taking this approach? Couldn't we just read all the bytes in one go and then decipher from there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like we're only reading the first 6 bytes of this specific message type and not reaching the total byte length, which causes an error signaling that more bytes are needed to match the payload size.
correct
But I'm wondering, why are we taking this approach? Couldn't we just read all the bytes in one go and then decipher from there?
this is technical debt we need to clean up
#903 is the first step of that journey
|
@plebhash Thanks alot for the call and the thorough explanation about this. |
c5afda3
to
40c54e9
Compare
done |
65b14fd
to
146b9ed
Compare
146b9ed
to
60b722a
Compare
utACK 60b722a |
e480c0a
to
0d962ce
Compare
0d962ce
to
6e3334b
Compare
utACK 6e3334b |
Note: Wait to merge until the following is merged: |
6e3334b
to
f1aef4d
Compare
f1aef4d
to
4a5105b
Compare
the tasks mentioned above have been de-prioritized @Shourya742 is working to remove therefore, I'm removing the draft status so we can merge this soon |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #977 +/- ##
=======================================
Coverage 19.20% 19.20%
=======================================
Files 166 166
Lines 11107 11107
=======================================
Hits 2133 2133
Misses 8974 8974
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Co-authored-by: pythcoiner <[email protected]>
4a5105b
to
0f73211
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tACK
This PR is refactoring the
examples/ping-pong-without-noise
example intoexamples/ping-pong
.I wrote it from scratch as an exercise to get familiar with
protocols
crates.The overall idea is quite similar to the previous implementation, but with some simplifications to make the example easier to understand.
ping-pong
is an example of how to encode and decode SV2 binary frames (without any encryption layer) while leveraging the following crates:binary_sv2
codec_sv2
framing_sv2
(which is actually just re-exported bycodec_sv2
)We establish a simple
Ping
-Pong
protocol with a server and a client communicating over a TCP socket.The server expects to receive a
Ping
message encoded as a SV2 binary frame.The
Ping
message contains anonce
, which is au8
generated randomly by the client.The client expects to get a
Pong
message in response, also encoded as a SV2 binary frame, with the samenonce
.The messages are assigned arbitrary values for binary encoding: