Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated RemotePingPing to also support .NET 9 #7392

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

Updated RemotePingPing to also support .NET 9 #7392

wants to merge 2 commits into from

Conversation

Aaronontheweb
Copy link
Member

Won't run on the build system without some updates, but figured we should give it a try

@Aaronontheweb
Copy link
Member Author

.NET 8

First run

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      42

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    118977,    1681.70,            42,              68
         5, 1000000,    372163,    2687.16,            76,              80
        10, 2000000,    737464,    2712.57,            88,              90
        15, 3000000,    766088,    3916.48,            98,              98
        20, 4000000,    765258,    5227.18,           106,             107
        25, 5000000,    777726,    6429.57,           115,              93
        30, 6000000,    801176,    7489.40,           101,              93

Second run:

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      43

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    108933,    1836.81,            43,              70
         5, 1000000,    322894,    3097.82,            78,              80
        10, 2000000,    718391,    2784.02,            88,              93
        15, 3000000,    754717,    3975.14,           101,             101
        20, 4000000,    749626,    5336.03,           109,             101
        25, 5000000,    769586,    6497.42,           109,              95
        30, 6000000,    758342,    7912.86,           103,              95

Third run:

OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      43

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    108578,    1842.73,            43,              68
         5, 1000000,    353858,    2826.18,            76,              80
        10, 2000000,    726745,    2752.39,            88,              94
        15, 3000000,    712082,    4213.18,           102,             102
        20, 4000000,    740056,    5405.47,           110,             102
        25, 5000000,    776037,    6443.32,           110,              94
        30, 6000000,    751127,    7988.91,           102,              94

.NET 9

First run

OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      42

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    117165,    1707.04,            42,              56
         5, 1000000,    398884,    2507.26,            64,              66
        10, 2000000,    691563,    2892.56,            74,              87
        15, 3000000,    729750,    4111.34,            95,              95
        20, 4000000,    737736,    5422.05,           103,              97
        25, 5000000,    735836,    6795.78,           105,              89
        30, 6000000,    736830,    8143.54,            97,              87
Done..

Second run:

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      42

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    132539,    1509.65,            42,              55
         5, 1000000,    331017,    3021.78,            63,              68
        10, 2000000,    696865,    2870.39,            76,              84
        15, 3000000,    710901,    4220.58,            92,              92
        20, 4000000,    730461,    5476.37,           100,              93
        25, 5000000,    717670,    6967.97,           101,              86
        30, 6000000,    710985,    8439.67,            94,              78

Third run:

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      42

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    117717,    1699.07,            42,              57
         5, 1000000,    448230,    2231.97,            65,              68
        10, 2000000,    727538,    2749.12,            76,              76
        15, 3000000,    706548,    4246.51,            84,              92
        20, 4000000,    710985,    5626.97,           100,             100
        25, 5000000,    720254,    6942.15,           108,              85
        30, 6000000,    723241,    8296.13,            93,              77

Takeaways

There are some data points that make it look like .NET 9 can really outperform .NET 8 (i.e. ~450k on the step 2 of the third iteration) but overall the results are quite a bit lower on .NET 9 than they are on .NET 8, by about 8-10%.

I'll retry this later from a cold start because I wonder if CPU heating is a factor - and my grafana dashboard for monitoring that machine doesn't give me that info right now - that would also explain why all of the .NET 8 numbers are lower than they were on the initial run too.

@Aaronontheweb
Copy link
Member Author

I'll re-run the numbers again from a cold start but I'm getting #5385 vibes from this so far

@Aaronontheweb
Copy link
Member Author

Was given a hint on Twitter that the culprit with the performance numbers might be the new GC mode DATAS introduced in .NET 9, so I've decided to re-run the numbers with that setting disabled via MSBuild properties.

.NET 9 w/ DATAS Disabled

Run 1

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      41

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    113379,    1764.75,            41,              66
         5, 1000000,    379795,    2633.39,            74,              76
        10, 2000000,    706714,    2830.85,            84,              84
        15, 3000000,    766088,    3916.06,            92,             107
        20, 4000000,    762486,    5246.25,           115,             115
        25, 5000000,    788147,    6344.19,           123,              99
        30, 6000000,    791975,    7576.56,           107,              88

Run 2

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority. 
OSVersion:                         Unix 6.9.3.76060903                                                               
ProcessorCount:                    16                                                                                
ClockSpeed:                        0 MHZ                                                                             
Actor Count:                       32                                                                                
Messages sent/received per client: 200000  (2e5)                                                                     
Is Server GC:                      True                                                                              
Thread count:                      41                                                                                
                                                                                                                     
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads                                           
         1,  200000,    125550,    1593.19,            41,              67                                           
         5, 1000000,    323311,    3093.21,            75,              78                                           
        10, 2000000,    776398,    2576.24,            86,              86                                           
        15, 3000000,    757768,    3959.69,            94,              99                                           
        20, 4000000,    772052,    5181.50,           107,             109                                           
        25, 5000000,    775555,    6447.73,           117,              93                                           
        30, 6000000,    781556,    7677.74,           101,              93

Run 3

Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion:                         Unix 6.9.3.76060903
ProcessorCount:                    16
ClockSpeed:                        0 MHZ
Actor Count:                       32
Messages sent/received per client: 200000  (2e5)
Is Server GC:                      True
Thread count:                      42

Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
         1,  200000,    119332,    1676.91,            42,              66
         5, 1000000,    330034,    3030.71,            74,              75
        10, 2000000,    732601,    2730.50,            83,              85
        15, 3000000,    770417,    3894.92,            93,              95
        20, 4000000,    793022,    5044.85,           103,             103
        25, 5000000,    774474,    6456.14,           111,              87
        30, 6000000,    810045,    7407.47,            95,              87

Takeaways

It definitely looks like DATAS, which is enabled by default in .NET 9, was responsible for the throughput degradation we observed. With it disabled it looks like .NET 9 performs slightly faster than .NET 8 and with a peak observed performance that is higher in absolute terms. This is only looking at throughput data - these benchmarks don't consider GC.

@Aaronontheweb
Copy link
Member Author

So if it's just a GC setting (and therefore, not a bug in the runtime), no risk of #5385 - we'll just need to see how things going with real-world applications that upgrade to .NET 9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
akka-remote perf
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Aaronontheweb