Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks for linecache #32

Merged
merged 3 commits into from
Jun 10, 2019
Merged

Add benchmarks for linecache #32

merged 3 commits into from
Jun 10, 2019

Conversation

Cogitri
Copy link
Collaborator

@Cogitri Cogitri commented Jun 10, 2019

No description provided.

Cogitri added 3 commits June 10, 2019 14:45
@Cogitri
chore(cache): Add benchmarks
afd458d
@Cogitri
style(fmt): run cargo-fmt
e807177
@Cogitri
chore(ci): be sure to run cargo fmt/clippy on all arches
8e6d665 
We allow nightly to fail on CI (due to fmt/clippy not being available sometimes
and to not block PRs on nightly oddnesses), but this makes CI give a green light
even if clippy and fmt complain
@Cogitri
Copy link
Collaborator Author

Cogitri commented Jun 10, 2019

The benchmarks can be invoked via cargo bench and look like this:

load_cargo_toml         time:   [26.959 us 27.023 us 27.099 us]                             
                        change: [+0.7661% +1.6023% +2.5279%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) low mild
  4 (4.00%) high mild
  7 (7.00%) high severe

edit_cargo_toml         time:   [182.20 us 182.62 us 183.07 us]                            
                        change: [-0.8213% -0.0546% +0.6730%] (p = 0.89 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  7 (7.00%) high mild
  2 (2.00%) high severe

@little-dude little-dude merged commit 71b4326 into xi-frontend:master Jun 10, 2019
@little-dude
Copy link
Collaborator

Very nice, thank you!

@little-dude
Copy link
Collaborator

Have you been able to bench gxi's line cache as well by any chance?

@Cogitri
Copy link
Collaborator Author

Cogitri commented Jun 10, 2019

I'll make some comparable benches with gxi's implementation

@Cogitri
Copy link
Collaborator Author

Cogitri commented Jun 10, 2019

xrl's current implementation:

load_cargo_toml         time:   [23.534 us 23.574 us 23.616 us]                             
                        change: [-0.7763% -0.2950% +0.1900%] (p = 0.24 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

edit_cargo_toml         time:   [158.03 us 158.31 us 158.62 us]                            
                        change: [-1.0611% -0.5844% -0.1090%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

gxi's implementation:

load_cargo_toml         time:   [25.095 us 25.184 us 25.299 us]                             
                        change: [+6.6959% +7.4277% +8.2195%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

edit_cargo_toml         time:   [487.47 us 488.97 us 490.63 us]                            
                        change: [+207.87% +210.28% +212.63%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

So yeah, I think xrl's current linecache is fine once we have #29 fixed :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Cogitri @little-dude