Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document business rules / conscious biases in README.md for clarity #1

Open
joepindar opened this issue Feb 18, 2019 · 0 comments
Open

Comments

@joepindar
Copy link

All data analysis has biases, because there are many ways to slice the data based on the hypothesis being tested or the analysis business rules. These biases are not wrong, but it is important to document them so the reader can decide if they agree with them.

For example in average-ranking-per-200-blocks-window, comparing Atom_Guide to layover_run, Atom_Guide had:

  • 4.4x more missed blocks
  • 4.4x more avgMissed per 200 block window

but Atom_Guide ranked higher owing to the compression effect of large groups of validators missing blocks at the same time as Atom_Guide - using the method described on Riot.

As @gamarin2 states "Your rank on a window really depends on how other perform during same window". While a perfectly valid method of slicing the data, it does mean that people who 'overachieved' by not missing blocks while many others around them did are not rewarded more. This is a conscious business rule which creates bias... and should be documented, along with the others in the analysis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant