Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

321 improve nomenclature for statistical outputs #2

Merged
merged 31 commits into from
Feb 9, 2024
Merged

321 improve nomenclature for statistical outputs #2

merged 31 commits into from
Feb 9, 2024

Conversation

hausman-gdit
Copy link
Contributor

@hausman-gdit hausman-gdit commented Jan 17, 2024 •
edited by shapiromatron
Loading

Closes #321

Multiple changes based on statisticians reviewing the UI and reports:

  • For BMD CDF, change the y-axis on the plot, as well as the table for values from "Percentile" to
    "Cumulative Probability"
  • For all output reports, remove ° from degree of multistage models
  • For all output reports, remove ° from degree of polynomial models
  • Standardize display of p-value to P-Value in column headers, and P-value elsewhere
  • changed all occurrences of Log likelihood to -2* Log(Likelihood Ratio)
    • Dichotomous - changed in Summary table; this is different than BMDS Excel
    • Dichotomous - changed in analysis of deviance table; this is different than BMDS Excel
    • Continuous - changed in Summary table; this is different than BMDS Excel
    • Continuous - changed in analysis of deviance table; this is different than BMDS Excel
    • Nested dichotomous - changed in bootstrap results; this is different than BMDS Excel
    • Multitumor (individual model) - changed in Summary table; this is different than BMDS Excel
    • Multitumor (individual model) - changed in analysis of deviance table; this is different than BMDS Excel
    • Multitumor (summary model) - KEPT Combined Log-Likelihood and Combined Log-Likelihood Constant
    • TODO- the multitumor summary models are inconsistent with others
    • TODO- did we do too many?
  • rename "Scaled Residual for Dose Group near BMD" to "Scaled Residual near BMD"
  • rename "Scaled Residual for Control Dose Group" to "Scaled Residual at Control"
  • rename "Likelihoods of Interest" table to "Likelihoods" table
  • rename "DOF" to "d.f" (always lowercase in all table headers)
    • TODO - should we change to DOF or DF? AIC doesn't have periods between?!
  • rename "Summary" to "Modeling Summary"
  • rename "Frequentist Model Results" to "Maximum Likelihood Approach Model Results"
  • rename "Abs(Residual of interest)" -> "|Residual near BMD|"
  • remove Lower Confidence and Upper Confidence from Model Parameters table
  • rename "Bounded" to "On Bound" on Model Parameters Table
  • change "NA" to "Not Reported" for SE column on Model Parameters Table
  • Change "Test of Interest" to "Tests of Mean and Variance Fits" for continuous data
    • TODO - LLR is still shown
  • Change "Confidence Level" to "Confidence Level (one-sided)" globally
  • remove Lower Confidence and Upper Confidence from Model Parameters table
  • change NA to Not Reported for SE on parameter on bound
  • For continuous summary plots, change "Mean ± 95% CI" to "Observed Mean ± 95% CI"
  • Conditionally add "On bound footnote" to continuous and dichotomous
  • Attempted to fix Goodness of Fit table headers/values for normal/lognormal continuous data
    • TODO is this correct? Did we use the correct values in the correct columns? Needs detailed review
  • Remove “Tail Probability” row unless the Hybrid approach was used for the BMR

@shapiromatron
Merge branch 'next' into 321-collated-nomenclature
2482580
@shapiromatron
Merge branch 'next' into 321-collated-nomenclature
7f1d575
@shapiromatron
rename percentile to cummulative probability
f7c9b0f
@shapiromatron
resize tables as needed
f885035
@shapiromatron
remove change
b04dddc
@shapiromatron
fix test w/ degree symbol
54c2e1a
@shapiromatron
rename log likelihood globally
9007c7f
@shapiromatron
fix bug in layout
eabf495
@shapiromatron
rename scaled residual
16553b0
@shapiromatron
rename DOF to d.f.
33126af
@shapiromatron
change Summary to Modeling Summary
e3a8853
@shapiromatron
rename "Abs(Residual of interest)" -> "|Residual near BMD|"
f6a93e9
@shapiromatron
rename "Frequentist Model Results" to "Maximum Likelihood Approach Mo…
cf10766 
…del Results"
@shapiromatron
title case
6876c7a
@shapiromatron
add vertical spacing
f2b32b5
@shapiromatron
Change "Confidence Level" to "Confidence Level (one-sided)" globally
7d7a464
@shapiromatron
add on bound footnote to UI
69126fd
@shapiromatron
handle goodness of fit table
6e5d96f
@shapiromatron
use < and > instead of less than and greater than
417b638
@shapiromatron shapiromatron changed the title 321 collated nomenclature 321 improve nomenclature for statistical outputs Feb 9, 2024
shapiromatron
shapiromatron approved these changes Feb 9, 2024
View reviewed changes
Copy link
Collaborator

@shapiromatron shapiromatron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - this was very difficult to crosscheck all the assorted outputs. Nice job!

@shapiromatron shapiromatron merged commit 71786b7 into next Feb 9, 2024
3 checks passed
@shapiromatron shapiromatron deleted the 321-collated-nomenclature branch February 9, 2024 23:11
shapiromatron added a commit that referenced this pull request Feb 12, 2024
@hausman-gdit @shapiromatron
321 improve nomenclature for statistical outputs (#2)
696efc7 
* 8

* 25a, 26, 30 comment

* 27, 29, 33, 34, 35

* 2, 6

* 31

* testfix

* rm comment, 31

* 3 note, 2, 5a, 12, 24, rm comment

* js eol lf

* revert MT

* rename percentile to cummulative probability

* resize tables as needed

* remove change

* fix test w/ degree symbol

* rename log likelihood globally

* fix bug in layout

* rename scaled residual

* rename DOF to d.f.

* change Summary to Modeling Summary

* rename "Abs(Residual of interest)" -> "|Residual near BMD|"

* rename "Frequentist Model Results" to "Maximum Likelihood Approach Model Results"

* title case

* add vertical spacing

* Change "Confidence Level" to "Confidence Level (one-sided)" globally

* add on bound footnote to UI

* handle goodness of fit table

* use < and > instead of less than and greater than

* only show tail probability if hybrid continuous bmr

* make format

---------

Co-authored-by: Andy Shapiro <[email protected]>
shapiromatron added a commit that referenced this pull request Feb 13, 2024
@hausman-gdit @shapiromatron
319 update epa style (#9)
bfa3d34 
* Multitumor and Nested Dichotomous (#264)

* troubleshooting

* format

* trusted host all the things

* update documentation

* future flag change, rm background, testfix

* lint

* rm more background, testfix

* unused import

* testfix

rm used for testing

* docs

* review fixes

* formatting

* formatting

* to remove

* remove testing

* rm fake failing data

* mt breadcrumbs

* enable nested dichotomous

* run all permutations of lsc/ilc

* updates

* build basic excel/word reports

* ui stubs

* litter data & bootstrap

* enable nested dichotomous on frontend

* sr, ll, chi^2

* model params, misc values matched

* combined pvalue, -9999 -> ???

* excel exports

* update UI for multitumor degree settings

* MT excel exports

* sketch placeholders

* col headings, isSummary modal jank

* more plug n play, formatting

* rm t specific, reuse individual

* tweak incompatibility, chasing loose ends

* summary & parameter settings

* add !isSummary

* slightly less clonky?

* misc cleanup

* latest updates

* updates from review

* checkout `next`; tests will still fail

---------

Co-authored-by: Andy Shapiro <[email protected]>

* BMDS Desktop settings flag (#261)

* sketch is_desktop

* notes, add to config test

* add further restrictions, override item limit, ui fix

* pin pydantic to version 1.10.* series

* updates

* revise

* wrap up desktop

* final review

---------

Co-authored-by: hausman-gdit <[email protected]>

* UI updates for nested dichotomous and multitumor (#265)

* set max height for dataset tables

* fix layout issues with ND

* add ND hover

* make dataset y-size resizable

* ND model options

* ND read only options

* add modal outputs

* deployment updates

* nested logistic UI updaets

* complete nested dichotomous modal

* fix resizing of dataset table

* a few summary table/option updates

* dont show multitumor model selection since there's only one

* add ms combo summary tables

* add CSF to summary table

* multitumor updates

* individual plot

* show all models in result table; highlight selected

* add multitumor plot

* 4 dose groups

* add bottom margin

* simplify legend

* expand use of TwoColumnTable

* 293 mt min max datasets (#266)

* increase MT datasets to 10

* decrease MT option set

* move logic to store

* update contraints on server to relax for desktop

---------

Co-authored-by: Andy Shapiro <[email protected]>

* Complete nested dichotomous summary table (#267)

* wip

* improve nested dichotomous table!

* switch to ruff format (#271)

* update python package versions (#268)

* karen dev ev fix, package updates

* sync frontend packages too

* fix integration tests

---------

Co-authored-by: Andy Shapiro <[email protected]>

* integration tests - nested dichotomous + multitumor (#269)

* basic ND MT test, tweak button language

* update test data, R-O sketch

* read-only update

* misc changes

* a few updates

* add back change

---------

Co-authored-by: Andy Shapiro <[email protected]>

* 297 dynamic word export (#270)

* change default download options

* forgot this

* big T little t, set defaults but still wonky model_type reset

* oops

* a little better

* move logic to one location

* delete

* change default type to continuous

---------

Co-authored-by: Andy Shapiro <[email protected]>

* update to pydantic v2 (#273)

* ran bump, tweaking

* err message change, mod deprecated

* future test tweak, forgot something, alias not working?

* deprecated

* more updates

* fix tests

* add key so that page updates

* revert changes

---------

Co-authored-by: hausman-gdit <[email protected]>

* multitumor display edits (#272)

* add residuals to summary table

* rename model name

* rename model average row

* add bmd diamonds to multitumor result

* lint

* draw csf to bmdl, not bmd

* (unrelated) update django version

* add slope factor for individual models; see bmds PR 123

* add years context (#274)

* remove zenodo.org badge (#278)

* 283 polyk data link (#275)

* try gitattribute fix before reset

* add button for polyk

* better

* </> --> null

* use flex layout for icons; update icon

---------

Co-authored-by: Andy Shapiro <[email protected]>

* polyk - copy dataset to clipboard for bmds modeling (#276)

* add copy to clipboard

* basic copied alert

* clean up

* missed one

* missed another one

* use ClipobardButton; add on optional click handler

* removed store info

---------

Co-authored-by: Andy Shapiro <[email protected]>

* Add Word and Excel reports to Poly K adjustments (#277)

* add download buttons and apis for polyk reports

* wip

* resketch

* update integration with new polyk adjustment type

* add tests for all api views

* sync w/ latest bmds branch

* pass dose_units

---------

Co-authored-by: hausman-gdit <[email protected]>

* add Desktop mode (#279)

* hello desktop

* start app

* add logging

* wip

* updates

* finish demo

* cleanup

* start server

* start next steps

* updates

* add bmdsdesktop cli option

* hacks for desktop demo

* lint

* enable wal

* update .gitignore

* wip - continue

* double getter?

* demo prep

* add a few updates to the desktop

* desktop updates

* add collections

* refactor homepage

* updates to desktop

* parsimony

* add collections

* include markdown in bundle

* add package versions

* fix integration tests

* 2, 6

* Revert "2, 6"

This reverts commit c8ecff5.

* Run tests in CI (#5)

* attempt to run tests in cicd

* add PAT

* update coverage reporting

* add coverage report to gitignore

* 324 random bootstrap seed (#1)

* set seed on dataset select & add options

* attempt to run tests in cicd

* add PAT

* update coverage reporting

* add coverage report to gitignore

* refactor - only set bootstrap_seed for nested dichotomous

---------

Co-authored-by: Andy Shapiro <[email protected]>

* 328 improve error message (#4)

* set seed on dataset select & add options

* basic functionality

* move to utils, test, add node

* attempt to run tests in cicd

* add PAT

* update coverage reporting

* add coverage report to gitignore

* refactor - only set bootstrap_seed for nested dichotomous

* refactorings

* tweak

* standardize server error parsing

* remove

---------

Co-authored-by: Andy Shapiro <[email protected]>

* 321 improve nomenclature for statistical outputs (#2)

* 8

* 25a, 26, 30 comment

* 27, 29, 33, 34, 35

* 2, 6

* 31

* testfix

* rm comment, 31

* 3 note, 2, 5a, 12, 24, rm comment

* js eol lf

* revert MT

* rename percentile to cummulative probability

* resize tables as needed

* remove change

* fix test w/ degree symbol

* rename log likelihood globally

* fix bug in layout

* rename scaled residual

* rename DOF to d.f.

* change Summary to Modeling Summary

* rename "Abs(Residual of interest)" -> "|Residual near BMD|"

* rename "Frequentist Model Results" to "Maximum Likelihood Approach Model Results"

* title case

* add vertical spacing

* Change "Confidence Level" to "Confidence Level (one-sided)" globally

* add on bound footnote to UI

* handle goodness of fit table

* use < and > instead of less than and greater than

* only show tail probability if hybrid continuous bmr

* make format

---------

Co-authored-by: Andy Shapiro <[email protected]>

* EPA Template CSS updates

* fix branch info

* pull in content for usepa/hawc

* fix animation

* add sticky top

---------

Co-authored-by: Andy Shapiro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shapiromatron shapiromatron shapiromatron approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hausman-gdit @shapiromatron