-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integrate common base mode colors into public dashboard #148
Merged
Merged
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
58d7697
Added in support for emission common to public dashboard, and integra…
louisg1337 ee7cc2e
Added in edge case to deal with bike to BICYCLE
louisg1337 25259d2
Removed emcommon dependency install and adjusted code to work for dyn…
louisg1337 7b456a9
Added in support for lighter colors if multiple of the same color appear
louisg1337 7137a01
Use exponential decay coefficient to account for infinite duplicate c…
louisg1337 6630214
Import and use emcommon's dedupe_colors function
louisg1337 998a725
Added support for emcommon default labels, but commented out for time…
louisg1337 ff703a9
Refactored plots and colors so they work off of internal values inste…
louisg1337 1808c69
Initial commit. Changes copied from Integrate common base mode colors…
iantei 3094859
Check for baseMode as key in dynamic_labels
iantei 86f3798
Update purpose-value mapping from label_options's Purpose's value ins…
iantei e2cd09c
Remove find_closest_key() since we are not using it anymore.
iantei b07ffd1
Remove dic_re, dic_pur as function parameter from mapping_color_label…
iantei 2e2bedc
Update mapping_color_labels() to only take dynamic_labels as the func…
iantei d4cbcc1
Merge remote-tracking branch 'Louis_Changes/common_colors' into Integ…
iantei 26c39a8
1. Update load_viz_notebook_data() as async function so we can call a…
iantei 8c8b7ff
Update Other to other since we pass the internal mode instead of Disp…
iantei 1ef4c3b
Update mapping_color_labels() to use await,and update its return valu…
iantei ac65528
Update mapping_colors_labels() to use await, and update return variab…
iantei fccc53d
Change the mapping_color_labels() to use await, and update return var…
iantei a0fae22
Update use of merge_mode_confirm over mode_confirm.
iantei 12d4795
Update await to scaffolding.load_viz_notebook_data()
iantei 51eb1da
Remove find_closest_key() since it's not being used anywhere.
iantei a5180c4
Add await in front of scaffolding.mapping_color_labels() since it is …
iantei 78d105a
Add values_to_translations mapping for text and html table for mappin…
iantei e634d8e
Use merge_mode_confirm column which has internal label, merged to oth…
iantei ed0c4c6
Handle cases when MODE, PURPOSE, or REPLACED_MODE is unavailable.
iantei 75b4ce2
Split mapping_color_labels() to two functions - mapping_color_labels(…
iantei a1620e4
Merge branch 'main' into Integrate_base_mode_colors
iantei 75d594c
Handle INVALID sensed mode color mapping. Use dedupe to differentiate…
iantei f1176f6
Extract color mapping for BLE modes. Set the default values for dyanm…
iantei 399cd5a
Update mapping_color_labels() to return colors_ble as another return …
iantei 60f5af3
Update mapping_color_labels() for ble modes to pass only the unique_k…
iantei 0e5d869
Add try-except block for ble processing for survey_metrics.ipynb like…
iantei b3a0391
Remove e-mission-common here since it is pulled from the server
shankari 0034d6f
Aggregate a single place to chose labels from either dynamic_labels o…
iantei 669d521
Merge branch 'Integrate_base_mode_colors' of https://github.com/iante…
iantei afaed09
Enforce coding standard for python as import X as x and then x.function.
iantei 9cefa33
Rename the column in expaneded_ct in scaffolding from merge_** to **_…
iantei ec355ca
Rename merge_** to **_w_other in generic_metrics and mode_specific_me…
iantei 821fe7d
Remove unnecessary conversion of defaultdict to dict.
iantei ac07f67
Rename edb for emcommon.diary.base.modes to emcdb. Rename eu for emco…
iantei 3cbd76b
Merge branch 'main' into Integrate_base_mode_colors
iantei File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -90,7 +90,8 @@ | |
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"colors_mode, colors_purpose, colors_sensed = scaffolding.mapping_color_labels(dynamic_labels, dic_re, dic_pur)" | ||
"colors_mode, colors_replaced, colors_purpose, colors_sensed, colors_ble = await scaffolding.mapping_color_labels(dynamic_labels)\n", | ||
"values_to_translations, value_to_translations_purpose, values_to_translations_replaced = await scaffolding.translate_values_to_labels(dynamic_labels)" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: why isn't this |
||
] | ||
}, | ||
{ | ||
|
@@ -108,7 +109,7 @@ | |
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"expanded_ct, file_suffix, quality_text, debug_df = scaffolding.load_viz_notebook_data(year,\n", | ||
"expanded_ct, file_suffix, quality_text, debug_df = await scaffolding.load_viz_notebook_data(year,\n", | ||
" month,\n", | ||
" program,\n", | ||
" study_type,\n", | ||
|
@@ -207,8 +208,8 @@ | |
" # We will have text results corresponding to the axes for simplicity and consistency\n", | ||
" text_results = [[\"Unmodified Alt Text\", \"Unmodified HTML\"], [\"Unmodified Alt Text\", \"Unmodified HTML\"]]\n", | ||
" \n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: (df.groupby(\"Mode_confirm\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False)), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax[0], text_results[0], colors_mode, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: (df.groupby(\"mode_confirm_w_other\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False)), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax[0], text_results[0], colors_mode, debug_df, values_to_translations)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_sensed, lambda df: (df.groupby(\"primary_mode\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False)), \n", | ||
" \"Sensed by OpenPATH\\n\"+stacked_bar_quality_text_sensed, ax[1], text_results[1], colors_sensed, debug_df_sensed)\n", | ||
" \n", | ||
|
@@ -258,8 +259,8 @@ | |
" # Plot entries\n", | ||
" fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15,2*1), sharex=True) \n", | ||
" text_results = [\"Unmodified Alt Text\", \"Unmodified HTML\"]\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_commute, lambda df: df.groupby(\"Mode_confirm\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n (Confirmed trips)\", ax, text_results, colors_mode, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_commute, lambda df: df.groupby(\"mode_confirm_w_other\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n (Confirmed trips)\", ax, text_results, colors_mode, debug_df, values_to_translations)\n", | ||
" set_title_and_save(fig, text_results, plot_title, file_name)\n", | ||
"except (AttributeError, KeyError, pd.errors.UndefinedVariableError) as e:\n", | ||
" plt.clf()\n", | ||
|
@@ -291,8 +292,8 @@ | |
"try:\n", | ||
" fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(15,2*1), sharex=True)\n", | ||
" text_results = [\"Unmodified Alt Text\", \"Unmodified HTML\"]\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: df.groupby(\"Trip_purpose\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax, text_results, colors_purpose, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: df.groupby(\"purpose_confirm_w_other\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax, text_results, colors_purpose, debug_df, value_to_translations_purpose)\n", | ||
" set_title_and_save(fig, text_results, plot_title_no_quality, file_name)\n", | ||
"except (AttributeError, KeyError, pd.errors.UndefinedVariableError) as e:\n", | ||
" plt.clf()\n", | ||
|
@@ -335,16 +336,16 @@ | |
"\n", | ||
" ## We do an existence check for the labeled df because we want to display the sensed value even if we don't have the labeled value\n", | ||
" ## but we don't need to have an existence check for sensed because in that case we will have no data to display\n", | ||
" expanded_ct_u80 = expanded_ct.loc[(expanded_ct['distance'] <= cutoff)] if \"Mode_confirm\" in expanded_ct.columns else None\n", | ||
" expanded_ct_u80 = expanded_ct.loc[(expanded_ct['distance'] <= cutoff)] if \"mode_confirm_w_other\" in expanded_ct.columns else None\n", | ||
" expanded_ct_sensed_u80 = expanded_ct_sensed.loc[(expanded_ct_sensed['distance'] <= cutoff)]\n", | ||
" sensed_u80_quality_text = f\"{len(expanded_ct_sensed_u80)} trips ({round(len(expanded_ct_sensed_u80)/len(expanded_ct_sensed)*100)}% of all trips)\\nfrom {scaffolding.unique_users(expanded_ct_sensed_u80)} {sensed_match.group(3)}\"\n", | ||
" labeled_u80_quality_text = f\"{len(expanded_ct_u80)} trips ({round(len(expanded_ct_u80)/len(expanded_ct)*100)}% of all labeled,\\n{round(len(expanded_ct_u80)/len(expanded_ct_sensed)*100)}% of all trips)\\nfrom {scaffolding.unique_users(expanded_ct_u80)} {sensed_match.group(3)}\" if \"Mode_confirm\" in expanded_ct.columns else \"0 labeled trips\"\n", | ||
" \n", | ||
" # Plot entries\n", | ||
" fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(15,2*2), sharex=True)\n", | ||
" text_results = [[\"Unmodified Alt Text\", \"Unmodified HTML\"], [\"Unmodified Alt Text\", \"Unmodified HTML\"]]\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_u80, lambda df: df.groupby(\"Mode_confirm\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+labeled_u80_quality_text, ax[0], text_results[0], colors_mode, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_u80, lambda df: df.groupby(\"mode_confirm_w_other\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+labeled_u80_quality_text, ax[0], text_results[0], colors_mode, debug_df, values_to_translations)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_sensed_u80, lambda df: df.groupby(\"primary_mode\").agg({distance_col: 'count'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Sensed by OpenPATH\\n\"+sensed_u80_quality_text, ax[1], text_results[1], colors_sensed, debug_df_sensed)\n", | ||
" set_title_and_save(fig, text_results, plot_title_no_quality, file_name)\n", | ||
|
@@ -383,8 +384,8 @@ | |
" fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(15,2*2), sharex=True)\n", | ||
" \n", | ||
" text_results = [[\"Unmodified Alt Text\", \"Unmodified HTML\"], [\"Unmodified Alt Text\", \"Unmodified HTML\"]]\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: df.groupby(\"Mode_confirm\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax[0], text_results[0], colors_mode, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct, lambda df: df.groupby(\"mode_confirm_w_other\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+stacked_bar_quality_text_labeled, ax[0], text_results[0], colors_mode, debug_df, values_to_translations)\n", | ||
" plot_and_text_stacked_bar_chart(expanded_ct_sensed, lambda df: df.groupby(\"primary_mode\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Sensed by OpenPATH\\n\"+stacked_bar_quality_text_sensed, ax[1], text_results[1], colors_sensed, debug_df_sensed)\n", | ||
" set_title_and_save(fig, text_results, plot_title_no_quality, file_name) \n", | ||
|
@@ -419,15 +420,15 @@ | |
"try:\n", | ||
" ## We do an existence check for the labeled df because we want to display the sensed value even if we don't have the labeled value\n", | ||
" ## but we don't need to have an existence check for sensed because in that case we will have no data to display\n", | ||
" labeled_land_trips_df = expanded_ct[expanded_ct['mode_confirm'] != \"air\"] if \"mode_confirm\" in expanded_ct.columns else None\n", | ||
" labeled_land_trips_df = expanded_ct[expanded_ct['mode_confirm_w_other'] != \"air\"] if \"mode_confirm_w_other\" in expanded_ct.columns else None\n", | ||
" sensed_land_trips_df = expanded_ct_sensed[expanded_ct_sensed['primary_mode'] != \"AIR_OR_HSR\"]\n", | ||
" \n", | ||
" sensed_land_quality_text = f\"{len(sensed_land_trips_df)} trips ({round(len(sensed_land_trips_df)/len(expanded_ct_sensed)*100)}% of all trips)\\nfrom {scaffolding.unique_users(sensed_land_trips_df)} {sensed_match.group(3)}\"\n", | ||
" labeled_land_quality_text = f\"{len(labeled_land_trips_df)} trips ({round(len(labeled_land_trips_df)/len(expanded_ct)*100)}% of all labeled,\\n{round(len(labeled_land_trips_df)/len(expanded_ct_sensed)*100)}%) of all trips)\\nfrom {scaffolding.unique_users(labeled_land_trips_df)} {sensed_match.group(3)}\" if \"Mode_confirm\" in expanded_ct.columns else \"0 labeled trips\"\n", | ||
" labeled_land_quality_text = f\"{len(labeled_land_trips_df)} trips ({round(len(labeled_land_trips_df)/len(expanded_ct)*100)}% of all labeled,\\n{round(len(labeled_land_trips_df)/len(expanded_ct_sensed)*100)}%) of all trips)\\nfrom {scaffolding.unique_users(labeled_land_trips_df)} {sensed_match.group(3)}\" if \"mode_confirm_w_other\" in expanded_ct.columns else \"0 labeled trips\"\n", | ||
"\n", | ||
" fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(15,2*2), sharex=True)\n", | ||
" plot_and_text_stacked_bar_chart(labeled_land_trips_df, lambda df: df.groupby(\"Mode_confirm\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+labeled_land_quality_text, ax[0], text_results[0], colors_mode, debug_df)\n", | ||
" plot_and_text_stacked_bar_chart(labeled_land_trips_df, lambda df: df.groupby(\"mode_confirm_w_other\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Labeled by user\\n\"+labeled_land_quality_text, ax[0], text_results[0], colors_mode, debug_df, values_to_translations)\n", | ||
" plot_and_text_stacked_bar_chart(sensed_land_trips_df, lambda df: df.groupby(\"primary_mode\").agg({distance_col: 'sum'}).sort_values(by=distance_col, ascending=False), \n", | ||
" \"Sensed by OpenPATH\\n\"+sensed_land_quality_text, ax[1], text_results[1], colors_sensed, debug_df_sensed)\n", | ||
" set_title_and_save(fig, text_results, plot_title_no_quality, file_name) \n", | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
don't we need the values_to_translations here? we should use the display modes in the energy and emission impact plots.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do not need the values_to_translations() here. It's being used only with the Stacked Bar Charts. We are using display modes in the energy and emission impact plots.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, so we have
mode_confirm
,mode_confirm_w_other
andMode_confirm
all at the same time? When we remove the csvs I think we should probably get rid ofMode_confirm
and just usevalues_to_translations
to prevent confusion.