Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated readme and added demographics table #63

Merged
merged 8 commits into from
Sep 12, 2023
Merged

Updated readme and added demographics table #63

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5f3eabf
Updated README with correct yml file.
achasmita Aug 23, 2023
bb5a122
added demographics table to data page and update README
achasmita Aug 31, 2023
af13070
addressing comments from previous pull request
achasmita Sep 6, 2023
b873a02
addressing comments from previous pull request
achasmita Sep 11, 2023
11620a0
Revert docker-compose-dev.yml
achasmita Sep 12, 2023
5ea421b
Addressing comments from previous pull request
achasmita Sep 12, 2023
1039d30
removed metadata columns
achasmita Sep 12, 2023
37590cf
Addressing previous comments
achasmita Sep 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions utils/db_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,13 @@ def query_demographics():
if not df.empty:
for col in constants.BINARY_DEMOGRAPHICS_COLS:
if col in df.columns:
df[col] = df[col].apply(str)
df.drop(columns=['data.xmlResponse', 'data.name', 'data.version', 'data.label'], inplace=True)
df[col] = df[col].apply(str)
columns_to_drop = [col for col in df.columns if col.startswith("metadata")]
df.drop(columns= columns_to_drop, inplace=True)
df.drop(columns=['data.xmlResponse', 'data.name', 'data.version', 'data.label', 'data.jsonDocResponse.aSfdnWs9LE6q8YEF7u9n85.attr.id','data.jsonDocResponse.aSfdnWs9LE6q8YEF7u9n85.attr.xmlns:jr','data.jsonDocResponse.aSfdnWs9LE6q8YEF7u9n85.attr.xmlns:orx'], inplace=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should hardcode any of the form ids (such as aSfdnWs9LE6q8YEF7u9n85) because I believe they can be different from form to form. But the other changes are fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. the form ID for the time use survey, for example, is time_use_survey_form_v9_1
https://github.com/sebastianbarry/nrel-openpath-deploy-configs/blob/surveys-info-and-surveys-data/survey-resources/data-json/time-use-survey-form-v9.json

modified_columns = perm_utils.get_demographic_columns(df.columns)
df.columns = modified_columns
df.columns = modified_columns
df.columns=[col.rsplit('.',1)[-1] if col.startswith('data.jsonDocResponse.') else col for col in df.columns]
return df


Expand Down