Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #88

Merged
merged 5 commits into from
Dec 3, 2024
Merged

Dev #88

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The DSI will be extending a data pipeline to format data from new experiments in
## Pipeline
The data pipeline for this project does the following standardizes data from multiple facilities for display on a dashboard displaying decomposition rates of different compostable plastics as well as operating conditions of the associated facilities.

Note: The pipeline was set up to handle multiple disparate files with varied input formats. Future data will come in a standardized format. The pipline is left as one script for ease of iteration and refactoring later when the new data format is known.
Note: The pipeline was set up to handle multiple disparate files with varied input formats. Future data will come in a standardized format. The pipeline is left as one script for ease of iteration and refactoring later when the new data format is known.

### Docker
The pipeline runs in Docker. If you use VS Code, this is set up to run in a [dev container](https://code.visualstudio.com/docs/devcontainers/containers), so build the container the way you normally would. Otherwise, just build the Docker image from the ```Dockerfile``` in the root of the directory.
Expand All @@ -37,7 +37,7 @@ To run the pipeline:
python scripts/pipeline-template.py
```

Cleaned data files will be output in ```data/```. To update the files displayed on the dashboard, follow the instuctions in [Updating the Dashboard Data](#updating-the-dashboard-data)
Cleaned data files will be output in ```data/```. To update the files displayed on the dashboard, follow the instructions in [Updating the Dashboard Data](#updating-the-dashboard-data)

## Dashboard
This is a [Next.js](https://nextjs.org/) project.
Expand Down Expand Up @@ -71,7 +71,7 @@ npm run dev

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

### Deplying the Dashboard
### Deploying the Dashboard
The dashboard is deployed via Vercel and is hosted on CFTP's site in an iframe.

Any update to the ```main``` branch of this repo will update the production deployment of the dashboard.
Expand Down
7 changes: 0 additions & 7 deletions dashboard/app/api/data/route.js
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ const getIntersectingTrialIDs = (...sets) => {
};

const prepareData = async (searchParams) => {
// console.log("searchParams", searchParams);
// Display params
const aggCol = searchParams.get("aggcol") || "Material Class I";
const displayCol = searchParams.get("displaycol") || "% Residuals (Mass)";
Expand Down Expand Up @@ -177,8 +176,6 @@ const prepareData = async (searchParams) => {
filteredData = filterData(filteredData, "Technology", technologies);
// Return empty object to preserve privacy if not enough trials (Except for Bulk Dose)
const technologyTrialIDs = new Set(filteredData.map((d) => d["Trial ID"]));
// console.log("technologyTrialIDs", technologyTrialIDs);
// console.log(technologies);
const trialThreshold = testMethod === "Bulk Dose" ? 1 : 3;
if (technologyTrialIDs.size < trialThreshold && testMethod !== "Bulk Dose") {
return {
Expand Down Expand Up @@ -242,8 +239,6 @@ const prepareData = async (searchParams) => {
trialDurationTrialIDs
);

console.log("combinedTrialIDs", combinedTrialIDs);

if (combinedTrialIDs.size === 0) {
filteredData = [];
} else {
Expand All @@ -252,8 +247,6 @@ const prepareData = async (searchParams) => {
);
}

console.log("filteredData.length", filteredData.length);

// Not enough data - return empty object
const dataThreshold = 1;
if (filteredData.length < dataThreshold) {
Expand Down
2 changes: 1 addition & 1 deletion dashboard/components/DashboardDisplayControls.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export default function DashboardDisplayControls() {
options={display2col}
title="Show Results by Mass or by Surface Area"
filterKey="displayCol"
infoText="Disintegration data can be based on the change in weight of a product, or the surface area before and after testing."
infoText="Field testing results can be based on the change in weight of a product, or the surface area before and after testing. The results by mass provided here are a combination of wet and dry weights. Future versions of the dashboard will further divide these data sets."
/>
<RadioSingleSelect
options={material2col}
Expand Down
4 changes: 2 additions & 2 deletions dashboard/components/DashboardFilterControls.js
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,14 @@ export default function DashboardFilterControls() {
selectedOptions={snap.filters.selectedTemperatureLevels}
filterKey="selectedTemperatureLevels"
title="Mean Temperature"
infoText="Select one or more options to filter the data by the average temperature range during the field test."
infoText="Select one or more options to filter the data by the average Temperature. Note that temperature measurements are taken in the area surrounding the items being tested"
/>
<CheckboxMenu
options={Object.keys(moistureFilterDict)}
selectedOptions={snap.filters.selectedMoistureLevels}
filterKey="selectedMoistureLevels"
title="Mean % Moisture"
infoText="Select one or more options to filter the data by the average % moisture."
infoText="Select one or more options to filter the data by the average % moisture. Note that moisture measurements are taken from material in the area surrounding the items being tested."
/>
<CheckboxMenu
options={Object.keys(trialDurationDict)}
Expand Down
6 changes: 3 additions & 3 deletions dashboard/lib/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ export const temperatureFilterDict = {
};

export const trialDurationDict = {
"30-45 Days": [30, 45, true],
"45-75 Days": [45, 75, true],
">75 Days": [75, Infinity, false],
"40-59 Days": [40, 59, true],
"59-89 Days": [59, 89, true],
">89 Days": [89, Infinity, false],
};

export const material2col = {
Expand Down
Loading