Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Code Inspection] Data Cleaning Prototype (HTMX + Alpine) #34846

Closed
wants to merge 121 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
e432ddc
add htmx to package.json
biyeun Jun 12, 2024
a8422ac
include htmx in our javascript as a core library
biyeun Jun 13, 2024
af4916f
add django tables
biyeun Jun 17, 2024
4a67cec
add multi-column sorting capability for django tables
biyeun Jun 18, 2024
cf7f42a
set up base templates for django tables, with HTMX support
biyeun Jun 18, 2024
4d89b5b
add styles for django tables
biyeun Jun 18, 2024
c614df0
add placeholder for htmx tests
biyeun Jun 18, 2024
f43ddff
add simple pagination of table example with HTMX
biyeun Jun 18, 2024
b59340d
add sortablejs
biyeun Jun 20, 2024
0e3e586
add alpine.js
biyeun Jun 20, 2024
c10d3c1
add alpine js decorator and include in base template
biyeun Jun 20, 2024
9afce2c
add ability to trigger a referesh of the table with the refreshTable …
biyeun Jun 20, 2024
9b96b03
add loading indicator for table
biyeun Jun 20, 2024
4beaeeb
update formatting
biyeun Jun 20, 2024
86b4ba2
split off saved pagination state to separate mixin, create SavedPagin…
biyeun Jun 20, 2024
6e41356
add models folder in prototype app
biyeun Jun 20, 2024
39bdbf2
add scaffolding for data cleaning prototype
biyeun Jun 20, 2024
8c32639
add forms folder
biyeun Jun 20, 2024
6e7938e
move alpinejs before core libraries
biyeun Jun 25, 2024
9979441
un-bundle htmx from requirejs
biyeun Jun 25, 2024
177b99c
propagate pagination changes to parent page
biyeun Jun 25, 2024
1381d4f
add styling for select column
biyeun Jun 25, 2024
26cf7ea
saas design sprint prototype, minus data editing forms
biyeun Jun 25, 2024
f3a3a9c
quick utility to reset data
biyeun Jun 27, 2024
ac70c78
increase timeout for cache stores
biyeun Jun 27, 2024
4a2093b
add ability to delete cache store data
biyeun Jun 27, 2024
87128b6
update permissions to prototype flag
biyeun Jun 27, 2024
65f9e7a
add data editing capabilities for prototype
biyeun Jun 27, 2024
31fbdc9
update decorators for pagination example
biyeun Jun 27, 2024
dee3cfb
move data_cleaning.js out of htmx folder
biyeun Jun 27, 2024
fa5dcc6
inline editing in table cell styling
biyeun Jun 28, 2024
8468c9d
inline editing of table cell value with htmx and alpine
biyeun Jun 28, 2024
895d136
thanks to hx-target being the parent table, hx-swap is not needed here
biyeun Jun 28, 2024
e370f2d
narrow the width of selected column
biyeun Jun 28, 2024
cfe8e0f
allow partial updates for cell data changes
biyeun Jun 28, 2024
5a91ecf
update translations
biyeun Jun 28, 2024
6413831
use self.data_store
biyeun Jun 28, 2024
892c84a
add support for handling multiple hx-action requests to a TemplateView
biyeun Jul 1, 2024
a405555
use HTMX actions with all table hx requests and merge DataCleaningCel…
biyeun Jul 1, 2024
43f5e08
add placeholder for htmx error handling
biyeun Jul 1, 2024
c81e063
update error handling modal to get information from htmx event
biyeun Jul 2, 2024
08eca27
rename view for clarity
biyeun Jul 2, 2024
4e0a8e8
cleanup variable names and attribute order. remove unused ids
biyeun Jul 2, 2024
e22cf54
remove unused event
biyeun Jul 2, 2024
a1e3b17
update container_id usage for forms
biyeun Jul 2, 2024
8a7b272
cleanup unused ids, add translations
biyeun Jul 2, 2024
419f569
add clarity to select all functionality
biyeun Jul 2, 2024
cddfd50
add fyi
biyeun Jul 2, 2024
cf4f9cc
update apply edits button when values are edited (or cancelled)
biyeun Jul 2, 2024
c969e5c
add translations
biyeun Jul 2, 2024
cbe8e23
disable clean data button unless records are selected
biyeun Jul 2, 2024
29107f4
update translations
biyeun Jul 2, 2024
4d0dc3b
a simpler way of upadating the apply edits button visibility
biyeun Jul 2, 2024
2a25621
rename update_edit to is_partial_update for clarity
biyeun Jul 2, 2024
dfbdd28
change htmx:beforeSend to htmx:configRequest instead
biyeun Jul 3, 2024
c755826
implement loading indicators
biyeun Jul 3, 2024
f015d3d
add an alpine js pre init block for testing out more advanced feature…
biyeun Jul 11, 2024
f8cb0a3
add ability to simulate slow htmx requests
biyeun Jul 11, 2024
a43b59f
additional info in response
biyeun Jul 11, 2024
9b2fa23
fix deep copy bugs with simple stores
biyeun Jul 11, 2024
22e0f98
update ability to adjust slow response times through admin endpoint
biyeun Jul 11, 2024
7ca5b5d
disable checkboxes when requests are happening to prevent rapid click…
biyeun Jul 11, 2024
6648355
add hq-hx-loading
biyeun Jul 11, 2024
70db2af
add loading indicator
biyeun Jul 11, 2024
8747249
fix translation block variables
biyeun Jul 11, 2024
7ae38ca
update table loading indicators for prototype
biyeun Jul 11, 2024
a2898a2
prevent rapid click errors on inline editing for editbable column
biyeun Jul 11, 2024
009e1df
fix typo
biyeun Jul 11, 2024
d81f011
fix whitespace changes
biyeun Jul 11, 2024
10a4f4c
add more htmx loading styles
biyeun Jul 11, 2024
86e6576
fix bug for htmx-request style on no swap requests
biyeun Jul 11, 2024
f62470e
add loading indicator id to fake case table
biyeun Jul 11, 2024
a59f1d8
add url for slow simulator view
biyeun Jul 11, 2024
d3c815d
update hx-action to hq-hx-action
biyeun Jul 11, 2024
e7baf9b
reorganize table related attrs
biyeun Jul 12, 2024
862ab68
move column management actions to separate model
biyeun Jul 15, 2024
dd5a106
update css_id to container_id
biyeun Jul 15, 2024
7e2693a
fix filtering
biyeun Jul 15, 2024
2a32aca
add support for displaying whitespaces
biyeun Jul 16, 2024
897f506
allow ability to replace blank values
biyeun Jul 16, 2024
55a718f
add support for removing whitespaces
biyeun Jul 16, 2024
7de0f1d
simulate problems with data
biyeun Jul 16, 2024
68e56e8
remove unnecessary class
biyeun Jul 16, 2024
0ec5822
update reset to clear all relevant stores
biyeun Jul 16, 2024
1a69df3
update data-refresh-table to hq-hx-refresh
biyeun Jul 17, 2024
d289703
add ability to use regex in find/replace
biyeun Jul 17, 2024
e4268b4
apply whitespace strip on edited value if one exists
biyeun Jul 17, 2024
be005b2
simplify logic
biyeun Jul 17, 2024
6bbd299
forgot to add use regex to crispy forms
biyeun Jul 17, 2024
b3f67f9
update submit experience
biyeun Jul 17, 2024
1fb2c96
only strip whitespaces at beginning or end of string
biyeun Jul 17, 2024
508af84
add htmx refresh plugin to data cleaning js
biyeun Jul 17, 2024
b386d2b
cleanup responses and interactions. actually allow reordering filters
biyeun Jul 17, 2024
14883f0
add store definition for whitespaces to alpine pre-init
biyeun Jul 17, 2024
da1e797
add ability to merge properties
biyeun Jul 17, 2024
f5b2077
select records update to use input group
biyeun Jul 17, 2024
aadf4e2
reorder buttons and move everything into input groups for table actions
biyeun Jul 17, 2024
168512d
move action to after swap
biyeun Jul 17, 2024
5d156b4
move apply edit to front
biyeun Jul 17, 2024
575f280
add system property column
biyeun Jul 17, 2024
a783c20
generate fake case data that better mimics real case data
biyeun Jul 18, 2024
5fe0a62
improve value styles
biyeun Jul 21, 2024
ac10714
add filters
biyeun Jul 21, 2024
709091c
add improvments to data cleaning actions
biyeun Jul 21, 2024
128c7f8
update diff styles
biyeun Jul 21, 2024
7aa61a8
simulate apply changes taking time
biyeun Jul 21, 2024
7d97784
dismiss num changes alert when offcanvas closes
biyeun Jul 21, 2024
a15693d
support capitalization changes
biyeun Jul 21, 2024
6cc097b
add data history capability
biyeun Jul 22, 2024
16bf2b7
single sort and allow sort deselection
biyeun Jul 22, 2024
3faeba3
improvements to histories and applying edits
biyeun Jul 22, 2024
ff01e2e
make values nullable
biyeun Jul 23, 2024
7f01fbf
iterating on changes from testing round 1
biyeun Jul 23, 2024
85cf1dc
update timeout to larger number
biyeun Jul 23, 2024
d1ac9bf
start moving some logic into column manager
biyeun Jul 23, 2024
3cdaa19
simplify edits even further
biyeun Jul 23, 2024
19b89ba
additional cleanup
biyeun Jul 23, 2024
c45b413
deal with flaky connection issues
biyeun Jul 23, 2024
08803f3
quick updates before demo
biyeun Jul 23, 2024
9a6f1f3
make clean data button blue when enabled
biyeun Jul 29, 2024
6823f1e
make last cleaned property sticky
biyeun Jul 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions corehq/apps/hqwebapp/decorators.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,52 @@
from corehq.apps.hqwebapp.utils.bootstrap import set_bootstrap_version5


def use_alpinejs(view_func):
"""Use this decorator on the dispatch method of a TemplateView subclass
to use Alpine.js on a page.
Note: Eventually this will be bundled with Webpack. The es6/esm module
is incompatible with Requirejs and didn't seem fitting to add additional tooling
to support this in Requirejs since alpine.js lives primarily in HTML attributes.
Can be referenced in javascript with `window.Alpine` if necessary.

Example:
@use_alpinejs
def dispatch(self, request, *args, **kwargs):
return super().dispatch(request, *args, **kwargs)

Or alternatively:
@method_decorator(use_alpinejs, name='dispatch')
class MyViewClass(MyViewSubclass):
...
"""
return set_request_flag(view_func, 'use_alpinejs')


def use_htmx(view_func):
"""Use this decorator on the dispatch method of a TemplateView subclass
to use HTMX on a page.

Note: Bundling HTMX with RequireJS can be done by setting window.htmx = htmx
after importing the htmx module. However, some page loads caused HTMX to not initialize
properly on a page, resulting in components that never loaded. Importing the HTMX script
directly in <head> first thing proves to be the more performant and reliable way to get
HTMX initialize reliably and quickly.

TODO: Investigate bundling HTMX with Webpack.

Example:
@use_htmx
def dispatch(self, request, *args, **kwargs):
return super().dispatch(request, *args, **kwargs)

Or alternatively:
@method_decorator(use_htmx, name='dispatch')
class MyViewClass(MyViewSubclass):
...
"""
return set_request_flag(view_func, 'use_htmx')


def use_daterangepicker(view_func):
"""Use this decorator on the dispatch method of a TemplateView subclass
to enable the inclusion of the daterangepicker library at the base template
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ requirejs.config({
"ko.mapping": "hqwebapp/js/lib/knockout_plugins/knockout_mapping.ko.min",
"sentry_browser": "sentry/js/sentry.browser.7.28.0.min",
"sentry_captureconsole": "sentry/js/sentry.captureconsole.7.28.0.min",
"sortablejs": "sortablejs/Sortable.min",
"underscore": "underscore/underscore",
"stripe": "https://js.stripe.com/v2/?noext",
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ requirejs.config({
"popper": "@popperjs/core/dist/umd/popper.min",
"sentry_browser": "sentry/js/sentry.browser.7.28.0.min",
"sentry_captureconsole": "sentry/js/sentry.captureconsole.7.28.0.min",
"sortablejs": "sortablejs/Sortable.min",
"tempusDominus": "@eonasdan/tempus-dominus/dist/js/tempus-dominus.min",
"underscore": "underscore/underscore",
"nvd3/nv.d3.latest.min": "nvd3-1.8.6/build/nv.d3.min",
Expand Down
2 changes: 2 additions & 0 deletions corehq/apps/hqwebapp/static/hqwebapp/scss/commcarehq.scss
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
@import "commcarehq/ace_cle";
@import "commcarehq/appcues";
@import "commcarehq/backgrounds";
@import "commcarehq/data_cleaning";
@import "commcarehq/datagrid";
@import "commcarehq/datatables";
@import "commcarehq/date_range_picker";
Expand All @@ -76,6 +77,7 @@
@import "commcarehq/form_steps";
@import "commcarehq/helpbubble";
@import "commcarehq/hubspot";
@import "commcarehq/htmx";
@import "commcarehq/icons";
@import "commcarehq/inline_edit";
@import "commcarehq/label";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
.dc-spaces {
opacity: .5;
background-color: transparent;
font-weight: bold;
font-size: $font-size-base * 1.5;
line-height: $font-size-base * 1.5;
position: relative;
vertical-align: middle;
transition: .5s ease-in-out background-color;

&::before {
position: absolute;
top: 0;
left: 0;
font-size: $font-size-lg;
display: none;
}

&:hover {
cursor: help;
color: $white;
background-color: rgba(0, 0, 0, .5);
opacity: 1.0;
}
}

.dc-value {
border: 1px solid rgba(0, 0, 0, .1);
display: inline-block;
font-family: $font-family-monospace;
letter-spacing: 1px;
line-height: 12px;
padding: $spacer *.25 $spacer *.5;
vertical-align: middle;
}

.dc-diff-view {
font-family: $font-family-monospace;
letter-spacing: 1px;
}

.dc-diff-before,
.dc-diff-after {
border: 1px solid white;
display: inline-block;
margin: $spacer *.13;
padding-right: $spacer *.5;
vertical-align: middle;

&::before {
display: inline-block;
color: $white;
line-height: 12px;
padding: $spacer *.25 $spacer *.5;
}
}

.dc-diff-before {
background-color: $red-100;
border-color: $red-500;

&::before {
background-color: $red-500;
content: "-";
}

.dc-spaces {
color: $red-800;

&:hover {
color: $white;
}
}
}

.dc-diff-after {
background-color: $green-100;
border-color: $green-500;

&::before {
background-color: $green-500;
content: "+";
}

.dc-spaces {
color: $green-800;
}
}

.dc-null-value {
display: block;
position: relative;

.dc-info {
background-color: $gray-800;
color: $white;
font-weight: bold;
position: absolute;
left: 0;
top: 0;
opacity: 0;
padding: 0 $spacer * .25;
transition: opacity .5s ease-in-out;
}
&:hover .dc-info {
display: block;
opacity: 1;
}
}

.dc-diff-before .dc-null-value,
.dc-diff-after .dc-null-value {
display: inline-block;

.dc-info {
top: -2px;
left: -9px;
font-size: $font-size-sm;
padding: 3px 6.5px;
}
}

.dc-diff-before .dc-null-value {
color: $red-500;

.dc-info {
background-color: $red-500;
}
}

.dc-diff-after .dc-null-value {
color: $green-500;

.dc-info {
background-color: $green-500;
}
}
72 changes: 72 additions & 0 deletions corehq/apps/hqwebapp/static/hqwebapp/scss/commcarehq/_htmx.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
@mixin htmx-spinner($width) {
position: absolute;
height: $width;
width: $width;
$offset: $width / 2;
left: calc(50% - $offset);
top: calc(50% - $offset);
}

form.htmx-request {
.htmx-loading {
position: relative;
&::before {
content: " ";
@extend .spinner-border;
@extend .spinner-border-sm;
@include htmx-spinner(22px);
}
&.btn-sm::before {
@include htmx-spinner(18px);
}
}
.input-group .btn.htmx-loading::before {
vertical-align: -0.28em;
}
}

.htmx-request.page-link {
background-color: $blue-300;
color: $white;
position: relative;

&::after {
content: " ";
@extend .spinner-border;
@extend .spinner-border-sm;
@include htmx-spinner(22px);
}
}

.htmx-request[type="checkbox"] {
position: relative;

&::after {
content: " ";
color: $blue-400;
@extend .spinner-border;
@extend .spinner-border-sm;
@include htmx-spinner(22px);
}
}

.htmx-request.form-check-input {
&::after {
@include htmx-spinner(30px);
}
}

button.htmx-request {
position: relative;
&::before {
content: " ";
@extend .spinner-border;
@extend .spinner-border-sm;
@include htmx-spinner(22px);
opacity: .5;
}

&.btn-sm::before {
@include htmx-spinner(18px);
}
}
Loading
Loading