enable importing fwdata projects as crdts (#863)

* use async scopes when creating project scopes to allow disposing of IAsyncDisposable services

* implement fwdata file import to FW lite, switch to new performance pass branch in harmony

* make CreateEntryLite method to improve import performance, tweak some queries to avoid inlining parameters in select Contains queries

* show fieldworks projects in project list, allow importing them by clicking on the project.

* remove sena-3 automatic project creation

* correct import parameter expectations

* change tests to use async service scope

* fix a bug where proxy updates wouldn't work on multi strings

* rename and move LexboxApiHub.cs to CrdtMiniLcmApiHub.cs

* allow editing fwdata files directly without importing them

* cache LcmCache project service between requests to avoid loading it each time, also ensure it's disposed properly on server shutdown.

* don't log user provided value

* refactor db init in tests to avoid wiping out the in memory db. Fix compile error due to merge conflict in CrdtLexboxApi.cs

* use application lifetime to cancel oauth tasks to prevent the app from hanging when a login request is hung

* change default authority to develop for now

* fix system threading package version mismatch

* init Icu on startup for dev, add an Icu binary project reference

* move crdt sync in lexbox into a controller, update the path in the sync client. Define api to list CRDT projects.

* rework project list, list projects from lexbox.

* run vite over https when running out of docker. Fix some forwarding issues so that dotnet gets the right port and scheme. enable oauth when lexbox is running in k8s.

* make list lexbox projects work while offline

* remove required crdt commit hash fields

* wire up project upload button from home page. This will upload a CRDT project to lexbox assuming the local project name matches the lexbox project code.

* update harmony to fix a sync bug related to commits getting synced when they are already in the remote

* setup download crdt project workflow

* local-web-app and viewer fixups and styling

* Fix invalid cast

* Fix: changes are diffed against the first viewed or last changed entry

* Set correct api features

* Adds https-proxy as oauth authority for local or k8s

* Add fwdata support for example sentnce translations
I think there's typically only 1, but there's a Find example sentence feature in FLEx that copies more from other sources. With my change, I think we safely support editing the first translation.

* enable https in k8s

* Use correct writing-system type in crd-api query

* Add exemplars and implementing querying for FwData API

* Update svelte-ux so the app doesn't crash when you navigate away from the viewer

* fix lockfile conflicts

* catch auth errors when trying to acquire token silently so we don't take down the app when offline.

* fix unit tests failing because there was no vernacular writing system

* fix `pnpm run -r build` faliing by removing build script from https-proxy package.json

* fix security issue caused by passing the project file name directly instead of looking up the project in the project list.

* sanitize create project name

---------

Co-authored-by: Tim Haasdyk <[email protected]>

.gitmodules

@@ -1,4 +1,4 @@
  [submodule "backend/harmony"]
 	path = backend/harmony
 	url = https://github.com/hahn-kev/harmony.git
-	branch = add-crdt
+	branch = chore/performance-pass

LexBox.sln

@@ -37,6 +37,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Crdt", "backend\harmony\src
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Crdt.Core", "backend\harmony\src\Crdt.Core\Crdt.Core.csproj", "{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}"
 EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FwDataMiniLcmBridge", "backend\FwDataMiniLcmBridge\FwDataMiniLcmBridge.csproj", "{279197B6-EC06-4DE0-94F8-625379C3AD83}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -95,6 +97,10 @@ Global
 		{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Release|Any CPU.Build.0 = Release|Any CPU
+		{279197B6-EC06-4DE0-94F8-625379C3AD83}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{279197B6-EC06-4DE0-94F8-625379C3AD83}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{279197B6-EC06-4DE0-94F8-625379C3AD83}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{279197B6-EC06-4DE0-94F8-625379C3AD83}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(NestedProjects) = preSolution
 		{E8BB768B-C3DC-4BE6-9B9F-82319E05AF86} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
@@ -104,5 +110,6 @@ Global
 		{7D874D9B-1CA9-49E9-8B03-91B5C324E938} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
 		{740C8FF5-8006-4047-8C52-53873C2DD7C4} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
 		{8B54FFB5-0BDF-403E-83CC-A3B3861EC507} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
+		{279197B6-EC06-4DE0-94F8-625379C3AD83} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
 	EndGlobalSection
 EndGlobal

Taskfile.yml

@@ -36,6 +36,7 @@ tasks:
       - echo "GOOGLE_OAUTH_CLIENT_ID=__REPLACE__.apps.googleusercontent.com" >> deployment/local-dev/local.env
       - echo "GOOGLE_OAUTH_CLIENT_SECRET=__REPLACE__" >> deployment/local-dev/local.env
       - kubectl --context=docker-desktop apply -f deployment/setup/namespace.yaml
+      - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.0/cert-manager.yaml
       - docker build -t local-dev-init data/
   setup-win:
     platforms: [ windows ]
@@ -78,5 +79,8 @@ tasks:
     deps: [ infra-up, api:only, k8s:infra-forward ]
     interactive: true
 
+  local-web-app-for-develop:
+    deps: [ ui:viewer-dev, api:local-web-app-for-develop, ui:https-oauth-authority ]
+
   local-web-app:
-    deps: [ ui:viewer-dev, api:local-web-app ]
+    deps: [ ui:viewer-dev, api:local-web-app, ui:https-oauth-authority ]