Skip to content

Commit

Permalink
bring FLEx lite poc into lexbox (#811)
Browse files Browse the repository at this point in the history
* add local web app

* setup LocalWebApp.csproj to run itself

* auto launch browser on startup. Enabled published trimmed to reduce size

* add local web app publish scripts to taskfile

* Show part of speech and semantic domains (not finished)

* Fix option/select fields. Add filter field.

* Redesign layout. Put view options in drawer

* Tidy up app header

* add build app to viewer package

* Add readonly mode. Redesign dictionary preview.

* Hacky search-bar centralizing

* Improve dictionary entry colors

* Fix: changes not being commited to model

* add LcmCrdt lib

* link viewer to local webapp for data, use crdt as source

* make main not async for browser builds. Add missing sqlite dependency

* add task to startup local web app dev

* fix bug due to blur being called twice

* fix multistring so it can be the target of JsonPatches as if it were a dictionary

* fix `CLR load callback is already in error state. Error HRESULT E_FAIL has been returned from a call to a COM component. The error code is E_FAIL, or EFail, or 0x80004005.` error by disabling build options in debug mode

* change how the Entry editor produces change events to help with generating diffs.

* implement field updates

* enable lexbox to act as a crdt sync server

* enable local web app to sync with lexbox server.

* make initial entry reactive

* add create and update apis for writing systems

* promote writing system to a fully fledged crdt object

* add create/update ws methods to frontend interfaces and lexbox api hub

* sync with lexbox when signalr connects

* enable sync to be optional and fail gracefully if we're unable to access the server

* make WritingSystemId read only

* Fix some styles

* refactor writing system into it's own file, fix warning on MultiString

* refactor code to introduce projects

* add index to ChangeEntry to order changes correctly

* move commits and changes into crdt core

* pull in CRDT tests and sample project, including Ycs

* store crdt data in postgres by project id on server side

* change sync state timestamp to be unix milliseconds instead of ticks

* setup lf next app routing to select the project, refactor WebComponent to depend on ProjectView which expects the lexboxApi to be injected in order to run.

* Fix some web-component issues

* Add basic infinite scrolling (not virtual scrolling)

* remove aspnet dependency from LcmCrdt

* Fix overlays

* restrict project creation api to only use alphanumerics

* change default service provider to none, create testing specific project component

* prevent creating snapshots that reference the same commit and entity

* enable creating new entries on the frontend

* fix issue where deleted entities would break the snapshot update logic trying to recreate each change made to the deleted entity

* simplify snapshot worker so it only works on commits starting with the first one added

* ensure changes and commits are loaded to prevent null references errors in update snapshots

* Add index exemplars and some design work

* Add new/delete/reorder buttons and scroll to added entitiy

* remove create event from editor since we want to only create senses and examples once we have data in the fields. Introduce an emptyId to detect when we need to create a new sense or example from a change

* update lexbox api hub to use JsonPatchDocument instead of a simple Operations record

* remove GetExemplars from client side api since it was removed in favor of putting exemplars on writingsystems instead

* remove some debug logs

* wireup delete buttons, fix some selected entry bugs where the entry wouldn't be selected after created, or would stay selected after being deleted

* define latin exemplars by default on writing systems

* remove api to get entries by exemplar and use query options instead

* implement sorting on headword in crdts, implementing filtering entries on headword exemplar

* make some notes about using IMemoryCache

* create LcmCrdt.Tests.csproj and move related tests into the project

* update LfClassic api handle new query options with exemplar options

* fix verify encoding and line endings in editorconfig and gitattributes

* use harmony from submodule

* remove field help icon from tabindex

* begin adding history view

* Suppress svlte/vite/rollup warnings in our dependencies we can't control

* Implement reordering. Toc design. Use headwords. Fix bubbling changes into dictionary-preview and Toc.

* Implement global search bar

* rename route builder class for crdt sync

* list change history for selected object.
add sense and example history support.

* include commits when loading snapshot to prevent null reference error

* Fix history view layout for senses and examples

* Make progress towards mobile view. Dictionary preview mode

* Clean up confusing icons/wording

* Make usable on mobile

* set tab title to 'FLEx lite'

* fix snapshot querying for history to use the exact timestamp to avoid issue with rounding down

* add simple activity view

* refactor routes into routes folder

* remove ProjectHubActivator.cs because it's no longer needed because of a middleware that's configuring the project context

* add commit metadata

* lock down sync api in lexbox to only be available to admins for now

* update pnpm-lock.yaml

* remove use of INewableObject since it's removal

* add sub module checkout during build of backend

* add xunit test logger, ensure that there test output files are distinct between assemblies

* make search filter on lexeme form and citation form

* store client id in ProjectData, use in CrdtLexboxApi to maintain a consistent client id on a single client.

* mock ProjectContext for tests

* migrate LcmCrdt to use new Create and Edit Change classes

* remove task delay in background sync

* remove exemplar options default

---------

Co-authored-by: Tim Haasdyk <[email protected]>
  • Loading branch information
hahn-kev and myieye authored May 28, 2024
1 parent fb43c1c commit 05f928d
Show file tree
Hide file tree
Showing 127 changed files with 6,484 additions and 743 deletions.
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,7 @@
# force LF line endings on all shell scripts, even on Windows
*.sh text eol=lf

# Verify settings
*.verified.txt text eol=lf working-tree-encoding=UTF-8
*.verified.xml text eol=lf working-tree-encoding=UTF-8
*.verified.json text eol=lf working-tree-encoding=UTF-8
2 changes: 2 additions & 0 deletions .github/workflows/integration-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ jobs:
runs-on: ${{ inputs.runs-on }}
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: testspace-com/[email protected]
with:
domain: ${{ github.repository_owner }}
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/lexbox-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,15 @@ jobs:

steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.x'
- name: Dotnet build
run: dotnet build
- name: Unit tests
run: dotnet test --logger xunit --results-directory ./test-results --filter "Category!=Integration" --blame-hang-timeout 10m
run: dotnet test --logger:"xunit;LogFileName={assembly}.results.xml" --results-directory ./test-results --filter "Category!=Integration" --blame-hang-timeout 10m
- name: Publish unit test results
uses: EnricoMi/publish-unit-test-result-action@8885e273a4343cd7b48eaa72428dea0c3067ea98 # v2.14.0
if: always()
Expand Down
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "backend/harmony"]
path = backend/harmony
url = https://github.com/hahn-kev/harmony.git
branch = add-crdt
35 changes: 35 additions & 0 deletions LexBox.sln
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LfClassicData", "backend\Lf
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "MiniLcm", "backend\MiniLcm\MiniLcm.csproj", "{6D4062DC-1B1A-4A24-9C61-0F6A18A39882}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LocalWebApp", "backend\LocalWebApp\LocalWebApp.csproj", "{2CDF7126-733B-46E8-9B93-7DC31DE39682}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LcmCrdt", "backend\LcmCrdt\LcmCrdt.csproj", "{6B857F83-72E3-43E6-91C3-A946F3F988D8}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LcmCrdt.Tests", "backend\LcmCrdt.Tests\LcmCrdt.Tests.csproj", "{7D874D9B-1CA9-49E9-8B03-91B5C324E938}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Crdt", "backend\harmony\src\Crdt\Crdt.csproj", "{740C8FF5-8006-4047-8C52-53873C2DD7C4}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Crdt.Core", "backend\harmony\src\Crdt.Core\Crdt.Core.csproj", "{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -65,9 +75,34 @@ Global
{6D4062DC-1B1A-4A24-9C61-0F6A18A39882}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6D4062DC-1B1A-4A24-9C61-0F6A18A39882}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6D4062DC-1B1A-4A24-9C61-0F6A18A39882}.Release|Any CPU.Build.0 = Release|Any CPU
{2CDF7126-733B-46E8-9B93-7DC31DE39682}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2CDF7126-733B-46E8-9B93-7DC31DE39682}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2CDF7126-733B-46E8-9B93-7DC31DE39682}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2CDF7126-733B-46E8-9B93-7DC31DE39682}.Release|Any CPU.Build.0 = Release|Any CPU
{6B857F83-72E3-43E6-91C3-A946F3F988D8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6B857F83-72E3-43E6-91C3-A946F3F988D8}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6B857F83-72E3-43E6-91C3-A946F3F988D8}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6B857F83-72E3-43E6-91C3-A946F3F988D8}.Release|Any CPU.Build.0 = Release|Any CPU
{7D874D9B-1CA9-49E9-8B03-91B5C324E938}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{7D874D9B-1CA9-49E9-8B03-91B5C324E938}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7D874D9B-1CA9-49E9-8B03-91B5C324E938}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7D874D9B-1CA9-49E9-8B03-91B5C324E938}.Release|Any CPU.Build.0 = Release|Any CPU
{740C8FF5-8006-4047-8C52-53873C2DD7C4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{740C8FF5-8006-4047-8C52-53873C2DD7C4}.Debug|Any CPU.Build.0 = Debug|Any CPU
{740C8FF5-8006-4047-8C52-53873C2DD7C4}.Release|Any CPU.ActiveCfg = Release|Any CPU
{740C8FF5-8006-4047-8C52-53873C2DD7C4}.Release|Any CPU.Build.0 = Release|Any CPU
{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Release|Any CPU.ActiveCfg = Release|Any CPU
{8B54FFB5-0BDF-403E-83CC-A3B3861EC507}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(NestedProjects) = preSolution
{E8BB768B-C3DC-4BE6-9B9F-82319E05AF86} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{6D4062DC-1B1A-4A24-9C61-0F6A18A39882} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{2CDF7126-733B-46E8-9B93-7DC31DE39682} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{6B857F83-72E3-43E6-91C3-A946F3F988D8} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{7D874D9B-1CA9-49E9-8B03-91B5C324E938} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{740C8FF5-8006-4047-8C52-53873C2DD7C4} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
{8B54FFB5-0BDF-403E-83CC-A3B3861EC507} = {7B6E21C4-5AF4-4505-B7D9-59A3886C5090}
EndGlobalSection
EndGlobal
3 changes: 2 additions & 1 deletion Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,4 +84,5 @@ tasks:
deps: [ infra-up, api:only, k8s:infra-forward ]
interactive: true


local-web-app:
deps: [ ui:viewer-dev, api:local-web-app ]
13 changes: 12 additions & 1 deletion backend/.editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -135,4 +135,15 @@ dotnet_diagnostic.VSTHRD200.severity = none
dotnet_diagnostic.IDE0058.severity = suggestion

[{*Kernel}.cs]
dotnet_diagnostic.IDE0058.severity = none
dotnet_diagnostic.IDE0058.severity = none


# Verify settings
[*.{received,verified}.{txt,xml,json}]
charset = "utf-8-bom"
end_of_line = lf
indent_size = unset
indent_style = unset
insert_final_newline = false
tab_width = unset
trim_trailing_whitespace = false
31 changes: 31 additions & 0 deletions backend/LcmCrdt.Tests/LcmCrdt.Tests.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>

<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="coverlet.collector" Version="6.0.0"/>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0"/>
<PackageReference Include="FluentAssertions" Version="6.12.0"/>
<PackageReference Include="xunit" Version="2.5.3"/>
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.3"/>
<PackageReference Include="XunitXml.TestLogger" Version="3.1.20" />
</ItemGroup>

<ItemGroup>
<Using Include="Xunit"/>
<Using Include="FluentAssertions"/>
<Using Include="Crdt"/>
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\LcmCrdt\LcmCrdt.csproj" />
</ItemGroup>

</Project>
Loading

0 comments on commit 05f928d

Please sign in to comment.