Trying to sync sena 3 twice via fw headless results in an error

[hahn-kev]

Describe the bug
I was testing out calling fw headless sync and if I call it twice for sena 3, the second time I get the following error:

   System.InvalidOperationException: Complex form component not found, searched for diso - LexEntry 9331
         at FwDataMiniLcmBridge.Api.FwDataMiniLcmApi.RemoveComplexFormComponent(ILexEntry lexEntry, ComplexFormComponent component) in /src/backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs:line 681
         at FwDataMiniLcmBridge.Api.FwDataMiniLcmApi.<>c__DisplayClass85_0.<DeleteComplexFormComponent>b__0() in /src/backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs:line 622
         at SIL.LCModel.Infrastructure.UndoableUnitOfWorkHelper.Do(String undoText, String redoText, IActionHandler actionHandler, Action task)
         at SIL.LCModel.Infrastructure.UndoableUnitOfWorkHelper.DoUsingNewOrCurrentUOW(String undoText, String redoText, IActionHandler actionHandler, Action task)
         at FwDataMiniLcmBridge.Api.FwDataMiniLcmApi.DeleteComplexFormComponent(ComplexFormComponent complexFormComponent) in /src/backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs:line 616
         at MiniLcm.SyncHelpers.EntrySync.<>c.<<Sync>b__3_2>d.MoveNext() in /src/backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs:line 86
      --- End of stack trace from previous location ---
         at MiniLcm.SyncHelpers.DiffCollection.Diff[T,TId](IMiniLcmApi api, IList`1 before, IList`1 after, Func`2 identity, Func`3 add, Func`3 remove, Func`4 replace) in /src/backend/FwLite/MiniLcm/SyncHelpers/DiffCollection.cs:line 26
         at MiniLcm.SyncHelpers.EntrySync.Sync(IList`1 afterComponents, IList`1 beforeComponents, IMiniLcmApi api) in /src/backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs:line 64
         at MiniLcm.SyncHelpers.EntrySync.Sync(Entry afterEntry, Entry beforeEntry, IMiniLcmApi api) in /src/backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs:line 34
         at MiniLcm.SyncHelpers.EntrySync.<>c.<<Sync>b__0_2>d.MoveNext() in /src/backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs:line 23
      --- End of stack trace from previous location ---
         at MiniLcm.SyncHelpers.DiffCollection.Diff[T,TId](IMiniLcmApi api, IList`1 before, IList`1 after, Func`2 identity, Func`3 add, Func`3 remove, Func`4 replace) in /src/backend/FwLite/MiniLcm/SyncHelpers/DiffCollection.cs:line 22
         at MiniLcm.SyncHelpers.DiffCollection.Diff[T](IMiniLcmApi api, IList`1 before, IList`1 after, Func`3 add, Func`3 remove, Func`4 replace) in /src/backend/FwLite/MiniLcm/SyncHelpers/DiffCollection.cs:line 47
         at MiniLcm.SyncHelpers.EntrySync.Sync(Entry[] afterEntries, Entry[] beforeEntries, IMiniLcmApi api) in /src/backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs:line 24
         at FwLiteProjectSync.CrdtFwdataProjectSyncService.Sync(IMiniLcmApi crdtApi, IMiniLcmApi fwdataApi, Boolean dryRun, Int32 entryCount, ProjectSnapshot projectSnapshot) in /src/backend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.cs:line 67
         at FwLiteProjectSync.CrdtFwdataProjectSyncService.Sync(IMiniLcmApi crdtApi, FwDataMiniLcmApi fwdataApi, Boolean dryRun) in /src/backend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.cs:line 24
         at Program.<<Main>$>g__ExecuteMergeRequest|0_1(ILogger`1 logger, IServiceProvider services, SendReceiveService srService, IOptions`1 config, FwDataFactory fwDataFactory, ProjectsService projectsService, ProjectLookupService projectLookupService, CrdtFwdataProjectSyncService syncService, CrdtHttpSyncService crdtHttpSyncService, IHttpClientFactory httpClientFactory, Guid projectId, Boolean dryRun) in /src/backend/FwHeadless/Program.cs:line 107
         at Microsoft.AspNetCore.Http.RequestDelegateFactory.ExecuteTaskResult[T](Task`1 task, HttpContext httpContext)
         at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|7_0(Endpoint endpoint, Task requestTask, ILogger logger)
         at Program.<>c.<<<Main>$>b__0_0>d.MoveNext() in /src/backend/FwHeadless/Program.cs:line 36

This is assuming you've never synced this project before.

To Reproduce

1. make sure you don't have any crdt commits for sena 3
2. execute the sync endpoint for sena 3
3. execute it again, there's an error when there should be 0 changes

Expected behavior
no error and a result saying there's no changes (considering we just synced)

[rmunn]

Integration test will:

• Create new project in Lexbox
• Download sena-3.zip backup file from the sena-3 project
• Upload sena-3.zip backup to the new project (via the FinishReset API) to seed it
• Have FwHeadless sync that project once
• Have FwHeadless sync that project a second time
• Verify that 0 changes happened on second sync and it did not throw

[rmunn]

Now I'm getting the same exception. At a first guess, this might be a problem of operation ordering: complex forms needs to be added after their component parts. But I'll start by writing the test so that we can prove that that's the issue.

[rmunn]

Discussed with @hahn-kev, we're going to solve this with a transaction system where updates can be batched and added in a single commit. That way the compoent and its parts would have all been added together and there wouldn't be a "missing component part" error. I'll just focus on writing the test, and then @hahn-kev will implement the transaction system to make the test pass.

[hahn-kev]

I realized last night while thinking about this that transactions aren't really going to help, though they might make it easier to fix, ultimately a create entry call is going to try to make all the changes at once (complex forms etc), and that design is always going to be difficult. So I'm starting to wonder if we need to split the create entry calls out so that they don't try to create complex form references, and then once we've created all new entries, then we can apply the rest of the entry data. For example a change log might look like this:

1. create entry 1
2. create entry 2
3. create entry 3...
4. create entry 1 complex forms and components
5. create entry 2 complex forms and components...

[hahn-kev]

fixed in #1256