CORE-17387: Merging forward updates from release/os/5.1 to release/interop/syntax - 2023-10-03 #4769
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![corda-jenkins-ci02[bot]](https://avatars.githubusercontent.com/u/22600631?s=60&v=4){kind=small}
A few improvements to the pre-install check: - Handle null when looking up secret - Output single-line resource check info messages - Always check both memory and CPU resources - Update developer values.yaml and values-prereqs.yaml to disable the preinstall check as they do not specify resources (as is the preference in a low-resource environment)
* Use Jackson object mapper to correctly format the vNodeCreate body
Setting minVersion to 5.1 for network endpoints that were introduced in 5.1 and didn't have one, so they were falling back to the default (5.0).
Issue found during resiliency testing: Registration with ID ‘ce42453a-c6c8-4e67-9711-e28cbbe703e9’ was received by MGM, however due to the cache being empty after a pod start-up, the processing stopped at RegistrationRequestHandler with Could not find member info exception. As part of this ticket I also added some additional fix to wait for member list processor being ready before starting the synchronization service.
… received and the mapper state is null (#4650) Adds the ability to send a session error event back to the counterparty if the session is unknown when receiving a data event. The flow mapper could receive a session data event with no session init information included. Ordinarily this is a normal part of the flow message exchange. However, if the session is unknown to the flow mapper then this is an error, as the mapper cannot route the message successfully. Previously the mapper would attempt to forward these messages anyway and fail. With this change, the mapper will return an error to the counterparty for inbound messages, and drop the event with a warning for outbound ones.
Change the failing integration test to be time insensitive by getting the timestamps from the persisted entities directly
…2e-tests repo (#4654) Introduce utilities needed for e2e testing in order to migrate some e2e tests into the e2e-tests repo (MultiClusterDynamicNetworkTest and SessionCertificateTest). As part of this, I extracted disabling the certificate revocation checks from the onboardMember / onboardMgm functions, so that it can be executed along with any other config changes needed within the test. Otherwise, this could revert any changes on the same section performed by within the test (e.g. when trying to change tls mode, which is in the same gateway section as revocation checks). I also removed usage of any proxy utils that are targeted for removal/deprecation from e2e testing and stuck to using the existing utilities (this meant I had to introduce a couple of new functions that were not available).
- Integration of a HTTP RPC server into the Uniqueness processor. - Smoketests include a test for the service outside the scope of a flow. - Bunch of fixes/changes to the JavalinServer implementation. - No integration of a client for this service yet, this will come in a subsequent change. --------- Co-authored-by: Emily Bowe <[email protected]>, mbrkic-r3
…4663) * Improve documentation for SandbxGroupContextServiceImpl. * Change broken() -> asBroken().
…processor-service for DBProcessor (#4674)
…ould report an error back to the flow (#4673) Before this change, flows would hang whenever there was an issue in the token selection service. This fix ensure that the flow status changes to FAILURE if an error is raised by the token selection service.
Disable UniquenessCheckerRPCSmokeTests as it started failing consistently recently, until it can be fixed.
* Uses the deduplication table (idempotency table) to deduplicate `PersistEntities` (coming from `PersistenceService.persist`) * Moves `EntityManager.transaction` around specific `EntityRequest`s to allow more fine-grained error control * Changes `PersistenceExceptionTests`: * uses `BeforeEach` handler as sandbox context needs resetting * use `VirtualNodeLoader` instead of `VirtualNodeService` in test class, because it seems `VirtualNodeService` creates (in memory) DB with name X but then we would also use `FakeDbConnectionManager` which creares a different (in memory) DB with name Y, so different DBs
… TransactionSignatureService (#4666) CORE-17166 Adding cpiSummary.filecheckSum to DigitalSignatureMetadata in TransactionSignatureService, because that is also required to identify the running CPI.
…4670) Deleting tests migrated to e2e-tests repo and migrate two remaining tests into the smoke test suite. I've deleted the tests that were testing p2p connectivity via Kafka, since they were disabled until direct access to Kafka was enabled (which is not in any plans at the moment) and connectivity is already tested via flows now in other tests. I've also switched the scripts in `app-simulator` to use 5.1 version, because as they stood they were failing with the following: ``` Release "corda" does not exist. Installing it now. Error: values don't meet the specifications of the schema(s) in the following chart(s): corda: - bootstrap: Additional property preinstallCheck is not allowed ```
#4624) CORE-17154 - Enable state manager to be configured as part of worker deployment. - New --stateManager boot option for all workers to configure a state manager connection. - State manager configuration is optional for the combined worker and defaults to the cluster database. - Worker helm template supports deployment of workers with state manager config, reading configuration from values.yaml. - state manager boot configuration provided at worker startup is validated against boot config json schema. - STATE_MANAGER schema is deployed as part of application db setup. - Refactor getBootstrapConfig so we can supply extraConfig, and we can supply a list of keys in the config that must be converted to Integer before boot config schema validation. - This change does not deploy an isolated database for the state manager. Co-authored-by: Juan José Ramos Cassella <[email protected]>
Adding some more information in the logs shown in the MGM side when a member provides a wrong serial number, e.g. to include the submitted serial and the latest one.
… when api is called. (#4676) Send a counterparty info request to start a session if no data is sent before requesting counterparty info. It is possible when starting a session with a counterparty to request counterparty information before sending the first data message. Previously if this happened with the new session protocol for 5.1, the flow would fail. This PR introduces specific messages to handle this case in the counterparty info request and response message types. These will now be sent if this condition is met. --------- Co-authored-by: James Higgs <[email protected]>
Fix `isCombinedWorker` parsing logic. - Swap default to be combined worker when running locally. - Rename properties to be more generic. - RE-enable unstable test.
* Introduces housekeeping functionality for the deduplication table as a `Scheduler` task * Adds delete query for "old" request ids * Adds clean up functionality in a Kafka consumer (`DurableProcessor`) to consume from the `Scheduler` * Adds the `Scheduler` to the `DBWorker`
…he state output table (#4680) The NotaryX500Name is being stored again in the DB.
As part of the multi-source event mediator implementation, we will have a new subscription which should be able to send messages (NB: non-transactionally, as of today) to endpoints which may be Kafka, DB or RPC; the subscription code should be agnostic of the implementation of these endpoints, and so we now require a more generic messaging system which isn't tied to Kafka. --------- Co-authored-by: Chris Rankin <[email protected]> Co-authored-by: Miljenko Brkic <[email protected]>
corda-jenkins-ci02
bot
requested review from
YashNabar,
williamvigorr3,
BenYip123,
ronanbrowne,
mbrkic-r3,
conalsmith-r3,
JamesHR3,
vkolomeyko,
blsemo,
emilybowe,
bpaunescu,
charlieR3,
josephzunigadaly,
anton-subbotin,
jujoramos,
dimosr,
kyriathar,
vlajos,
davidcurrie,
jennyang-r3,
filipesoliveira,
malachyb,
relyafi,
owenstanford,
ben-millar,
LWogan,
yift-r3 and
nikinagy
ac101m
deleted the
merge-release/os/5.1-release/interop/syntax-2023-10-03-84
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR was created by the merge bot.
Files with conflicts to resolve manually:
components/flow/flow-mapper-impl/src/main/kotlin/net/corda/flow/mapper/impl/RecordFactoryImpl.ktcomponents/flow/flow-mapper-impl/src/main/kotlin/net/corda/flow/mapper/impl/executor/SessionEventExecutor.ktcomponents/flow/flow-mapper-impl/src/main/kotlin/net/corda/flow/mapper/impl/executor/SessionInitExecutor.ktcomponents/flow/flow-mapper-impl/src/test/kotlin/net/corda/flow/mapper/impl/executor/SessionEventExecutorTest.ktcomponents/flow/flow-mapper-impl/src/test/kotlin/net/corda/flow/mapper/impl/executor/SessionInitExecutorTest.ktcomponents/flow/flow-mapper-impl/src/test/kotlin/net/corda/flow/mapper/impl/executor/SessionInitProcessorTest.ktcomponents/flow/flow-service/build.gradlecomponents/flow/flow-service/src/main/kotlin/net/corda/flow/service/FlowService.ktcomponents/flow/flow-service/src/test/kotlin/net/corda/flow/pipeline/impl/FlowGlobalPostProcessorImplTest.ktcomponents/flow/flow-service/src/test/kotlin/net/corda/flow/service/FlowServiceTest.ktgradle.propertieslibs/flows/flow-mapper-impl/src/integrationTest/kotlin/net/corda/flow/mapper/integration/FlowMapperIntegrationTest.ktlibs/flows/session-manager-impl/src/main/kotlin/net/corda/session/manager/impl/processor/SessionCounterpartyInfoRequestProcessorSend.ktlibs/flows/session-manager-impl/src/main/kotlin/net/corda/session/manager/impl/processor/SessionInitProcessorReceive.ktprocessors/flow-mapper-processor/build.gradleprocessors/flow-processor/build.gradleIncludes:
-pfvoption inVerifyCLI plugin (CORE-15064: Rename-pfvoption inVerifyCLI plugin #4749)-voption inVerifyCLI plugin (CORE-15064: Rename-voption inVerifyCLI plugin #4732)Schedulererror goingDOWNprevents k8s replacing the pod (CORE-17367 OnSchedulererror goingDOWNprevents k8s replacing the pod #4714)mixinStandardHelpOptions = true(CORE-15064: SetmixinStandardHelpOptions = true#4711)