Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA Exec Pipeline #897

Open
wants to merge 59 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
8fc7014
Refactor exec job
dotasek May 10, 2024
d15635c
Add condition for qa publishing
dotasek May 10, 2024
beee62a
Add expected qa outputs + use commit hash
dotasek May 14, 2024
b707f5b
Publish json artifact
dotasek May 14, 2024
1403353
Try another character escape
dotasek May 14, 2024
9a4c1bf
First stab at waiting for all qa jobs
dotasek May 14, 2024
8f77eb5
First stab at waiting for all qa jobs 2
dotasek May 14, 2024
1d8820d
Replace illegal job name characters
dotasek May 14, 2024
16b897e
The other illegal character
dotasek May 14, 2024
12226e6
Use ubuntu pool for final publish job
dotasek May 14, 2024
a2a97ae
Try again with different syntax
dotasek May 14, 2024
ecf2454
Bad parenthesis
dotasek May 14, 2024
c72c2a7
Use all artifacts download step
dotasek May 15, 2024
e05ad51
Get txt files too + check contents
dotasek May 15, 2024
e64b565
Check contents of pipeline workspace
dotasek May 15, 2024
a9f366c
Different syntax
dotasek May 15, 2024
2672e05
Publish txt artifacts + add directory path to artifacts
dotasek May 15, 2024
fd30d53
Drop artifacts in target directory
dotasek May 15, 2024
ca35f26
Download json artifacts too, make new target directory
dotasek May 15, 2024
aac7557
Copy expected qa files
dotasek May 15, 2024
eba2670
Verify our created directories have our artifacts
dotasek May 15, 2024
a26893d
Fix verify step
dotasek May 15, 2024
5e42241
Try final diff publish
dotasek May 16, 2024
7633d40
Some extra conditions for template re-use
dotasek May 16, 2024
6a53aa3
Combine txt and json packaging
dotasek May 16, 2024
6c4b306
Try different extension glob
dotasek May 16, 2024
c1bef4e
Rename qa files in first step
dotasek May 16, 2024
9f8feff
More copying and artifact matching
dotasek May 16, 2024
1220b34
Make parent directories
dotasek May 16, 2024
780bea6
Avoid using parenthesis in script
dotasek May 16, 2024
02a7852
Refactor templates 1
dotasek May 16, 2024
8215ac3
Refactor templates 2
dotasek May 16, 2024
3fae9a0
More refactor and restore original os execs
dotasek May 16, 2024
b94f48e
Fix missing params in display name
dotasek May 16, 2024
37a2936
Simplify qa jobs + better exec job name
dotasek May 16, 2024
2468570
Move params to each template
dotasek May 16, 2024
959b1d1
Start adding real IGs for QA
dotasek May 16, 2024
0b0dd64
Increase java mem for IGs
dotasek May 16, 2024
1f9e46e
Try another format for exec
dotasek May 16, 2024
cbf2813
Java specific exec
dotasek May 16, 2024
0703226
Add mainClass
dotasek May 16, 2024
ac890a8
Try optional 10G xmx on macos
dotasek May 16, 2024
1dec87f
Try again
dotasek May 16, 2024
187f807
Make macos explicit + even more memory
dotasek May 16, 2024
d52b454
Add final IGs for QA tests
dotasek May 16, 2024
5e6a9dd
Add ipa IG, add compare.txt qa file
dotasek May 17, 2024
a736674
Merge remote-tracking branch 'origin/master' into do-20240510-qa-enab…
dotasek May 30, 2024
4643283
Merge remote-tracking branch 'origin/master' into do-20240510-qa-enab…
dotasek Jun 20, 2024
9a1252a
Merge remote-tracking branch 'origin/master' into do-20240510-qa-enab…
dotasek Aug 28, 2024
7c8bfd6
Merge branch 'master' into do-20240510-qa-enabled-exec-pipeline
dotasek Oct 8, 2024
c648602
Remove unneeded template from master merge
dotasek Oct 8, 2024
f0e9fdd
Bump maven task version to 4
dotasek Oct 8, 2024
e247f66
Catch NullPointer
dotasek Oct 8, 2024
218cc28
Merge branch 'master' into do-20240510-qa-enabled-exec-pipeline
dotasek Dec 13, 2024
0f9706a
Make all qa.compare.txt files relative
dotasek Dec 13, 2024
fce8806
Merge remote-tracking branch 'origin/master' into do-20240510-qa-enab…
dotasek Dec 13, 2024
2224232
Merge remote-tracking branch 'origin/master' into do-20240510-qa-enab…
dotasek Dec 16, 2024
da9da1b
Merge branch 'master' into do-20240510-qa-enabled-exec-pipeline
dotasek Jan 2, 2025
f030173
Merge branch 'master' into do-20240510-qa-enabled-exec-pipeline
dotasek Jan 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .azure/qa-diff/expected/example.fhir.uv.myig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"url" : "http://somewhere.org/fhir/uv/myig/ImplementationGuide/example.fhir.uv.myig",
"name" : "YourComputableMyIGNameHere",
"title" : "Your User Friendly Name for MyIG Here",
"description" : "A brief description of what MyIG is about (probably the same text as in your readme)",
"status" : "draft",
"package-id" : "example.fhir.uv.myig",
"ig-ver" : "0.2.0",
"date" : "Sun, 12 May, 2024 07:09:00 +1000",
"dateISO8601" : "2024-05-12T07:09:00+10:00",
"errs" : 21,
"warnings" : 19,
"hints" : 3,
"suppressed-hints" : 1,
"suppressed-warnings" : 1,
"version" : "4.0.1",
"tool" : "5.0.0 (3)",
"maxMemory" : 3491051984
}
123 changes: 123 additions & 0 deletions .azure/qa-diff/expected/example.fhir.uv.myig.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
YourComputableMyIGNameHere : Validation Results
=========================================

err = 21, warn = 19, info = 3
$--
Generated $--. FHIR version $-- for example.fhir.uv.myig#$-- (canonical = http://somewhere.org/fhir/uv/myig)
Build Errors : 1 / 4 / 0
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/Binary-example : 0 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/Binary-image-example : 0 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/Library-example : 2 / 6 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/Observation-example : 10 / 2 / 3
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/Patient-example : 0 / 1 / 3
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/examples/binary-logical-example : 1 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/history/h1 : 0 / 1 / 2
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/myig : 4 / 1 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/ActorDefinition-example : 1 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/MyLogical : 1 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/codesystem-fake : 0 / 1 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/library-example-image : 0 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/library-example-sql : 0 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/myObservation : 0 / 4 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/valueset-fake : 0 / 0 / 1
/Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/resources/valueset-no-codesystem : 0 / 2 / 1
not relevant : 1 / 1 / 1
</table>

== n/a ==
ERROR: /Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/cql/notincluded.cql: CQL source was not associated with a library resource in the IG.
WARNING: 1: The HTML fragment 'ip-statements.xhtml' is not included anywhere in the produced implementation guide
WARNING: 2: An HTML fragment from the set [cross-version-analysis.xhtml, cross-version-analysis-inline.xhtml] is not included anywhere in the produced implementation guide
WARNING: 3: An HTML fragment from the set [dependency-table.xhtml, dependency-table-short.xhtml] is not included anywhere in the produced implementation guide
WARNING: 4: The HTML fragment 'globals-table.xhtml' is not included anywhere in the produced implementation guide


== input/examples/Binary-example.json ==


== input/examples/Binary-image-example.jpg ==


== input/examples/Library-example.json ==
WARNING: /Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/cql/example.cql: The function FHIRHelpers.ToString has multiple overloads and due to the SignatureLevel setting (None), the overload signature is not being included in the output. This may result in ambiguous function resolution at runtime, consider setting the SignatureLevel to Overloads or All to ensure that the output includes sufficient information to support correct overload selection at runtime.
WARNING: /Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/cql/example.cql: The function FHIRHelpers.ToInterval has multiple overloads and due to the SignatureLevel setting (None), the overload signature is not being included in the output. This may result in ambiguous function resolution at runtime, consider setting the SignatureLevel to Overloads or All to ensure that the output includes sufficient information to support correct overload selection at runtime.
WARNING: /Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/cql/example.cql: The function FHIRHelpers.ToDateTime has multiple overloads and due to the SignatureLevel setting (None), the overload signature is not being included in the output. This may result in ambiguous function resolution at runtime, consider setting the SignatureLevel to Overloads or All to ensure that the output includes sufficient information to support correct overload selection at runtime.
ERROR: Library/example: Library.relatedArtifact[6].resource: Canonical URLs must be absolute URLs if they are not fragment references (TBD)
ERROR: Library/example: Library.dataRequirement[4].codeFilter[0].valueSet: Canonical URLs must be absolute URLs if they are not fragment references (TBD)


== input/examples/Observation-example.xml ==
ERROR: Observation/example: Observation.code: As specified by profile http://somewhere.org/fhir/uv/myig/StructureDefinition/myObservation|0.2.0, Element 'code' is out of order (found after subject)
ERROR: Observation/example: Observation.code.coding[2].display: Wrong Display Name 'VS1' for http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01. Valid display is one of 2 choices: 'CS1' or 'CS2' (for the language(s) '--')
ERROR: Observation/example: Observation.code.coding[4].display: Wrong Display Name 'VS2' for http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01. Valid display is one of 2 choices: 'CS1' or 'CS2' (for the language(s) '--')
ERROR: Observation/example: Observation.code: None of the codings provided are in the value set 'ValueSet fake' (http://somewhere.org/fhir/uv/myig/ValueSet/valueset-fake|0.2.0), and a coding from this value set is required) (codes = http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01, http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01, http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01, http://loinc.org#85354-9, http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake#01)
WARNING: Observation/example: Observation: Best Practice Recommendation: In general, all observations should have a performer
WARNING: Observation/example: Observation: Best Practice Recommendation: In general, all observations should have an effective[x] ()
ERROR: Observation/example: Observation.code: As specified by profile http://hl7.org/fhir/StructureDefinition/bp|4.0.1, Element 'code' is out of order (found after subject)
ERROR: Observation/example: Observation: Observation.category: minimum required = 1, but only found 0 (from http://hl7.org/fhir/StructureDefinition/bp|4.0.1)
ERROR: Observation/example: Observation: Slice 'Observation.category:VSCat': a matching slice is required, but not found (from http://hl7.org/fhir/StructureDefinition/bp|4.0.1). Note that other slices are allowed in addition to this required slice
ERROR: Observation/example: Observation: Observation.effective[x]: minimum required = 1, but only found 0 (from http://hl7.org/fhir/StructureDefinition/bp|4.0.1)
INFORMATION: Observation/example: Observation.code: Reference to experimental CodeSystem http://somewhere.org/fhir/uv/myig/CodeSystem/cs-fake|0.2.0
ERROR: Observation/example: Observation.component[0].value.ofType(Quantity): Observation.component:SystolicBP.value[x].unit: minimum required = 1, but only found 0 (from http://hl7.org/fhir/StructureDefinition/bp|4.0.1)
ERROR: Observation/example: Observation.component[1].value.ofType(Quantity): Observation.component:DiastolicBP.value[x].unit: minimum required = 1, but only found 0 (from http://hl7.org/fhir/StructureDefinition/bp|4.0.1)


== input/examples/Patient-example.xml ==
INFORMATION: Patient/example: Patient.name[0]: This element does not match any known slice defined in the profile http://somewhere.org/fhir/uv/myig/StructureDefinition/mypatient|0.2.0 (this may not be a problem, but you should check that it's not intended to match a slice)
INFORMATION: Patient/example: Patient.name[1]: This element does not match any known slice defined in the profile http://somewhere.org/fhir/uv/myig/StructureDefinition/mypatient|0.2.0 (this may not be a problem, but you should check that it's not intended to match a slice)
WARNING: Patient/example: Patient.maritalStatus: No code provided, and a code should be provided from the value set 'MaritalStatus' (http://hl7.org/fhir/ValueSet/marital-status|4.0.1)


== input/examples/Binary-logical-example.xml ==
ERROR: Binary/logical-example: /?:MyLogical/?:element2: Text should not be present ('Text')


== input/history/h1.json ==
WARNING: Bundle/h1: Bundle.entry[0].resource/*Provenance/h1-1*/.reason[0]: No code provided, and a code should be provided from the value set 'PurposeOfUse' (http://terminology.hl7.org/ValueSet/v3-PurposeOfUse|3.0.0)


== input/myig.xml ==
WARNING: /Users/grahamegrieve/work/test-igs/example.fhir.uv.myig/input/myig: Unknown Parameters: [showsource=true]
ERROR: ImplementationGuide/example.fhir.uv.myig: ImplementationGuide.definition.extension[5].extension[0].value.ofType(string): The value provided ('showsource') was not found in the value set 'ig-parameters Codes ValueSet' (http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0), and a code is required from this value set (error message = The System URI could not be determined for the code 'showsource' in the ValueSet 'http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0'; The provided code '#showsource' was not found in the value set 'http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0')
ERROR: ImplementationGuide/example.fhir.uv.myig: ImplementationGuide.definition.extension[40].extension[0].value.ofType(code): The value provided ('showsource') was not found in the value set 'ig-parameters Codes ValueSet' (http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0), and a code is required from this value set (error message = The System URI could not be determined for the code 'showsource' in the ValueSet 'http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0'; The provided code '#showsource' was not found in the value set 'http://hl7.org/fhir/tools/ValueSet/ig-parameters|0.2.0')
ERROR: ImplementationGuide/example.fhir.uv.myig: ImplementationGuide.definition.page.page[2]: The extension http://hl7.org/fhir/StructureDefinition/structuredefinition-fmm is not allowed to be used at this point (allowed = e:DomainResource, e:Resource; this element is [ImplementationGuide.definition.page, ImplementationGuide.definition.page.page])
ERROR: ImplementationGuide/example.fhir.uv.myig: ImplementationGuide.definition.page.page[2].page[0]: The extension http://hl7.org/fhir/StructureDefinition/structuredefinition-fmm is not allowed to be used at this point (allowed = e:DomainResource, e:Resource; this element is [ImplementationGuide.definition.page, ImplementationGuide.definition.page.page, ImplementationGuide.definition.page.page.page])


== input/resources/ActorDefinition-example.json ==
ERROR: ActorDefinition/example: ActorDefinition.type: The value provided ('system') was not found in the value set 'Example Scenario Actor Type' (http://hl7.org/fhir/ValueSet/examplescenario-actor-type|5.0.0), and a code is required from this value set (error message = The System URI could not be determined for the code 'system' in the ValueSet 'http://hl7.org/fhir/ValueSet/examplescenario-actor-type|5.0.0'; The provided code '#system' was not found in the value set 'http://hl7.org/fhir/ValueSet/examplescenario-actor-type|5.0.0')


== input/resources/MyLogical.xml ==
ERROR: StructureDefinition/MyLogical: StructureDefinition.snapshot: Constraint failed: sdf-3: 'Each element definition in a snapshot must have a formal definition and cardinalities'


== input/resources/codesystem-fake.xml ==
WARNING: CodeSystem/cs-fake: CodeSystem.concept[0].designation[0]: The designation 'CS2' has no use and no language, so is not differentiated from the base display ('CS1')


== input/resources/library-example-image.xml ==


== input/resources/library-example-sql.xml ==


== input/resources/myObservation.xml ==
WARNING: StructureDefinition/myObservation: StructureDefinition.identifier[0].type.coding[0].system: A definition for CodeSystem 'http://example.org/some-id-type-system' could not be found, so the code cannot be validated
WARNING: StructureDefinition/myObservation: StructureDefinition.identifier[1].type.coding[0]: A definition for CodeSystem 'http://example.org/some-id-type-system' could not be found, so the code cannot be validated
WARNING: StructureDefinition/myObservation: StructureDefinition.identifier[2].type.coding[0]: A definition for CodeSystem 'http://example.org/some-id-type-system' could not be found, so the code cannot be validated
WARNING: StructureDefinition/myObservation: StructureDefinition.identifier[3].type.coding[0]: A definition for CodeSystem 'http://example.org/some-id-type-system' could not be found, so the code cannot be validated


== input/resources/valueset-fake.xml ==


== input/resources/valueset-no-codesystem.xml ==
WARNING: ValueSet/valueset-no-codesystem: ValueSet.compose.include[0]: Unknown System/Version 'http://not-a-known-code-system#2.36' specified, so Concepts and Filters can't be checked (Details: A definition for CodeSystem 'http://not-a-known-code-system' version '2.36' could not be found, so the code cannot be validated. Valid versions: [])
WARNING: ValueSet.where(id = 'valueset-no-codesystem'): Error from http://tx-dev.fhir.org/r4: Unable to provide support for code system http://not-a-known-code-system version 2.36


== input/resources/resources-spreadsheet.xml ==
ERROR: StructureDefinition.snapshot.element[12]: The slice definition for Patient.name has a minimum of 0 but the slices add up to a minimum of 1


9 changes: 9 additions & 0 deletions .azure/qa-diff/expected/fhir.base.template.ig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"url" : "http://fhir.org/templates/fhir.base.template",
"package-id" : "fhir.base.template",
"ig-ver" : "0.7.0",
"date" : "Sun, 12 May, 2024 07:02:36 +1000",
"dateISO8601" : "2024-05-12T07:02:36+10:00",
"version" : "5.0.0",
"tool" : "5.0.0 (3)"
}
3 changes: 3 additions & 0 deletions .azure/qa-diff/expected/fhir.base.template.ig.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
url = http://fhir.org/templates/fhir.base.template
package-id = fhir.base.template
ig-ver = 0.7.0
9 changes: 9 additions & 0 deletions .azure/qa-diff/expected/hl7.base.template.ig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"url" : "http://fhir.org/templates/hl7.base.template",
"package-id" : "hl7.base.template",
"ig-ver" : "0.9.0",
"date" : "Sun, 12 May, 2024 07:02:00 +1000",
"dateISO8601" : "2024-05-12T07:02:00+10:00",
"version" : "5.0.0",
"tool" : "5.0.0 (3)"
}
3 changes: 3 additions & 0 deletions .azure/qa-diff/expected/hl7.base.template.ig.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
url = http://fhir.org/templates/hl7.base.template
package-id = hl7.base.template
ig-ver = 0.9.0
19 changes: 19 additions & 0 deletions .azure/qa-diff/expected/hl7.cda.uv.core.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"url" : "http://hl7.org/cda/stds/core/ImplementationGuide/hl7.cda.uv.core",
"name" : "ClinicalDocumentArchitecture",
"title" : "Clinical Document Architecture",
"description" : null,
"status" : "active",
"package-id" : "hl7.cda.uv.core",
"ig-ver" : "2.0.0-sd",
"date" : "Sun, 12 May, 2024 06:27:19 +1000",
"dateISO8601" : "2024-05-12T06:27:19+10:00",
"errs" : 2,
"warnings" : 1,
"hints" : 0,
"suppressed-hints" : 16,
"suppressed-warnings" : 570,
"version" : "5.0.0",
"tool" : "5.0.0 (3)",
"maxMemory" : 2294626760
}
Loading
Loading