Added history file downloader

.gitignore

@@ -9,3 +9,4 @@
 build
 target
 testerra-report
+/src/test/resources/local.properties

README.md

@@ -60,13 +60,15 @@ Maven:
 
 ## Documentation
 
-### TeamCity configuration
+### Improvement of build status in TeamCity
+
+#### TeamCity configuration
 
 Please ensure that you have `Failure Conditions > Common Failure Conditions > at least one test failed` deactivated on your TeamCity
 build configuration,  
 because TeamCity Connector will announce the build status on report generation based on test execution statistics.
 
-### Gradle / Maven configuration
+#### Gradle / Maven configuration
 
 When using TeamCity Connector you have to ensure that Gradle/Maven will ignore test failures.
 
@@ -95,7 +97,7 @@ Maven:
 </build>
 ````
 
-### Impacts on TeamCity
+#### Impacts on TeamCity
 
 *Changes in TeamCity*
 
@@ -124,6 +126,42 @@ The following tables shows some more examples how the result could be.
 | ![](doc/teamcity_connector_result_failure_corr.png) | The Failure corridor was matched, the status is OK although a test failed.|
 | ![](doc/teamcity_connector_result_exp_failed.png) | A test was marked as expected failed, all other tests passed. The restult of the test run is still passed.|
 
+### Support of Testrun history of Testerra Report
+
+#### Properties
+
+Specify the following properties in `test.properties` to control the history file download for Testerra Report:
+
+| Property                              | Description                                                                                          |
+|---------------------------------------|------------------------------------------------------------------------------------------------------|
+| `tt.teamcity.history.download.active` | Activate the history file download, default: `false`                                                 |
+| `tt.teamcity.url`                     | URL of your TeamCity server                                                                          |
+| `tt.teamcity.rest.token`              | TeamCity Access token needed for REST API                                                            |
+| `tt.teamcity.buildTypeId`             | BuildType ID for the current Build configuration                                                     |
+| `tt.teamcity.build.branch`            | Specify the branch of the build job from which the history file has to be downloaded, default: `all` |
+
+#### TeamCity configuration
+
+It is recommended that the REST token is stored in a Configuration Parameter in your Build Configuration.
+
+![teamcity_connector_resttoken_parameter.png](doc/teamcity_connector_resttoken_parameter.png)
+
+All the other properties can be setup as 'Additional Maven command line parameters':
+
+![teamcity_connector_history_parameter.png](doc/teamcity_connector_history_parameter.png)
+
+#### Selecting the branch
+
+You can specify the Git branch of the build job which is used to download the latest history file.
+
+Add the property `tt.teamcity.build.branch` to your setup: 
+
+| Value         | Description                                                                                                        |
+|---------------|--------------------------------------------------------------------------------------------------------------------|
+| `all`         | TeamCity connector takes the last build job independent of the used branch. This is default.                       |
+| `default`     | Only build jobs of the default branch are used for download. The default branch is configured in your VCS setting. |
+| `<any other>` | Any other value is used as a branch name, eg. `development`, `fix/a-bug`                                           |
+
 ---
 
 ## Publication
@@ -132,15 +170,15 @@ This module is deployed and published to Maven Central. All JAR files are signed
 
 The following properties have to be set via command line or ``~/.gradle/gradle.properties``
 
-| Property                      | Description                                         |
-| ----------------------------- | --------------------------------------------------- |
-| `moduleVersion`               | Version of deployed module, default is `1-SNAPSHOT` |
-| `deployUrl`                   | Maven repository URL                                |
-| `deployUsername`              | Maven repository username                           |
-| `deployPassword`              | Maven repository password                           |
-| `signing.keyId`               | GPG private key ID (short form)                     |
-| `signing.password`            | GPG private key password                            |
-| `signing.secretKeyRingFile`   | Path to GPG private key                             |
+| Property                    | Description                                         |
+|-----------------------------|-----------------------------------------------------|
+| `moduleVersion`             | Version of deployed module, default is `1-SNAPSHOT` |
+| `deployUrl`                 | Maven repository URL                                |
+| `deployUsername`            | Maven repository username                           |
+| `deployPassword`            | Maven repository password                           |
+| `signing.keyId`             | GPG private key ID (short form)                     |
+| `signing.password`          | GPG private key password                            |
+| `signing.secretKeyRingFile` | Path to GPG private key                             |
 
 If all properties are set, call the following to build, deploy and release this module:
 ````shell

build.gradle

@@ -11,7 +11,8 @@ ext {
     // Minimum required Testerra version
     testerraCompileVersion = '2.0'
     // Unit tests use the latest Testerra version
-    testerraTestVersion = '[2,3-SNAPSHOT)'
+//    testerraTestVersion = '[2,3-SNAPSHOT)'
+    testerraTestVersion = '2-hist-SNAPSHOT'
     moduleVersion = '2-SNAPSHOT'
     if (System.properties.containsKey('moduleVersion')) {
         moduleVersion = System.getProperty('moduleVersion')
@@ -30,7 +31,8 @@ java {
 apply from: rootProject.file('publish.gradle')
 
 dependencies {
-    compileOnly 'io.testerra:driver-ui-desktop:' + testerraCompileVersion
+    compileOnly 'io.testerra:driver-ui:' + testerraCompileVersion
+    implementation 'com.jayway.jsonpath:json-path:2.9.0'
 
     testImplementation 'io.testerra:driver-ui-desktop:' + testerraTestVersion
     testImplementation 'io.testerra:report-ng:' + testerraTestVersion

src/main/java/eu/tsystems/mms/tic/testerra/plugins/teamcity/TeamCityHistoryDownloader.java

@@ -0,0 +1,128 @@
+/*
+ * Testerra
+ *
+ * (C) 2024, Martin Großmann, Deutsche Telekom MMS GmbH, Deutsche Telekom AG
+ *
+ * Deutsche Telekom AG and all other contributors /
+ * copyright owners license this file to you under the Apache
+ * License, Version 2.0 (the "License"); you may not use this
+ * file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package eu.tsystems.mms.tic.testerra.plugins.teamcity;
+
+import eu.tsystems.mms.tic.testframework.common.IProperties;
+import eu.tsystems.mms.tic.testframework.common.Testerra;
+import eu.tsystems.mms.tic.testframework.logging.Loggable;
+import eu.tsystems.mms.tic.testframework.report.Report;
+import eu.tsystems.mms.tic.testframework.utils.FileDownloader;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.StandardCopyOption;
+import java.util.UUID;
+
+public class TeamCityHistoryDownloader implements Loggable {
+
+    private static final String REPORT_MODEL_DIRECTORY = "report-ng/model/";
+    private static final String HISTORY_FILENAME = "history";
+
+    public enum Properties implements IProperties {
+        TEAMCITY_HISTORY_DOWNLOAD("tt.teamcity.history.download.active", false),
+        TEAMCITY_URL("tt.teamcity.url", ""),
+        TEAMCITY_REST_TOKEN("tt.teamcity.rest.token", ""),
+        TEAMCITY_BUILD_TYPE_ID("tt.teamcity.buildTypeId", ""),
+
+        // Define the type or name of the branch from which the last history file should download
+        // all = all branches
+        // default = only default branch
+        // <any other> = value is used as a branch name
+        TEAMCITY_BUILD_BRANCH("tt.teamcity.build.branch", "all")
+        ;
+
+        private final String property;
+        private final Object defaultValue;
+
+        Properties(String property, Object defaultValue) {
+            this.property = property;
+            this.defaultValue = defaultValue;
+        }
+
+        @Override
+        public String toString() {
+            return property;
+        }
+
+        @Override
+        public Object getDefault() {
+            return defaultValue;
+        }
+    }
+
+    /**
+     * This workflow get the history file of the latest finished build job
+     * and move it to the final report directory
+     */
+    public void downloadHistoryFileToReport() {
+        if (!Properties.TEAMCITY_HISTORY_DOWNLOAD.asBool()) {
+            return;
+        }
+
+        log().info("Trying to download the Report History file of the last TeamCity build...");
+
+        try {
+            final String historyFilePath = this.getHistoryFilePath();
+            if (historyFilePath == null) {
+                return;
+            }
+            File historyFile = this.downloadHistoryFile(historyFilePath);
+            Report report = Testerra.getInjector().getInstance(Report.class);
+            File finalReportDirectory = new File(report.getFinalReportDirectory(), REPORT_MODEL_DIRECTORY);
+            File finalHistoryFile = new File(finalReportDirectory, historyFile.getName());
+            if (finalReportDirectory.getAbsoluteFile().exists() && !finalReportDirectory.isDirectory()) {
+                finalReportDirectory.delete();
+            }
+            Files.createDirectories(finalReportDirectory.getAbsoluteFile().toPath());
+            Files.move(historyFile.toPath(), finalHistoryFile.getAbsoluteFile().toPath(), StandardCopyOption.REPLACE_EXISTING);
+            log().info("History file moved to {}", finalHistoryFile.getAbsoluteFile().toPath());
+        } catch (Exception e) {
+            log().warn("Cannot download history file to report directory: {}: {}", e.getClass(), e.getMessage());
+        }
+    }
+
+    private String getHistoryFilePath() {
+        final String teamCityUrl = Properties.TEAMCITY_URL.asString();
+        final String restToken = Properties.TEAMCITY_REST_TOKEN.asString();
+        final String buildTypeId = Properties.TEAMCITY_BUILD_TYPE_ID.asString();
+        final String branchType = Properties.TEAMCITY_BUILD_BRANCH.asString();
+
+        TeamCityRestClient client = new TeamCityRestClient(teamCityUrl, restToken);
+        final String buildId = client.findLatestBuildId(buildTypeId, branchType);
+        if (buildId == null) {
+            return null;
+        }
+        return client.getHistoryFilePath(buildId);
+    }
+
+    private File downloadHistoryFile(final String path) throws IOException {
+        final String restToken = Properties.TEAMCITY_REST_TOKEN.asString();
+        FileDownloader downloader = new FileDownloader();
+        downloader.setConnectionConfigurator(connection -> {
+            connection.setRequestProperty("Authorization", "Bearer " + restToken);
+        });
+        return downloader.download(path, UUID.randomUUID().toString() + "/" + HISTORY_FILENAME);
+    }
+
+
+}

src/main/java/eu/tsystems/mms/tic/testerra/plugins/teamcity/TeamCityRestClient.java

@@ -0,0 +1,130 @@
+/*
+ * Testerra
+ *
+ * (C) 2024, Martin Großmann, Deutsche Telekom MMS GmbH, Deutsche Telekom AG
+ *
+ * Deutsche Telekom AG and all other contributors /
+ * copyright owners license this file to you under the Apache
+ * License, Version 2.0 (the "License"); you may not use this
+ * file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package eu.tsystems.mms.tic.testerra.plugins.teamcity;
+
+import com.jayway.jsonpath.DocumentContext;
+import com.jayway.jsonpath.JsonPath;
+import eu.tsystems.mms.tic.testframework.logging.Loggable;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.util.List;
+import java.util.Objects;
+
+public class TeamCityRestClient implements Loggable {
+
+    private final String tcUrl;
+    private final String token;
+
+    public TeamCityRestClient(final String tcUrl, final String token) {
+        this.tcUrl = tcUrl;
+        this.token = token;
+    }
+
+    public String getHistoryFilePath(final String buildId) {
+        final String path = "/app/rest/builds/id:" + buildId + "/artifacts/children?locator=recursive:true";
+        try {
+            log().info("Get history file path from build id {}", buildId);
+            HttpRequest request = this.buildRequest(path);
+            HttpResponse<String> response = HttpClient.newBuilder()
+                    .build()
+                    .send(request, HttpResponse.BodyHandlers.ofString());
+            if (response.statusCode() > 204) {
+                log().error("Error getting history file path of build id {}:", buildId);
+                log().error(response.body());
+                return null;
+            }
+            Object value = this.getValueFromJson(response.body(), "$['file'][?(@['name'] == 'history')]['content']['href']");
+            if (value instanceof List && !((List) value).isEmpty()) {
+                final String historyPath = ((List<Object>) value).get(0).toString();
+                return this.tcUrl + historyPath;
+            } else {
+                log().warn("Cannot find history file in artifacts of build id {}", buildId);
+                return null;
+            }
+
+        } catch (IOException | InterruptedException e) {
+            log().error("Cannot get history file path from {}{}", this.tcUrl, path);
+            log().error(e.getMessage());
+            return null;
+        }
+    }
+
+    public String findLatestBuildId(final String buildTypeId, final String branchType) {
+        String branchLocator = "name:" + branchType;
+        switch (branchType.toLowerCase()) {
+            case "all":
+                branchLocator = "policy:ALL_BRANCHES";
+                break;
+            case "default":
+                branchLocator = "default:true";
+        }
+
+        final String path = "/app/rest/buildTypes/id:" + buildTypeId + "/builds?locator=running:false,canceled:false,branch:(" + branchLocator + "),count:1";
+        try {
+            log().info("Get last build id from {}", buildTypeId);
+            HttpRequest request = this.buildRequest(path);
+            HttpResponse<String> response = HttpClient.newBuilder()
+                    .build()
+                    .send(request, HttpResponse.BodyHandlers.ofString());
+            if (response.statusCode() > 204) {
+                log().error("Error getting build id of {}:", buildTypeId);
+                log().error(response.body());
+                return null;
+            }
+            final String count = Objects.requireNonNull(this.getValueFromJson(response.body(), "$['count']")).toString();
+            if ("0".equals(count)) {
+                log().warn("Cannot find a build id of buildtype {} and branch locator ({})", buildTypeId, branchLocator);
+                return null;
+            }
+
+            return Objects.requireNonNull(this.getValueFromJson(response.body(), "$['build'][0]['id']")).toString();
+        } catch (IOException | InterruptedException e) {
+            log().error("Cannot get build id from {}{}", this.tcUrl, path);
+            log().error(e.getMessage());
+            return null;
+        }
+    }
+
+    private Object getValueFromJson(final String jsonString, final String path) {
+        DocumentContext jsonContext = JsonPath.parse(jsonString);
+        if (jsonContext == null) {
+            return null;
+        }
+        return jsonContext.read(path);
+//        if() {}
+//        return value.toString();
+    }
+
+    private HttpRequest buildRequest(final String path) throws IOException {
+        return HttpRequest.newBuilder()
+                .uri(URI.create(this.tcUrl + path))
+                .header("Authorization", "Bearer " + token)
+                .header("Accept", "application/json")
+                .GET()
+                .build();
+    }
+
+}