Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tap reporting of or1k-tests #92

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Tap reporting of or1k-tests #92

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
67c51b0
Jenkinsfile modified to include Tap reporting of or1k-tests'
Aug 4, 2019
b9f8943
Added comments
Aug 6, 2019
320df8e
changing name of report.tap generated i.e it is displayed with unique…
Aug 9, 2019
49d2e29
Revert last commit
Aug 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis/run-or1k-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,8 @@ result=$?
if [ $result != 0 ] ; then
cat runtests.log
fi

# copy the report.tap (which has TAP test result) for or1k-tests to /src location
cp $HOME/src/tools/or1k-tests/native/report.tap /src

exit $result
15 changes: 15 additions & 0 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 2") {
Expand All @@ -40,6 +41,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 3") {
Expand All @@ -52,6 +54,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 4") {
Expand All @@ -64,6 +67,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 5") {
Expand All @@ -76,6 +80,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 6") {
Expand All @@ -88,6 +93,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 7") {
Expand All @@ -100,6 +106,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 8") {
Expand All @@ -112,6 +119,7 @@ pipeline {
}
steps {
dockerrun()
tapReporting()
}
}
stage("testing 9") {
Expand All @@ -125,6 +133,7 @@ pipeline {
//TODO: remove try/catch once the failing test is fixed (https://github.com/openrisc/mor1kx/issues/71)
try {
dockerrun()
tapReporting()
} catch (Exception e) {
echo "Allowed failure"
}
Expand All @@ -136,6 +145,12 @@ pipeline {
}
}

// data of test.sh is persisted in docker container librecores-ci-openrisc
void dockerrun() {
sh 'docker run --rm -v $(pwd):/src -e "JOB=$JOB" -e "SIM=$SIM" -e "PIPELINE=$PIPELINE" -e "EXPECTED_FAILURES=$EXPECTED_FAILURES" -e "EXTRA_CORE_ARGS=$EXTRA_CORE_ARGS" librecores/librecores-ci-openrisc /src/.travis/test.sh'
}

// TAP Plugin is used which adds the support to TAP test result files to Jenkins which can be seen at https://ci.librecores.org/job/Projects/job/OpenRISC/
void tapReporting() {
step([$class: "TapPublisher", testResults: "report.tap"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The report.tap name is the same for every run. It's hard to tell which report is for which run. Just like you use env vars in dockerrun() we should create a report name that is unique for each test. One thought, is the stage name available in an env var? We could make stage name more meaningful too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @stffrdhrn I am already on it report-cappuccino--feature-immu-none.tap
does this sounds reasonable. Just $PIPELINE $EXTRA_CORE_ARGS paramaters are passed with some modifications

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok. Sorry the comment was stuck in my browser history, I posted on accident this morning.

}