Skip to content
This repository has been archived by the owner on Sep 12, 2023. It is now read-only.

Feat puppet only #73

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

Feat puppet only #73

Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
5266e37
echo -> info when checking MODIFIED_CONTENT_FILE
pcfe May 15, 2017
74ca304
fold at 240 columns instead of 80
pcfe May 15, 2017
e134fe6
copy of config.xml but with RPM part removed
pcfe May 14, 2017
fafc741
added one forced puppet run before we start waiting
pcfe May 15, 2017
23df4e4
adjusted copy of buildtestvms.sh
pcfe May 15, 2017
9148467
added off for libvirt to buildtestvms.sh
pcfe Aug 22, 2016
cb38bc5
introduce sleep flag to puppet-done-test.sh
pcfe May 15, 2017
9bf8a6b
adjusted copy of buildtestvms.sh
pcfe May 15, 2017
e929214
wait 30 seconds betwen checks, and wait after instead of before the c…
pcfe May 15, 2017
b8cfea0
as used in testing on home server, add power control
pcfe May 15, 2017
5bd61ed
as used in testing on home server, add power control
pcfe May 15, 2017
1297df0
doc only
pcfe May 14, 2017
537782f
removed tight polling used during testing
pcfe May 16, 2017
fc5b353
Merge branch 'master' of github.com:RedHatSatellite/soe-ci into feat-…
pcfe May 16, 2017
e32d57b
replaced post build action with conditional build step (#69)
jeichler Jun 17, 2017
933767e
SELinux on Jenkins server works just fine
pcfe Jun 25, 2017
0ebca8f
add link to yEd
pcfe Jun 25, 2017
6e28d7a
puppet-only workflow makes this rev 0.4
pcfe Jun 25, 2017
a5d5a8b
clarify output
pcfe Jun 25, 2017
f2ce067
typo fixed; >> not »
pcfe Jun 27, 2017
82c6feb
force immediate poweroff of SUT plus note RHBZ 1417976
pcfe Jun 28, 2017
dcdc7e4
fixup
pcfe Jun 30, 2017
2e4b609
Merge branch 'master' into feat-puppet-only
pcfe Jul 2, 2017
d66fed3
Merge branch 'master' into feat-puppet-only
pcfe Jul 3, 2017
3e125d4
adjust hammer command for Sat 6.3.2
pcfe Jul 10, 2018
1686a15
fixup, clarify log message
pcfe Jul 11, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
Continuous Integration Scripts for Satellite 6
==============================================

- - -

* Author: Patrick C. F. Ernzer
* Email: <[email protected]>
* Revision: 0.4
* Add puppet-only workflow

- - -

* Domain Architect: Eric Lavarde
* Email: <[email protected]>
* Consultant: Patrick C. F. Ernzer
Expand Down Expand Up @@ -35,14 +44,14 @@ The CII system consists of the following components:
* Red Hat Satellite 6. This acts as the repository for Red Hat-provided and 3rd party packages, kickstarts and puppet modules. The Foreman module is also used to deploy test clients.
* A virtualisation infrastructure to run test clients. I have used KVM/Libvirt, VMware and RHEV in different engagements.

The architecture is shown in [this YeD diagram](https://github.com/RedHatEMEA/soe-ci/blob/master/Engineering%20Platform.graphml).
The architecture is shown in [this YeD diagram](https://github.com/RedHatEMEA/soe-ci/blob/master/Engineering%20Platform.graphml). View it with [yEd](http://www.yworks.com/downloads#yEdLive).

## Setup
The following steps should help you get started with CII.

### Jenkins Server

NB I have SELinux disabled on the Jenkins server as I ran into too many problems with it enabled and didn't have the time to fix them.
NB SELinux in _Enforcing_ mode with the RHEL7 _Targeted_ policy works just fine,

#### Installation

Expand Down Expand Up @@ -87,8 +96,10 @@ NB I have SELinux disabled on the Jenkins server as I ran into too many problems
* Restart Jenkins
* Add the `jenkins` user to the `mock` group (`usermod -a -G mock jenkins`). This will allow Jenkins to build RPMs.
* Create `/var/www/html/pub/soe-repo` and `/var/www/html/pub/soe-puppet` and assign their ownership to the `jenkins` user. These will be used as the upstream repositories to publish artefacts to the satellite.
* Create `/var/www/html/pub/soe-puppet-only` for the puppet only workflow and assign their ownership to the `jenkins` user. These serve for the puppet only workflow.
* `su` to the `jenkins` user (`su jenkins -s /bin/bash`) and use `ssh-keygen` to create an ssh keypair. These will be used for authentication to both the git repository, and to the satellite server.
* Create one build plan per release you want to build for in Jenkins by creating the directory `/var/lib/jenkins/jobs/SOE-<release> (e.g. SOE-el7)` and copying in the [config.xml] file
* create a similar build plan for the puppet only workflow, omitting the RPM stuff.
* Check that the build plan is visible and correct via the Jenkins UI, you will surely need to adapt the parameter values to your environment.
* you might need to reload the configuration from disk using 'Manage Jenkins -> Reload Configuration from Disk'.

Expand Down
2 changes: 1 addition & 1 deletion common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ SRPMBUILD_ERR=6
MODBUILD_ERR=7

function tell() {
echo "${@} [$(date)]" | fold -s
echo "${@} [$(date)]" | fold --spaces --width=240
}

function info() {
Expand Down
259 changes: 259 additions & 0 deletions config-puppet-only.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
<?xml version='1.0' encoding='UTF-8'?>
<project>
<actions/>
<description>Generic plan to create puppet module environments</description>
<keepDependencies>false</keepDependencies>
<properties>
<jenkins.model.BuildDiscarderProperty>
<strategy class="hudson.tasks.LogRotator">
<daysToKeep>7</daysToKeep>
<numToKeep>100</numToKeep>
<artifactDaysToKeep>-1</artifactDaysToKeep>
<artifactNumToKeep>-1</artifactNumToKeep>
</strategy>
</jenkins.model.BuildDiscarderProperty>
<hudson.model.ParametersDefinitionProperty>
<parameterDefinitions>
<hudson.model.StringParameterDefinition>
<name>PUSH_USER</name>
<description>The username to use when pushing artefacts to the satellite. The build job will need to be able to ssh to the satellite as this user so you need to have keys set up.</description>
<defaultValue>jenkins</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>RSA_ID</name>
<description>RSA ID of the Jenkins user.</description>
<defaultValue>/var/lib/jenkins/.ssh/id_rsa</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>KNOWN_HOSTS</name>
<description>Known hosts file of the Jenkins user.</description>
<defaultValue>/var/lib/jenkins/.ssh/known_hosts</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>SATELLITE</name>
<description>The FQDN of the Red Hat Satellite.</description>
<defaultValue>satellite.example.com</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>ORG</name>
<description>This is the name of the organisation you are using on the Red Hat Satellite.</description>
<defaultValue>Default_Organization</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>PUPPET_REPO_ID</name>
<description>The numeric ID of the puppet module repository on the Red Hat Satellite that you want to use for your custom puppet modules.</description>
<defaultValue></defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>TESTVM_HOSTCOLLECTION</name>
<description>The name of the Satellite 6 host collection containing your test machines.</description>
<defaultValue>Test Servers puppet only</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>TESTVM_ENV</name>
<description>The numeric ID of the environment you want to promote CI builds to. This will typically be a &apos;Crash&apos; or &apos;EngTest&apos; environment.

You can find the IDs of your environments using:
hammer lifecycle-environment list --organization &quot;Default_Organization&quot;</description>
<defaultValue></defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>TEST_ROOT</name>
<description>This is the root password of your test machines, as set in your hostgroup definition.</description>
<defaultValue>redhat</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>MODAUTHOR</name>
<description>This is the authorname for your puppet modules.</description>
<defaultValue>acme</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>PUPPET_REPO</name>
<description>This is the location of the puppet repository on the Jenkins server used to export puppet modules to the satellite server.</description>
<defaultValue>/var/www/html/pub/soe-puppet-only</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CV</name>
<description>This is the name of the Content View on the satellite that you want to publish your build into.</description>
<defaultValue>cv-puppet-only</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CV_PASSIVE_LIST</name>
<description>A comma separated list of content view names to publish and optionally promote as part of the build.
Those CVs won&apos;t get new Puppet Modules assigned to them, meaning you must (and are free to) assign (and unassign) them yourself directly in Satellite.</description>
<defaultValue></defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CCV_NAME_PATTERN</name>
<description>Enter here a name pattern if you want Composite Content Views using the given CV(s) to be updated with the latest version and as well published and promoted.
CCVs respecting the pattern but not containing any CV will not be touched.
Example of pattern: &quot;ccv-test-*&quot;</description>
<defaultValue></defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.BooleanParameterDefinition>
<name>CONDITIONAL_VM_BUILD</name>
<description>If this parameter is true/set, only the hosts existing in Satellite with a comment matching
the modified Puppet module, RPM package or kickstart file, each surrounded by hashes (#), will be built and tested.
&lt;br/&gt;
Examples: &lt;code&gt;#ssh#bats#kickstart.erb#&lt;/code&gt;
&lt;br/&gt;
&lt;b&gt;CAUTION:&lt;/b&gt; be aware that in this case, no host will be tested if only the scripts have been modified.
&lt;br/&gt;
If unset/false, all existing hosts, within the given Host Collection, will be rebuilt independent of changes done in Git.</description>
<defaultValue>true</defaultValue>
</hudson.model.BooleanParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>MOCK_CONFIG</name>
<description>This will be passed to mock --root
e.g. mock --root rhel-6-x86_64
needed to ensure we can build RPMs against more then just the default mock chroot.</description>
<defaultValue>rhel-7-x86_64</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>PUPPET_DONE_SLEEP</name>
<description>Number of seconds to wait in puppet-done-test.sh
if you set up things with puppet that take a while to settle (e.g. NTP), you may want to wait a bit (e.g. 75 seconds is good for ntp to settle)</description>
<defaultValue>0</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
<scm class="org.jenkinsci.plugins.multiplescms.MultiSCM" plugin="[email protected]">
<scms>
<hudson.plugins.git.GitSCM plugin="[email protected]">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>git@localhost:soe-ci.git</url>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>*/master</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<submoduleCfg class="list"/>
<extensions>
<hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
<relativeTargetDir>scripts</relativeTargetDir>
</hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
</extensions>
</hudson.plugins.git.GitSCM>
<hudson.plugins.git.GitSCM plugin="[email protected]">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>git@localhost:acme-soe.git</url>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>*/master</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<submoduleCfg class="list"/>
<extensions>
<hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
<relativeTargetDir>soe</relativeTargetDir>
</hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
</extensions>
</hudson.plugins.git.GitSCM>
</scms>
</scm>
<canRoam>true</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
<blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
<triggers>
<hudson.triggers.SCMTrigger>
<spec>* * * * * </spec>
<ignorePostCommitHooks>false</ignorePostCommitHooks>
</hudson.triggers.SCMTrigger>
</triggers>
<concurrentBuild>false</concurrentBuild>
<builders>
<hudson.tasks.Shell>
<command>echo &quot;#####################################################&quot;
echo &quot;# BUILDING PUPPET MODULES #&quot;
echo &quot;#####################################################&quot;
/bin/bash -x ${WORKSPACE}/scripts/puppetbuild.sh ${WORKSPACE}/soe/puppet
</command>
</hudson.tasks.Shell>
<hudson.tasks.Shell>
<command>echo &quot;#####################################################&quot;
echo &quot;# PUSH PUPPET MODULES TO SATELLITE #&quot;
echo &quot;#####################################################&quot;
/bin/bash -x ${WORKSPACE}/scripts/puppetpush.sh ${WORKSPACE}/artefacts
echo &quot;#####################################################&quot;
echo &quot;# REPUBLISH CONTENT VIEW #&quot;
echo &quot;#####################################################&quot;
/bin/bash -x ${WORKSPACE}/scripts/publishcv.sh</command>
</hudson.tasks.Shell>
<hudson.tasks.Shell>
<command>echo &quot;#####################################################&quot;
echo &quot;# MAKING SURE TEST VMs ARE ON #&quot;
echo &quot;#####################################################&quot;

/bin/bash ${WORKSPACE}/scripts/starttestvms.sh
</command>
</hudson.tasks.Shell>
<hudson.tasks.Shell>
<command>echo &quot;#####################################################&quot;
echo &quot;# PUSHING TESTS to TEST VMS #&quot;
echo &quot;#####################################################&quot;
/bin/bash -x ${WORKSPACE}/scripts/pushtests.sh
</command>
</hudson.tasks.Shell>
<org.jenkinsci.plugins.conditionalbuildstep.singlestep.SingleConditionalBuilder plugin="[email protected]">
<condition class="org.jenkins_ci.plugins.run_condition.core.StatusCondition" plugin="[email protected]">
<worstResult>
<name>SUCCESS</name>
<ordinal>0</ordinal>
<color>BLUE</color>
<completeBuild>true</completeBuild>
</worstResult>
<bestResult>
<name>SUCCESS</name>
<ordinal>0</ordinal>
<color>BLUE</color>
<completeBuild>true</completeBuild>
</bestResult>
</condition>
<buildStep class="hudson.tasks.Shell">
<command>echo &quot;#####################################################&quot;
echo &quot;# POWER OFF TEST VMs #&quot;
echo &quot;#####################################################&quot;
/bin/bash ${WORKSPACE}/scripts/powerofftestvms.sh

echo &quot;#####################################################&quot;
echo &quot;# CLEAN-UP AFTER SUCCESS BUILD #&quot;
echo &quot;#####################################################&quot;
/bin/bash -x ${WORKSPACE}/scripts/cleanup.sh
</command>
</buildStep>
<runner class="org.jenkins_ci.plugins.run_condition.BuildStepRunner$Fail" plugin="[email protected]"/>
</org.jenkinsci.plugins.conditionalbuildstep.singlestep.SingleConditionalBuilder>
</builders>
<publishers>
<org.tap4j.plugin.TapPublisher plugin="[email protected]">
<testResults>test_results/*.tap</testResults>
<failIfNoResults>false</failIfNoResults>
<failedTestsMarkBuildAsFailure>false</failedTestsMarkBuildAsFailure>
<outputTapToConsole>false</outputTapToConsole>
<enableSubtests>false</enableSubtests>
<discardOldReports>true</discardOldReports>
<todoIsFailure>false</todoIsFailure>
<includeCommentDiagnostics>false</includeCommentDiagnostics>
<validateNumberOfTests>false</validateNumberOfTests>
<planRequired>true</planRequired>
<verbose>true</verbose>
<showOnlyFailures>false</showOnlyFailures>
<stripSingleParents>false</stripSingleParents>
<flattenTapResult>false</flattenTapResult>
<skipIfBuildNotOk>false</skipIfBuildNotOk>
</org.tap4j.plugin.TapPublisher>
</publishers>
<buildWrappers/>
</project>
Loading