Git and GitHub
First steps with Git and GitHub
diff --git a/612/404.html b/612/404.html new file mode 100644 index 0000000000..9baf45b2b4 --- /dev/null +++ b/612/404.html @@ -0,0 +1,15 @@ +
Oops! This page doesn't exist. Try going back to our home page.
The EGI documentation is a static site built using +Hugo +from Markdown source files. Hugo uses +goldmark to parse and render markdown, +which is compliant with CommonMark and +GitHub Flavored Markdown (also based +on CommonMark).
The EGI documentation is organized into sections and +pages. Read below to understand when to use each of these, +and how to create new sections and add new pages to a section.
Sections are those pages that can have subpages. They always appear in +bold in the left-side navigation tree:
Sections are also pages, meaning that selecting them in the +navigation tree will show their content.
To create a new section, create a folder under /content/<language>/
,
+add a file named _index.md
to it, then author content for it as described
+below.
/content/<language>/
can show up in the top-level navigation bar.
+See below for details on how to control this.Pages are Markdown files that contain the
+documentation about a specific topic. They hold the content for a section
+(in which case are named _index.md
and the containing folder is the section),
+or a stand-alone page that is immediately under a section (the containing
+folder is the section).
This is how stand-alone pages appear in the left-side navigation tree: +
Creating a documentation page is done by creating a Markdown file (with
+.md
extension) under the relevant section (in the section’s folder).
Each page needs some metadata that controls where the page appears and how +its content will be rendered. The beginning of the Markdown file contains a +front matter in YAML, +holding the metadata of the page:
---
+title: "Style"
+linkTitle: "Style Guide"
+description: "Style guide for EGI documentation"
+type: docs
+weight: 30
+---
+
The parameter weight
controls the order of the pages on the same level in
+the left-side navigation tree. Pages will appear in ascending order of their
+weight.
The above metadata produces the following result (relevant elements highlighted): +
Pages can be added to the top navigation bar by using a front matter like this:
---
+title: "About"
+description: "About EGI Documentation"
+type: docs
+menu:
+ main:
+ weight: 50
+---
+
Pages will be added to the top navigation bar in ascending order of +their menu weight, from left to right.
If you also want to add an icon to the entry in the top navigation bar:
---
+title: "About"
+description: "About EGI Documentation"
+type: docs
+menu:
+ main:
+ weight: 50
+ pre: <i class='fa fa-info'></i>
+---
+
Hugo organizes content +for each page into a subfolder with the same name as the page’s filename. +This allows authors to easily keep track of the resources used by each page.
Let’s assume we have a section named About with a subpage Concepts, using +the following hierarchy of files:
/documentation
+ /content
+ /en
+ /about
+ _index.md
+ concepts.md
+ map.png
+ /concepts
+ metadata.png
+
Embedding an image in the page of
+the section (the file _index.md
) can be done with:
![Image title](map.png)
+
Embedding an image in a subpage can
+be done by editing the Markdown file of the page (concepts.md
in our case):
![Image title](metadata.png)
+![Image title](../map.png)
+
You can include hyperlinks in the documentation that will +link to any documentation page, +or to external resources.
Assuming we have the same section named About with a subpage Concepts
+as used above, and the file _index.md
+contains this:
This is a link to a [page](concepts)
+This is another link to the same [page](concepts.md)
+
while the page concepts.md
contains this:
This is link to a [section](../)
+This another link to a [section](../../about)
+
the links in these pages exemplify how to link to a different page, +be it a section or a stand-alone page.
.md
will be assumed.You can also include hyperlinks in the documentation that will +link to any heading (aka chapter) +from any documentation page.
To link to a heading in a documentation page, you need to point the hyperlink +to the anchor created automatically by Markdown for the targeted heading.
Below you have an example of a page with sample headings and links to +those headings:
This is a link to a [chapter](#some-chapter-title)
+This is a link to another [subchapter](#some-subchapter-title)
+
+## Some chapter title
+...
+### Some subchapter title
+...
+
Assuming we have the same section named About with a subpage Concepts
+as used above, and the file _index.md
+contains this:
## Section chapter
+This is a link to a [page chapter](concepts#page-chapter)
+This is another link to the same [page chapter](concepts.md#page-chapter)
+
while the page concepts.md
contains this:
## Page chapter
+This s link to a [section chapter](../#section-chapter)
+This s link to a [section chapter](../../about#section-chapter)
+
the links in these pages exemplify how to link to headings in a +different page or section.
The EGI Glossary is available on the +EGI Glossary space.
For collaboration purposes, it is best if you create a GitHub account and fork +the repository to your own account. Once you do this, you will be able to push +your changes to your GitHub repository for others to see and use, and you will +be able to create pull requests (PRs) in the official EGI documentation +repository based on branches in your fork.
If you are new to git
and GitHub you are advised to start by the two
+following articles providing simple tutorials:
GitHub official documentation is available at +docs.github.com.
Additional documentation about the main steps for working with GitHub is also +available in this section.
In order to be able to send code update to the repository you need to:
In this process three git repositories are used:
The most convenient way to authenticate with GitHub is to use SSH keys over the +SSH protocol.
You can add an SSH public key to your GitHub account in the Settings
on
+GitHub, at https://github.com/settings/keys.
Refer to +Connecting to GitHub with SSH +for an extensive documentation on using SSH keys with GitHub.
It’s worth to mention that your ssh public keys can easily be retrieved using
+a URL like https://github.com/<your_username>.keys
.
In order to manage repositories over ssh, you will have to clone them via +SSH, not HTTPS.
If you already have a local clone of a repository created via HTTPS, you can +switch it to SSH by following +Switching remote URLs from HTTPS to SSH.
The GitHub command-line interface greatly helps with working with GitHub +repositories from a terminal.
It can be installed using the packages available on +their homepage. There is also a +manual.
Once installed you will have to start by setting up authentication.
# Authenticate with GitHub, favor SSH protocol
+$ gh auth login
+$ gh config set git_protocol ssh
+
The easiest way is to do it via the GitHub CLI that will also clone it locally. +But it can also be done via the web interface, using the fork button and then +cloning it locally manually.
This command will fork the repository to your GitHub account and clone a local +copy for you to work with.
$ gh repo fork EGI-Federation/documentation
+
If you want to clone an existing fork you should use:
$ gh repo clone <your_username>/documentation
+
If your local clone of you fork is correctly setup you should see references to +the origin and upstream repositories.
$ git remote -v
+origin git@github.com:<your_username>/documentation (fetch)
+origin git@github.com:<your_username>/documentation (push)
+upstream git@github.com:EGI-Federation/documentation.git (fetch)
+upstream git@github.com:EGI-Federation/documentation.git (push)
+
The documentation site is built from the source files using +Hugo. The repository +README +can be used as a reference for building instructions.
To install npm+Node.js please check the +official instructions.
Everything has been tested with Node.js 12.
The dependencies of the docsy theme can be installed as follows:
# From the root of the repository clone
+$ npm ci
+
Hugo can be installed following +the official documentation.
Hugo (extended) releases can be downloaded at +the Hugo releases page.
To build and run the site, from the repository root:
$ git submodule update --init --recursive --depth 1
+$ hugo --minify
+
To launch the site locally, from the repository root:
$ hugo serve -D
+
The site is available locally at: +http://localhost:1313/.
You should submit your patch as a git branch ideally named with a meaningful +name related to the changes you want to propose. This is called a feature +branch (sometimes also named topic branch). You will commit your +modifications to this feature branch and submit a Pull Request (PR) based on +the differences between the upstream main branch and your feature branch.
Try to avoid committing changes to the main branch of your clone to simplify +management, creating a dedicated feature branch helps a lot. Try to pick a +meaningful name for the branch (my_nice_update in the example).
# This should be done from the up-to-date main branch
+# Read furthermore to see documentation on updating a local clone
+$ git checkout -b my_nice_update
+
The documentation being made of plain text files you are free to use whatever +text editor or Integrated Development Environment (IDE) suits you, from +neovim to +Visual Studio Code.
Some environments may provide you plugins helping with syntax or offering a +preview, they are worth checking.
Be sure to commit files with having been formated using +Prettier as documented in our style guide.
It is the best practice to have your commit message have a summary line that +includes the issue number, followed by an empty line and then a brief +description of the commit. This also helps other contributors understand the +purpose of changes to the code.
#3 - platform_family and style
+
+ * use platform_family for platform checking
+ * update notifies syntax to "resource_type[resource_name]" instead of
+ resources() lookup
+ * GH-692 - delete config files dropped off by packages in conf.d
+ * dropped debian 4 support because all other platforms have the same
+ values, and it is older than "old stable" debian release
+
# Select the modified files to be committed
+$ git add files1 path2/
+# Commit the changes
+$ git commit -m <commit_message>
+
From inside a feature branch you can push it to your remote fork.
# Ask git to keep trace of the link between local and remote branches
+$ git push --set-upstream
+
Once done, the output will show a URL that you can click to generate a Pull +Request (PR). Accessing GitHub upstream of forked repositories may also propose +you to submit a PR.
If needed GitHub CLI can also be used to prepare the PR:
$ gh pr create <your_username>:<feature_branch> --web
+
If a repository maintainer adds the label safe for preview
to a pull request
+it will be possible to preview it using a pull request-specific URL:
+https://docs.egi.eu/documentation/[PR_NUMBER]
The preview can be used as an alternative to testing a pull request locally, and +the preview can easily be shared with other contributors.
Only collaborators having write permission to the repository are able to mark a +pull request as safe for review.
This should be carefully considered, especially for external and first time +contributors.
Once you PR have been opened it will be reviewed, and reviewers can propose and +commit changes to your PR. If you need to make further changes be sure to update +the local clone with the remote changes.
# Retrieve changes made on your PR in the upstream repository
+$ git pull
+
Then you can commit new changes and push them to your remote fork.
# If you are still in a branch created for a previous PR, move to main
+$ git checkout main
+# Get the latest data from the upstream repository
+$ git fetch upstream
+# Update your local copy with this data
+$ git rebase upstream/main main
+# Update your remote GitHub fork with those changes
+$ git push
+
In case the main branch evolved since the feature branch was created, it may +be required to merge the new changes in the feature branch.
It can easily be done via the PR page on the GitHub web interface, but it can
+also be done in your repository clone using git rebase
.
# Retrieve changes made in the upstream repository
+$ git fetch upstream
+# Check out the feature branch
+$ git checkout feature_branch
+# Apply the new changes on main to your feature branch
+$ git rebase upstream/main
+
In case some files have been changed on both sides you will have to merge +the conflicts manually.
The repository leverages GitHub Actions to do +automatic checks.
The main checks are:
It’s possible to run those linters locally, it can be useful to test and debug why they +are reporting errors, and test without waiting on the automatic checks.
To save time, we recommend running the specific you are interested in, but as documented +later for Check-Spelling, you can use
act
to run the real GitHub actions workflows.
markdownlint-cli can be +installed locally on some platforms, in that case it can be used like this:
$ markdownlint -c .github/linters/.markdownlint.json \
+ content/en/about/_index.md
+
If you cannot or don’t want to install it locally, you can rely on using +Docker and GitHub Packages:
$ docker run -v $PWD:/workdir:ro --rm -i ghcr.io/igorshubovych/markdownlint-cli:latest \
+ --config .github/linters/.markdownlint.json \
+ content/en/about/_index.md
+
You can use Docker and +GitHub Packages:
$ docker run -v $PWD:/tmp:ro --rm -i ghcr.io/tcort/markdown-link-check:stable \
+ --config /tmp/.github/linters/mlc_config.json \
+ /tmp/content/en/about/_index.md
+
For spelling, it may be easier to rely on a spell checker integrated in your content editor.
Nevertheless, if you are adventurous, you can use act, +that will use docker to run the checks locally.
While the spell check should work, other part of the job interacting with GitHub will +likely fail, like when errors are identified, so be sure to properly scan through the +command output.
# Listing available jobs
+$ act -l
+# Running the spelling job
+$ act -j spelling
+
Should you want to do this, Super-Linter can be run locally.
You can try using act:
# Listing available jobs
+$ act -l
+# Running the lint job
+$ act -j super-lint
+
If it doesn’t work as expected, or if you prefer another solution, you should look at the +Super-Linter documentation for running locally.
It’s possible to clone a Pull Request to a local branch to test it locally. It’s +done using the PR number.
# List available PR and their identifiers.
+$ gh pr list
+# Clone specific PR, updating sudmodules
+$ gh pr checkout XX --recurse-submodules
+
Once done it’s possible to build and run the site locally:
# From the root of the repository clone
+# Here on MacOS X, adapt depending on your platform
+$ hugo serve -D
+
The documentation will then be accessible on +http://localhost:1313.
People having write access to the repository hosting the branch related to the +PR (ie. usually the PR author) will be able to add and edit files.
# From the local clone of the repository
+$ gh pr checkout XXX --recurse-submodules
+$ vim yyy.zz
+$ git add yyy.zz
+$ git commit yyy.zz -m <commit_message>
+$ git push
+
# It will ask you to merge changes
+$ git pull
+
Then you can refer to the README.md
to see how to test it locally.
In case the PR got commits that were forced pushed you may have troubles, in +that case it may be easier to +delete the local branch and do another +checkout of the PR.
In case you have troubles updating the local clone, as it can happens if changes +were forced pushed to it, it maybe easier to delete the local copy of the PR and +recreate it.
# Switch to main branch
+$ git checkout main
+# Check local branches
+$ git branch -vv
+# Delete a specific branch
+$ git branch -d <branch_name>
+# If you need to force the deletion use -D
+$ git branch -D <branch_name>
+
Sometimes we realise just before committing a change that we are not in the
+correct branch (ie. that we forgot to create a dedicated feature branch), when
+this happens git stash
can be helpful.
# Saving a change
+$ git stash save <optional message>
+# Creating the forgotten branch
+$ git checkout -b <my_feature_branch>
+# Reviewing the saved changes, use TAB completion
+$ git stash show <TAB>
+# Applying the saved changes, use TAB completion
+$ git stash pop <TAB>
+# Review the changes to be committed
+$ git diff
+
If you already committed your change(s) you may have to look at git reset
.
# Viewing the diff of the two last commits
+$ git log -n 2 -p
+# Reverting the last change, keeping the change in the local directory
+$ git reset HEAD^
+
Thank you for taking the time to contribute to this project. The maintainers +greatly appreciate the interest of contributors and rely on continued engagement +with the community to ensure this project remains useful. We would like to +take steps to put contributors in the best possible position to have their +contributions accepted. Please take a few moments to read this short guide on +how to contribute.
If you wish to discuss anything related to the project, please open a +GitHub issue or +start a topic on the EGI Community Forum.
All contributions have to go through a review process, and contributions can be +made in two ways:
If you need to discuss your changes +beforehand (e.g. adding a new section or if you have any doubts), please +consult the maintainers by creating a +GitHub issue.
You can also create an issue by navigating to a documentation page, and +clicking the Create documentation issue link +in the top-right corner.
Before proposing a contribution via the so-called Pull Request (PR) workflow, +there should be an open issue +describing the need for your contribution (refer to this issue number when you +submit the PR). We have a three-step process for contributions:
Code review takes place in GitHub pull requests (PRs). See +this article if you’re +not familiar with GitHub PRs.
Once you open a PR, automated checks will verify the style and syntax +of your changes and maintainers will review your code using the built-in code +review process in GitHub PRs.
The process at this point is as follows:
main
branch.request changes
in the review and provide an explanation.The documentation is using a rolling release model, all changes merged to the
+main
branch are directly deployed to the live production environment.
The main
branch is always available. Tagged versions may be created as needed
+following semantic versioning when applicable.
EGI benefits from a strong community of developers and system administrators, +and vice-versa. If you have any questions or if you would like to get involved +in the wider EGI community you can check out:
First steps with Git and GitHub
Style guide for EGI documentation
Helpers for writing EGI documentation
In addition to the formatting support provided by +Markdown, Hugo adds support for +shortcodes, which are Go templates for easily including or displaying content +(images, notes, tips, advanced display blocks, etc.).
For reference, the following shortcodes are available:
This is achieved using +Docsy shortcodes.
The following code:
{{% pageinfo %}} This is a placeholder. {{% /pageinfo %}}
+
Will render as:
The following code:
{{% alert title="Note" color="info" %}} This is a Note. {{% /alert %}}
+
Will render as:
The following code:
{{% alert title="Important" color="warning" %}} This is a warning.
+{{% /alert %}}
+
Will render as:
The code or instructions should be surrounded with three backticks, followed by +an optional highlighting type parameter.
The supported languages are dependent on the syntax highlighter, which depends +itself on the Markdown parser.
The following Markdown creates a shell excerpt:
```shell
+$ ssh-keygen -f fedcloud
+$ echo $HOME
+```
+
Will render as:
$ ssh-keygen -f fedcloud
+$ echo $HOME
+
This is also achieved using +Docsy shortcodes.
When you need to include code snippets, and you want to provide the same code in +multiple programming languages, you can use a tabbed pane for code snippets:
{{< tabpane >}}
+{{< tab header="C++" lang="C++" >}}
+#include <iostream>
+int main()
+{
+ std::cout << "Hello World!" << std::endl;
+}
+{{< /tab >}}
+{{< tab header="Java" lang="Java" >}}
+class HelloWorld {
+ static public void main( String args[] ) {
+ System.out.println( "Hello World!" );
+ }
+}
+{{< /tab >}}
+{{< tab header="Kotlin" lang="Kotlin" >}}
+fun main(args : Array<String>) {
+ println("Hello, world!")
+}
+{{< /tab >}}
+{{< tab header="Go" lang="Go" >}}
+import "fmt"
+func main() {
+ fmt.Printf("Hello World!\n")
+}
+{{< /tab >}}
+{{< /tabpane >}}
+
Will render as:
#include <iostream>
+int main()
+{
+ std::cout << "Hello World!" << std::endl;
+}
class HelloWorld {
+ static public void main( String args[] ) {
+ System.out.println( "Hello World!" );
+ }
+}
fun main(args : Array<String>) {
+ println("Hello, world!")
+}
import "fmt"
+func main() {
+ fmt.Printf("Hello World!\n")
+}
When you need to include multiple variants of the same content, other than code +snippets in multiple programming languages, you can use the following shortcode:
{{< tabpanex >}}
+
+{{< tabx header="Linux" >}}
+
+You can list all files in a folder using the command:
+```shell
+$ ls -a -l
+```
+{{< /tabx >}}
+
+{{< tabx header="Mac" >}}
+To get a list of all files in a folder, press **Cmd** + **Space** to open
+a spotlight search, type terminal, then press Enter. In the terminal window
+then run the command:
+
+```shell
+$ ls -a -l
+```
+
+{{< /tabx >}}
+
+{{< tabx header="Windows" >}}
+
+You can list all files in the current folder using the command:
+
+```shell
+> dir
+```
+
+or you can use PowerShell:
+
+```powershell
+> Get-ChildItem -Path .\
+```
+
+{{< /tabx >}}
+
+{{< /tabpanex >}}
+
Will render as:
You can list all files in a folder using the command:
$ ls -a -l
+
To get a list of all files in a folder, press Cmd + Space to open a +spotlight search, type terminal, then press Enter. In the terminal window then +run the command:
$ ls -a -l
+
You can list all files in the current folder using the command:
> dir
+
or you can use PowerShell:
> Get-ChildItem -Path .\
+
[EGI Cloud Compute](https://www.egi.eu/service/cloud-compute/)
+
Documentation pages have to be written in Markdown, compliant with +CommonMark and +GitHub Flavored Markdown.
##
), as level 1 (#
) is the title of the
+page-
not *
1.
for each line (automatic numbering)\n
)$
or >
) in front of commands, to
+make it easy to understand which is the command and which is the output\
) on each line
+that continues on the nextStyle should be enforced via the usage of Prettier. +Prettier can be integrated with +various editors.
Configuration is provided in .prettierrc
, options can be set as follows:
--print-width 80 --tab-width 2 --prose-wrap always
+
When a contribution is received (via a pull request), the proposed changes are +checked using +various linters.
Follow the guidelines below to ensure readability and consistency of the EGI +documentation. These are based on the +OpenStack documentation writing style +guidelines, released under a +Creative Commons license.
Use standard British English (UK) throughout all technical publications. When in +doubt about the spelling of a word, consult the Merriam-Webster’s Collegiate +Dictionary and the +IBM developerWorks editorial style guide.
Follow the principles of minimalism. If you can describe an idea in one word, do +not use two words. Eliminate all redundant modifiers, such as adjectives and +adverbs.
Do not use humor, jargon, exclamation marks, idioms, metaphors, and other +colloquialisms.
Put the most common case in the main clause and at the beginning of a paragraph +or section. You can introduce additional use cases by starting a sentence with +“however” or “if”.
In general, write in active voice rather than passive voice. Active voice +identifies the agent of action as the subject of the verb, usually the user. +Passive voice identifies the recipient (not the source) of the action as the +subject of the verb.
Active-voice sentences clarify the performer of an action and are easier to +understand than passive-voice sentences. Passive voice is usually less engaging +and more complicated than active voice. When you use passive voice, the actions +and responses of the software can be difficult to distinguish from those of the +user. In addition, passive voice usually requires more words than active voice.
Do not use | Use |
---|---|
After the software has been installed, the computer can be started. | After you install the software, start the computer. |
The Configuration is saved when you click OK. | Click OK to save the configuration. |
A server is created by you. | Create a server. |
However, passive voice is acceptable in the following situations:
Do not use | Use |
---|---|
If the build fails, you probably omitted the flavor. | If the build fails, the flavor might have been omitted. |
Do not use | Use |
---|---|
The product, OS, or database returns the messages. | The messages are returned [by the database]. |
Do not use | Use |
---|---|
In 2009, engineers developed a software that simplifies the installation. | A software that simplifies the installation was developed in 2009. |
Users are more engaged with documentation when you use second person (that is, +you address the user as “you”).
Writing in second person has the following advantages:
Do not use first person to avoid naming the product or to avoid using passive +voice. If the product is performing the action, use third person (the product as +an actor). If you want to de-emphasize the agent of action and emphasize the +object on which the action is performed, use passive voice.
The first-person singular pronoun “I” is acceptable in the question part of FAQs +and when authors of blogs or signed articles are describing their own actions or +opinions.
Do not use | Use |
---|---|
Creating a server involves specifying a name, flavor, and image. | To create a server, specify a name, a flavor, and image. |
To create a server, the user specifies a name, flavor, and image. | To create a server, you specify a name, flavor, and image. |
Users read documentation to perform tasks or gather information. For users, +these activities take place in their present, so the present tense is +appropriate in most cases. Additionally, the present tense is easier to read +than the past or future tense.
Do not use | Use |
---|---|
The product will prompt you to verify the deletion. After you log in, your account will then begin the verification process. | The product prompts you to verify the deletion. After you log in, your account begins the verification process. |
Do not give human characteristics to non-human subjects or objects.
Do not use | Use |
---|---|
This guide assumes… | This guide describes… |
Do not express your fears or feelings in technical writing. Avoid the adverbs +such as “probably”, “hopefully”, “basically”, and so on.
Each title should include a clear description of the page’s or chapter’s +subject.
Do not use | Use |
---|---|
Update metadata | Update object metadata |
Do not use “please” and “thank you” in technical documentation.
Write in a positive tone. Positive sentences improve readability. Try to avoid +the following words as much as possible:
Do not use | Use |
---|---|
damage | affect |
catastrophic | serious |
bad | serious (or add explanation) |
fail | unable to |
kill | cancel or stop |
fatal | serious |
destroy | remove or delete |
wrong | incorrect or inconsistent |
Generally, do not contract the words.
Do not use | Use |
---|---|
can’t | cannot |
don’t | do not |
Use these pronouns sparingly. Overuse contributes to readers’ confusion. To fix +the ambiguity, rephrase the sentence.
Do not use | Use |
---|---|
The monitoring system should perform regular checks to verify that the Ceph cluster is healthy. This can be achieved using the Ceph health command. | The monitoring system performs regular checks to ensure the Ceph cluster is functioning correctly. Use the Ceph health command to run a health check. |
Do not place modifiers between “to” and the verb. Typically, placing an adverb +or an adjective between “to” and a verb adds ambiguity to a sentence.
As much as possible, avoid trailing prepositions in sentences by avoiding +phrasal verbs.
Do not use | Use |
---|---|
The image registration window will open up. | The image registration window opens. |
To fix the verb-preposition constructions, replace them with active verbs.
Do not use | Use |
---|---|
written up | composed |
pop up | appear |
Use consistent terms across all content. Avoid multiple variations or spellings +to refer to the same service, function, UI element, and so on.
Run text through spelling and grammar checking tools, if available. Correcting +mistakes, especially to larger sections of new content, helps eliminate rework +later.
When reading a document for the first time, users scan through pages stopping +only on the content that stands out, such as titles, lists, links, diagrams, and +so on. Lists help to organize options, as well as help readers to find +information easily.
When listing items, follow these guidelines:
When listing options in a paragraph, add and or or before the last item in a +list. Use a serial (Oxford) comma before these conjunctions if they connect +three or more items.
In bulleted lists:
In numbered lists:
Successfully passing the checks is a firm requirement, but for the following +cases it is possible to +add exceptions and +bypass some checks in Markdown files:
In some specific cases it is impossible to use anything but in-line HTML tags:
Blocks with in-line HTML tags should be preceded by a HTML comment starting with
+markdownlint-disable
+to disable the no-inline-html
check, as in the following example:
<!-- markdownlint-disable no-inline-html -->
+
+| Action | OCCI | OpenStack | This is a very long column with important data |
+| ----------- | ------------------------ | ---------------------- | ---------------------------------------------- |
+| List images | `occi -a list -r os_tpl` | `openstack image list` | <ul><li>Lorem</li><li>ipsum</li></ul> |
+
+<!-- markdownlint-enable no-inline-html -->
+
no-inline-html
check.<ul>
and <li>
, not
+<br />
.When the same procedure needs to be described for multiple platforms, or when +the same code has to be exemplified for multiple languages, it is possible that +the automatic checkers will flag these as duplicates.
For example, describing the following procedure will result in duplicates being +reported:
{{< tabpanex >}}
+
+{{< tabx header="Linux" >}}
+ To run the FedCloud client in a container, make sure
+ [Docker is installed](https://docs.docker.com/engine/install/#server),
+ then run the following commands:
+
+ ```shell
+ $ docker pull tdviet/fedcloudclient
+ $ docker run -it tdviet/fedcloudclient bash
+ ```
+{{< /tabx >}}
+
+{{< tabx header="Mac" >}}
+ To run the FedCloud client in a container, make sure
+ [Docker is installed](https://docs.docker.com/desktop/mac/install/),
+ then run the following commands:
+
+ ```shell
+ $ docker pull tdviet/fedcloudclient
+ $ docker run -it tdviet/fedcloudclient bash
+ ```
+{{< /tabx >}}
+
+{{< tabx header="Windows" >}}
+ To run the FedCloud client in a container, make sure
+ [Docker is installed](https://docs.docker.com/desktop/windows/install/),
+ then run the following commands:
+
+ ```shell
+ > docker pull tdviet/fedcloudclient
+ > docker run -it tdviet/fedcloudclient bash
+ ```
+{{< /tabx >}}
+
+{{< /tabpanex >}}
+
This type of content should be preceded by a HTML comment that disables the +check for duplicates, and followed by another HTML comment that enables it +again.
<!--
+// jscpd:ignore-start
+-->
+
+...content with duplicates here...
+
+<!--
+// jscpd:ignore-end
+-->
+
The EGI documentation is written in Markdown, +uses the Docsy theme, and is built +using Hugo.
Concepts used when writing documentation
Contributing to EGI documentation
EGI is an international e-Infrastructure +providing advanced computing and data analytics services for research +and innovation. The EGI Federation offers a wide range of services for +compute, storage, data management, training, and support.
This is the start page for the user and provider documentation of the +EGI Services.
EGI Accounting tracks and reports usage of EGI +services, offering insights and control over resource consumption. EGI +Federation members can use it to account for the resource usage of their own +services.
EGI Accounting consists of two main components:
Property | Value |
---|---|
Name | Accounting (Repository and Portal) |
Description | Tracking, in an APEL-based repository, and reporting, via a web portal, of the usage of EGI resources and services. |
URL | https://accounting.egi.eu (portal) |
Support Email | apel-admins <at> stfc.ac.uk |
Helpdesk Support Unit | EGI Services and Service Components I__ Accounting Portal I__ APEL Client and Accounting Repository |
Configuration Database entries | Repository Portal |
Suppliers | UKRI (Repository), CESGA (Portal) |
Roadmap | N/A |
Release notes | N/A |
Source code | APEL, Portal: N/A |
Issue tracker for developers | APEL, Portal: Helpdesk |
Licence | Apache 2.0 |
Privacy Notice | Privacy Notice |
EGI Foundation is having access to a +GÉANT Trusted Certificate Service +subscription.
Sectigo is the current Certificate Authority (CA) providing certificates to +GÉANT TCS.
This can be used to issue IGTF-trust (AKA eScience certificates) and
+public-trust certificates, for the domains managed by EGI Foundation, like
+egi.eu
.
Operators of central services can request two types of certificates:
In some specific cases, like for +Cloud Compute providers not having access +to an IGTF CA, it’s possible for them to request a robot certificate, as an +IGTF certificate is required for sending accounting records.
You can contact scs-ra@egi.eu (or operations@egi.eu) if you need support.
Open a ticket to Collaboration Tools SU in EGI Helpdesk, +providing:
An operator will follow with the request, and help you getting the certificate.
Certificates are usually valid for one year. Auto-renewal of certificates is +enabled by default, 30 days before expiration, notifications will be sent to the +contact address provided when requesting the certificate.
The notifications contain links allowing to renew the certificate.
In order to get a certificate, Service Providers may be requested to send a +Certificate Signing Request (CSR).
In the CSR only the Common Name (CN) is important, it should be the (FQDN) of +the service Most of other fields will be replaced by the CA while generating the +certificate.
It can be done via different ways, some are documented below.
DigiCert provides an online +web application.
It can be done with the cfssl
tool from
+CloudFlare.
# Replace #FQDN# by the FQDN of the service
+$ cfssl genkey <(echo '{"hosts":["#FQDN#"],"CN"#FQDN#","key":{"algo":"rsa","size":4096}}')
+ | cfssljson -bare ##FQDN##.rsa
+
It can be done using the following OpenSSL
command (This will generate a
+password-protected key.
You will be asked for various questions, but the only important ones are the +Common Name (CN) and Subject Alternative Names (SAN) (in case you want to +request a certificates covering different FQDNs), as other values will be +overwritten by the CA.
$ openssl req -out CSR.csr -new -newkey rsa:4096 -keyout privateKey.key
+
Adding the
-nodes
option will disable password protection for the key, +beware if using it.
It is possible to automate the certificate request and renewal using the +ACME protocol via +certbot or similar tools.
Two things should be considered:
The EGI SDIS team will have to create and register an ACME client ID for you.
Once you will have the credential it should be possible to request a +certificate, using the standard certbot client, as +documented below.
This will interactively register your certbot client. Even if the ACME +credentials are shared, only a given client is able to manage the certificates +it requested. Email address is used for urgent renewal and security notices. +This is preferred way if you need to get notifications on certificate expiration +and if you are doing some manual management or testing.
$ sudo certbot register --no-eff-email \
+ --server https://acme.sectigo.com/v2/OV \
+ --eab-kid <EAB_KID> \
+ --eab-hmac-key <EAB_HMAC_KEY> \
+ --email <CONTACT_EMAIL>
+
+# Checking existing account
+$ sudo certbot show_account --server https://acme.sectigo.com/v2/OV
+
+# Unregistering an account
+# Beware: you won't any more be able to revoke certificate issued with the account
+$ sudo certbot unregister --server https://acme.sectigo.com/v2/OV
+
$ sudo certbot certonly --standalone --non-interactive \
+ --server https://acme.sectigo.com/v2/OV \
+ --domain fakedomaindonotexist.egi.eu
+
$ sudo certbot revoke \
+ --server https://acme.sectigo.com/v2/OV \
+ --cert-name fakedomaindonotexist.egi.eu
+
This is useful when you don’t want interactive registration, like for one shot +scripts. Email address is used for urgent renewal and security notices.
Apparently the notification to use this email is not visible in cert-manager, +and the first option with explicit registration may be safer to get the +notifications, if it’s a required feature.
$ sudo certbot certonly --standalone --non-interactive --agree-tos \
+ --server https://acme.sectigo.com/v2/OV \
+ --eab-kid <EAB_KID> --eab-hmac-key <EAB_HMAC_KEY> \
+ --rsa-key-size 4096 \
+ --email <CONTACT_EMAIL> \
+ --domain fakedomaindonotexist.egi.eu
+
Other usual certbot options should also work. You may also be able to use other +tools that can speak the ACME protocol, it should be standard.
The EGI Collaboration Tools are a set of online services maintained by the EGI +Foundation and meant to support collaboration across the EGI Federation, +supporting its daily activities (sharing information and documents, organising +meetings, communicating in task or project specific groups…).
The EGI SSO is a central authentication and authorisation service +allowing to use a single username and password to access various Collaboration +Tools.
Group owners can manage group members, and send invitations to new users. Groups +are used in Collaboration Tools services for authorisation.
EGI SSO is now being deprecated in favour of +EGI Check-in, providing a federation +authentication and authorisation service.
Domain entries can be created for the service as long as they are relevant for
+the EGI infrastructure. All EGI central services are usually available under the
+egi.eu
domain.
Requests needs to be discussed with the +EGI Operations team.
EGI Foundation is having access to a +GÉANT Trusted Certificate Service +subscription.
This can be used to issue IGTF-trust (AKA eScience certificates) and
+public-trust certificates, for the domains managed by EGI Foundation, like
+egi.eu
.
Please refer to documentation on requesting a certificate.
Information about the EGI Federation activities is published on the +public site.
A Confluence-based +“Wiki” space is available, allowing the EGI members to work collaboratively on +documenting many aspects like Service Management, +Polices and Procedures or activities +from Boards and Groups.
Separate instances for EGI and EOSC are operated by EGI Foundation.
EGI internal request tracking system, meant to support the Service Management +System and projects.
EGI Foundation is operating dedicated instances for EGI and EOSC.
EGI internal request tracking system, being replaced for many activities by +Jira.
Many mailing lists exists to cover specific areas of collaboration. New ones can +be crated on request as documented below.
Membership in mailing lists is usually determined by membership in EGI SSO +groups. Only group owners can manage the group membership.
Note that canonical addresses are list-name@mailman.egi.eu
, not
+list-name@egi.eu
.
See the dedicated page on mailing list for more information +like how to request the creation of a new mailing list.
EGI Documentation database, hosting many published +document relevant to the EGI Federation.
Documents have metadata describing them and can be versioned. EGI SSO groups are +used for restricting access to documents.
Event planner, used for various conferences and meetings, powered by the +Indico product.
All users with an SSO account can use it.
Indico also has its own local accounts, which can be used by people who do not +want an EGI SSO account, but want to register to some meeting in Indico.
EGI Collaboration Tools support can be contacted via:
Technical details of the Collaboration Tools
Certificate provisioning
EGI SSO usage
Mailing lists usage
Mailing lists are managed by software called Mailman.
The software provides a web interface and a +public list of mailing lists is +available.
By default all mailing lists are managed using groups in EGI SSO. Only for +very specific they may be managed directly in mailman.
Open a ticket to Collaboration Tools SU in EGI Helpdesk, +providing:
After validation of those information the list will be created.
Name and description of the mailing list will be validated, you may be +recommended to change name or description to follow already existing best +practices for naming, or to improve description.
EGI mailing lists are usually managed by EGI SSO, with some exceptions for lists +that include members without SSO accounts.
List administrators of every SSO-managed list are set to the owners of the +corresponding EGI SSO group when a new list is created.
Only the EGI IT support can change who are the owners of an SSO group, please +contact us if you want to add a new list +administrator.
Each list can be administered over the web interface by following its link from +the page Admin links. You can use your EGI SSO password to access the +administration pages of the lists for which you are the administrator.
Members of lists are synchronized with members of same named groups in the EGI +SSO system.
Managing list members must be done by managing members of corresponding groups +in EGI SSO. Log into your EGI SSO account, and you will see a list of groups of +which you are the owner. Click on the “»manage” link, it will display a page +where you can
If you manage more then one list, and you need to add many new users to several +lists at once, it may be helpful to follow the link titled “»Create new users in +groups”. It directs to a page where you can enter many addresses at once, and +specify the groups to which they should be added. Existing users will be added +immediately, non-existing users will be invited to create an account and they +will be assigned to the groups when they create the account.
Property | Value |
---|---|
Name | Collaboration Tools |
Description | Supporting Collaboration across the EGI Federation |
URL | N/A |
Support Email | it-support@egi.eu |
Helpdesk Support Unit | EGI Services and Service Components I__ Collaboration Tools |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=983 |
Supplier | EGI Foundation and CESNET |
Roadmap | N/A |
Release notes | N/A |
Source code | N/A |
Issue tracker for developers | N/A |
License | Every component is having its own licence |
Privacy Policy | Parent policy, components usually have their own policy |
EGI SSO is a legacy central Identity Provider mainly +intended to access the EGI Collaboration Tools.
For most services people should look into using federated authentication via +EGI Check-in. Still, for some services not yet +integrated with Check-in, an EGI SSO account should be used, in that case you +can create an EGI SSO account.
On purpose a user cannot manually edit a certificate DN
You usually just need to link your new DN to your EGI SSO account, as documented +above.
You may have to do this by in a private browser session to ensure that your +previous certificate is used.
Once you have done this, you can request us to remove the former DN by opening +an Helpdesk ticket to the Collaboration Tools Support +Unit.
Legacy DNs shouldn’t prevent you accessing any service with a newer +certificate, as long as its DN is linked to your account.
To access the web interface of the EGI Configuration +Database (GOCDB), users can either:
Users can access the system as soon as they are authenticated. However, they +will only be able to update information based on their roles, and only once they +will have registered a new user account.
More information about roles and associated permission is available in the +Users and roles section of the documentation.
Applications requesting a specific role have to be validated by parent roles or +administrators. Once granted, users can access and/or modify relevant +information, according to the roles granted to them.
In order to be able to access the Configuration Database with their +institutional account, users need to:
In the case the user cannot use +an IdP compliant with REFEDS R&S +and REFEDS Sirtfi, the user will have to request +joining a specific group, by performing the steps below. Using a compliant IdP +is the preferable solution.
To access the Configuration Database using a digital certificate, first obtain a +certificate from one of the recognised EU-Grid-PMA Certification Authorities +(CAs), then install it in your browser of choice (or import it into the +certificate store of your local machine, on Windows).
X.509 certificates do not support single +or double quotes in the certificate’s Distinguished Name (DN). The DN below is +rejected because of the single quote:
/C=UK/O=STFC/OU=SomeOrgUnit/CN=David Mc'Donald
This is in accordance with RFC1778, which +also disallows single quotes in all Relative Distinguished Name (RDN) +components, and the OGF Certificate Authority Working Group (CAOPS) who strongly +discourage any type of quote in a certificate DN as specified by their +Grid Certificate Profile document.
Being authenticated in one of the two ways described above is enough to have +read-only access to all the public features of the EGI Configuration Database. +If you need to edit data in and request roles, you will need to fill in the +registration form.
To Register:
GOCDB, the software powering the EGI Configuration Database, is multi-tenanted; +it can host multiple projects in the same instance. There are a number of +different deployment scenarios that can be used to support new projects detailed +below. Please contact the EGI Configuration Database admins and EGI Operations +to discuss the options available.
get_services&scope=SubGroupX
+
Note, resources can be tagged multiple times to declare support for multiple +projects and sub-groups:
get_services&scope=SubGroupX,EGI&scope_match=all
+
get_services&scope=ProjectX
+
Note, resources can be tagged multiple times to declare support for multiple +projects:
get_services&scope=ProjectX,EGI&scope_match=all
+
For monitoring purposes, each service endpoints registered into the +Configuration Database, and having the flags production and monitored should +include the endpoint URL information in order to be contacted by the +corresponding service-specific nagios probe.
The information needed for service type are:
GlueServiceEndpoint
published in the
+Configuration Database or BDII (e.g. httpg://se.egi.eu:8444/srm/managerv2
)endpoint URL
must contain the Keystone v3 URL:
+https://hostname:port/url/v3
endpoint URL
must contain the Keystone v3 URL:
+https://hostname:port/url/v3
GlueServiceEndpoint
+published in the BDIIIt is also possible to register additional endpoints for every services, they +will also be monitored if the “Monitored” flag is set.
For having more information about managing the Service endpoints in the +Configuration Database, please consult the +service endpoints documentation.
For retrieving the queue URL from the BDII, you can use the command
+lcg-infosites
, to be executed from a UI. Be sure to query a production Top
+BDII: you can either use the one provided by your Operations Centre or choose
+one from
+the Configuration Database
For example:
$ export LCG_GFAL_INFOSYS=egee-bdii.cnaf.infn.it:2170
+
+$ lcg-infosites --vo ops ce | grep nikhef
+ 5680 15 0 0 0 dissel.nikhef.nl:2119/jobmanager-pbs-infra
+ 5680 17 1 1 0 gazon.nikhef.nl:8443/cream-pbs-infra
+ 5680 15 0 0 0 juk.nikhef.nl:8443/cream-pbs-infra
+ 5680 15 0 0 0 klomp.nikhef.nl:8443/cream-pbs-infra
+ 5680 16 0 0 0 stremsel.nikhef.nl:8443/cream-pbs-infra
+
In order to find the GlueServiceEndpoint
URL of your SRM service, you can
+launch a LDAP query to your Site BDII (or directly to the SRM service):
$ ldapsearch -x -LLL -H ldap://sbdii01.ncg.ingrid.pt:2170 \
+ -b "mds-vo-name=NCG-INGRID-PT,o=grid" \
+ '(&(objectClass=GlueService)(GlueServiceType=SRM))' \
+ GlueServiceEndpoint
+
+dn: GlueServiceUniqueID=httpg://srm01.ncg.ingrid.pt:8444/srm/managerv2,Mds-Vo-name=NCG-INGRID-PT,o=grid
+GlueServiceEndpoint: httpg://srm01.ncg.ingrid.pt:8444/srm/managerv2
+
In a similar way, by just changing the value of GlueServiceType
, you can
+retrieve the endpoint URLs of other services.
An alternative way for retrieving the GlueServiceEndpoint
URL is using the
+GLUE2 information browser provided by
+VAPOR: select
+your NGI, then your site and hence the Storage service; click on the endpoint
+details button for finding the URL associated to the SRM interface.
This is the home page regarding your site. You need to fill in the URL +information.
Click on a service for displaying its page (e.g. the CREAM-CE).
Click on the EDIT button in the top right corner
fill in the Service URL field with the queue URL
Now the CREAM-CE service endpoint contains the required queue information.
Proceed in a similar way for the other services.
In case you need to register an additional endpoint for a service, go on the +service summary page and add the proper information. In the example below it is +shown the case of a computing element.
This is the service summary page.
You need to click on the Add endpoint button for registering additional +endpoint URLs.
Fill in the proper information and don’t forget to select the “Monitored” flag +for making Nagios to detect the new endpoint.
The summary page of the endpoint just added should look like this one.
And this is the summary page of the service reporting the information about all +its endpoints registered: the first one in the Grid Information section and +the additional ones in the Service Endpoints section.
In order to properly monitor your webdav endpoint:
https://darkstorm.cnaf.infn.it:8443/webdav/ops
or
+https://hepgrid11.ph.liv.ac.uk/dpm/ph.liv.ac.uk/home/ops/
GLUE2EndpointURL
+(containing the used port and without the VO folder);https://darkstorm.cnaf.infn.it:8443/webdav
) is properly accessible.The EOS service endpoints expose an XrootD interface, so in order to properly +monitor them, even in case you provide a plain XrootD endpoint, please do the +following:
root://eosatlas.cern.ch//eos/atlas/opstest/egi/
,
+root://recas-se-01.cs.infn.it:1094/dpm/cs.infn.it/home/ops/
,
+root://dcache-atlas-xrootd-ops.desy.de:2811/pnfs/desy.de/ops
or similar).
+Pay attention to the port configured for the interface.In order to properly monitor your gridftp endpoint for ops VO
globus-GRIDFTP
, with the “production” flag disabled;/dpm/ui.savba.sk/home/ops
(this is an example, set the proper path)The SURL value needed by the SRM monitoring probes is the following structure:
srm://<hostname>:<port>/srm/managerv2?SFN=<GlueSAPath or GlueVOInfoPath>
For example:
srm://ccsrm.in2p3.fr:8443/srm/managerv2?SFN=/pnfs/in2p3.fr/data/dteam/
GlueServiceEndpoint
URL information.GlueSAPath
information, use that. To retrieve it:$ ldapsearch -x -LLL -H <ldap://sbdii01.ncg.ingrid.pt:2170> \
+ -b "mds-vo-name=NCG-INGRID-PT,o=grid" \
+ '(&(objectClass=GlueSA)(GlueSAAccessControlBaseRule=VO:ops))' \
+ GlueSAPath GlueChunkKey
+
+dn: GlueSALocalID=opsdisk:replica:online,GlueSEUniqueID=srm01.ncg.ingrid.pt,Mds-Vo-name=NCG-INGRID-PT,o=grid
+GlueChunkKey: GlueSEUniqueID=srm01.ncg.ingrid.pt
+GlueSAPath: /gstore/t2others/ops
+
GlueSAPath
information, use instead the
+GlueVOInfoPath
one:$ ldapsearch -x -LLL -H <ldap://ntugrid5.phys.ntu.edu.tw:2170> \
+ -b "Mds-Vo-name=TW-NTU-HEP,o=grid" \
+ (&(objectClass=GlueVOInfo)(GlueVOInfoAccessControlBaseRule=VO:ops)) \
+ GlueVOInfoLocalID GlueChunkKey
+
+dn: GlueVOInfoLocalID=ops:SRR,GlueSALocalID=SRR:SR:replica:*****,GlueSEUniqueID=ntugrid6.phys.ntu.edu.tw,Mds-Vo-name=TW-NTU-HEP,o=grid
+GlueVOInfoPath: /dpm/phys.ntu.edu.tw/home/ops
+GlueChunkKey: GlueSALocalID=SRR:SR:replica:*****
+GlueChunkKey: GlueSEUniqueID=ntugrid6.phys.ntu.edu.tw
+GlueVOInfoLocalID: ops:SRR
+
+dn: GlueVOInfoLocalID=ops:data01,GlueSALocalID=data01:replica:online,GlueSEUniqueID=ntugrid6.phys.ntu.edu.tw,Mds-Vo-name=TW-NTU-HEP,o=grid
+GlueVOInfoPath: /dpm/phys.ntu.edu.tw/home/ops
+GlueChunkKey: GlueSALocalID=data01:replica:online
+GlueChunkKey: GlueSEUniqueID=ntugrid6.phys.ntu.edu.tw
+GlueVOInfoLocalID: ops:data01
+
srm://srm01.ncg.ingrid.pt:8444/srm/managerv2?SFN=/gstore/t2others/ops
The GOCDB Programmatic Interface (PI) is available under /gocdbpi
.
The GOCDB PI has two main components:
The Read API provides programmatic access to the data. Access to some +information (security/critical, personal details, otherwise sensitive +information) is restricted, more details are available in the section about +data protection levels.
The Write API provides limited functionality to add, update, and delete +entities. Access is restricted, more details can be found in the section about +authentication and authorisation.
API calls can be tested in a browser or done from the command-line interface,
+using curl
.
Below are some examples, including methods with different +data protection levels.
API calls starting with
https://goc.egi.eu/gocdbpi/private
require the +client to present a valid credential.
It is possible to filter content using xpath
. Download
+information about endpoints under the EGI and FedCloud scopes
+as egi_fedcloud_service_endpoints.xml
.
# Extracting endpoints in production
+$ xpath -q -e "//SERVICE_ENDPOINT[IN_PRODUCTION='Y']/HOSTNAME/text()" \
+ egi_fedcloud_service_endpoints.xml | sort | uniq
+
Querying information about a specific site using CURL, and authenticating with +an X.509 client certificate.
$ curl -v --cert ~/.globus/usercert.pem --key ~/.globus/userkey.pem \
+ 'https://goc.egi.eu/gocdbpi/private/?method=get_site&sitename=CESGA'
+
See +python script from cloud-info-provider repository.
Examples of using the Write API can be found on the +GOCDB PI site.
A downtime is a period of time for which a service is declared to be inoperable. +Downtimes may be scheduled (e.g. for software/hardware upgrades), or unscheduled +(e.g. power outages). The Configuration Database stores the following +information about downtimes (non exhaustive list):
There are different pages in the Configuration Database where downtimes are +listed:
Each downtime has its own page providing details, accessible by clicking on the
+Downtime Id
link or similar in downtime listing pages.
The EGI Operations Portal provides a +publicly-accessible page allowing to view and filter downtimes: +Operations Portal.
Authenticated users can +subscribe to downtimes +affecting sites selected using a filter. The downtimes notifications can be sent +by email, RSS and iCal, allowing to easily integrate with your calendar.
Provided you have proper permissions (check the
+permissions matrix
+section), you can add a downtime by clicking on the Add Downtime
link in the
+sidebar.
This is done in 2 steps:
edit
link on top of the downtime’s
+details page.To delete a downtime, simply click the delete
link on top of the downtime’s
+details page. For integrity reasons, it is only possible to remove downtimes
+that have not started.
Depending on the planning of the intervention, downtimes can be:
Scheduled
: planned and agreed in advanceUnscheduled
: planned or unplanned, usually triggered by an unexpected
+failure or at a short term noticeEGI defines precise rules about what should be declared as scheduled or +unscheduled, based on how long in advance the downtime is declared. These +rules are described in +MAN02 Service intervention management +and are enforced as follows:
UNSCHEDULED
SCHEDULED
SCHEDULED
, it is good practice to declare it at least 5 working days before
+it starts.When declaring a downtime, you will be presented the choice of a “severity”,
+which can be either WARNING
or OUTAGE
. Please consider the following
+definitions:
WARNING
means the resource is considered available, but the quality of
+service might be degraded. Such downtimes generate notifications, but are not
+taken into account by monitoring and availability calculation tools. In case
+of a service failure during the WARNING
period an OUTAGE
downtime has to
+be declared, cancelling the rest of the WARNING
downtime.
OUTAGE
means the resource is considered as unavailable. Such downtimes will
+be considered as in maintenance
by monitoring and availability calculation
+tools.
Limitation rules to downtime extensions are enforced as follows:
If for any reason a downtime already declared needs to be extended, the +procedure is to add another adjacent downtime, before or after.
Examples:
To return all sites that define VO with a value of Alice:
?method=get_site&extensions=(VO=Alice)
+
Use no value to define a wildcard search, i.e. all sites that define the VO +property regardless of value:
?method=get_site&extensions=(VO=)
+
NOTE: From version 5.7 (Autumn/Winter 2016) keys must be unique for a given +site, service, or service endpoint, or service group. The following section of +documentation has not yet been changed to reflect this.
Extensions also supports OR/AND/NOT operators. This can be used to search +against multiple key values eg:
?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)
+
These can be used together:
?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)
+
?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)OR(CPU_HS02_HOUR=2)
+
When no operator is specified the default is AND, therefore the following:
?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)
+
Is the same as:
?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)
+
The extensions parameter can also be used in conjunction with the existing +parameters previously supported:
?method=get_site&extensions=(VO=Alice)NOT(VO=LHCB)&scope=EGI&roc=NGI_UK
+
The site_extensions and service_extensions can also be used on the +get_downtime and get_downtime_nested_services methods using same logic +described above. Note, the EXTENSIONS element is not rendered in the XML +output for these queries.
?method=get_downtime_nested_services&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)
+
?method=get_downtime&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)
+
For EGI Services, the Standard Extension property “HostDN” has been defined to +allow the fetching the DNs of multiple hosts behind a load balanced service from +the endpoint properties of a single GOCDB Service, rather than creating multiple +GOCDB Services with different host DNs.
To supply multiple or alternate DN(s) for a service, for example of the multiple +hosts supporting a single service entry, the Service Extension Property +(hereafter Ext) “HostDN” SHOULD be used. If Ext “HostDN” is present it MUST +contain one or more x.509 DN values. Multiple values MUST be delimited by +enclosing each within “<>” characters. If Ext “HostDN” is present, the Service +“Host DN” SHOULD contain the x.509 SubjectAltName used in the X509 +certificate(s) presented by the hosts identified by the Ext “HostDN” values.
If you were registered but are not recognised anymore (e.g. because your +certificate DN changed), do not register again!
Instead, follow the steps +Lost access to your Configuration Database account +section.
Only registered users with an approved role on an NGI can add a new site. If you +are the site administrator, the first thing to do is to contact your NGI staff +and ask them to add the site for you. Then, register to EGI Configuration +Database (see the user account section) and ask for a site admin role for your +site (see the requesting a role section). Once your role approved, you will be +able to edit and change your site information.
Because of EGI policies it is not possible to extend a downtime. Recommended +good practice for any downtime extension is to declare a new unscheduled +downtime, starting just when the first one finishes. Please refer to the +downtimes section of this documentation for more information, especially the +“downtime extension” paragraph.
If you have declared the downtime as being at risk and an outage actually +happens half way through, you need to update the Configuration Database to +reflect the fact that your site is now down. There is currently no way of doing +this by updating the downtime on the fly without having the system considering +the whole downtime as being an outage. The best way to proceed is:
Monitoring status in Configuration Database cannot always be switched off. If a +node is declared as delivering a production service, rules apply and the node +has to be monitored. If you are running a test node and want to switch +monitoring off, set both “monitoring” and “production” to “N”.
Someone has to approve any request you make, in order to ensure nobody is trying +to get inappropriate roles. If yours is not getting approved, this can either be +because your request was not legitimate, or most likely because the people that +are supposed to do it forgot about it. Please refer to the Roles permissions +definitions section of this documentation to determine who should validate your +role, and try to get in touch with them. If you are requesting a site admin +role, they are likely to be your fellow site admins or your NGI operators.
Accessing the backend through another way than the +Configuration Database web interface is out of the scope +of this documentation. Please refer to the technical documentation instead, +which is available from GOCDB Documentation.
The EGI Configuration Database (GOCDB) is a central +registry that records topology information about all sites participating in +the EGI infrastructure.
The configuration database also provides different rules and grouping mechanisms +for filtering and managing the information associated to resources. This can +include entities such as operations and resource centres, service endpoints and +their downtimes, contact information and roles of staff responsible for +operations at different levels.
The configuration database is used by all the actors (end users, site managers, +NGI managers, support teams, VO managers), by other tools, and by third party +middleware to discover information about the infrastructure topology.
Configuration Database identity card
Accessing the Configuration Database
Adding service endpoints information into Configuration Database
Managing and consulting downtimes
Extension Properties
Managing NGIs entities
Managing the NGI Core Services entries in Configuration Database
Understanding and manipulating scopes
Managing Service entities
Understanding and manipulating service groups
Description of Service types.
Managing Sites entities
Guide about user accounts and the roles
Information required for registering a service into the Configuration Database
How to create a new project in EGI Configuration Database.
Accessing the Configuration Database API
Frequently Asked Questions
NGIs can register a number of ‘NGI-Core’ services in the Configuration Database. +A core NGI service is one that is used to calculate the availability and +reliability of the NGI. These services fall under the responsibility of the NGI +and provide production quality (no testing instances). NGIs can distinguish/flag +their core services from their other (non-core) services using one of two ways +(see A and B below).
The service instance MUST:
The following service types are mandatory to support the central operations and +all NGIs in the EGI scope should define instances of these services:
Other Mandatory services, depending on middleware deployed by sites under NGI +responsibility, are the following:
NGIs should also register their custom core services like accounting, helpdesk +if provided.
NGI core services can be grouped/flagged in one of two ways:
It is important that these core service Sites/ServiceGroups adhere to the +‘NGI_XX_SERVICES’ naming scheme. The list of existing Service Groups is +available on +GOCDB.
An NGI forms a grouping of Sites in EGI Configuration Database. The +Configuration Database stores the following information about these groups. The +main page listing groups actually shows NGIs/ROCs, and is available from “List +of NGIs/ROCs and associated contacts”, linked from the main menu.
Each NGI has its own listing page, accessible by clicking on the “view” link in +group listing pages. A group details page shows users with a role on that group, +as well as member sites and associated contacts and roles.
Adding groups is not possible through the Input System web interface. If you +want to start the registration process of a new NGI, please follow the procedure +described on:
Integration of the new group in the EGI Configuration Database is part of the +procedure but has to be done by the Configuration Database admins.
To edit a group, simply click on the “edit” link at the top of the group’s +details page.
This operation is not allowed.
Sites
, Services
and
+ServiceGroups
into flexible categories. A single entity can define multiple
+scope tags, allowing the resource to be associated with different categories
+without duplication of information. This is essential to maintain the
+integrity of topology information across different infrastructures and
+projects.It is important to understand that scopes and Projects are distinct:
Site
, Service
or ServiceGroup
visible to EGI, the resource’s
+‘EGI’ scope tag checkbox must be ticked. EGI scoped resources are exposed to
+the central operational tools for monitoring and will appear in the central
+operations portal.Site
->Service
relationship.Tag for resources that are part of the EGI Core services. To request this tag, +please raise an EGI Helpdesk ticket against the Operations SU.
A Service entity is formed by a hostname, a hosted service and a URL.
The EGI Configuration Database stores the following information about Service +entities (non exhaustive list):
ServiceTypes
below)As a machine can host many services, there can be many Service entities per +machine.
The machine myhost.domain.org
runs a CE, an UI and a UnicoreX service. This
+will show up in the EGI Configuration Database as 3 Service entities:
fully qualified hostname | ServiceType |
---|---|
myhost.domain.org | CE |
myhost.domain.org | UI |
myhost.domain.org | unicore6.Gateway |
Note that a single host can also specify multiple services of the same
+ServiceType
.
There are different pages in GOCDB where Service entities are listed:
Each Service entity also has its own listing page. By clicking the link to view +it, you can see all associated information.
Provided you have proper permissions (check the permissions matrix in the +Permissions_associated_to_roles section), you can add a Service entity by:
The editing process will show you the same form as the adding process. To edit +Service entities, simply click the “edit” link on top of the entities' +details page.
To delete a Service entity you have permissions on, simply click on the +“delete” link on top of the entities’ details page. The interface asks for +confirmation before proceeding.
A Service entity may optionally define Service Endpoint entities which model
+network locations for different service functionalities that can’t be described
+by the main ServiceType
and URL
alone.
For example: The Service entity +goc.egi.eu (of +ServiceType egi.GOCDB) defines the following Service Endpoint entities:
Name | URL | Interface Name |
---|---|---|
ProductionPortalInstance | https://goc.egi.eu/portal | egi.GOCDB.Portal |
Production PI base URL | https://goc.egi.eu/gocdbpi | egi.GOCDB.PI |
This indicates whether the Service entity is a beta service or not (part of the +staged rollout process).
This is the DN of the host certificate for the service. The format of the DN
+follows that defined by the
+OGF Interoperable Certificate Profile
+which restricts allowed chars to a PrintableString
that does NOT contain
+characters that cannot be expressed in printable 7-bit ASCII. For a list of
+allowed chars.
To supply multiple or alternate DN(s) for a service, for example of the multiple +hosts supporting a single Service entity, see +standard extension properties.
The Service entities’ Production flag indicates if this service delivers a +production quality service to the infrastructure it belongs to (EGI).
ServiceType
is a VOMS
or emi.ARGUS
)This flag is taken into account by monitoring tools.
All production Service entities MUST be monitored (except for emi.ARGUS
and
+VOMS
ServiceTypes).
ServiceType
associated to the
+entities is one of those included in Availability computation)YES
, the
+Monitoring Service will test the Service entity, but the
+test results are ignored by the Availability Computation Engine (ACE).NO
, the Service entities is
+ignored by the Monitoring Service, and no alarms are
+raised in the Operations Dashboard in case of CRITICAL
failure.CRITICAL
failure of the test. These
+alarms are visible in the Operations Dashboard and are tagged as “non
+production”.A service group is an arbitrary grouping of existing service endpoints that can +be distributed across different physical sites and users that belong to the SG +(SGs were previously known as ‘Virtual Sites’):
Property | Value |
---|---|
Name | Configuration Database |
Description | Central registry of the infrastructure topology |
URL | https://goc.egi.eu |
Support Email | gocdb-admin <at> mailman.egi.eu |
Helpdesk Support Unit | EGI Services and Service Components I__ Configuration Database (GOCDB) |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=335 |
Supplier | UKRI |
Roadmap | N/A |
Release notes | Release notes |
Source code | https://github.com/GOCDB |
Issue tracker for developers | https://github.com/GOCDB/gocdb/issues |
Licence | Apache 2.0 |
Privacy Notice | Privacy notice |
Information to provide when registering a new service (the fields marked with an +asterisk are mandatory):
In the EGI Configuration Database, a service type is a technology used to +provide a service. Each service endpoint is associated with a service type. +Service types are pieces of software while service endpoints are a particular +instance of that software running in a certain context.
Service.Type
attribute and is
+defined as the “Type of service according to a namespace based classification
+(the namespace MAY be related to a middleware name, an organisation or other
+concepts)”.org.openstack.swift
).These service types are used at some grid sites within EGI but aren’t EGI +operational tools or a part of the core middleware distributions.
To request a new service type, please submit a request for a new service type +(see the section “Adding a new service type”).
In the following section there is the list of “middleware agnostic” service +types. You can obtain the whole list of service types by browsing +Poem, or by launching the +following query to the GOCDBPI interface:
Please feel free to make a request for a new service type. All new service type +requests need to be assessed by EGI via lightweight review process (by +EGI OMB and EGI Operations) so that only suitable types +are added, and to prevent duplication.
You can submit your request via EGI Helpdesk to the +“Configuration and Topology Database (GOCDB)” support unit.
Please specify the following information as part of your request:
Note: please provide a suggested service type name following the naming scheme +described above (technology provider’s reversed domain . software name) and a +brief sentence to describe the service type.
A site (also known as a Resource Centre) +is a grouping of grid resources collating multiple Service Endpoints (SEs). +Downtimes are recorded on selected SEs of a site. GOCDB stores the following +information about sites (non exhaustive list). Note, when editing values in the +portal, mandatory fields are marked with ‘*’:
A site listing page shows a listing of all the sites in the database, with +controls to page through the listing. The table headers can be clicked to set +the ordering (ascending or descending).
Each site also has its own listing page. By clicking the link to view a site, +you can see all of the site’s information
Provided you have proper permissions (check the permissions matrix in the +related section +, you can add a site by clicking on the Add a New Site link in the sidebar. +Simply fill the form and validate.
Note: If you just registered as site admin and want your new site to be +registered in GOCDB, please contact your NGI representative.
The editing process will show you the same form as the adding process. To edit a +site, simply click the “edit” link on top of the site’s details page.
Provided you have permissions, you can change the Short Name, Official Name and +GIIS URL to the new Resource Center details. For more information regarding the +site renaming procedure please see PROC15
Site deletion is not allowed in GOCDB. If a site stops operation, its +certification status should be set to “closed” (see the next section for more +information).
For each site that delivers to the ‘Production’ Target Infrastructure, GOCDB +stores and shows information about its certification status. This reflects the +different steps of the official SA1 site certification procedure which typically +follows:
Candidate -> Uncertified -> Certified.
The different possible certification statuses are:
Clarifications:
The following site state transitions are allowed:
The following transitions are explicitly forbidden:
Going with the definition of the suspended status, +Operations Centre managers have to regularly give +their attention to all their suspended sites, so that they are processed within the +given maximum time of four months. Sites being in suspended should either be set to +closed or brought back in production via the uncertified status.
More information about site certification statuses can be found in the +EGI Federation Procedures:
Note: Site certification status cannot be changed by site administrators, and +requires intervention of Operations Centre staff.
When a registered user applies for a role, the request has to be validated by +someone who has the proper permissions to grant such a role. If you request a +role on a given entity, any user with a valid role on that entity or above will +be able to approve your request.
Example - If you request a “site administrator” role on site X, then the +following users can approve your request:
Role requests you can approve are listed on the Manage roles page +(accessible by clicking the Manage roles link in the user status panel in +the sidebar).
In order to approve or decline role requests, simply click on the accept or +deny links in front of each role request.
If a user within your scope has a role that needs to be revoked, you can do this +from the user’s page, where user’s details are listed along with his/her current +roles. To revoke a role, simply click on the role name then on the revoke +link at the top right of the role’s details page.
Note: This works for other users within your scope but also for yourself. +However just note that if you revoke your own roles you may not have proper +permissions to recover them afterwards.
In the sub-pages there is an explanation over the EGI Configuration Database +user accounts, how to manage them, and the roles defined.
Approving/revoking accounts, roles and other actions
Understanding and manipulating user accounts
Understanding and manipulating roles
The EGI Configuration Database UI attempts to authenticate you in one of two +ways (the REST style API applies X.509 only):
The editing process is the same as the registration process. To edit your user +account, simply follow these steps:
Each user account has its own user details page which is accessible to anyone +with a valid certificate.
There is currently no facility for listing all users in the database. List of +users that have a role on a given site appears on site details pages (see +section about sites). It is also possible to search for a user’s account using +the search feature on the sidebar.
If you wish to unregister from the Configuration Database, follow these steps:
Your account will then be deleted along with any roles the account has.
Under the following circumstances it is possible to lose access to an account:
In these situations, it is usually possible to regain access using to your +certificate based account by following one of the following procedures:
If for any reason you were unable to complete the relevant procedure (e.g. mail +confirmations problems) please open an EGI Helpdesk ticket +addressed to the “Configuration and Topology Database (GOCDB)” support unit.
You can only associate one X.509 DN with your account at any given time.
It’s possible to link this identity with your “other” +EGI Check-in identity at the level of the EGI +Check-in, see +account linking documentation.
Once your identity is linked at the EGI Check-in level, if you are still having +problems accessing, please reassign the ticket to “Configuration and Topology +Database (GOCDB)”
You can link these identities at the EGI Configuration Database level by +following these steps. The steps assume an existing X.509 based account and that +you are currently authenticated via EGI Check-in, though the steps should hold +for any pair of supported authentication methods.
Registered users with a user account will need at least one role in order to +perform any useful tasks.
The only difference between C and C’ users is that:
The difference between D and D’ users is that:
Roles and permissions are based on whether the considered object is owned or +not. In the table below the following definitions apply:
Each role has a set of associated permissions which apply on the role’s scope +(site, region or project). Main permissions are summarised in the table below
Action | A) Unregistered users | B) Registered users with no role | C) Site level users | C’ ) Site Management Level Users | D) NGI level users | D’ ) NGI Management Level Users | E) Project level users |
---|---|---|---|---|---|---|---|
Add a site to an owned group | irr. | irr. | irr. | irr. | no | yes | irr. |
Add a site to a non owned group | no | no | no | no | no | no | no |
Add a service endpoint to an owned site | irr. | irr. | yes | yes | yes | yes | irr. |
Add a service endpoint to a non owned site | no | no | no | no | no | no | no |
Add a downtime to an owned service endpoint | irr. | irr. | yes | yes | yes | yes | irr. |
Add downtime to a non owned service endpoint | no | no | no | no | no | no | no |
Update information of an owned site | irr. | irr. | yes | yes | yes | yes | irr. |
Update information of a non owned site | no | no | no | no | no | no | no |
Update certification status of an owned site | irr. | irr. | no | no | no | yes | yes |
Update certification status of a non owned site | no | no | no | no | no | no | yes |
Update information of a owned service endpoint | irr. | irr. | yes | yes | yes | yes | irr. |
Update information of a non owned service endpoint | no | no | no | no | no | no | no |
Update information of an owned group | irr. | irr. | irr. | irr. | yes | yes | irr. |
Update information of a non owned group | no | no | no | no | no | no | no |
Update own user account details | irr. | yes | yes | yes | yes | yes | yes |
Update other user’s account | no | no | no | no | no | no | no |
Update a downtime on an owned service endpoint | irr. | irr. | yes | yes | yes | yes | irr. |
Update a downtime on a non owned service endpoint | no | no | no | no | no | no | no |
Delete an owned site | irr. | irr. | no | no | no | no | no |
Delete a non owned site | no | no | no | no | no | no | no |
Delete an owned service endpoint | irr. | irr. | yes | yes | yes | yes | irr. |
Delete a non owned service endpoint | no | no | no | no | no | no | no |
Delete an owned group | irr. | irr. | irr. | no | no | no | irr. |
Delete a non owned group | no | no | no | no | no | no | no |
Delete a downtime on an owned service endpoint | irr. | irr. | yes | yes | yes | yes | irr. |
Delete a downtime on a non owned service endpoint | no | no | no | no | no | no | no |
Delete your own user account | irr. | yes | yes | yes | yes | yes | yes |
Delete other user’s account | no | no | no | no | no | no | no |
Register a new user account | yes | irr. | irr. | irr. | irr. | irr. | irr. |
Request a new role | no | yes | yes | yes | yes | yes | yes |
Approve a role request on an owned group | irr. | irr. | no | no | no | yes | yes |
Approve a role request on an owned site | no | no | no | yes | no | yes | irr |
Approve a role request on a non owned site or group | no | no | no | no | no | no | no |
Reject a role request on an owned group | no | no | no | no | no | yes | irr. |
Reject a role request on an owned site | no | no | no | yes | no | yes | irr |
Reject a role request on a non owned site or group | no | no | no | no | no | no | no |
Revoke an existing role on an owned object | irr. | irr. | no | yes | no | yes | irr. |
Revoke an existing role on a non owned object | no | no | no | no | no | no | no |
Retrieve an existing account/ change certificate DN | yes | yes | yes | yes | yes | yes | yes |
There are 2 ways to request new roles.
Once made, role requests have to be validated before the role is granted to you. +This part of the process is described in the next section.
Use this section to get started quickly with internal EGI services:
This section contains guidelines for software development to be considered when +developing a product for the EGI Federation.
You can ask for more information about them by contacting us via +our site.
GGUS users should be able to manage their GGUS account data by themselves.
Information about registration and account management is collected on +GGUS registration page and could be +reached by clicking the “Registration” link on top of +GGUS home page.
For managing the GGUS account data the user has to click the link “Check your +GGUS account” at the bottom of this page.
Users should update their certificate information before their old certificate +expires. They can navigate to the +registration page, click the link “Check +your GGUS account” and update their data. Modifying the DN field will result in +an additional DN field: the old DN will be kept and the new DN will be added. +Subsequently, the user is identified with the two DNs. If the old DN is no more +valid please just clear the DN field and save the change. Once the old +certificate has expired before updating the account data, users can request an +update via GGUS ticket against Helpdesk(GGUS) support unit.
Pre-requisites: the email address of your EGI SSO account must be equal to the +email address registered in GGUS!
For adding your EGI SSO persistent ID to your account data please do the +following:
In order to process tickets assigned to the support unit you belong to, or in +general tickets submitted by other users and assigned to other support units, +you need to own supporter privileges. Therefore users need to register an +account at GGUS. Registration can be done either using an X509 personal +certificate or using the EGI AAI account.
For registering with an X509 certificate the user should go to +GGUS home and click the +registration link in the menu bar at the +left. This link opens the registration information page which gives some +additional information about registration process. Clicking on the +registration link guides to the form that +the user has to fill in. After filling in the registration form, GGUS team will +check whether support privileges can be granted. The user will receive an email +from the GGUS team (usually) confirming their supporter privileges.
Users who do not have a valid X509 certificate can access GGUS via +EGI Check-in. For getting support privileges the +user needs to be member of the +ggus-supporters group +in EGI Check-in. However the user should fill in the +registration form for creating an account +at GGUS. Support privileges will be granted automatically. The user will receive +an automated email from GGUS system confirming their support privileges.
Both authentication methods, X509 certificate and EGI Check-in account, will be +guaranteed in the future. Since the use of login and password will no longer be +guaranteed in the future, it is recommended to access GGUS with a valid digital +certificate or EGI Check-in.
Pre-requisites: the email address of your EGI SSO account must be equal to the +email address registered in GGUS!
Unregistered users have to register first. On +registration page you will see this text: +“You are member of EGI AAI CheckIn ggus supporter. You will get automatically +support right for the GGUS portal once you will submit this form.“ Support +permissions will be granted automatically.
Click on the “Add my ….” button.
You can decide to add the Shibboleth ID to the existing account and check the +updated account. In case the email addresses are different you may either +harmonize your email address in EGI SSO and GGUS account or create a new account +using EGI SSO data.
The preferred way to get in contact with the GGUS team is by submitting a GGUS +ticket against Helpdesk (GGUS) support unit.
People who want to become an alarmer have to
GGUS system synchronizes its user database once per night with the VOMS servers. +The synchronization is based on the DN string. Please make sure the DN of your +GGUS account is the same than the one registered in VOMS.
Tier-1 site admins can become alarmers for testing the alarm process for their +site. They have to fill in the +registration form for supporters. Please add +an appropriate comment (e.g. “I’m a xxx tier-1 site admins and I want to become +an alarmer for testing purposes.”) in the registration form.
This section describes the workflows of alarm tickets from a technical point of +view.
Alarm tickets can be submitted using the GGUS web portal. On top of the ticket +submit form in GGUS web portal there is a link to the submit form for alarm +tickets.
As alarm ticket submitters are experts who will hopefully provide all necessary +information, the number of fields on the alarm ticket submit form is reduced to +a minimum compared to the number of fields on the user ticket submit form.
Three fields on this form are mandatory:
All other fields are optional.
The processing of a team ticket consists of two main parts: the notification of +the notified site and the routing of the ticket to the NGI/ROC the site belongs +to.
In parallel to the creation of an alarm ticket, the GGUS system sends an alarm +email directly to the tier 1 site specified in field “Notify SITE”. This email +is sent to a specific site alarm mail address and signed with the GGUS +certificate. The tier-0 alarm mail address is based on the VO name. It is +“voname”-operator-alarm"atnospam"cern.ch. Tier-1 site alarm mail addresses are +taken from the “Emergency Email” field in GOC DB. For tier-1 sites registered in +the OSG OIM DB the alarm email address is taken from field “SMSAddress” of the +“Administrative Contact” in OIM DB. The DN of the GGUS certificate is +/C=DE/O=GermanGrid/OU=KIT/CN=ggusmail/ggus.eu. The alarm mail looks like the +following figure.
Alarm tickets are bypassing the TPMs and routed to the appropriate NGI/ROC +automatically. The decision to which NGI/ROC a ticket has to be routed is done +automatically, based on the value of the “Notify SITE” field. The “Notify SITE” +drop-down menu shows both the tier 1 site names from GOC DB and the tier 1 site +names used by WLCG.
Once the tier 1 site received the alarm email the receipt should be confirmed. +Sending a reply mail containing the typical GGUS identifier and the ticket ID +“GGUS-Ticket-ID: #00000000” in the subject is sufficient. Such a reply will be +added to the alarm ticket.
For working on alarm tickets and resolving them please use the GGUS portal. A +reference link to the alarm ticket is given in the alarm email notification.
Alarm ticket testing is documented in +WLCG twiki.
The pages of this section detail multiple features of the EGI Helpdesk.
List of the values of the Issue Type field for the several ticket types
Selecting a Resource Centre as recipient and submitting tickets to multiple RCs
Definition and usage of the Alarm tickets
Quality of Support (QoS) levels
Features and usage of the Report generator
Definition and usage of the Team tickets
Definition and computation of the ticket priority in relation to the QoS levels
Description of ticket scopes
// jscpd:ignore-start
For generic user tickets selectable issue types are explained in the table +below.
Issue type | Description |
---|---|
Accounting | All kind of issues related to accounting tools (APEL, Accounting Portal and so on). |
AppDB | All kind of issues related to the AppDB or to the interactions with the AppDB. |
Authorization/Authentication | All issues related to authorization and authentication e.g. certificates |
COD Operations | WLCG Coordinator On Duty Operations |
Catalogue | Issues related to file catalogues, like LFC |
Computing Services | All issues related to execution of jobs. It combines two other type of problems (‘Workload management’ and ‘Local batch system’). The ‘Computing Services’ should be used when it is not clear to which subcategory the ticket refersConfiguration Issues of system configuration. |
Data Management - generic | All kind of issues related to data management tools (GFAL, LCG_Util and so on). |
Databases | Issues related to Databases. |
Deployment - other | Issues of software deployment that do not fit any other category. |
Documentation | Issues of missing, wrong, outdated documentation. |
File Access | File access issues. |
File Transfer | File transfer issues, e.g. related to FTS |
GGUS | Bugs and feature requests related to GGUS system. |
IaaS Operations | Issues related to VMs and clients (OCCI, rOCCI, etc.), block storage, networking. |
Information System | Issues related to the BDII. |
Installation | Installation issues. |
Local Batch System | Issues related to the local batch systems. |
Middleware | Issues of middleware stacks like gLite, globus and others. |
Monitoring | Issues of infrastructure and service monitoring. |
Network problem | Issues of network connectivity. |
Operations | Issues of general processes, procedures, information and so on of the entire infrastructure. |
Other | Requests that do not match to any other issue type. |
Security | Issues of infrastructure and operation security. |
Storage Systems | Issues concerning storage systems, like EOS, dCache, DPM |
VO Specific Software | Issues of VO specific software tools and packages. |
Virtual Appliance Management | Issues related to vmcatcher/vmcaster tools, and VA management in general. |
Workload Management | Issues related to workload management system and tools. |
Team and alarm tickets have a reduced number of issue types in the drop-down +menu.
Issue type | Description |
---|---|
Databases | Issues related to Databases. |
File Access | File access issues. |
File Transfer | File transfer issues, e.g. related to FTS |
Local Batch System | Issues related to the local batch systems. |
Middleware | Issues of middleware stacks like gLite, globus and others. |
Monitoring | Issues of infrastructure and service monitoring. |
Network problem | Issues of network connectivity. |
Other | Requests that do not match to any other issue type |
Storage Systems | Issues related to storage systems, like EOS, dCache or DPM |
Issue type | Description |
---|---|
Databases | Issues related to Databases. |
File Access | File access issues. |
File Transfer | File transfer issues, e.g. related to FTS |
Local Batch System | Issues related to the local batch systems. |
Middleware | Issues of middleware stacks like gLite, globus and others. |
Monitoring | Issues of infrastructure and service monitoring. |
Network problem | Issues of network connectivity. |
Other | Requests that do not match to any other issue type |
Storage Systems | Issues related to storage systems, like EOS, dCache or DPM |
The CMS VO has an own ticket submit form in the GGUS system. Although this form +provides CMS specific issue types no special privilege is required to use it. +This does not apply for the TEAM ticket submit form: Here the CMS specific issue +types are only selectable for registered CMS TEAM members.
Issue type | Description |
---|---|
CMS_AAA WAN Access | Issues around WAN |
CMS_CAF Operations | Issues around CAF operations at CERN, incl EOS space requests |
CMS_Central Workflows | Issues around centrally managed MC production and processing |
CMS_Data Transfers | Issues around data transfers, e.g. via Phedex or ASO |
CMS_Facilities | Typically issues at CMS sites |
CMS_HammerCloud | Issues around CMS HammerCloud |
CMS_Register New CMS Site | Chosen when a new site gets registered with CMS |
CMS_SAM tests | Issues around CMS SAM tests |
CMS_Submission Infrastructure | Issues around CMS_Submission Infrastructure |
CMS_Tier-1 Tape Families | For creation of Tape families/groups at Tier-1 archives |
The ATLAS specific issue types are only selectable for registered ATLAS TEAM +members.
Issue type | Description |
---|---|
ATLAS_ADC Central | Issues around ADC in general |
ATLAS_Databases | Issues around databases |
ATLAS_Deletion | Issues around file deletions |
ATLAS_File Access/Transfer | Issues around file access or file transfers |
ATLAS_Frontier-Squid | Issues around frontier or squid |
ATLAS_Local Batch System | Issues around batch systems |
ATLAS_Middleware | Issues related to middleware |
ATLAS_Monitoring | Issues around monitoring |
ATLAS_Network Problem | Issues with network infrastructure |
ATLAS_Staging | Issues around file staging |
ATLAS_Storage Systems | Issues around storage |
// jscpd:ignore-end
QoS stands for Quality of Support. It describes the level of support provided by +Support Units in GGUS system.
It has an impact on the ticket +priority colour in GGUS and the warnings are sent +to SUs if 75% of the maximum response time of the QoS level are over.
There are three different QoS levels, each defining different response times for +given Ticket Priority.
The default QoS level, if not declared differently, is Base.
Base QoS level defines a response time of 5 working days regardless of the +ticket priority.
Ticket Priority | Response time |
---|---|
less urgent | 5 working days |
urgent | 5 working days |
very urgent | 1 working day |
top priority | 1 working day |
Ticket Priority | Response time |
---|---|
less urgent | 5 working days |
urgent | 1 working day |
very urgent | 1 working day |
top priority | 4 working hours |
The QoS level for all of the SUs are available +here
For the several EGI services, the QoS levels are defined in the specific +Operational Level Agreements linked in the +EGI OLA SLA framework page
The GGUS Report Generator is available +from the support section.
The implementation of the report generator started in October 2011. Hence, the +report generator does not provide data for tickets submitted before December +2011!
The response time is a performance figure calculated from the support unit’s +point of view. It describes how quick a support unit is reacting on tickets. +Response time is the time from
The response time is calculated as difference between the timestamp changing the +status or re-assigning the ticket and the assign timestamp. While assigning a +ticket to a support unit the expected response timestamp is calculated by adding +an amount of time to the assign timestamp. The amount of time added depends on +the ticket priority and the kind of support unit. For support units that have +declared a quality of service level the response times are defined by the +QoS level. For all the other support units a +medium QoS is assumed for calculating the expected response timestamp. In case +the actual response timestamp is greater than the “expected response” timestamp +for middleware support units the “violate” flag is set.
Response times are based on office hours. Hence the results unit is working +days.
The solution time is a performance figure also calculated from the support +unit’s point of view. It describes how long it took the support unit for +providing a solution. Solution time is the time from assigning a ticket to a +support unit until it provides a solution for the problem described. “Providing +a solution” means setting the ticket status to “solved” or “unsolved”. The +solution time is calculated as the difference between the solution timestamp and +the assign timestamp.
Solution times are based on office hours. Hence the results unit is working +days.
Waiting time is the sum of all time slots the ticket was set to “waiting for +reply”. Calculating the waiting time has started in July 2012. For tickets +submitted before July 2012 no waiting time calculation was done. The waiting +time for these tickets may be zero. The waiting time can be excluded when +calculation solution times by ticking the checkbox “exclude waiting time”.
The ticket lifetime is calculated from the user’s point of view. It describes +how long it takes to provide a meaningful solution for a problem reported by the +user. Ticket lifetime is the time from ticket submission to ticket solution +(status “solved/unsolved”).
The ticket lifetime is based on calendar days.
GGUS support units are spread over a wide range of time zones. Some of the +support units themselves are spread over several time zones. However most +support units are located in European time zones. Support units and their time +zones are listed on a dedicated page. +For middleware product teams GGUS assumes time zone “UTC +1” for all support +units. For all the other support units the system uses the relevant time zone +for calculating timestamps as far as possible.
The systems assumes usual office hours from 09:00 to 17:00 Monday to Friday for +all support units. National holidays are not taken into account. For middleware +product teams the timezone UTC+1 is used as default timezone.
For the calculation of performance figures the original priority set during +ticket submission is used. This priority value is kept as long as the support +unit is in charge of the ticket. Updating the priority value during ticket +lifetime doesn’t affect the calculation of performance figures.
[wd]
means working days[d]
means calendar daysAlthough some of the drop-down lists request the selection of values a query +can be started anyway. In this case the query is equivalent to a query over +all tickets!
The “Reset” button resets all fields to their default settings besides the time +frame.
This metric gives the number of tickets submitted within the specified time +frame. Major criteria is the submit timestamp. The result lists shows the +current status of the tickets.
The open tickets time report calculates the time from ticket submission until +now in calendar days. The submit timestamp must match the specified time +frame for the report.
This metric is focused on the ticket life time. It gives the number of +tickets that reached the status “solved” or “unsolved” within the specified time +frame. Major criteria is the solution timestamp which is the timestamp +setting a ticket to “solved” or “unsolved”. Tickets in other terminal status +like “verified” or “closed” appear in the result list as long as they have been +set to “solved/unsolved” in the given time frame. Besides date, status and the +number of closed tickets the results list displays the average ticket lifetime +and the median ticket lifetime. The result lists shows the current status of the +tickets.
This metric focuses on the responsiveness of support units. Major criteria for +this report is the submit timestamp which must match the selected time +frame. The result list shows:
for the specified time frame. The result list shows all support units that have +ever been in charge of a ticket. Hence the same ticket ID may appear several +times. The number of responses may be greater than the number of tickets.
Response times are based on office hours. Hence an average response time of 1d +3h 23min means 13 hours and 23 minutes in total.
Specific privileges are required for this report. Only people belonging to a +dedicated technology provider (TP) are able doing reports for this TP. Major +criteria for this report are the expected response time defined in SLAs and +the TP.
Major criteria for this report is the solution timestamp. The result list +shows:
for the specified time frame. The result lists shows the current status of the +tickets.
Solution times are based on office hours. Hence, an average solution time of +12d 3h 5min means 99 hours and 5 minutes in total.
The waiting time can be excluded by ticking the checkbox “exclude waiting time”. +In case a ticket gets re-opened the metrics calculation starts again from +scratch. Hence, the same ticket can appear several times in the solution times +calculation.
The input parameters vary depending on the report type chosen. Possible input +parameters are:
The time frame defines begin and end date of the report. The begin date starts
+at 00:00:00
. The end date ends at 00:00:00
. As the implementation of the
+report generator started in October 2011 the report generator does not provide
+data for tickets submitted before December 2011!
The drop-down list offers all responsible units integrated in GGUS system. They +can be filtered by keywords. Responsible Units can be either selected all, one +by one or by checking the boxes in front of the responsible unit groups. In case +no responsible units are selected all responsible units are considered in the +reports.
The drop-down list offers all status values available in GGUS system. Multiple +selections are possible.
The drop-down list offers all priority values available in GGUS system. Multiple +selections are possible.
“Concerned VO” provides a drop-down list of all VOs supported by GGUS. Multiple +selections are possible.
This drop-down list lists all ticket types in GGUS. Multiple selections are +possible.
Selectable categories are “Incident”, “Change request”, “Documentation”, “Test”. +Multiple selections are possible. The category “Test” should not be considered +for all reports and therefore, if necessary, be excluded from the selection.
To distinguish between tickets under the responsibility of EGI or WLCG. See +ticket scope.
“Notified site” lists all sites integrated in GGUS. They are derived from GOC DB +and OIM DB. In case of reporting exclusively on tickets without any site value +specified please ticket the “blank” value in the drop-down list.
This input parameter is only visible if choosing the “violated response time” +report. For accessing this report specific privileges are required. The +drop-down list offers all technology providers currently integrated in GGUS and +the “DMSU”. Multiple selections are possible. In case no technology provider is +selected the reports will be done for tickets without any technology provider +specified.
The results aggregation level can be chosen from the drop down list “Choose date +aggregation”.
Results can be grouped by one or more of the input parameters.
The results are displayed below the input parameter area. They can be sorted in +different ways by clicking on the column labels. A drill-down is possible by +clicking on any row of the results list. The detail results open in a new +window. They can be sorted by clicking the column labels too. Clicking on the +ticket ID opens the ticket in GGUS system. At the bottom of the result panel +there are various icons offering features like:
Doing a simple “Export” saves the data in a csv file stripping all column +headers. For exporting the data including the column headers please use “Export +what you see”. The “Search” feature allows searching in the result list. +Possible search parameters are the columns of the result list.
People who want to become a team member have to:
GGUS system synchronizes its user database once per night with the VOMS servers. +The synchronization is based on the DN string. Please make sure the DN of your +GGUS account is the same to the one registered in VOMS.
This section describes the workflows of team tickets from a technical point of +view.
Team tickets can either be submitted using the GGUS web portal. On top of the +ticket submit form in GGUS web portal there is a link to the submit form for +team tickets.
As team ticket submitters are expected to be experts who will hopefully provide +all necessary information, the number of fields on the team ticket submit form +is reduced to a minimum, compared to the number of fields on the user ticket +submit form
Three fields on this form are mandatory:
All of the other fields are optional.
The processing of a team ticket consists of two main parts, the notification of +the notified site and the routing of the ticket to the NGI/ROC the site belongs +to.
In parallel to the creation of a team ticket the GGUS system sends an email +notification directly to the site specified in field “Notify Site”. This email +is sent to a specific site contact mail address.
Team tickets are bypassing the TPMs and routed to the appropriate NGI/ROCs +automatically as long as field “Routing type” is not changed. The decision to +which NGI/ROC a ticket has to be routed is done automatically, based on the +value of the “Notify Site” field.
For working on team tickets and resolving please use the GGUS portal. A +reference link to the team ticket is given in the team notification mail. Team +tickets can be upgraded to alarm tickets clicking on the button “Click to +convert to ALARM”.
See the +schema of the Team Tickets process.
If you plan to adjust the ticket priority from the default value ’less urgent' +to ‘urgent’, ‘very urgent’ or ’top priority’, please make sure you add a +justification for this change in the problem description field.
The following table will help you to find an appropriate priority value:
Priority | Comment |
---|---|
top priority | service interrupted; needs to be addressed as soon as possible |
very urgent | service degraded; no workaround available |
urgent | service degraded; workaround available |
less urgent | wishes and enhancements that are “nice to have” |
In particular, be very economical when choosing ’top priority’. This value, when +reaching the supporters via another ticketing system interface, might become a +beep or phone alert even in the middle of the night. This level of support is +ONLY committed by WLCG Tier0 and Tier1s and ONLY for ALARM tickets.
Finally, please be aware that the supporter who will try to solve your problem +may change the value you have chosen to a more realistic one, putting their +justification in the ticket’s public diary.
The priority colours in GGUS should help supporters getting an overview of the +most important tickets. The priority colours have no impact on the metrics +generated with the GGUS report generator or manually by the GGUS team.
Currently the following colours are used for open states:
For terminal states GGUS uses:
The priority colour algorithm is affected by the status value and the priority +of a ticket. The priority colour calculation for tickets in status “assigned” is +based on different values than for tickets in any other open status.
Office hours and weekends are taken into consideration when calculating the +priority colour. The system assumes usual office hours from 07:00 to 15:00 +o’clock UTC from Monday to Friday for all support units. Different time zones +are considered as far as possible.
The priority colour calculation starts after a ticket gets assigned to any +support unit. The support units should change the ticket status to “in progress” +or any other open status after receiving the ticket. Thus the support unit has +responded to the ticket (response time). After such a status change the priority +is reset to “green” and the calculation of the priority colour starts again from +scratch.
The priority colour for tickets dealing with middleware related problems is +calculated according to the Quality of Support the PTs voted for. The priority +colour changes after a dedicated amount of working hours as listed in the table +below. The priority colour calculation is processed every 15 minutes.
QoS Level | Ticket priority | Colour | Working hours | Colour | Working hours | Colour | Working hours |
---|---|---|---|---|---|---|---|
Base | less urgent | Yellow | 20 | Amber | 30 | Red | 40 |
urgent | 20 | 30 | 40 | ||||
very urgent | 20 | 30 | 40 | ||||
top priority | 20 | 30 | 40 | ||||
Medium | less urgent | Yellow | 20 | Amber | 30 | Red | 40 |
urgent | 20 | 30 | 40 | ||||
very urgent | 4 | 6 | 8 | ||||
top priority | 4 | 6 | 8 | ||||
Advanced | less urgent | Yellow | 20 | Amber | 30 | Red | 40 |
urgent | 4 | 6 | 8 | ||||
very urgent | 4 | 6 | 8 | ||||
top priority | 2 | 3 | 4 |
The priority colour changes after a dedicated amount of working hours as listed +in the table below. The priority colour calculation is running every 15 minutes.
Ticket priority | Colour | Working hours |
---|---|---|
less urgent | Yellow | 20 |
Amber | 30 | |
Red | 40 | |
urgent | Yellow | 20 |
Amber | 30 | |
Red | 40 | |
very urgent | Yellow | 16 |
Amber | 24 | |
Red | 32 | |
top priority | Yellow | 12 |
Amber | 18 | |
Red | 24 |
The GGUS ticket scope filter is available in GGUS ticket search. It offers +filtering for tickets in EGI or WLCG scope. The ticket scope field is +automatically set by the system based on the qualifications listed below.
WLCG scope tickets are either:
TEAM
or ALARM
related to VOs alice
, atlas
, cms
, lhcb
,
+belle
VOSupport
related to VOs alice
, atlas
, cms
, lhcb
,
+belle
OSG Software Support
, USCMS
, USATLAS
, USBELLE
, CRIC
CMS_
, ATLAS_
All other tickets are EGI scope.
Support for EGI services is available through the +EGI Helpdesk.
The EGI Helpdesk is a distributed tool with central coordination, which provides +the information and support needed to troubleshoot product and service problems. +Users can report incidents, bugs or request changes.
The support activities are grouped into first and second level support.
Technical details of EGI Helpdesk
Managing account and privileges
Helpdesk features
Guide for supporters
Guide for users
Workflows
Property | Value |
---|---|
Name | Helpdesk |
Description | Central helpdesk providing a single interface to EGI support |
URL | https://helpdesk.egi.eu |
Support Email | support at ggus.eu |
Helpdesk Support Unit | EGI Services and Service Components I__ Helpdesk (GGUS) |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=247 |
Supplier | KIT |
Roadmap | N/A |
Release notes | https://ggus.eu/index.php?mode=release_notes |
Source code | Not available |
Issue tracker for developers | N/A |
License | BMC Software Inc. |
Privacy Policy | https://ggus.eu/?mode=privacy |
Please see the release notes for +further information.
The EGI Helpdesk is reachable via a welcome page. +Another way for accessing the ticket management interface is +using the direct link provided in the notification emails sent after ticket +updates. However either a valid X509 personal certificate or a Check-in account +are required for accessing the system.
GGUS home provides a quick overview over tickets submitted by the current user, +the latest tickets of other users and a collection of news and links to useful +information. The navigation bar is located on the left side of GGUS pages. It +provides links to:
After each release, a major change or new feature is explained in some sentences +here.
In the documentation section there is a collection of links providing useful +information around GGUS system.
All information about registration can be found on the +registration page. For +registering as support staff, click the link “Apply” and fill in the +registration form. After registering successfully you will receive a +confirmation mail from GGUS team.
If you want to update your account data, you can do this using the link “Check +your GGUS account” on the registration page. If you can’t access GGUS web +interface due to changed DN string of your personal certificate, you can log +into the system using your Check-in account. Then go to the registration page +and click on Check/update your GGUS account. The system shows you all your user +data. It detects the new DN string of your browser automatically. Just save the +changes by clicking on button Update now. Additional information on GGUS +accounts is available here.
Access to the support staff page is restricted to users having +support privileges. Depending on further privileges +people may have there are links to e.g. news administration and other features. +All support staffs can use the GGUS report generator and the GGUS ticket +timeline tool as well as links to other information useful for support staffs.
The link to the GGUS ticket timeline tool is located on support staff page. The +ticket timeline tool provides a graphical overview of tickets concerning a +selected site and time range. It shows all tickets that have been updated during +the selected time range. When moving the mouse over one of the colored bars some +additional information is displayed. Clicking on the ticket ID opens a new +window showing the ticket details and the modify section of the ticket.
The link to the GGUS Report Generator is located on support staff page and on +GGUS home page in section “GGUS tools/reports”. The GGUS Report Generator could +be used for generating statistics and reports for all support units in GGUS. +Further information on the report generator is available on the +report generator.
Depending on the user’s privileges GGUS offers different ticket submit forms:
GGUS offers two fields which help to classify various tickets into categories +and types.
The ticket category is for differentiating between incidents and service +requests. This distinction is helpful for supporters as well as for the GGUS +reporting, e.g. for excluding test tickets. Other categories were added over the +time. Currently the following values are available for selection:
When submitting a ticket the ticket category field is not visible. It defaults +to “Incident” and is only editable for supporters, not for users. So it is up to +the supporter to check and, if necessary, classify the ticket correctly. Please +see details in section Changing ticket category.
The ticket type field is for differentiating between standard user tickets and +tickets for achieving the special requirements of various groups like the LHC +VOs or EGI operations. It can’t be set manually, but is set automatically by the +system based on several rules. The ticket type field could not be changed during +ticket lifetime. Possible ticket types in GGUS are:
The ticket type USER is the default ticket type. User tickets are the usual +tickets which can be submitted by everyone. They are visible to everybody. They +can be updated either by the submitter himself or by any supporter.
The purpose of TEAM tickets is to allow a group of people to submit and modify +ticketseditable by the whole group. TEAM tickets can only be submitted by people +who have the appropriate permissions in the GGUS user database. These people +belong to either one of the four LHC VOs (ALICE, ATLAS, CMS, LHCB) or to the +BIOMED or BELLE VO and are nominated by the particular VO management. TEAM +tickets are editable by all members of the VO team (which are the so called VO +shifters) regardless own the ticket. TEAM tickets are visible to everyone. They +can be submitted for all tier-1 sites in EGI and are routed to the appropriate +NGI/ROC automatically, bypassing the TPM. Additionally the site is notified by +mail about the ticket. By default TEAM tickets are routed to the appropriate +NGI/ROC directly, bypassing TPM. But the submitter could also choose to route it +to the TPM instead. Further information on TEAM tickets is available in the +TEAM tickets page.
The purpose of ALARM tickets is to notify tier-1 administrators about serious +problems of the site at any time, independent from usual office hours. They can +only be submitted by experts who have the appropriate permissions in the GGUS +user database. These people belong to one of the four LHC VOs (ALICE, ATLAS, +CMS, LHCB) and are nominated by the particular VO management. They are about 3 +to 4 people per VO. ALARM tickets are editable by all members of the VO team +(which are the so called VO shifters) regardless they own the ticket. They are +visible to everyone. ALARM tickets can be submitted for all tier-1 sites in EGI +and are routed to the appropriate NGI/ROC automatically, bypassing the TPM. +Additionally the tier-1 site is notified by an ALARM email. It is up to the +tier-1 site how to deal with this ALARM email. Further information on ALARM +tickets is available in the ALARM tickets page.
This ticket type is used for operations tickets submitted via the operations +portal.
This dashboard can be used for getting quick access to any ticket of interest. +Each ticket has to be added manually.
The search engine provides a large number of fields for creating dedicated query +strings. Many of them have additional information hidden behind a question mark +icon. The results of a query are shown in a results list. The default search +displays all open tickets of last week. They were ordered by date of creation in +descending manor. For further details on using the search engine see chapter +Searching for tickets. The result list is showing a color schema reflecting the +priority of tickets. The algorithm used for setting the priority colours is +explained in chapter Reminder emails.
Various possibilities of searching tickets in GGUS are described in this FAQ. +Please avoid searching for “all” tickets or “solved” tickets without any +time-frame if not necessary for some reasons as these searches cause heavy load +on the machine.
Searching via ticket ID is the easiest and fastest way to look at a ticket. When +searching via Ticket ID all other search parameters are ignored. Besides +searching for all open tickets this is the recommended kind of search, because +it avoids needless workload on the system. When searching via ticket ID the +ticket details are shown in the same window. For getting back to the main page +use the “Back” button of your browser. For Firefox users there is a nice add-on +for adding a customized search for any web page to the browser’s search bar +available here.
The search parameters can be combined in any way wanted. Description fields +“Keyword”, “Involved supporter” and “Assigned to person” trigger a LIKE search +to the database. Concatenating keywords with “AND” or “OR” is currently not +possible. The result of a search by parameters is shown in the result list. For +viewing ticket details just click on the ID. A new window opens showing ticket +details. For getting back to the search result just close the window with the +ticket details.
You can customize the result list in various ways. One way to customize the +result list is by checking or un-checking the appropriate boxes in the blue bar. +The related columns will then be added or removed. Another way for customizing +the result list is by selecting another ticket order in field Order tickets by. +After changing the result list layout you have to refresh the search result by +clicking the Go! button.
Search results can be exported in csv or xml format for further processing. +After clicking on the appropriate link a new window opens showing the results in +the specified format. Out of this window you can save a local copy of this file.
By clicking on the ticket ID of a ticket in the results list of the search +engine you can access the ticket data. The ticket data is divided into 3 main +sections:
The system shows the information section after clicking on a ticket ID.It +provides an overview of all relevant ticket parameters and could be divided into +5 areas:
Additional features of the ticket information section are:
Figure 5: Ticket duplication Supporters have the opportunity to duplicate an +existing ticket up to 15 times. The duplicate feature is located right below +ticket information. It is useful if a ticket concerns not only one support unit +but has to be handled by several support units. The fields that are copied into +the duplicated tickets are:
Attachments are not duplicated physically but linked to all duplicated tickets. +The Responsible Unit is set to “TPM” by default.
Figure 3: Convert team to alarm ticket Support staffs with ALARM privileges are +able to convert TEAM tickets to ALARM tickets clicking on a button in the ticket +information section. This feature is only available for the WLCG VOs.
The ticket history is located below ticket information. It shows all relevant +changes of the ticket in chronological manner. Changes of these fields lead to a +new entry in ticket history:
For making the history more readable solution entries and entries in the public +diary are marked with green colour, entries of the internal diary with orange +colour.
The ticket modification area offers several fields for modifying a ticket. The +fields are described in detail below.
GGUS system offers various possibilities for participating in tickets. They are
An overview on these fields is given in the table below. Ticket participation +can be done by adding a valid mail address to one of these fields. Please avoid +adding closed mailing lists as such produce a lot of mail errors! Several mail +addresses have to be separated by semicolon.
User submit | User modify | Supporter modify | |
---|---|---|---|
CC | Yes | No | Yes |
Involve others | No | No | Yes |
Subscribe | No | Yes | Yes |
The CC field can be set by the user in the ticket submit form. Updates are only +possible for supporters for correcting or removing invalid mail addresses. Every +ticket update triggers a notification email to the mail address specified in the +“CC” field.
The “Involve others” field is only for supporters use. Every ticket update +triggers a notification email to the mail address specified in the “Involve +others” field.
This field can be used by any user for participating in tickets at any time. The +user has just to add his mail address. He will receive notifications for updates +of the public diary or the solution for following the whole ticket life cycle. +“Internal Diary” entries never go to the people who subscribed to a ticket.
Several tickets describing the same issue can be put into a master-slave +relation. One of them can be marked as master, the other ones as slave. Only the +master ticket has to be dealt with. The slave tickets are set to “on hold”. They +can’t be updated directly as long as they are in a master-slave relation. The +user gets an automated notification if a ticket is marked as slave. All updates +of the master were pushed to the slaves. When solving the master the slaves are +solved to. The master-slave relation is kept after the master is solved. +Nevertheless each ticket can be reopened separately. Updates on reopened slave +tickets are possible. A master-slave relation can be reset manually either by +removing the master ID of a slave ticket or by un-checking the master checkbox +of the master. If a master is unmarked as master all slaves were reset to +“standard” tickets automatically.
Marking a ticket as slave is only possible if there is already a master ticket. +If a ticket is marked as slave a popup window opens showing available master +tickets. For selecting a master just click on the ID. The master ID is set +automatically. Once you have chosen a wrong master ID click Reset Master-Slave +relation and select another one.
To show all related slave tickets click on link “show slaves for this ticket”. A +popup window opens showing the IDs of all slave tickets.
If you want to search for master or slave tickets you can do this using field +“Special attributes” of the search engine. The status value for searching is set +to an appropriate value accordingly.
Parent-child relationships work in reverse to master-slave relationships. The +parent ticket cannot be resolved until all of its child tickets are resolved. +The parent ticket is set to the status “on hold” while the child tickets are +waiting for their solution. For each solved child ticket a note is added to the +parent ticket history including the solution of the child ticket. After the last +open child ticket has been solved the status of the parent ticket changes to “in +progress” automatically. In addition, the system sends a notification mail to +the responsible support unit that all child tickets have been solved now. So the +parent ticket can be “solved” too.
A ticket can be selected as child ticket by checking the box “Mark this ticket +as child of ticket” and adding the ticket ID of the parent ticket. A comment is +added to the ticket history automatically stating “This ticket is a child ticket +of GGUS ticket # 18492”. Multiple child tickets can be related to one parent +ticket by repeating this procedure. The parent ticket is flagged as “parent” +automatically. A comment is added to the ticket history automatically stating +“This ticket is a parent ticket. It has to wait the solving of all its child +tickets. GGUS ticket #18493 is a child to this ticket.".
For resetting child tickets just remove the tick from the checkbox “Mark this +ticket as child of ticket”.
Selecting a parent ticket explicitly is not possible. The parent tickets are +flagged automatically by the system while the parent ID is specified for a child +ticket.
The search for parent or child tickets is similar to the search for master or +slave tickets. It can be done using field “Special attributes” of the search +engine.
This section is a description of how the GGUS ticketing system behaves. There +are other documents which describe the system in more detail and include more of +the implementation details. One of the most important fields of the system is +the status field. Many workflows are triggered by status value changes. Please +read the Short Guide for getting information on status values. Tickets are +normally assigned to a support unit. This means that the ticket notification is +sent to a mailing list composed of many supporters. One of these supporters +assigns the ticket to himself and takes responsibility for it; the supporter +changes the status to “in progress”. This person is then in charge of the +ticket. He either solves it or reassigns the ticket to somebody else. The status +of the ticket stays set to “in progress” if the ticket is under the +responsibility of one supporter and until the ticket has been solved.
A graphical view of the ticket flow in GGUS is shown here: +
The GGUS Support is organized with two main lines of support:
The first line support is provided by an organisation called TPM – Ticket +Processing Manager. The TPM team has members who have very good general grid +knowledge. It is an organisation populated by people provided from the Czech +NGI. This organisation is responsible for the routing and processing of all +active tickets.
The second line support is formed by many support units. Each support unit is +formed from members who are specialists in various areas of grid middleware, or +NGI/ROC supporters for operations problems, or VO specific supporters. The +membership of the support units is maintained on mailing lists. If the user +responds to any email received from GGUS, then the reply is added to the ticket +history. The subject of the email includes metadata to ensure the association of +the response with the ticket.
The workflows for tickets waiting for user input is described in this FAQ.
The following advice is intended for people working on TPM.
When submitting a ticket the ticket category field could not be set but default +to “Incident”. It is up to the supporters to decide whether a ticket describes +an “Incident” or a service ticket. Service tickets are tickets that request +something be done like:
They do not report issues. This differentiation is compliant to ITIL. +Differentiating between incident tickets and service tickets can help supporters +to order tickets they are responsible for by urgency. The GGUS reporting also +relies on the correct setting of the ticket category field as it does ignore +tickets of category “Test”.
Tickets assigned to a support unit by error or tickets that need actions from +other support units should be either assigned back to the TPM or assigned to the +relevant support unit directly. In both cases an explanation in the public diary +will avoid confusion.
In this section the usage of the different status values and input fields is +described.
The system offers two groups of meta states, open states and terminal states.
Single steps of the solution process can be documented in field “Public Diary”. +Information and comments which should not be visible to the user can be put into +the “Internal diary”. When a solution is found, the modifier types the solution +into the solution field and changes status to “solved”. The “Solution” field +provides 4000 characters. If 4000 characters are not sufficient, please add an +attachment. After changing status to “solved” and saving all changes, the +solution is sent to the submitter via mail automatically. Tasks for solving a +ticket:
Reminder mails are based on the priority colours. The algorithm of setting +priority colours is described in the following chapters. Reminder mails are sent +with the reply-to address ignored - atnospam - ggus.eu. All mails sent to this +mail box are deleted regularly without reading them.
Priority colours are:
Priority colours depend on the expected response time and the expected solution +time of a ticket.
The expected response times for support units that did not agree on a dedicated +quality of service are listed in the +relevant FAQ. This means, the status of a ticket should be set to another value +than “assigned” for indicating that the support unit has acknowledged the +ticket.
The expected solution times are driven by the priority values of the tickets. +All values are working days. The higher the priority the shorter is the duration +within which the ticket is expected to be solved. For further details please see +the Ticket Priority page.
Ticket follow-up Ticket follow-up is done by a team at KIT (Germany) for all +tickets besides operations tickets. More information on the ticket follow-up +processes are available at the pages about +DMSU ticket follow-up and +Ticket monitoring.
GGUS is the Helpdesk service of the EGI Infrastructure. Incident and Service +request tickets can be recorded, and their progress is tracked until the +solution. The users of the service should not need to know any of the details of +what happens to the ticket in order to get it from creation to solution. +However, an understanding of the operation of the system may be helpful to +explaining what happens when you request help.
Tickets can be created through the GGUS web interface, which +is described in section “Accessing the web interface to GGUS” of this note.
Once the ticket has entered GGUS, it is processed by assigning it to the +appropriate group to deal with the issue. The groups are generally addressed via +mailing lists, and so GGUS assigns the ticket to a group, and an email message +is sent to people on the appropriate list. Sometimes, a ticket is simple and it +is assigned to the correct group immediately and dealt with immediately.
Before using this route, it is essential to have either a digital certificate +installed in the appropriate manner in the web browser or an EGI Check-in +account.
If the user carries out all of the steps above, but is not registered to use +GGUS, then the home page is like the one shown in the following figure:
If the user faces problems with their certificate, they may get help at +wiki.egi.eu: Getting certificate
For accessing GGUS users have to register first. Registration process is +described in chapter Registering at GGUS.
For getting supporter privileges users need to be registered. For registering at +GGUS the user should go to GGUS home and +click the registration link. +This link opens the registration form that the user has to fill in. In addition, +there are some links where the user can find information about X509 personal +certificates. Users who do not have a valid digital certificate can access GGUS +via EGI SSO. After filling in the registration form, the user receives an email +from the GGUS team with his access data to GGUS.
The primary address of the GGUS portal is: https://ggus.eu. +If the user enters this in the browser, a warning will be displayed by the +browser prompting the user to specify which certificate to use (if the user has +not selected to automatically select the certificate in the browser settings). +The reason for this warning is that GGUS has to validate the user in order to +allow access. After that, the user will no longer be asked to identify himself.
The user identifies themselves with their digital certificate. At this point in +time, GGUS has identified the user and displays the start page as shown in the +following figure:
Note that in this case, the user’s identity is displayed on the page and GGUS +has recognized that this user is registered to submit tickets. If authentication +by certificate fails, you will be taken to the login page.
When choosing “Login via EGI AAI CheckIn” on the login page the user is guided +to the EGI Check-in page were the users Identity Provider can be selected. After +authenticating there, the user will be redirected to GGUS.
The tickets submit form offers a set of fields which should help the user to +describe his issue as detailed as possible. Most of the fields on the submit +form are optional. Only field “Subject” is required. The submit form consists of +three main sections: the user information, the issue information and the +routing information.
Most fields in the user information section are pre-filled by GGUS system.
Although only one field is mandatory in this section, as much fields as possible +should be filled with information.
After clicking the “Submit” button the user gets a confirmation page showing the +information submitted and the ticket ID.
The TPM (Ticket Processing Manager) is the 1st Line Support in GGUS. Users can +bypass the TPM if they have good knowledge about where the problem is. For this +purpose at the bottom of ticket submit form there is a section “Routing +information”. Selecting either a site from the +“Notify SITE” drop-down menu +or a support unit from the “Assign to support unit” drop-down menu routes the +ticket directly to the selected support unit. If selecting a site name the +NGI/ROC the site belongs to is set automatically. Hence the ticket is assigned +to the relevant NGI/ROC. Additionally the site will receive a notification about +the ticket. Selecting both, the “Notify SITE” and the “Assign to support unit” +is not possible.
Tickets of type TEAM and +ALARM are always routed to +the relevant NGI/ROC by default.
After authenticating themselves the user has access to the GGUS homepage. On +this page they see a list of their own open tickets and a list of the latest +open tickets of all users (Figure 2). Below the list of the user’s own open +tickets there are two links for further browsing possibilities of the user’s own +tickets:
The system only shows the user tickets which have been created with the same +authentication information the user is currently logged in. This means if a user +submits tickets with different certificates he does not see all of their +tickets. The reason for this is that the DN string of the certificate is stored +in each ticket. Showing all tickets of a user can be done by using the GGUS +search engine. In the GGUS search engine users can search by username amongst +others. This search will show all tickets of a user independent from the +authentication information.
This link opens a new window showing tables of all open and closed tickets of +the user and all tickets of other users the user has subscribed to. Information +on how to subscribe to a ticket is available in section “Subscribing to a ticket +of another user.”
For modifying a ticket the user can just click on the ticket ID. He is guided to +another page. On this page are three main sections:
The ticket information gives an overview of the personal data the user provided, +the issue description and the ticket status. The ticket history shows all +actions that have been taken to solve the ticket, the date and time these +actions have been taken and the name of the supporter who did them. In the +modify section the user can add some additional information or comment on a +question of a supporter to them. The user can add attachments, change the email +notification behavior of the system and change some other attributes of his +ticket.
Between the information section and the ticket history there is a button which +allows the user to escalate a ticket (Figure 9).
Three escalation levels are available in GGUS:
The escalation levels are reached one by one. It is not possible to choose one +of them. Additional information is available by clicking on the little question +mark on the left hand side of the button.
If a ticket is already closed it could be reopened by adding a comment and +changing the status to “reopened” in the “Modify section” (Figure 8). In this +case the support unit which solved the ticket and the TPM receive an email about +the ticket reopening.
When a ticket is solved, the user could confirm that the solution has solved +their issue by verifying it. A solution could be verified by either:
Verifying a solution can help to increase the quality of solutions in GGUS.
Updating a ticket using email is also possible if one fundamental requirement is +achieved. The mail subject must contain the typical GGUS string “GGUS-Ticket-ID: +#Ticket-ID” where “Ticket-ID” is the ID of the ticket which should be updated. +The easiest way to do this is to reply to an update notification received from +GGUS. When updating a ticket using email the whole mail body will be added to +the ticket. Changing any other field (e.g. status, priority,…) is impossible
GGUS system offers various possibilities for participating in tickets. They are
An overview on these fields is given in the table below. Ticket participation +can be done by adding a valid mail address to one of these fields. Please avoid +adding closed mailing lists as such produce a lot of mail errors! Several mail +addresses have to be separated by semicolon.
User submit | User modify | Supporter modify | |
---|---|---|---|
CC | Yes | No | Yes |
Involve others | No | No | Yes |
Subscribe | No | Yes | Yes |
The CC field can be set by the user in the ticket submit form. Updates are only +possible for supporters for correcting or removing invalid mail addresses. Every +ticket update triggers a notification email to the mail address specified in the +“CC” field.
The “Involve others” field is only for supporters use. Every ticket update +triggers a notification email to the mail address specified in the “Involve +others” field.
Figure 11: Ticket subscribe Every user could subscribe to tickets of other users +if he is interested in the solution of any. For subscribing a valid email +address has to be provided. The user gets a notification once the ticket is +solved. After subscribing to a ticket the user could change the notification +mode or unsubscribe if he wants to (Figure 11). Accessing the system with the +same credentials as used for subscription is necessary for this. Additional +information on subscribing to tickets is available by clicking on the question +mark at the right hand side of the label “Subscribe to this ticket”.
The ticket “submitter” gets emails according to the “Notification mode” value +(s)he selected when submitting the ticket. If the selected “Notification mode” +value is “on every change” then all updates are sent to the “submitter”. “Public +Diary” entries are sent to the submitter regardless the value of the +“Notification mode”. “Internal Diary” entries never go to the “submitter”. They +are reserved for exchanges amongst supporters.
The email addresses in the “Cc:” field can be entered by the “submitter” and +receive the same notifications as the submitter. “Public Diary” entries are sent +to the addresses in the “Cc:” field. “Internal Diary” entries never go to the +people in the “Cc:” field. They are reserved for exchanges amongst supporters.
The email addresses in the “Involve others:” field can be entered by supporters +only and receive the same notifications as the Support Unit (SU) to which the +ticket is assigned. “Internal Diary” entries are sent to the relevant SU members +AND the people in the “Involve others:” field, as they are supposed to be +experts and contribute to the ticket solution.
The email address in the “Assign to one person:” field can be entered by +supporters only and receive the same notifications as the Support Unit (SU) to +which the ticket is assigned. “Internal Diary” entries are sent to the relevant +SU members AND the people in the “Involve others:” field AND the email address +in the “Assign to one person:” field as they are, all, supposed to be experts +and contribute to the ticket solution.
Every ticket update triggers an email to the addresses in the “Cc:”, “Involve +others:” and to ticket subscribers, i.e. GGUS users, unrelated to the specific +ticket, who entered their own email in the “Subscribe to this ticket” field.
GGUS email notifications highlight the fields changed with the specific update.
Please avoid including closed mailing lists, e-groups in these fields as mail +delivery will fail.
At the bottom of the home page there are additional links for browsing:
For browsing all tickets the GGUS search engine is a useful tool.
The GGUS search engine can be entered by clicking on link “Search ticket +database”. When accessing the search engine a default search is performed like +shown in Figure 12: GGUS search engine. Searching via Ticket ID is the easiest +and fastest way to look at a ticket. When searching via Ticket ID all other +search parameters were ignored. Besides searching for all open tickets this is +the recommended kind of search, because it avoids needless workload on the +system. When searching via ticket ID the ticket details are shown in the same +window. For getting back to the main page use the “Back” button of your browser. +The various search parameters can be combined in any way wanted except +“Untouched since”. Description fields “Keyword”, “Involved supporter” and +“Assigned to person” trigger a LIKE search to the database. Concatenating +keywords with “AND” or “OR” is currently not possible. The search can either be +started by clicking on “go” or just hitting the return key. The result of a +search by parameters is shown in the result list. For viewing ticket details +just click on the ID. A new window opens showing ticket details. For getting +back to the search result just close the window with the ticket details.
Clicking on this link shows all open tickets that are currently in the system. +Unlike “Showing all open tickets” the default search in GGUS search engine uses +a timeframe of one week for showing open tickets.
Users can click on the “Envelope” icon in the menu bar for sending an email to +the GGUS team with any comments. Note: This must not be used for submitting +support requests as it does not create a ticket in the system!
On GGUS home page there are a couple of links where users can get more +information on the +GGUS development plans as well as +submit own feature requests. Feature requests are collected in the GGUS shopping +lists in JIRA (only for users having a CERN account) and EGI RT.
The GGUS system is running on servers located at Karlsruhe Institute of +Technology (KIT), Germany. Besides the GGUS production system a backup system is +in place. Switching from production system to backup system currently needs +manual interaction. An automatic switch in case of fail-over will be +implemented. GGUS staffs at Karlsruhe are not providing support apart from +requests concerning GGUS system itself. They can be contacted by email to +support “at” ggus.eu. Usual office hours for GGUS staffs are from 07:00 to 15:00 +o’clock (UTC).
TPM (Ticket Processing Manager) is the most important part of the support system +for the grid. The purpose of TPM is:
The TPM teams consist of people with a broad knowledge of the Grid.
Besides developing and maintaining GGUS system the GGUS team is also doing the +ticket monitoring. The ticket monitoring team is responsible for:
Description of some of the Helpdesk workflows.
Process for closing tickets lacking of submitter’s input
After discussions in the GGUS-AB and the +OMB meetings, members agreed on +implementing a work flow for closing tickets waiting for submitter input after a +reasonable amount of time. This work flow was also presented at the +UCB meeting.
The status value “waiting for reply” should only be used when input from the +ticket submitter is required. Setting the ticket status to “waiting for reply” +in GGUS triggers an email notification to the submitter’s email address +registered in the ticket. After the submitter has replied either by
the ticket status changes to “in progress” automatically. Updates on the ticket +done using different credentials/certificates or email addresses than registered +in the ticket will not be recognized as reply by the system and hence have no +impact on the work flow. The status value “waiting for reply” must not be used +in case of waiting for input of any other person involved in the solving +process.
5 working days after setting the ticket status to “waiting for reply” the first +reminder is sent to the ticket submitter requesting input. The ticket will also +be added to the +ticket monitoring dashboard. +The ticket monitoring team will make sure that the status value “waiting for +reply” is used in a correct sense.
In case the submitter does not reply, the second reminder is sent to the ticket +submitter requesting input after 5 more working days.
In case the submitter does not reply after another 5 more working days the +ticket monitoring team gets notified. The monitoring team will check the ticket +for any updates by the submitter not recognized by the system and set the ticket +status to “unsolved” if none. The ticket will follow the usual process for +“solved”/“unsolved” tickets and be closed after 10 working days without +re-opening the ticket again.
Action | Ticket Status | Working Days | Work Flow |
---|---|---|---|
1 | waiting for reply | 5 | first reminder to submitter; adding ticket to monitoring dashboard |
2 | waiting for reply | 5 | second reminder to submitter |
3 | waiting for reply | 5 | notification to monitoring team; the monitoring team sets status “unsolved” |
4 | unsolved | 10 | manuel status change to “closed” by ticket monitoring team |
In this workflow there is always a human intervention before closing a ticket. +The submitter has 15 working days in total for replying to a ticket. +Additionally, they have 10 more working days for re-opening the ticket in case +they do not agree with setting the ticket to “unsolved”.
This section contains documentation about the +internal EGI services. These services +are being operated centrally on behalf of EGI, and are supporting the +coordination of the EGI Federation.
We provide +Guidelines for software development to be +considered when developing a product for the EGI Federation.
You can ask for more information about the internal EGI services +on our site.
Introduction to internal EGI services
Resource usage accounting for EGI services
Fostering Collaboration across the EGI Federation
Topology and configuration registry for sites in EGI infrastructure
EGI Helpdesk
Messaging service supporting other central services
Central portal supporting EGI infrastructure operations
Enhance local security for a safer global infrastructure
Monitor performance of EGI services
Guidelines for software development
The EGI Messaging Service is powered by ARGO Messaging Service (AMS), a +real-time messaging service that allows the user to send and receive messages +between independent applications.
It’s a Publish/Subscribe Service implementing the Google PubSub protocol and +providing an HTTP API that enables Users/Systems to implement message oriented +service using the Publish/Subscribe Model over plain HTTP.
This central service is used by other EGI central services in order to exchange +messages, like for sending information about accounting or resources available +at a cloud site. More specifically, the services that use the Messaging service +are:
ams-authN
. The entry point for users, topics and
+subscriptions is the Configuration Database.It supports both push and pull message delivery. In push delivery, the Messaging +Service initiates requests to the subscriber application to deliver messages. In +pull delivery, the subscription application initiates requests to the server to +retrieve messages.
Apart from the main service a number of valuable components are also supported. +These components are extensively used by the connected services.
Instead of focusing on a single Messaging service specification for handling the +logic of publishing/subscribing to the broker network, the service focuses on +creating nodes of Publishers and Subscribers as a Service. In the +Publish/Subscribe paradigm, Publishers are users/systems that can send messages +to named-channels called Topics. Subscribers are users/systems that create +Subscriptions to specific topics and receive messages.
As shown in Figure below, the current deployment of messaging service comprises +a haproxy server, which acts as a load balancer for the 3 AMS servers running in +the backend.
Property | Value |
---|---|
Name | Messaging service |
Description | Facilitate messages exchange between EGI services |
URL | N/A |
Support Email | argo-ggus-support at grnet.gr |
Helpdesk Support Unit | EGI Services and Service Components I__ Messaging |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=429 |
Supplier | GRNET, SRCE |
Roadmap | |
Release notes | https://github.com/ARGOeu/argo-messaging/releases |
Source code | https://github.com/ARGOeu/argo-messaging |
Issue tracker for developers | https://github.com/ARGOeu/argo-messaging/issues |
License | Apache 2.0 |
Privacy Policy |
The EGI Service Monitoring keeps an eye on the +performance of the EGI services to quickly detect and resolve issues.
The service monitors the infrastructure by collecting data generated by +functional probes. The raw data is merged into statistics and available through +the user interface in a user-friendly way. It provides automated reporting tools +with minimal development or operational effort for setting up monitoring.
In order to access the information it’s required to have an X509 client +certificate provided by an IGTF-accredited Certificate +Authority.
For people not having access to an IGTF client certificate it’s possible to +access Availability and Reliability information on the +EGI ARGO page.
Different service instances are available for different purposes:
The endpoint for uncertified sites is using a certificate from a Certificate +Authority (CA) that is part of the +IGTF distribution but that is +not in the default Operating System and browser stores.
Your browser may be presenting you a security warning about an unknown CA, +it’s a known issue with certificate having IGTF trust but not public trust +(ie. not by default in the Operating Systems and browsers’ trust stores).
If you want to address this you can try to manually download the certificate for +the ROOT CA and add it to your trust store and mark it as trusted. The exact +process is dependant on the Operating System and browser that you use.
For an individual site, people having specific roles for the site can access +using the X509 certificate linked to their account in +Configuration Database:
For all sites, the members of the +dteam VO can access.
Property | Value |
---|---|
Name | ARGO Monitoring Service |
Description | Service Monitoring for Availability and Reliability |
URL | https://argo.egi.eu/ |
Support Email | argo-ggus-support at grnet.gr |
Helpdesk Support Unit | EGI Services and Service Components I__ Monitoring (ARGO) |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=641 |
Supplier | CNRS, GRNET, SRCE |
Roadmap | |
Release notes | |
Source code | https://github.com/ARGOeu |
Issue tracker for developers | https://github.com/ARGOeu |
License | Apache 2.0 |
Privacy Policy | https://argo.egi.eu/egi/Critical/policies |
The EGI Operations Portal is a central +portal for supporting the operations and coordination of the EGI Infrastructure. +It offers a bundle of different capabilities, such as:
Property | Value |
---|---|
Name | Operation Portal |
Description | Provision of VO management functions and other capabilities supporting the daily operations of EGI |
URL | https://operations-portal.egi.eu/ |
Support Email | cic-information at in2p3.fr |
Helpdesk Support Unit | EGI Services and Service Components I__ Operations Portal |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=568 |
Supplier | CNRS |
Roadmap | |
Release notes | Release notes |
Source code | https://gitlab.in2p3.fr/opsportal/sf3 |
Issue tracker for developers | https://gitlab.in2p3.fr/opsportal/sf3/-/issues |
License | Apache 2 |
Privacy Policy | https://operations-portal.egi.eu/home/a/aup |
VAPOR is a component of the Operations Portal allowing to query the information +system, aggregating information from Top BDII and +EGI Configuration Database.
VAPOR, a component of the +Operations Portal, provides a graphical +resources explorer.
It can be used as an alternative to querying the Top BDII using ldapsearch
.
Using the left menu you can select a VO and filter the different resources +types.
VAPOR will list the matching resources.
Security Coordination improves the capabilities of local security activities for +a safer federated infrastructure environment.
The EGI Computer Security Incident Response Team +(EGI CSIRT) has the tools and the knowledge to run +Security Coordination on behalf of the federation. The EGI CSIRT is a certified +Trusted Introducer since 2015.
Security Coordination is especially important in a federated environment, where +incidents are often not isolated and can affect several service providers. A +coordinated response is essential to minimize the impact of incidents and +vulnerabilities.
This service provides:
Technical details of EGI Security Coordination
Security Monitoring for EGI Resources Providers and Services
EGI is an interconnected federation where a single vulnerable place may have a +huge impact on the whole infrastructure. In order to recognise the risks and to +address potential vulnerabilities in a timely manner, the EGI Security +Monitoring provides an oversight of the infrastructure from the security +standpoint.
Also, sites connected to EGI differ significantly in the level of security and +detecting weaknesses exposed by the sites allows the EGI security operations to +contact the sites before the issue leads to an incident.
Information produced by security monitoring is also important during assessment +of new risks and vulnerabilities since it enables to identify the scope and +impact of a potential security incident.
This service includes the following components.
A Nagios-based service provided to monitor a range of assets like CRLs, file +system permissions, vulnerable file permissions etc.
Ad-hoc probes are deployed to support incident management, to assess the +vulnerability of the infrastructure with regards to specific security issues and +for proactive security management.
The results produced are available to the EGI Security dashboard of the +Operations Portal for visualisation.
Pakiti is the monitoring and notification service which is +responsible for checking the patching status of systems.
The results produced are available to the EGI Security dashboard of the +Operations Portal for visualisation.
Ticketing system for tracking of incident.
Security challenges are a mechanism to check the compliance of sites/NGIs/EGI +with security requirements. Runs of Security Service Challenges need a set of +tools that are used during various stages of the runs.
Pakiti is a client-server tool to +collect and evaluate data about packages installed on Linux machines, primarily +meant to identify vulnerable SW that have not been properly updated. The +EGI CSIRT operates the +EGI Pakiti instance that is used to monitor the state +of the EGI sites.
The pakiti-client
can be used to send package informations to
+pakiti.egi.eu.
If you have the proper credentials in the
+Configuration Database and submit your report
+with the correct SITE_NAME
, you, your NGI-CSIRT and the EGI-CSIRT will be
+able to monitor the packages installed on your hosts and potentially
+vulnerabilities. The results can be accessed on the
+EGI Pakiti central instance.
If you have CVMFS installed and configured to mount grid.cern.ch
, you can run
+pakiti by simply running:
$ /cvmfs/grid.cern.ch/pakiti/bin/pakiti-client \
+ --url "https://pakiti.egi.eu/feed/" \
+ --site SITE_NAME
+
Please remember to replace SITE_NAME by your actual site name
The pakiti-client
is now available from
+EPEL. If your machine already has
+EPEL enabled, the following command is enough to install it:
$ yum install pakiti-client
+
With the package and the configuration, the following commands will run the
pakiti-client
and transmit all its data to the EGI CSIRT pakiti instance!
$ pakiti-client --url "https://pakiti.egi.eu/feed/" --site SITE_NAME
+
Please remember to replace SITE_NAME by your actual site name
The simplest way to configure and run the pakiti-client
on a cluster is to use
+puppet: You just need to create a file and a manifest.
package { 'pakiti-client':
+ ensure => 'present',
+}
+cron { 'pakiti-egi':
+ ensure => 'present',
+ command => 'pakiti-client --url "https://pakiti.egi.eu/feed/" --site SITE_NAME',
+ user => 'nobody',
+ hour => fqdn_rand(24),
+ minute => fqdn_rand(60),
+}
+
Property | Value |
---|---|
Name | Security coordination |
Description | Enhance local security for a safer global infrastructure |
URL | https://csirt.egi.eu |
Support Email | abuse at egi.eu |
Helpdesk Support Unit | EGI Services and Service Components I__ Security Coordination I__ Security Monitoring |
Configuration Database entry | https://goc.egi.eu/portal/index.php?Page_Type=Site&id=968 https://goc.egi.eu/portal/index.php?Page_Type=Site&id=1127 |
Supplier | UKRI, FOM-Nikhef, CERN, CESNET, GRNET, IJS |
Roadmap | N/A |
Release notes | N/A |
Source code | N/A |
Issue tracker for developers | N/A |
License | N/A |
Privacy Policy | N/A |
This page contains information about integrating your identity provider (IdP) +with Check-in in order to allow users +in your community to access EGI tools and services.
Organisations who want to register their IdP in Check-in needs to fill this
+form in case the IdP is not publishing
+REFEDS R&S and
+Sirtfi compliance in eduGAIN. A PDF scan of a
+printed and signed copy should be sent to operations_at_egi.eu
To integrate your Identity Provider with the EGI Check-in service, you need to +submit a GGUS ticket indicating your request. The responsible +support unit is AAI Support. +The integration follows a two-step process:
The most important URLs for each environment are listed in the table below but +more information can be found in the protocol-specific sections that follow.
Protocol | Production environment |
---|---|
SAML | https://aai.egi.eu/proxy/module.php/saml/sp/metadata.php/sso |
OpenID Connect | See client registration |
Protocol | Demo environment |
---|---|
SAML | https://aai-demo.egi.eu/proxy/module.php/saml/sp/metadata.php/sso |
OpenID Connect | See client registration |
Protocol | Development environment |
---|---|
SAML | https://aai-dev.egi.eu/proxy/module.php/saml/sp/metadata.php/sso |
OpenID Connect | See client registration |
An institution or a community may connect their IdP with Check-in to allow their +users to access EGI services, or any other services that have enabled Check-in +as an authentication provider. This section presents the general requirements +for integrating an IdP with EGI Check-in, while protocol-specific instructions +are provided in the sections that follow.
As a bare minimum, the IdP of a user’s Home Organisation or Community is +expected to release a non-reassignable identifier that uniquely identifies the +user within the scope of that organisation or community. The unique identifier +must be accompanied with a minimum set of attributes which the Check-in Service +Provider Proxy will attempt to retrieve from the user’s IdP. If this is not +possible, the missing user attributes will be acquired and verified through the +user registration process with the EGI Account Registry. The following table +describes the data requested from the user’s Home Organisation, which are +communicated to the Check-in SP as either SAML attributes or OIDC claims, +depending on the protocol supported by the authenticating IdP.
Description | Notes |
---|---|
At least one of the following unique user identifiers:
| |
Preferred name for display purposes | For example to be used in a greeting or a descriptive listing |
First name | |
Surname | |
Email address | |
Affiliation within Home Organisation or Community | To be released only if relevant for accessing EGI services |
Note that the above set of requested attributes, particularly the identifier, +name, email and affiliation information, complies with the +REFEDS R&S attribute bundle.
Information about group membership and role information released by your IdP +should follow the URN scheme below (see also +AARC-G002):
<NAMESPACE>:group:<GROUP>[:<SUBGROUP>*][:role=<ROLE>]#<GROUP-AUTHORITY>
+
where:
<NAMESPACE>
is in the form of
+urn:<NID>:<DELEGATED-NAMESPACE>[:<SUBNAMESPACE>*]
, where<NID>
is the namespace identifier associated with a URN namespace
+registered with
+IANA,
+as per RFC8141, ensuring global uniqueness. Implementers can and should use
+one of the existing registered URN namespaces, such as
+urn:geant
+and
+urn:mace
;<DELEGATED-NAMESPACE>
is a URN sub-namespace delegated from one of the
+IANA registered NIDsto an organisation representing the e-infrastructure, research infrastructure or +research collaboration.
<GROUP>
is the name of a VO, research collaboration or a top level arbitrary
+group. <GROUP>
names are unique within the urn:mace:egi.eu:group
+namespace;<SUBGROUP>
components represent the hierarchy of subgroups in
+the <GROUP>
; specifying sub-groups is optional<ROLE>
component is scoped to the rightmost (sub)group; if no
+group information is specified, the role applies to the VO<GROUP-AUTHORITY>
is a non-empty string that indicates the authoritative
+source for the entitlement value. For example, it can be the FQDN of the group
+management system that is responsible for the identified group membership
+informationExample entitlement values expressing VO/group membership and role +information:
urn:geant:dariah.eu:group:egi-interop:role=member#aaiproxy.de.dariah.eu
+urn:geant:dariah.eu:group:egi-interop:role=vm_operator#aaiproxy.de.dariah.eu
+
The IdP needs to comply with additional requirements to achieve a higher level +of assurance and allow its users to gain access to a wider set of EGI services. +A first group of additional requirements are defined by the +Sirtfi framework v1.0. +Adherence to these requirements can be asserted either by publishing Sirtfi +compliance in the eduGAIN metadata or by declaring it in this +form. These requirements are in the +areas of operational security, incident response, traceability and IdPs and +users responsibility.
Check-in provides a central Discovery Service (or “Where Are You From” - WAYF) +page where users in your Home Organisation or Community will be automatically +redirected when necessary to select to authenticate at your IdP. You can provide +us with a logo of your Organisation or Community (in high-res PNG or preferably +in svg format) to include a dedicated login button that will allow users to +easily identify your IdP.
To allow users in your community to sign into federated EGI applications, you +need to connect to the EGI AAI SP Proxy as a SAML Identity Provider (IdP). Users +of the application will be redirected to the central Discovery Service page of +the EGI AAI Proxy where they will able to select to authenticate at your IdP. +Once the user is authenticated, the EGI AAI Proxy will return a SAML assertion +to the application containing the information returned by your IdP about the +authenticated user.
SAML authentication relies on the use of metadata. Both parties (you as an IdP +and the EGI AAI SP) need to exchange metadata in order to know and trust each +other. The metadata include information such as the location of the service +endpoints that need to be invoked, as well as the certificates that will be used +to sign SAML messages. The format of the exchanged metadata should be based on +the XML-based +SAML 2.0 specification. +Usually, you will not need to manually create such an XML document, as this is +automatically generated by all major SAML 2.0 IdP software solutions (e.g., +Shibboleth, SimpleSAMLphp). It is important that you serve your metadata over +HTTPS using a browser-friendly SSL certificate, i.e. issued by a trusted +certificate authority.
To exchange metadata, please send an email including the following information:
Depending on the software you are using, the authoritative XML metadata URL for +your IdP might be in the following form:
https://your.idp.example.eu/idp/shibboleth
(Shibboleth)https://your.idp.example.eu/simplesaml/module.php/saml2/idp/metadata.php
+(SimpleSAMLphp)Note that if your IdP is part of a federation, then it would be preferred to +send us the URL to a signed federation metadata aggregate. We can then cherry +pick the appropriate entityID from that.
You can get the metadata of the EGI Check-in SP Proxy on a dedicated URL that +depends on the integration environment being used:
Production environment |
---|
https://aai.egi.eu/proxy/module.php/saml/sp/metadata.php/sso |
Development environment |
---|
https://aai-dev.egi.eu/proxy/module.php/saml/sp/metadata.php/sso |
For the production environment, it is recommended that you get the metadata for
+the EGI Check-in SP (entityID:
+https://aai.egi.eu/proxy/module.php/saml/sp/metadata.php/sso
)
+from a signed eduGAIN metadata aggregate. For example, the following aggregates
+are provided by GRNET:
The SAML based Identity Provider of your Home Organisation or Community is +expected to release a non-reassignable identifier that uniquely identifies the +user within the scope of that organisation or community, along with a set of +additional information as described in the following table (see also +general attribute release requirements):
Description | SAML attribute |
---|---|
At least one of the following unique user identifiers:
|
|
Preferred name for display purposes | displayName |
First name | givenName |
Surname | sn |
Email address | mail |
Affiliation within Home Organisation or Community | eduPersonScopedAffiliation |
Group(s)/role(s) within Home Organisation or Community | eduPersonEntitlement |
Users in your community can sign into federated EGI applications through the +Check-in service using your OpenID Connect or OAuth 2.0 based Identity Provider.
To enable your OIDC Identity Provider for user login, Check-in needs to be +registered as a client in order to obtain OAuth 2.0 credentials, such as a +client ID and client secret, and to register one or more redirect URIs. Once +Check-in is registered as a client, your users will be redirected to the central +Discovery Service page of Check-in when logging into EGI federated applications, +where they will able to select to authenticate at your IdP. Once the user is +authenticated, Check-in will be responsible for communicating the information +returned by your IdP about the authenticated user to the connected application. +Depending on the protocol, this information will be expressed through a SAML +assertion, a set of OIDC claims or a (proxy) X.509 certificate.
Check-in needs to obtain your OpenID Provider's configuration information,
+including the location of the Authorisation, Token and UserInfo endpoints. Your
+OpenID Provider is expected to make a JSON document available at the path formed
+by concatenating the string /.well-known/openid-configuration
to the Issuer,
+following the
+OpenID Connect Discovery 1.0 specification.
The OpenID Connect or OAuth 2.0 based Identity Provider of your Home +Organisation or Community is expected to release a non-reassignable identifier +that uniquely identifies the user within the scope of that organisation or +community, along with a set of additional information as described in the +following table (see also +general attribute release requirements):
Description | OIDC claim |
---|---|
At least one of the following unique user identifiers:
|
|
Preferred name for display purposes | name |
First name | given_name |
Surname | family_name |
Email address | email |
Affiliation within Home Organisation or Community | eduperson_scoped_affiliation |
Group(s)/role(s) within Home Organisation or Community | eduPerson_entitlement |