From 697ad50daacbf14d71d78807100b63912ecfb916 Mon Sep 17 00:00:00 2001 From: Harshil Gajera Date: Mon, 12 Aug 2024 12:30:25 +0530 Subject: [PATCH 1/3] ci: updating readme --- README.md | 63 +++++++++++++++++++- README.rst | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 228 insertions(+), 3 deletions(-) create mode 100644 README.rst diff --git a/README.md b/README.md index 8d2496aca..1e736e604 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # pytest-splunk-addon -![PyPI](https://img.shields.io/pypi/v/pytest-splunk-addon) -![Python](https://img.shields.io/pypi/pyversions/pytest-splunk-addon.svg) +![PyPI]() +![Python]() ## What is PSA @@ -9,4 +9,61 @@ A Dynamic test tool for Splunk Apps and Add-ons. ## Usage -For full usage instructions, please visit the [documentation](https://splunk.github.io/pytest-splunk-addon). +For full usage instructions, please visit the \[documentation\](). + +# Requirements + +- Docker or an external single instance Splunk deployment + +# Installation + +You can install "pytest-splunk-addon" via [pip] from [PyPI]: + +``` +$ pip install pytest-splunk-addon +``` + +# Run e2e tests locally + +- For e2e tests we are using a functionality of pytest which creates a temp dir and copies all the required file to that dir and then runs the pytest cmd from the tests. +- e2e tests can be found under /tests/e2e + +Prerequisites: + +- Docker version: 25.0.3 +- Docker Compose version: v2.24.6-desktop.1 + +``` +git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git $ cd pytest-splunk-addon $ poetry install $ poetry run pytest -v --splunk-version=${splunk-version} -m docker -m ${test-marker} tests/e2e +``` + +Troubleshooting: + +1. If you face an error like this: + + > argparse.ArgumentError: argument -K/--keepalive: conflicting option strings: -K, --keepalive + + - This is likely to happen if you have older version of PSA requirements installed, to solve this try to uninstall lovely-pytest-docker and pull the latest main branch and then do `poetry install` + +2. If while running the tests you face an exception like this: + + > Exception: Command ['docker', 'compose', '-f', '/docker-compose.yml', '-p', '', 'down', '-v'] returned 125: """unknown shorthand flag: 'f' in -f + + - This happens due to misconfigurations in docker, try to follow below steps: + - sudo mkdir -p /usr/local/lib/docker + - sudo ln -s /Applications/Docker.app/Contents/Resources/cli-plugins /usr/local/lib/docker/cli-plugins + +3. If you face error like this: + + > ERROR: no match for platform in manifest: not found + + - Try adding platform: `linux/amd64` to docker-compose.yml file + +# Contributing + +Contributions are very welcome. Tests can be run with [pytest], please ensure +the coverage at least stays the same before you submit a pull request. + +# License + +Distributed under the terms of the [Apache Software License 2.0] license, "pytest-splunk-addon" is free and open source software diff --git a/README.rst b/README.rst new file mode 100644 index 000000000..391d1c111 --- /dev/null +++ b/README.rst @@ -0,0 +1,168 @@ +# pytest-splunk-addon + +![PyPI](https://img.shields.io/pypi/v/pytest-splunk-addon) +![Python](https://img.shields.io/pypi/pyversions/pytest-splunk-addon.svg) + +## What is PSA + +A Dynamic test tool for Splunk Apps and Add-ons. + +## Usage + +For full usage instructions, please visit the [documentation](https://splunk.github.io/pytest-splunk-addon). + +Requirements +------------ + +* Docker or an external single instance Splunk deployment + + +Installation +------------ + +You can install "pytest-splunk-addon" via `pip`_ from `PyPI`_:: + + $ pip install pytest-splunk-addon + +Developing +------------ + +Note: Must install docker desktop. + +.. code:: bash + $ git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git + $ cd pytest-splunk-addon + $ poetry install + $ ... (change something) + # run unit tests + $ poetry run pytest tests/unit + # run some of the docker-based tests to verify end-to-end behaviour, example: + $ poetry run pytest -v --splunk-version=8.2 -m docker tests/test_splunk_addon.py::test_splunk_app_requirements_modinput +Usage +----- + +Installation for external Splunk + +.. code:: bash + pip install pytest-splunk-addon +Installation with built in docker orchestration + +.. code:: bash + pip install pytest-splunk-addon[docker] +Basic project structure + +The tool assumes the Splunk Add-on is located in a folder "package" in the project root + +Triggering the tool: + +Create a test file in the tests folder + +.. code:: python3 + from pytest_splunk_addon.standard_lib.addon_basic import Basic + class Test_App(Basic): + def empty_method(): + pass +Create a Dockerfile-splunk file + +.. code:: Dockerfile + ARG SPLUNK_VERSION=latest + FROM splunk/splunk:$SPLUNK_VERSION + ARG SPLUNK_APP=TA_UNKNOWN + ARG SPLUNK_APP_PACKAGE=package + COPY deps/apps /opt/splunk/etc/apps/ + COPY $SPLUNK_APP_PACKAGE /opt/splunk/etc/apps/$SPLUNK_APP +Create a docker-compose.yml update the value of SPLUNK_APP + +.. code:: yaml + version: "3.7" + services: + splunk: + build: + context: . + dockerfile: Dockerfile-splunk + args: + - SPLUNK_APP=xxxxxxx + ports: + - "8000" + - "8089" + environment: + - SPLUNK_PASSWORD=Changed@11 + - SPLUNK_START_ARGS=--accept-license +Run pytest with the add-on and SA-eventgen installed and enabled in an external Splunk deployment + +.. code::: bash + pytest \ + --splunk-type=external \ + --splunk-type=external \ + --splunk-host=splunk \ + --splunk-port=8089 \ + --splunk-password=Changed@11 \ + -v +Run pytest with the add-on and SA-eventgen installed and enabled in docker + +.. code::: bash + pytest \ + --splunk-password=Changed@11 \ + -v +For full usage instructions, please visit the `pytest-splunk-addon documentation pages over at readthedocs`_. + +Run e2e tests locally +--------------------- + +* For e2e tests we are using a functionality of pytest which creates a temp dir and copies all the required file to that dir and then runs the pytest cmd from the tests. +* e2e tests can be found under /tests/e2e + +Prerequisites: + +* Docker version: 25.0.3 +* Docker Compose version: v2.24.6-desktop.1 + +.. code:: bash + $ git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git + $ cd pytest-splunk-addon + $ poetry install + $ poetry run pytest -v --splunk-version=${splunk-version} -m docker -m ${test-marker} tests/e2e +Troubleshooting: + +1. If you face an error like this: + + argparse.ArgumentError: argument -K/--keepalive: conflicting option strings: -K, --keepalive + + * This is likely to happen if you have older version of PSA requirements installed, to solve this try to uninstall lovely-pytest-docker and pull the latest main branch and then do `poetry install` + +2. If while running the tests you face an exception like this: + + `Exception: Command ['docker', 'compose', '-f', '/docker-compose.yml', '-p', '', 'down', '-v'] returned 125: """unknown shorthand flag: 'f' in -f` + + * This happens due to misconfigurations in docker, try to follow below steps: + * sudo mkdir -p /usr/local/lib/docker + * sudo ln -s /Applications/Docker.app/Contents/Resources/cli-plugins /usr/local/lib/docker/cli-plugins + +3. If you face error like this: + + ERROR: no match for platform in manifest: not found + + * Try adding platform: `linux/amd64` to docker-compose.yml file + +Contributing +------------ +Contributions are very welcome. Tests can be run with `pytest`_, please ensure +the coverage at least stays the same before you submit a pull request. + +License +------- + +Distributed under the terms of the `Apache Software License 2.0`_ license, "pytest-splunk-addon" is free and open source software + + +Issues +------ + +If you encounter any problems, please `file an issue`_ along with a detailed description. + +.. _`pytest-splunk-addon documentation pages over at readthedocs`: https://pytest-splunk-addon.readthedocs.io/en/latest/ +.. _`Apache Software License 2.0`: http://www.apache.org/licenses/LICENSE-2.0 +.. _`file an issue`: https://github.com/splunk/pytest-splunk-addon/issues +.. _`pytest`: https://github.com/pytest-dev/pytest +.. _`pip`: https://pypi.org/project/pip/ +.. _`PyPI`: https://pypi.org/project \ No newline at end of file From 1239f81c4f5760a08bb5580c0f8369d9ad314abf Mon Sep 17 00:00:00 2001 From: Harshil Gajera Date: Mon, 12 Aug 2024 13:27:40 +0530 Subject: [PATCH 2/3] ci: updating readme --- README.rst | 168 ----------------------------------------------------- 1 file changed, 168 deletions(-) delete mode 100644 README.rst diff --git a/README.rst b/README.rst deleted file mode 100644 index 391d1c111..000000000 --- a/README.rst +++ /dev/null @@ -1,168 +0,0 @@ -# pytest-splunk-addon - -![PyPI](https://img.shields.io/pypi/v/pytest-splunk-addon) -![Python](https://img.shields.io/pypi/pyversions/pytest-splunk-addon.svg) - -## What is PSA - -A Dynamic test tool for Splunk Apps and Add-ons. - -## Usage - -For full usage instructions, please visit the [documentation](https://splunk.github.io/pytest-splunk-addon). - -Requirements ------------- - -* Docker or an external single instance Splunk deployment - - -Installation ------------- - -You can install "pytest-splunk-addon" via `pip`_ from `PyPI`_:: - - $ pip install pytest-splunk-addon - -Developing ------------- - -Note: Must install docker desktop. - -.. code:: bash - $ git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git - $ cd pytest-splunk-addon - $ poetry install - $ ... (change something) - # run unit tests - $ poetry run pytest tests/unit - # run some of the docker-based tests to verify end-to-end behaviour, example: - $ poetry run pytest -v --splunk-version=8.2 -m docker tests/test_splunk_addon.py::test_splunk_app_requirements_modinput -Usage ------ - -Installation for external Splunk - -.. code:: bash - pip install pytest-splunk-addon -Installation with built in docker orchestration - -.. code:: bash - pip install pytest-splunk-addon[docker] -Basic project structure - -The tool assumes the Splunk Add-on is located in a folder "package" in the project root - -Triggering the tool: - -Create a test file in the tests folder - -.. code:: python3 - from pytest_splunk_addon.standard_lib.addon_basic import Basic - class Test_App(Basic): - def empty_method(): - pass -Create a Dockerfile-splunk file - -.. code:: Dockerfile - ARG SPLUNK_VERSION=latest - FROM splunk/splunk:$SPLUNK_VERSION - ARG SPLUNK_APP=TA_UNKNOWN - ARG SPLUNK_APP_PACKAGE=package - COPY deps/apps /opt/splunk/etc/apps/ - COPY $SPLUNK_APP_PACKAGE /opt/splunk/etc/apps/$SPLUNK_APP -Create a docker-compose.yml update the value of SPLUNK_APP - -.. code:: yaml - version: "3.7" - services: - splunk: - build: - context: . - dockerfile: Dockerfile-splunk - args: - - SPLUNK_APP=xxxxxxx - ports: - - "8000" - - "8089" - environment: - - SPLUNK_PASSWORD=Changed@11 - - SPLUNK_START_ARGS=--accept-license -Run pytest with the add-on and SA-eventgen installed and enabled in an external Splunk deployment - -.. code::: bash - pytest \ - --splunk-type=external \ - --splunk-type=external \ - --splunk-host=splunk \ - --splunk-port=8089 \ - --splunk-password=Changed@11 \ - -v -Run pytest with the add-on and SA-eventgen installed and enabled in docker - -.. code::: bash - pytest \ - --splunk-password=Changed@11 \ - -v -For full usage instructions, please visit the `pytest-splunk-addon documentation pages over at readthedocs`_. - -Run e2e tests locally ---------------------- - -* For e2e tests we are using a functionality of pytest which creates a temp dir and copies all the required file to that dir and then runs the pytest cmd from the tests. -* e2e tests can be found under /tests/e2e - -Prerequisites: - -* Docker version: 25.0.3 -* Docker Compose version: v2.24.6-desktop.1 - -.. code:: bash - $ git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git - $ cd pytest-splunk-addon - $ poetry install - $ poetry run pytest -v --splunk-version=${splunk-version} -m docker -m ${test-marker} tests/e2e -Troubleshooting: - -1. If you face an error like this: - - argparse.ArgumentError: argument -K/--keepalive: conflicting option strings: -K, --keepalive - - * This is likely to happen if you have older version of PSA requirements installed, to solve this try to uninstall lovely-pytest-docker and pull the latest main branch and then do `poetry install` - -2. If while running the tests you face an exception like this: - - `Exception: Command ['docker', 'compose', '-f', '/docker-compose.yml', '-p', '', 'down', '-v'] returned 125: """unknown shorthand flag: 'f' in -f` - - * This happens due to misconfigurations in docker, try to follow below steps: - * sudo mkdir -p /usr/local/lib/docker - * sudo ln -s /Applications/Docker.app/Contents/Resources/cli-plugins /usr/local/lib/docker/cli-plugins - -3. If you face error like this: - - ERROR: no match for platform in manifest: not found - - * Try adding platform: `linux/amd64` to docker-compose.yml file - -Contributing ------------- -Contributions are very welcome. Tests can be run with `pytest`_, please ensure -the coverage at least stays the same before you submit a pull request. - -License -------- - -Distributed under the terms of the `Apache Software License 2.0`_ license, "pytest-splunk-addon" is free and open source software - - -Issues ------- - -If you encounter any problems, please `file an issue`_ along with a detailed description. - -.. _`pytest-splunk-addon documentation pages over at readthedocs`: https://pytest-splunk-addon.readthedocs.io/en/latest/ -.. _`Apache Software License 2.0`: http://www.apache.org/licenses/LICENSE-2.0 -.. _`file an issue`: https://github.com/splunk/pytest-splunk-addon/issues -.. _`pytest`: https://github.com/pytest-dev/pytest -.. _`pip`: https://pypi.org/project/pip/ -.. _`PyPI`: https://pypi.org/project \ No newline at end of file From 177b3d199c11410bae3c9f98f8aaebdfd098d4a6 Mon Sep 17 00:00:00 2001 From: Harshil Gajera Date: Wed, 14 Aug 2024 11:20:17 +0530 Subject: [PATCH 3/3] ci: updating doc --- docs/contributing.md | 101 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/contributing.md diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 000000000..173ae34b1 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,101 @@ +# Contributing Guidelines + + +# Prerequisites: + +- Docker or an external single instance Splunk deployment +- Poetry 1.5.1. [Installation guide](https://python-poetry.org/docs/#installing-with-the-official-installer) + +# Installation + +You can install "pytest-splunk-addon" via [pip] from [PyPI]: + +``` +$ pip install pytest-splunk-addon +``` + + +### Unit tests + +```bash +poetry run pytest -v tests/unit +``` + +### e2e tests + +- For e2e tests we are using a functionality of pytest which creates a temp dir and copies all the required file to that dir and then runs the pytest cmd from the tests. +- e2e tests can be found under /tests/e2e + +Prerequisites: + +- Docker version: 25.0.3 +- Docker Compose version: v2.24.6-desktop.1 + +Steps to run e2e tests locally: + +``` +git clone --recurse-submodules -j8 git@github.com:splunk/pytest-splunk-addon.git +cd pytest-splunk-addon +poetry install +poetry run pytest -v --splunk-version=${splunk-version} -m docker -m ${test-marker} tests/e2e +``` + +### Troubleshooting: + +1. If you face an error like this: + + > argparse.ArgumentError: argument -K/--keepalive: conflicting option strings: -K, --keepalive + + - This is likely to happen if you have older version of PSA requirements installed, to solve this try to uninstall lovely-pytest-docker and pull the latest main branch and then do `poetry install` + +2. If while running the tests you face an exception like this: + + > Exception: Command ['docker', 'compose', '-f', '/docker-compose.yml', '-p', '', 'down', '-v'] returned 125: """unknown shorthand flag: 'f' in -f + + - This happens due to misconfigurations in docker, try to follow below steps: + - sudo mkdir -p /usr/local/lib/docker + - sudo ln -s /Applications/Docker.app/Contents/Resources/cli-plugins /usr/local/lib/docker/cli-plugins + +3. If you face error like this: + + > ERROR: no match for platform in manifest: not found + + - Try adding platform: `linux/amd64` to docker-compose.yml file + + +## Linting and Type-checking + +`Pytest-splunk-addon` uses the [`pre-commit`](https://pre-commit.com) framework for linting and type-checking. +Consult with `pre-commit` documentation about what is the best way to install the software. + +To run it locally: + +```bash +pre-commit run --all-files +``` + +## Documentation changes + +Documentation changes are also welcome! + +To verify changes locally: + +```bash +poetry run mkdocs serve -a localhost:8001 +``` + +## Issues and bug reports + +You can create an [issue](https://github.com/splunk/pytest-splunk-addon/issues) on GitHub. + +## Pull requests + +We love to see pull requests! + +We are using [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). +The two most important types: "fix" and "feat", would result in the new version of the `Pytest-splunk-addon` once merged. + +To do the changes you think are needed, run the previous steps (build / test / linting / documentation). +After you create a PR, all the needed reviewers will be added automatically by GitHub. + +Note: The `semgrep` and `fossa` steps might fail if you are an external contributor. This is expected for now.