diff --git a/docs/changes.rst b/docs/changes.rst deleted file mode 100644 index d9e113ec..00000000 --- a/docs/changes.rst +++ /dev/null @@ -1 +0,0 @@ -.. include:: ../CHANGES.rst diff --git a/docs/contribute/changes.rst b/docs/contribute/changes.rst new file mode 100644 index 00000000..d76c92b6 --- /dev/null +++ b/docs/contribute/changes.rst @@ -0,0 +1 @@ +.. include:: ../../CHANGES.rst diff --git a/docs/developer.rst b/docs/contribute/developer.rst similarity index 97% rename from docs/developer.rst rename to docs/contribute/developer.rst index 1d24243a..67591138 100644 --- a/docs/developer.rst +++ b/docs/contribute/developer.rst @@ -29,7 +29,7 @@ OpenTAXII repository has the following layout: Database schema =============== -.. image:: db_schema_diagram.png +.. image:: ../db_schema_diagram.png :width: 1000 Compiling from source diff --git a/docs/contribute/index.rst b/docs/contribute/index.rst new file mode 100644 index 00000000..b7b1a1e1 --- /dev/null +++ b/docs/contribute/index.rst @@ -0,0 +1,11 @@ +====================== +Contribute and develop +====================== + +.. toctree:: + :maxdepth: 1 + :titlesonly: + :glob: + + developer + changes diff --git a/docs/index.rst b/docs/index.rst index a9052a55..6e0f3f05 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -32,16 +32,11 @@ OpenTAXII architecture follows TAXII specification in its idea of TTA (TAXII tra .. toctree:: :maxdepth: 1 - installation - configuration - running - docker - auth - public-apis - opentaxii-apis - developer - changes + installation/index + usage/index + contribute/index license + references/index .. rubric:: External links diff --git a/docs/configuration.rst b/docs/installation/configuration.rst similarity index 70% rename from docs/configuration.rst rename to docs/installation/configuration.rst index f6899e9f..d3536ffb 100644 --- a/docs/configuration.rst +++ b/docs/installation/configuration.rst @@ -164,7 +164,6 @@ Example custom configuration: parameters: rest_api: http://rest.mydomain.com/api - The built-in implementation of the Persistence and Authentication APIs support SQLite, PostgreSQL, MySQL, and other SQL databases. Check `SQLAlchemy website `_ to get the full list. OpenTAXII CLI tools are implemented to call corresponding API methods and support any API implementation. @@ -183,72 +182,8 @@ You can (re)define any configuration option with environment variables. Start va Environment variables applied after all other configs and can be used to redefine any option. -Services, collections and accounts -================================== - -Services, collections and accounts can be created with CLI command ``opentaxii-sync-data`` or with custom code talking to a specific Persistent API implementation/backend. - -Step 1 ------- -Create YAML file with collections/services/accounts configuration. See provided example from `OpenTAXII git repo `_ — file :github-file:`examples/data-configuration.yml ` that contains: - -Services: - * 2 Inbox Services (with ids ``inbox_a`` and ``inbox_b``), - * 1 Discovery Service (with id ``discovery_a``), - * 1 Collection Management Service (with id ``collection_management_a``), - * 1 Poll Service (with id ``poll_a``). - -.. note:: - Services have relative path in the address field, which means OpenTAXII will prepend it with domain configured in server configuration file (``localhost:9000`` in `Default configuration`_). - -Collections: - * ``collection-a`` that has type ``DATA_SET``, accepts all content types and is attached to services - ``inbox_a``, ``collection_management_a``, and ``poll_a``. - * ``collection-b`` that accepts only content types specified in field ``content_bindings`` and is attached to services ``inbox_a``, ``inbox_b``, ``collection_management_a`` and ``poll_a``. - * ``collection-c`` that accepts not only STIX v1.1.1 content type but also custom content type ``urn:custom.bindings.com:json:0.0.1``. It is attached to services ``inbox_a``, ``collection_management_a`` and ``poll_a``. - * ``col-not-available`` that is marked as not available, even though it is attached to ``inbox_b`` and ``collection_management_a``. - -Accounts: - * account with username ``test`` and password ``test``, with ability to modify collection ``collection-a``, read ``collection-b`` and ``coll-stix-and-custom``, and unknown permission ``some`` for non-existing collection ``collection-xyz``. Incorrect settings will be ignored during sync. - * account with username ``admin`` and password ``admin`` that has admin permissions because ``is_admin`` is set to ``yes``. - -.. note:: - Without an account you can't access services with ``authentication_required`` enabled. - - -Step 2 ------- -Use ``opentaxii-sync-data`` command to synchorize data configuration in provided file and in DB. - -Usage help:: - - (venv) $ opentaxii-sync-data --help - usage: opentaxii-sync-data [-h] [-f] config - - Create services/collections/accounts - - positional arguments: - config YAML file with data configuration - - optional arguments: - -h, --help show this help message and exit - -f, --force-delete force deletion of collections and their content blocks - if collection is not defined in configuration file - (default: False) - -To sync data run:: - - (venv) $ opentaxii-sync-data examples/data-configuration.yml - -.. note:: - To drop the databases, just delete sqlite3 files ``/tmp/data.db``, ``/tmp/auth.db`` and restart OpenTAXII server. - -Now OpenTAXII has services, collections and accounts configured and can function as a TAXII server. -Check :doc:`Running OpenTAXII ` to see how to run it. - .. rubric:: Next steps Continue to the :doc:`Running OpenTAXII ` page to see how to run OpenTAXII. - .. vim: set spell spelllang=en: diff --git a/docs/docker.rst b/docs/installation/docker.rst similarity index 96% rename from docs/docker.rst rename to docs/installation/docker.rst index 6a723faa..320a08f9 100644 --- a/docs/docker.rst +++ b/docs/installation/docker.rst @@ -1,5 +1,5 @@ -Docker -====== +Run OpenTAXII on Docker +======================= OpenTAXII can also be run using docker. This guide assumes that you have access to a local or remote docker server, and won't go into the setup of docker. @@ -147,8 +147,6 @@ This configuration starts two containers: ``opentaxii`` and ``db``, and creates .. rubric:: Next steps -Continue to :doc:`Authentication ` page to learn how OpenTAXII authentication process works. - - +Continue to :doc:`Configuration ` page to learn how to configure OpenTAXII. .. vim: set spell spelllang=en: diff --git a/docs/installation/index.rst b/docs/installation/index.rst new file mode 100644 index 00000000..9967c85c --- /dev/null +++ b/docs/installation/index.rst @@ -0,0 +1,13 @@ +================= +Install OpenTAXII +================= + +.. toctree:: + :maxdepth: 1 + :titlesonly: + :glob: + + installation + configuration + docker + running diff --git a/docs/installation.rst b/docs/installation/installation.rst similarity index 100% rename from docs/installation.rst rename to docs/installation/installation.rst diff --git a/docs/installation/running.rst b/docs/installation/running.rst new file mode 100644 index 00000000..17db08a6 --- /dev/null +++ b/docs/installation/running.rst @@ -0,0 +1,77 @@ +================= +Running OpenTAXII +================= + +.. highlight:: sh + +Here we describe how to run OpenTAXII in a *development* or *production* mode. Development mode activates `Flask `_ debug mode, simplifies log messages (for humans) and runs the server in one thread. While in production mode there are no debug messages, proper json log messages and multithreaded if configured. + +Development mode +================ + +To run the server in development mode use the CLI command shipped with OpenTAXII package:: + + (venv) $ opentaxii-run-dev + +This will start OpenTAXII in a development mode and bind it to ``localhost:9000``. + + +Production mode +=============== + +To run OpenTAXII in production it is recommended to use `standalone WSGI +container +`_ +that works with Flask. In this example we use `Gunicorn WSGI HTTP Server +`_. For complete logging configuration we recommend +Gunicorn 19.8 and above. + +.. note:: + Run ``pip install gunicorn`` to install gunicorn. Yes, it's that simple. + +To run OpenTAXII with Gunicorn execute:: + + (venv) $ gunicorn opentaxii.http:app \ + --bind localhost:9000 --config python:opentaxii.http + +Common practice is to wrap gunicorn execution into `supervisord `_, to be able to monitor, start, and stop it easily. + +Example supervisord configuration file: + +.. code-block:: ini + + [program:opentaxii] + + command = + /opt/eclecticiq/opentaxii-venv/bin/gunicorn opentaxii.http:app + --workers 2 + --log-level info + --log-file - + --timeout 300 + --bind localhost:9000 + --config python:opentaxii.http + environment = + OPENTAXII_CONFIG="/opt/eclecticiq/custom-opentaxii-configuration.yml" + + stdout_logfile = /var/log/opentaxii.log + redirect_stderr = true + autostart = true + autorestart = true + + +Using SSL/TLS +============= + +If you want to run OpenTAXII with SSL, you need to use a web server like `Nginx `_, that provides SSL/TLS layer. You can find details on how to run Nginx with SSL `here `_. + +Make sure you configure your TAXII services in OpenTAXII with proper protocol bindings: + + * use ``urn:taxii.mitre.org:protocol:https:1.0`` if you're planning on serving data via HTTPS. + * use ``urn:taxii.mitre.org:protocol:http:1.0`` if the server is going to support unsecure HTTP as well. + +You can use multiple protocol bindings per service. That would tell OpenTAXII you want to advertise services over both HTTPs and HTTP. TAXII services create external URLs according to their protocol bindings, using ``http://`` or ``https://`` schemas. + +Continue to :doc:`Manage OpenTAXII <../usage/manage>` page to learn how to OpenTAXII create services, collections, and accounts. + +.. vim: set spell spelllang=en: + diff --git a/docs/references/index.rst b/docs/references/index.rst new file mode 100644 index 00000000..b8b1965f --- /dev/null +++ b/docs/references/index.rst @@ -0,0 +1,11 @@ +========== +References +========== + +.. toctree:: + :maxdepth: 1 + :titlesonly: + :glob: + + opentaxii-apis + public-apis diff --git a/docs/opentaxii-apis.rst b/docs/references/opentaxii-apis.rst similarity index 100% rename from docs/opentaxii-apis.rst rename to docs/references/opentaxii-apis.rst diff --git a/docs/public-apis.rst b/docs/references/public-apis.rst similarity index 100% rename from docs/public-apis.rst rename to docs/references/public-apis.rst diff --git a/docs/auth.rst b/docs/usage/auth.rst similarity index 96% rename from docs/auth.rst rename to docs/usage/auth.rst index f5cc35ec..373894e5 100644 --- a/docs/auth.rst +++ b/docs/usage/auth.rst @@ -72,7 +72,7 @@ does that by encoding account ID inside the token. .. rubric:: Next steps -Continue to :doc:`Public code-level APIs ` page for the details about OpenTAXII APIs. +Continue to :doc:`Public code-level APIs <../references/public-apis>` page for the details about OpenTAXII APIs. .. vim: set spell spelllang=en: diff --git a/docs/usage/index.rst b/docs/usage/index.rst new file mode 100644 index 00000000..bef77188 --- /dev/null +++ b/docs/usage/index.rst @@ -0,0 +1,12 @@ +============= +Use OpenTAXII +============= + +.. toctree:: + :maxdepth: 1 + :titlesonly: + :glob: + + manage + auth + use-opentaxii diff --git a/docs/usage/manage.rst b/docs/usage/manage.rst new file mode 100644 index 00000000..0f50f770 --- /dev/null +++ b/docs/usage/manage.rst @@ -0,0 +1,69 @@ +================ +Manage OpenTAXII +================ + +Services, collections and accounts +================================== + +Services, collections and accounts can be created with CLI command ``opentaxii-sync-data`` or with custom code talking to a specific Persistent API implementation/backend. + +.. Note:: + The services and collections created with CLI command ``opentaxii-sync-data`` are only available with TAXII 1. + +Step 1 +------ +Create YAML file with collections/services/accounts configuration. See provided example from `OpenTAXII git repo `_ — file :github-file:`examples/data-configuration.yml ` that contains: + +Services: + * 2 Inbox Services (with ids ``inbox_a`` and ``inbox_b``), + * 1 Discovery Service (with id ``discovery_a``), + * 1 Collection Management Service (with id ``collection_management_a``), + * 1 Poll Service (with id ``poll_a``). + +.. note:: + Services have relative path in the address field, which means OpenTAXII will prepend it with domain configured in server configuration file (``localhost:9000`` in Default configuration). + +Collections: + * ``collection-a`` that has type ``DATA_SET``, accepts all content types and is attached to services + ``inbox_a``, ``collection_management_a``, and ``poll_a``. + * ``collection-b`` that accepts only content types specified in field ``content_bindings`` and is attached to services ``inbox_a``, ``inbox_b``, ``collection_management_a`` and ``poll_a``. + * ``collection-c`` that accepts not only STIX v1.1.1 content type but also custom content type ``urn:custom.bindings.com:json:0.0.1``. It is attached to services ``inbox_a``, ``collection_management_a`` and ``poll_a``. + * ``col-not-available`` that is marked as not available, even though it is attached to ``inbox_b`` and ``collection_management_a``. + +Accounts: + * account with username ``test`` and password ``test``, with ability to modify collection ``collection-a``, read ``collection-b`` and ``coll-stix-and-custom``, and unknown permission ``some`` for non-existing collection ``collection-xyz``. Incorrect settings will be ignored during sync. + * account with username ``admin`` and password ``admin`` that has admin permissions because ``is_admin`` is set to ``yes``. + +.. note:: + Without an account you can't access services with ``authentication_required`` enabled. + + +Step 2 +------ +Use ``opentaxii-sync-data`` command to synchorize data configuration in provided file and in DB. + +Usage help:: + + (venv) $ opentaxii-sync-data --help + usage: opentaxii-sync-data [-h] [-f] config + + Create services/collections/accounts + + positional arguments: + config YAML file with data configuration + + optional arguments: + -h, --help show this help message and exit + -f, --force-delete force deletion of collections and their content blocks + if collection is not defined in configuration file + (default: False) + +To sync data run:: + + (venv) $ opentaxii-sync-data examples/data-configuration.yml + +.. note:: + To drop the databases, just delete sqlite3 files ``/tmp/data.db``, ``/tmp/auth.db`` and restart OpenTAXII server. + +Now OpenTAXII has services, collections and accounts configured and can function as a TAXII server. +Check :doc:`Use OpenTAXII <../installation/running>` to see how to use it. \ No newline at end of file diff --git a/docs/running.rst b/docs/usage/use-opentaxii.rst similarity index 68% rename from docs/running.rst rename to docs/usage/use-opentaxii.rst index 9e6155bd..1405e590 100644 --- a/docs/running.rst +++ b/docs/usage/use-opentaxii.rst @@ -1,76 +1,6 @@ -================= -Running OpenTAXII -================= - -.. highlight:: sh - -Here we describe how to run OpenTAXII in a *development* or *production* mode. Development mode activates `Flask `_ debug mode, simplifies log messages (for humans) and runs the server in one thread. While in production mode there are no debug messages, proper json log messages and multithreaded if configured. - -Development mode -================ - -To run the server in development mode use the CLI command shipped with OpenTAXII package:: - - (venv) $ opentaxii-run-dev - -This will start OpenTAXII in a development mode and bind it to ``localhost:9000``. - - -Production mode -=============== - -To run OpenTAXII in production it is recommended to use `standalone WSGI -container -`_ -that works with Flask. In this example we use `Gunicorn WSGI HTTP Server -`_. For complete logging configuration we recommend -Gunicorn 19.8 and above. - -.. note:: - Run ``pip install gunicorn`` to install gunicorn. Yes, it's that simple. - -To run OpenTAXII with Gunicorn execute:: - - (venv) $ gunicorn opentaxii.http:app \ - --bind localhost:9000 --config python:opentaxii.http - -Common practice is to wrap gunicorn execution into `supervisord `_, to be able to monitor, start, and stop it easily. - -Example supervisord configuration file: - -.. code-block:: ini - - [program:opentaxii] - - command = - /opt/eclecticiq/opentaxii-venv/bin/gunicorn opentaxii.http:app - --workers 2 - --log-level info - --log-file - - --timeout 300 - --bind localhost:9000 - --config python:opentaxii.http - environment = - OPENTAXII_CONFIG="/opt/eclecticiq/custom-opentaxii-configuration.yml" - - stdout_logfile = /var/log/opentaxii.log - redirect_stderr = true - autostart = true - autorestart = true - - -Using SSL/TLS ============= - -If you want to run OpenTAXII with SSL, you need to use a web server like `Nginx `_, that provides SSL/TLS layer. You can find details on how to run Nginx with SSL `here `_. - -Make sure you configure your TAXII services in OpenTAXII with proper protocol bindings: - - * use ``urn:taxii.mitre.org:protocol:https:1.0`` if you're planning on serving data via HTTPS. - * use ``urn:taxii.mitre.org:protocol:http:1.0`` if the server is going to support unsecure HTTP as well. - -You can use multiple protocol bindings per service. That would tell OpenTAXII you want to advertise services over both HTTPs and HTTP. TAXII services create external URLs according to their protocol bindings, using ``http://`` or ``https://`` schemas. - +Use OpenTAXII +============= Sending requests to services ============================ @@ -214,7 +144,6 @@ OpenTAXII has an endpoint that can be used to check health of the service:: .. rubric:: Next steps -Continue to :doc:`Authentication ` page to learn how OpenTAXII authentication process works. +Continue to :doc:`Authentication <../usage/auth>` page to learn how OpenTAXII authentication process works. .. vim: set spell spelllang=en: -