Add typing to confidant unit tests (#392)

Typing was added to confidant unit tests. This is to further improve the
service.

`run-piptools` was utilized for adding the new requirements and the mypy
configurations were added per the infradocs.

Some versions were pinned due to conflicting deprecation of functions,
mainly flask (and the flask dependencies) due to a change in
[werkzeug](https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-0)
which deprecated the `safe_str_cmp` function.

These changes were tested locally by running the service per the
[docs](https://lyft.github.io/confidant/contributing.html).
`mypy` was added to the `pre-commit` configuration. `pre-commit` was
utilized as another avenue for testing everything is working correctly.
Finally `pytest` was ran to ensure all the unit tests were passing
correctly.

.github/workflows/pull_request.yml

@@ -50,7 +50,7 @@ jobs:
       - name: Install apt dependencies
         run: sudo apt-get update && sudo apt-get install -y python3-dev openssl libssl-dev gcc pkg-config libffi-dev libxml2-dev libxmlsec1-dev
       - name: Install dependencies
-        run: pip install -r piptools_requirements.txt && pip install -r requirements.txt
+        run: pip install -r piptools_requirements.txt && pip install -r piptools_requirements3.txt && pip install -r requirements3.txt 
       - name: Run python unit tests
         run: make test_unit
   test-integration:

.pre-commit-config.yaml

@@ -7,3 +7,18 @@ repos:
         additional_dependencies:
         -   flake8
         -   flake8-tidy-imports
+-   repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.4.1
+    hooks:
+    -   id: mypy
+        additional_dependencies:
+        -   --no-compile
+        -   cffi==1.15.1
+        -   cryptography==41.0.1
+        -   pycparser==2.21
+        -   types-pyopenssl==23.2.0.1
+        -   types-pytz==2023.3.0.0
+        -   types-pyyaml==6.0.12.10
+        -   types-redis==4.6.0.0
+        -   types-requests==2.31.0.1
+        -   types-urllib3==1.26.25.13

Dockerfile

@@ -22,12 +22,13 @@ COPY package.json /srv/confidant/
 RUN npm install grunt-cli && \
     npm install
 
-COPY piptools_requirements.txt requirements.txt /srv/confidant/
+COPY piptools_requirements.txt piptools_requirements3.txt requirements3.txt /srv/confidant/
 
 ENV PATH=/venv/bin:$PATH
 RUN virtualenv /venv --python=/usr/bin/python3.8 && \
     pip install --no-cache -r piptools_requirements.txt && \
-    pip install --no-cache -r requirements.txt
+    pip install --no-cache -r piptools_requirements3.txt && \
+    pip install --no-cache -r requirements3.txt
 
 COPY .jshintrc Gruntfile.js /srv/confidant/
 COPY confidant/public /srv/confidant/confidant/public

actions_run_integration.sh

@@ -2,5 +2,5 @@
 
 cd /srv/confidant
 apt-get update && apt-get install -y python3-dev openssl libssl-dev gcc pkg-config libffi-dev libxml2-dev libxmlsec1-dev
-pip install -r piptools_requirements.txt && pip install -r requirements.txt
+pip install -r piptools_requirements.txt && pip install -r piptools_requirements3.txt && pip install -r requirements3.txt
 make test_integration

piptools_requirements.txt

@@ -1,9 +1,11 @@
 #
-# This file is autogenerated by pip-compile with python 3.6
-# To update, run:
+# This file is autogenerated by pip-compile with Python 3.8
+# by the following command:
 #
-#    pip-compile
+#    cd ~/src/confidant-private/upstream && run-piptools
 #
 
-pip==21.3.1
-setuptools==59.1.1
+pip==23.1.2
+    # via -r /code/piptools/bootstrap_ins/requirements.in
+setuptools==68.0.0
+    # via -r /code/piptools/bootstrap_ins/requirements.in

piptools_requirements3.txt

@@ -0,0 +1,11 @@
+#
+# This file is autogenerated by pip-compile with Python 3.8
+# by the following command:
+#
+#    cd ~/src/confidant-private/upstream && run-piptools
+#
+
+pip==23.1.2
+    # via -r /code/piptools/bootstrap_ins/requirements.in
+setuptools==68.0.0
+    # via -r /code/piptools/bootstrap_ins/requirements.in

requirements.in

@@ -3,13 +3,13 @@
 # License: BSD
 # Upstream url: http://github.com/mitsuhiko/flask/
 # Use: For API.
-Flask
+Flask==1.1.4
 
 # Flask Scripting support for Flask
 # License: BSD
 # Upstream url: http://github.com/techniq/flask-script
 # Use: For CLI scripts.
-Flask-Script
+Flask-Script==2.0.5
 
 # Dispatching system for parties to subscribe to events
 # Supports per-endpoint timing in flask
@@ -22,14 +22,14 @@ blinker
 # License: Apache2
 # Upstream url: https://github.com/boto/botocore
 # Use: For boto3
-botocore>=1.4.58
+botocore==1.12.227
 
 # Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK)
 # for Python.
 # License: Apache2
 # Upstream url: https://github.com/boto/boto3
 # Use: For KMS
-boto3>1.5.0
+boto3==1.9.227
 
 # A Pythonic interface for Amazon's DynamoDB that supports Python 2 and 3.
 # License: MIT
@@ -54,20 +54,21 @@ cffi>1.10.0
 # License: BSD
 # Upstream url: https://github.com/fengsp/flask-session
 # Use: For shared sessions
-Flask-Session
+Flask-Session==0.2.3
 
 # Redis
 # License: MIT
 # Upstream url: https://github.com/andymccurdy/redis-py
 # Use: For shared sessions
 redis
+types-redis
 
 # Flask extension that configures the Flask application to redirect all
 # incoming requests to https
 # License: BSD
 # Upstream url: https://github.com/kennethreitz/flask-sslify
 # Use: For SSL redirection and HSTS
-Flask-SSLify
+Flask-SSLify==0.1.5
 
 # Authomatic is a framework agnostic library for Python web applications
 # with a minimalistic but powerful interface which simplifies
@@ -88,6 +89,7 @@ python3-saml>=1.15.0
 # Upstream url: http://python-requests.org
 # Use: REST calls to external services
 requests>=2.22.0,<3.0.0
+types-requests
 
 # Provides enhanced HTTPS support for httplib and urllib2 using PyOpenSSL
 # License: BSD
@@ -116,6 +118,7 @@ guard
 # Upstream url: http://pyyaml.org/wiki/PyYAML
 # Use: For parsing users.yaml
 PyYAML
+types-PyYAML
 
 # License: MIT
 # Upstream url: http://gunicorn.org/
@@ -190,8 +193,12 @@ pytest-gevent
 
 # Timezones
 pytz
+types-pytz
 
 # for jwt
 pyjwt>=2.6.0
 jwcrypto
 cerberus
+
+# for typing
+mypy