Skip to content

Commit

Permalink
Update service to use shared drives (#5)
Browse files Browse the repository at this point in the history
* Update service to use shared drives

* Add delete endpoint
  • Loading branch information
dzaslavskiy authored Apr 5, 2023
1 parent d5535b1 commit 36ac157
Show file tree
Hide file tree
Showing 11 changed files with 428 additions and 111 deletions.
18 changes: 18 additions & 0 deletions .github/dependabot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
version: 2
updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 10
labels:
- dependencies
- python
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 10
labels:
- dependencies
- github-actions
38 changes: 38 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# GitHub Actions CI/CD workflows

## Python Checks
The Python-Checks workflow will run a series of checks on the python code
in this repository.

### Bandit
The Bandit workflow will run the Bandit security linter tool against this
project. A failed run indicates that Bandit found at least one vulnerability.

### Black
The workflow outlined in `black.yml` checks to ensure that the Python style
for this project is consistent and fully implemented in all Python files.
For more information about this workflow, see
https://black.readthedocs.io/en/stable/github_actions.html

## CodeQL-Analysis
The codeql-analysis workflow the CodeQL semantic code analysis engine to help
find security issues very early on in the development process. See
[CodeQL](https://securitylab.github.com/tools/codeql) for more details.

## Deploy
Deploys the project to the correct GIVE environment within Cloud.gov. The
deploy workflow will run unit-tests and only deploy if those test are
successful. Deployment will also only be triggered in the 18F repository. This
will prevent forks from needlessly running workflows that will always fail
(forks won't be able to authenticate into the dev environment).

## Stale Items
The stale-items workflow will run once per day and mark issues and PR's as
stale if they have not seen any activity over the last 30 days. After being
marked stale for 5 days, the workflow will close the item.

## Unit Tests
The unit-tests workflow will install the project runtime dependencies and run
the unit test suite against the code. This workflow is used to run unit tests
for the application against pull requests before merging takes place. Additional
unit testing will take place on merging.
52 changes: 52 additions & 0 deletions .github/workflows/codeql-analysis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
name: "CodeQL"

on:
push:
branches: [main]
paths-ignore:
- '**.md' # All markdown files in the repository
pull_request:
# The branches below must be a subset of the branches above
branches: [main]
paths-ignore:
- '**.md'
schedule:
# weekly run at arbitrary time
- cron: '43 22 * * 2'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
language: ['python']

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java)
# If this step fails, then remove it and run the build manually. See below
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# If the Autobuild fails above, remove it and uncomment the following
# three lines and modify them (or add more) to build your code if your
# project uses a compiled language

# - run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
33 changes: 33 additions & 0 deletions .github/workflows/deploy-app.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
# This workflow will run unit tests and deploy the application to a
# target environment

name: Deploy

on:
push:
branches:
- main
tags:
- "*"
paths-ignore:
- "**.md" # All markdown files in the repository
release:
types: [released]

jobs:
deploy:
if: github.repository_owner == 'GSA-TTS'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: GSA-TTS/identity-idva-cf-setup@v2
id: cf-setup
with:
cf-username: ${{ secrets.CF_USERNAME }}
cf-password: ${{ secrets.CF_PASSWORD }}
cf-org: ${{ secrets.CF_ORG }}

- name: Deploy application
run: cf push --vars-file vars.yaml --var ENVIRONMENT=${{ steps.cf-setup.outputs.target-environment }} --strategy rolling
46 changes: 46 additions & 0 deletions .github/workflows/python-checks.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
# This workflow will run the Black Python formatter as well as the
# Bandit security linter. See the following pages for details:
# See https://black.readthedocs.io/en/stable/github_actions.html
# https://github.com/PyCQA/bandit
name: Python-Checks

on:
push:
branches:
- main
paths:
- '**.py' # All python files in the repository
pull_request:
paths:
- '**.py'

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11'

- uses: psf/black@stable

bandit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11'

- uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements-dev.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Scan
run: |
pip install -r requirements-dev.txt
bandit --exclude ./.venv/,./test -r .
23 changes: 23 additions & 0 deletions .github/workflows/stale-items.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
name: 'Stale-Items'
on:
schedule:
# daily run at arbitrary time
- cron: '30 1 * * *'

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v7
with:
stale-issue-message: >-
This issue has been automatically marked as stale because it has
not had any activity in the last 30 days. Remove stale label or
comment or this will be closed in 5 days.
stale-pr-message: >-
This issue has been automatically marked as stale because it has
not had any activity in the last 30 days. Remove stale label or
comment or this will be closed in 5 days.
days-before-stale: 30
days-before-close: 5
36 changes: 36 additions & 0 deletions .github/workflows/unit-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
# This workflow will install Python dependencies and run tests so that
# unit tests can be run against pull requests.

name: Unit-Tests

on:
pull_request:
paths-ignore:
- '**.md' # All markdown files in the repository
workflow_call:

jobs:
unit-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest
- name: Test with Pytest unit tests
run: |
export DEBUG=True
python -m pytest
43 changes: 35 additions & 8 deletions gdrive/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
import logging

import fastapi
from fastapi import Body
from fastapi import Response, status
from googleapiclient.http import HttpError
from starlette.requests import Request

from . import client, settings
Expand All @@ -16,18 +17,44 @@

router = fastapi.APIRouter()

client.init()

@router.post("/upload")
async def upload_file(id, filename, request: Request, base64: bool = False):
async def upload_file(id, filename, request: Request, response: Response, base64: bool = False):
"""
Upload file to gdrive.
"""
body = await request.body()

if base64:
body = base64decoder.b64decode(body)
try:
body = await request.body()

stream = io.BytesIO(body)
if base64:
body = base64decoder.b64decode(body)

parent = client.create_folder(id, settings.ROOT_DIRECTORY)
client.upload_basic(filename, parent, stream)
stream = io.BytesIO(body)

parent = client.create_folder(id, settings.ROOT_DIRECTORY)
client.upload_basic(filename, parent, stream)

except HttpError as error:
log.error(f"An error occurred: {error}")
response.status_code = error.status_code

@router.delete("/upload")
async def delete_file(filename, response: Response):
"""
Delete file from gdrive.
"""

try:
files = client.get_files(filename)
if files:
for file in files:
client.delete_file(file['id'])
else:
response.status_code = status.HTTP_404_NOT_FOUND

except HttpError as error:
log.error(f"An error occurred: {error}")
response.status_code = error.status_code

Loading

0 comments on commit 36ac157

Please sign in to comment.