Skip to content
This repository has been archived by the owner on Jul 16, 2022. It is now read-only.

Feat implement generate gdpr export python #232

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions python/generate-gdpr-export/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# 📷 Object Detection using Cloudmersive Vision API
A sample Python Cloud Function for exporting doctuments to csv format and saving it to the user storage.

## 📝 Environment Variables
Add the following environment variables in your Cloud Functions settings.

* **APPWRITE_API_KEY** - Create a key from the Appwrite console with the following scope (`collections.read, documents.read,files.write`)
* **APPWRITE_ENDPOINT** - Your Appwrite Endpoint


## 🚀 Building and Packaging

To package this example as a cloud function, follow these steps.

```bash
$ cd demos-for-functions/python/generate-gdpr-export

$ PIP_TARGET=./.appwrite pip install -r ./requirements.txt --upgrade --ignore-installed
```

* Ensure that your folder structure looks like this
```
.
├── .appwrite/
├── main.py
└── requirements.txt
```

* Create a tarfile

```bash
$ cd ..
$ tar -zcvf code.tar.gz generate-gdpr-export
```

* Upload the tarfile to your Appwrite Console and use the following entrypoint command

```bash
python main.py
```

## 🎯 Trigger

To trigger this function you will need to create an execution by means of the WebSDK in order to get the user context.
56 changes: 56 additions & 0 deletions python/generate-gdpr-export/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Perform all your imports

import os
import csv
import datetime
from appwrite.client import Client
from appwrite.services.database import Database
from appwrite.services.storage import Storage

# Initialise the Appwrite client
client = Client()
client.set_endpoint(os.environ["APPWRITE_ENDPOINT"]) # Your API Endpoint
client.set_project(os.environ["APPWRITE_FUNCTION_PROJECT_ID"]) # Your project ID
client.set_key(os.environ["APPWRITE_API_KEY"]) # Your secret API key

all_documents = []

user_id = os.environ['APPWRITE_FUNCTION_USER_ID']
user = f'user:{user_id}'

database = Database(client)
off_col = 0 # For the API pagination - collections
while True:
collections = database.list_collections(limit=100, offset=off_col)
for collection in collections['collections']:
collection_id = collection['$id']
off_doc = 0 # For the API pagination - documents
while True:
documents = database.list_documents(collection_id, limit=100, offset=off_doc)
for document in documents['documents']:
if user in document['$permissions']['read'] or '*' in document['$permissions']['read']:
all_documents.append(document)
off_doc += 100
if documents['sum'] <= off_doc:
break

off_col += 100
if collections['sum'] <= off_col:
break

# Create csv from the list of dictionaries
columns = []
for d in all_documents:
for key in d.keys():
if key not in columns:
columns.append(key)
timestamp = datetime.datetime.now().isoformat()
file_name = f"{user_id}_{timestamp}.csv"
with open(file_name, 'w', newline='') as f:
csv_writer = csv.DictWriter(f, fieldnames=columns, restval='')
csv_writer.writeheader()
csv_writer.writerows(all_documents)

storage = Storage(client)
result = storage.create_file(open(file_name, 'rb'), [user])
os.remove(file_name)
1 change: 1 addition & 0 deletions python/generate-gdpr-export/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
appwrite