GET: feedback/score (#25)

* init test feedback score post

* get feedback tests

* rename

* declare list of labels once instead of multiple lists

* feedback table

* feedback tests refined

* feedback init

* get feedback sore

* add feedback route

* add min for player name

* daily, tests failing, no errors

* Update feedback.py

* Update feedback.py

* Update feedback.py

* Update 01_tables.sql

* Update test_feedback_api.py

* sytnax

* black formatting

* checkpoint

* PredictionFeedback refactor from Feedback

* create procedue to insert feedback test data

* Update 02_data.sql

* merge conflict

* unable to locate player error checkpoint

* fixing references

* working get feedback

* dbPlayer not needed as an import

* daily checkpoint

* checkpoint

* change to dbPlayer

* working response

* working test case

* cleanup

* refactor AppModelFeedback to Feedback

* rename variables to be more descriptive

* remove unneeded select

* move create table to 01_tables.sql

* removing feedback out

* remove foreign keys not needed for given and received

* remove PredictionFeedbackOut and foreign keys

* fixing tabs and spaces

* remove debug statements

* cleanup of print statements, feedback get tests done

* 404 not found on players not returned

* do no randomize data so it's testable

* PredictionFeedbackRespo refactor FeedbackResponse

* feedback name for v2 fastapi datastructure verify

* feedback change to players/feedback/score

* test cases overhaul for test feedback

* remove unneeded kafka engine for feedback

* app model feedback updates for feedback input

* removd unused imports

* change route to be /players/feedback

* count checkpoint for feedback score resposne

* refactor score to just be get feedback added count

* make sure all ranges from 1-100 are populated

* need to optomize feedback query, count passing

* prevent sql injection and alias usage for speed

* alias for speed and prevent injection

* alias for speed and prevent injection

* test cases were failing, fixes

* remove ds_store add to ignore

* black formatting

* need setup-detatched for workflow

* str between 1-13 and at least 1 str, player/report/score

* lower player names, name for invalid player must be between 1-13 char

* player/feedback and player/feedback/score refactor

* players/report/score  player/report/score refactor

* prediction to player/prediction refactor

* reports to report refactor

* cleanup

* player cleanup comments

* refactor player_name alias to player_db, remove bindparams

* name consistancy and correct join

* prediction user_name to name, remove bind, revert list match, fix test cases

* cleanup aliases

* Keep class name formatting standard

* add longer deadline for large multi feedback response

* add multi player test strategy for players

* comment out get feedback for now

* insert possible ban and confirm ban randomly 0 1, feedback vote 1 -1 0

* cleanup feedback

* cleanup of model, feedback score with vote working

* cleanup feedback test cases

* all working - manual tested

* pulli n dev

* Revert "make sure all ranges from 1-100 are populated"

This reverts commit 8e69afa.

* Revert "insert possible ban and confirm ban randomly 0 1, feedback vote 1 -1 0"

This reverts commit ac9fe40.

* better data creation

* fix random functions etc

* semantics

* fix random data

* fix comment

* more comments fixed

* 1G buffer pool instead of 16G for local machine

* test_get_feedback_score_invalid_players fixed

* player id list

* update expire_logs_days and innodb_log_file_size to new values

* feedback tests passing, removing duplicate as multi covered 1

* remove duplicate cases, logging consistancy

* test prediction changes

* add timestamp using rand(42)

* do not need docker requirements

* specify mysql version

* Revert "update expire_logs_days and innodb_log_file_size to new values"

This reverts commit 134ce9a.

* naming without registery

* add clean docker

* removing big chunks of commented code

* cleanup code

* cleanup code

* cleanup code

* cleanup code

* remove debian

* fix invalid player test case for 404 and must return 'player not found'

* consistant logs from pytest if failure

* formatted

* add pre commit to make file

* cleanup

* print syntax changes

* missed a new line

* added health check for mysql & kafka

* revert print statements

* dont use make

* remove -d for testing

* forgot to remove -d

* mysql image trying -debian

* Revert "mysql image trying -debian"

This reverts commit 36f511a.

* lets try to remove this

* added -d back in

---------

Co-authored-by: extreme4all <>
Co-authored-by: extreme4all <[email protected]>

.github/workflows/pytest-workflow-manual.yml

@@ -33,10 +33,16 @@ jobs:
       uses: actions/setup-python@v3
       with:
         python-version: "3.11.6"
-
+
+    - name: install testing requirements
+      run: |
+        python3 -m pip install pytest 
+        python3 -m pip install requests 
+        python3 -m pip install hypothesis
+
     - name: Setup Docker Containers
       run: |
-        make setup-detached
+        docker-compose --verbose up --build -d
 
     - name: Check API is Up
       uses: cygnetdigital/[email protected]
@@ -48,7 +54,7 @@ jobs:
 
     - name: Generate Pytest Report
       run: |
-        make test-report
+        python3 -m pytest --junit-xml=pytest_report.xml
 
       # https://github.com/marketplace/actions/pytest-results-actions
     - name: Surface Failing Tests

.github/workflows/pytest-workflow.yml

@@ -43,9 +43,15 @@ jobs:
         with:
           python-version: "3.11.6"
 
+      - name: install testing requirements
+        run: |
+          python3 -m pip install pytest 
+          python3 -m pip install requests 
+          python3 -m pip install hypothesis
+
       - name: Setup Docker Containers
         run: |
-          make setup-detached
+          docker-compose --verbose up --build -d
 
       - name: Check API is Up
         uses: cygnetdigital/[email protected]
@@ -57,7 +63,7 @@ jobs:
 
       - name: Generate Pytest Report
         run: |
-          make test-report
+          python3 -m pytest --junit-xml=pytest_report.xml
 
         # https://github.com/marketplace/actions/pytest-results-actions
       - name: Surface Failing Tests

.gitignore

@@ -159,4 +159,4 @@ cython_debug/
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
 
-# .vscode/*
+# .vscode/*.DS_Store

.pre-commit-config.yaml

@@ -1,10 +1,10 @@
 repos:
-    - repo: https://github.com/psf/black
-      rev: 23.10.1
-      hooks:
-          - id: black
-    - repo: https://github.com/pycqa/isort
-      rev: 5.12.0
-      hooks:
-          - id: isort
-            args: ["--profile", "black"]
+  - repo: https://github.com/psf/black
+    rev: 23.10.1
+    hooks:
+      - id: black
+  - repo: https://github.com/pycqa/isort
+    rev: 5.12.0
+    hooks:
+      - id: isort
+        args: ["--profile", "black"]

Makefile

@@ -31,8 +31,15 @@ clean-test: ## cleanup pytests leftovers
 	rm -f test-results.html
 	rm -f output.xml
 
+docker-clean:
+	@read -p "Are you sure you want to prune all Docker data (y/n)? " -n 1 -r; \
+	echo; \
+	if [[ $$REPLY =~ ^[Yy]$$ ]]; then \
+		docker system prune -a -f; \
+	fi
+
 test: clean-test ## Run pytest unit tests
-	python3 -m pytest --verbosity=1
+	python3 -m pytest --verbosity=1 -s
 
 test-report:
 	python3 -m pytest --junit-xml=pytest_report.xml
@@ -52,17 +59,17 @@ docker-build: ## Startup docker with build switch
 docker-build-detached: ## Startup docker with build switch
 	docker-compose up --build -d
 
-setup: venv-create requirements pre-commit-setup docker-build test-setup ## setup & run after downloaded repo
+setup: test-setup requirements## setup requirements
 
-setup-detached: venv-create requirements pre-commit-setup docker-build-detached test-setup ## setup & run after downloaded repo detached
+setup-detached: test-setup docker-build-detached ## setup & run after downloaded repo detached
 
 pre-commit-setup: ## Install pre-commit
 	python3 -m pip install pre-commit
 	pre-commit --version
 
-test-setup: ## installs pytest singular package for local testing
-	python3 -m pip install pytest
-	python3 -m pip install requests
+test-setup: pre-commit-setup## installs pytest singular package for local testing
+	python3 -m pip install pytest 
+	python3 -m pip install requests 
 	python3 -m pip install hypothesis
 
 requirements: ## installs all requirements
@@ -82,11 +89,14 @@ docs: # opens your browser to the webapps testing docs
 	xdg-open http://localhost:5000/docs
 	. http://localhost:5000/docs
 
-venv-create: venv-remove ## cleans the .venv then creates a venv in the folder .venv
+venv-create: ## creates a venv in the folder .venv
 	python3 -m venv .venv
 
 venv-remove: ## removes the .venv folder
 	rm -rf .venv
 
 test-loud: ## runs pytest with verbose output
-	python3 -m pytest --verbose -s
+	python3 -m pytest --verbose -s
+
+pre-commit:
+	pre-commit run --all-files

docker-compose.yaml

@@ -42,6 +42,7 @@ services:
     container_name: kafka_setup
     build:
       context: ./kafka_setup
+    image: kafka_setup
     environment:
       - KAFKA_BROKER=kafka:9092
     networks:
@@ -54,7 +55,7 @@ services:
     container_name: database
     build:
       context: ./mysql
-    image: bot-detector/mysql:latest
+    image: mysql
     environment:
       - MYSQL_ROOT_PASSWORD=root_bot_buster
     volumes:
@@ -65,9 +66,13 @@ services:
       - 3306:3306
     networks:
       - botdetector-network
+    healthcheck:
+      test: "mysqladmin ping -h localhost -u root -proot_bot_buster"
+      interval: 30s
+      timeout: 10s
+      retries: 5
 
   public_api:
-    image: bot-detector/public_api
     container_name: public_api
     build:
       context: .
@@ -76,8 +81,11 @@ services:
       args:
         root_path: /
         api_port: 5000
+    image: public_api
     # command: bash -c "apt update && apt install -y curl && sleep infinity"
     command: uvicorn src.core.server:app --host 0.0.0.0 --reload --reload-include src/*
+    volumes:
+      - ./src:/project/src
     ports:
       - 5000:5000
     networks:
@@ -92,8 +100,10 @@ services:
     # env_file:
     #   - .env
     depends_on:
-      - kafka_setup
-      - mysql
-
+      kafka:
+        condition: service_healthy
+      mysql:
+        condition: service_healthy
+
 networks:
   botdetector-network:

mysql/Dockerfile

@@ -1,3 +1,3 @@
-FROM mysql:latest
+FROM mysql:8.0.32
 
 EXPOSE 3306