Next (#564)

* ffmpeg platform-agnostic hardware-acceleration * clear CUDA cache after swapping on low VRAM + ffmpeg cuda acceleration, clearing cache prevent cuda out-of-memory error * check torch gpu before clearing cache * torch check nvidia only * syntax error * Adjust comment * Normalize ARGS * Remove path normalization * Remove args overrides * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Revert to Ubuntu test only as Windows hangs * Simplified the way to maintain aspect ratio of the preview, and maintaining aspect ratio of the miniatures * Change face and target images from contain to fit * Improve status output * Massive utilities and core refactoring * Fix sound * Fix sound part2 * Fix more * Move every UI related thing to ui.py * Refactor UI * Introduce render_video_preview() * Add preview back part1 * Add preview back part2, Introduce --video-quality for CLI * Get the preview working * Couple if minor UI fixes * Add video encoder via CLI * Change default video quality, Integrate recent directories for UI * Move temporary files to temp/{target-name} * Fix fps detection * Rename method * Introduce suggest methods for args defaults, output mode and core/threads count via postfix * Fix max_memory and output memory in progress bar too * Turns out mac has a different memory unit * Add typing to swapper * Fix FileNotFoundError while deleting temp * Updated requirements.txt for macs. (cherry picked from commit fd00a18) * Doing bunch of renaming and typing * Just a cosmetic * Doing bunch of renaming again * Introduce execution provider to support DirectML * enhancer update * remove useless code * remove useless code * remove useless code * fix * reslove some errors in code review. * methods rename * del Unused import * recover the conditional installation for darwin! * recover the import module * del try catch in unrelate codes * fix error in argument and potential infinity loop * remove the ROCM check before face-enhancer * fix lint error * add the process for image * conditional process according to --frame-processor * Hotfix usage of --frame-processor face-swapper face-enhancer * Auto download models * typo fixed * Fix framerate and audio sync issues * Limit the video encoders according to -crf support * Limit the video encoders according to -crf support part2 * Migrate to theme based UI using customtkinter * Show full preview according to video frames total * Simplify is_image and is_video, close preview on source/target change, show preview slider on video only, fix start button error * Fix linter * Use float over int for more accurate fps * introduce a pre_check() to enhancer... * update * Update utilities.py * move the model_path to the method * Fix model paths * Fix linter * Fix images scaling * Update onnxruntime-silicon and monkey patch ssl for mac * Downgrade onnxruntime-silicon again * Introduce a first version of CONTRIBUTING.md * Update CONTRIBUTING.md * Add libvpx-vp9 to video encoder's choices * Update CONTRIBUTING.md * Migrate gpu flags to execution flags * Fix linter * Encode and decode execution providers for easier usage * Fix comment * Update CLI usage * Fixed wrong frame colors for preview * Introduce dynamic frame procesors * Remove unused imports * Use different structure * modified: roop/core.py , enhancer and swapper * fix link error * modified core.py ui.py frame/core.py * fix get_frame_processors_modules() * fix lint error in core.py * fix face_enhancer.py * fix enhancer.py * fix ui.py * modified: roop/ui.py * fix ui.py * Update ui.py * Fix preview to work with multiple frame processors * Remove multi processing as the performance is equal but memory consumtion better * Extract common methods to processors.frame.core * Add comments and todos * Minor fixes * Minor fixes * Limit enhancer to 1 thread but restore afterwards * Limit enhancer to 1 thread but restore afterwards * Update README and GUI demo * Implementation check for frame processors * Improve path validation * Introduce normalize output path * Fix GUI startup * Introduce pre_start() and move globals check to pre_check() * Flip the hooks and update CLI usage * Introduce bool returns for pre_check() and pre_start(), Scope for terminal output * Cast deprecated args and warn the user * Migrate to ctk.CTkImage * readme update: old keys fixed * Unused import made CI test to fail * Give gfpgan a shot * Update dependencies * Update dependencies * Update dependencies * Update dependencies * Fix ci (#498) * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Fix preview (#499) * Minor changes * Fix override of files using restore_audio() * Host the models in our huggingface space * Remove everything source face related from enhancer (#510) * Remove everything source face related from enhancer * Disable upscale for enhancer to double fps * Using futures for multi threading * Introduce predicter (#530) * Hotfix predicter * Fix square brackets in the target path (#532) * fixes the problem with square brackets in the target file path * fixes the problem with square brackets in the target file path * Ok, here is the fix without regexps * unused import * fix for ci * glob.escape fits here * Make multiple processors work with images * Fix output normalize for deprecated args * Make roop more or less type-safe (#541) * Make roop more or less type-safe * Fix ci.yml * Fix urllib type error * Rename globals in ui * Update utilities.py (#542) Updated the extraction process with the ffmpeg command that corrects the colorspace while extracting to png, and corrected the ffmpeg command, adding '-pix_fmt', 'rgb24', '-sws_flags', '+accurate_rnd+full_chroma_int', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1' '-pix_fmt rgb24', means treat the image as RGB (or RGBA) '-sws_flags +accurate_rnd+full_chroma_int', means use full color and chroma subsampling instead of 4:2:0 '-colorspace 1', '-color_primaries 1', '-color_trc 1' put the metadata color tags to the png * Use GFPGANv1.4 for enhancer * Fixing the colorspace issue when writing the mp4 from the extracted pngs (#550) * Update utilities.py Updated the extraction process with the ffmpeg command that corrects the colorspace while extracting to png, and corrected the ffmpeg command, adding '-pix_fmt', 'rgb24', '-sws_flags', '+accurate_rnd+full_chroma_int', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1' '-pix_fmt rgb24', means treat the image as RGB (or RGBA) '-sws_flags +accurate_rnd+full_chroma_int', means use full color and chroma subsampling instead of 4:2:0 '-colorspace 1', '-color_primaries 1', '-color_trc 1' put the metadata color tags to the png * Fixing color conversion from temp png sequence to mp4 '-sws_flags', 'spline+accurate_rnd+full_chroma_int', ' use full color and chroma subsampling -vf', 'colorspace=bt709:iall=bt601-6-625:fast=1', keep the same rec709 colorspace '-color_range', '1', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1', put the metadata color tags to the mp4 * Revert "Fixing the colorspace issue when writing the mp4 from the extracted pngs (#550)" This reverts commit cf5f27d. * Revert "Update utilities.py (#542)" This reverts commit d57279c. * Restore colorspace restoring * Add metadata to cli and ui * Introduce Face and Frame typing * Update CLI usage --------- Co-authored-by: Phan Tuấn Anh <[email protected]> Co-authored-by: Antoine Buchser <[email protected]> Co-authored-by: Eamonn A. Sweeney <[email protected]> Co-authored-by: Moeblack <Moeblack@[email protected]> Co-authored-by: Pozitronik <[email protected]> Co-authored-by: Pikachu~~~ <[email protected]> Co-authored-by: K1llM@n <[email protected]> Co-authored-by: NickPittas <[email protected]>
s0md3v · Jun 20, 2023 · ec12665 · ec12665
1 parent 7010822
commit ec12665
Show file tree

Hide file tree

Showing 29 changed files with 1,079 additions and 748 deletions.
diff --git a/.flake8 b/.flake8
@@ -1,2 +1,3 @@
 [flake8]
-select = E3, E4, F
+select = E3, E4, F
+per-file-ignores = roop/core.py:E402
diff --git a/.github/examples/face.jpg → .github/examples/source.jpg b/.github/examples/face.jpg → .github/examples/source.jpg
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -13,7 +13,9 @@ jobs:
     with:
      python-version: 3.9
   - run: pip install flake8
-  - run: flake8 run.py core
+  - run: pip install mypy
+  - run: flake8 run.py roop
+  - run: mypy --config-file mypi.ini run.py roop
  test:
   runs-on: ubuntu-latest
   steps:
@@ -25,8 +27,7 @@ jobs:
     uses: actions/setup-python@v2
     with:
      python-version: 3.9
-  - run: pip install -r requirements.txt gdown
-  - run: gdown 13QpWFWJ37EB-nHrEOY64CEtQWY-tz7DZ
-  - run: ./run.py -f=.github/examples/face.jpg -t=.github/examples/target.mp4 -o=.github/examples/output.mp4
-  - run: ffmpeg -i .github/examples/snapshot.mp4 -i .github/examples/output.mp4 -filter_complex "psnr" -f null -
+  - run: pip install -r requirements-ci.txt
+  - run: python run.py -s=.github/examples/source.jpg -t=.github/examples/target.mp4 -o=.github/examples/output.mp4
+  - run: ffmpeg -i .github/examples/snapshot.mp4 -i .github/examples/output.mp4 -filter_complex psnr -f null -
 
diff --git a/.gitignore b/.gitignore
@@ -1,3 +1,4 @@
 .idea
+models
+temp
 __pycache__
-*.onnx
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,21 @@
+## Pull Requests
+
+### Do
+
+- ...consider to fix bugs over adding features
+- ...one pull request for one feature or improvement 
+- ...consult us about implementation details
+- ...proper testing before you submit your code
+- ...resolve failed CI pipelines
+
+### Don't
+
+- ...introduce fundamental changes in terms of software architecture
+- ...introduce OOP - we accept functional programming only
+- ...ignore given requirements or try to work around them
+- ...submit code to a development branch without consulting us
+- ...submit massive amount of code changes
+- ...submit a proof of concept
+- ...submit code that is using undocumented and private APIs
+- ...solve third party issues in our project
+- ...comment what your code does - use proper naming instead
diff --git a/README.md b/README.md
@@ -36,26 +36,32 @@ Additional command line arguments are given below:
 ```
 options:
   -h, --help            show this help message and exit
-  -f SOURCE_IMG, --face SOURCE_IMG
-                        use this face
+  -s SOURCE_PATH, --source SOURCE_PATH
+                        select an source image
   -t TARGET_PATH, --target TARGET_PATH
-                        replace this face
-  -o OUTPUT_FILE, --output OUTPUT_FILE
-                        save output to this file
-  --keep-fps            maintain original fps
-  --keep-frames         keep frames directory
-  --all-faces           swap all faces in frame
+                        select an target image or video
+  -o OUTPUT_PATH, --output OUTPUT_PATH
+                        select output file or directory
+  --frame-processor {face_swapper,face_enhancer} [{face_swapper,face_enhancer} ...]
+                        pipeline of frame processors
+  --keep-fps            keep original fps
+  --keep-audio          keep original audio
+  --keep-frames         keep temporary frames
+  --many-faces          process every face
+  --video-encoder {libx264,libx265,libvpx-vp9}
+                        adjust output video encoder
+  --video-quality VIDEO_QUALITY
+                        adjust output video quality
   --max-memory MAX_MEMORY
-                        maximum amount of RAM in GB to be used
-  --cpu-cores CPU_CORES
-                        number of CPU cores to use
-  --gpu-threads GPU_THREADS
-                        number of threads to be use for the GPU
-  --gpu-vendor {apple,amd,intel,nvidia}
-                        choice your GPU vendor
+                        maximum amount of RAM in GB
+  --execution-provider {cpu,...} [{cpu,...} ...]
+                        execution provider
+  --execution-threads EXECUTION_THREADS
+                        number of execution threads
+  -v, --version         show program's version number and exit
 ```
 
-Looking for a CLI mode? Using the -f/--face argument will make the program in cli mode.
+Looking for a CLI mode? Using the -s/--source argument will make the program in cli mode.
 
 ## Credits
 - [henryruhs](https://github.com/henryruhs): for being an irreplaceable contributor to the project

diff --git a/gui-demo.png b/gui-demo.png
diff --git a/mypi.ini b/mypi.ini
@@ -0,0 +1,7 @@
+[mypy]
+check_untyped_defs = True
+disallow_any_generics = True
+disallow_untyped_calls = True
+disallow_untyped_defs = True
+ignore_missing_imports = True
+strict_optional = False
diff --git a/requirements-ci.txt b/requirements-ci.txt
@@ -0,0 +1,14 @@
+numpy==1.23.5
+opencv-python==4.7.0.72
+onnx==1.14.0
+insightface==0.7.3
+psutil==5.9.5
+tk==0.1.0
+customtkinter==5.1.3
+torch==2.0.1
+torchvision==0.15.2
+onnxruntime==1.15.0
+tensorflow==2.12.0
+opennsfw2==0.10.2
+protobuf==4.23.2
+tqdm==4.65.0
diff --git a/requirements.txt b/requirements.txt
@@ -6,14 +6,18 @@ onnx==1.14.0
 insightface==0.7.3
 psutil==5.9.5
 tk==0.1.0
+customtkinter==5.1.3
 pillow==9.5.0
 torch==2.0.1+cu118; sys_platform != 'darwin'
 torch==2.0.1; sys_platform == 'darwin'
+torchvision==0.15.2+cu118; sys_platform != 'darwin'
+torchvision==0.15.2; sys_platform == 'darwin'
 onnxruntime==1.15.0; sys_platform == 'darwin' and platform_machine != 'arm64'
 onnxruntime-silicon==1.13.1; sys_platform == 'darwin' and platform_machine == 'arm64'
 onnxruntime-gpu==1.15.0; sys_platform != 'darwin'
 tensorflow==2.13.0rc1; sys_platform == 'darwin'
 tensorflow==2.12.0; sys_platform != 'darwin'
 opennsfw2==0.10.2
 protobuf==4.23.2
-tqdm==4.65.0
+tqdm==4.65.0
+gfpgan==1.3.8
diff --git a/roop/__init__.py b/roop/__init__.py
@@ -1 +0,0 @@
-

diff --git a/roop/analyser.py b/roop/analyser.py
diff --git a/roop/capturer.py b/roop/capturer.py
@@ -0,0 +1,20 @@
+from typing import Any
+import cv2
+
+
+def get_video_frame(video_path: str, frame_number: int = 0) -> Any:
+    capture = cv2.VideoCapture(video_path)
+    frame_total = capture.get(cv2.CAP_PROP_FRAME_COUNT)
+    capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
+    has_frame, frame = capture.read()
+    capture.release()
+    if has_frame:
+        return frame
+    return None
+
+
+def get_video_frame_total(video_path: str) -> int:
+    capture = cv2.VideoCapture(video_path)
+    video_frame_total = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
+    capture.release()
+    return video_frame_total