Inpaint workflow: add cv2 mask creator and args for --masked-image-path #134
Conversation
anthonywu
changed the title
| @@ -68,6 +68,7 @@ def add_image_to_image_arguments(self, required=False) -> None: | |||
| self.supports_image_to_image = True | |||
| self.add_argument("--init-image-path", type=Path, required=required, default=None, help="Local path to init image") | |||
| self.add_argument("--init-image-strength", type=float, required=False, default=ui_defaults.INIT_IMAGE_STRENGTH, help=f"Controls how strongly the init image influences the output image. A value of 0.0 means no influence. (Default is {ui_defaults.INIT_IMAGE_STRENGTH})") | |||
| self.add_argument("--masked-image-path", type=Path, required=False, default=None, help="Local path to separate masked image as complement to --init-image-path") | |||
There was a problem hiding this comment.
@filipstrand I'll leave it to you to remove the hard coding in #129's generate.py
There was a problem hiding this comment.
Absolutely, I'll clean this up!
| if self.original_image is None: | ||
| raise FileNotFoundError(f"Could not open or find the image: {image_path}") | ||
|
|
||
| self.mask_output_path = image_path.with_name(f"{image_path.stem}_mask").with_suffix(".png") |
There was a problem hiding this comment.
this will save the mask image in the same parent dir as the source image (it does assume user has write access to the source image's parent directory)
trying to not over-complicate things for now, this IMO is the most obvious/friendly implementation
| self.display_image, | ||
| text, | ||
| (10, 30), | ||
| cv2.FONT_HERSHEY_DUPLEX, |
There was a problem hiding this comment.
font is funky but the Hershey font family is the only one that ships with opencv and therefore no additional dependencies
|
@anthonywu Looks really good, thanks a lot for the help with the GUI! I'll continue finishing up the last parts of the in-painting feature and include it in the release after v.0.6.0. There is very little work left for the in-context lora feature now and I'm planning on releasing v.0.6.0 tomorrow. Currently, only the boring parts (release notes, readme, tests etc.) are left haha :). Though I have to say that cursor + sonnet 3.7 are now very helpful with this kind of work that I have a habit of putting off to the last minute! |
This PR bases on #129 and can be merged into @filipstrand's WIP branch to continue his work as-is without merge issues.
With the inpaint feature, we're going to expect the user or client GUIs to provide the complementary mask image.
In this PR I propose:
--masked-image-pathargparser additionopencv-python/import cv2library to display a basic window that lets the user draw a binary mask over their original imageFor discussion but not implemented: maybe the
--masked-image-pathcan be set to a value of literalinteractive(or empty?) and the CLI can bring up the cv2 GUI to allow a live mask drawing. IMO should be pursued in a separate PR after the basic inpaint functionality is done, to not scope creep.Also, if we move the
cv2tool inside the library, maybe there should be amflux.pre_processingpackage for this and alike tools. (while reasonably restricting to existing library dependencies)Usage
python tools/inpaint_mask_tool.py ~/Downloads/presidio-view.webpGUI
the mask image can be seen in Preview.app as soon as you hit
sfor save.Example
via
mflux-generate -m dev-fill --init-image-path ~/Downloads/presidio-view.webp --masked-image-path ~/Downloads/presidio-view_mask.png --prompt "godzilla towering over the hills" --steps 8 --seed 1(in my WIP workspace)