New release

[vladmandic]

Update for 2024-11-21

Highlights for 2024-11-21

Three weeks is a long time in Generative AI world - and we're back with ~140 commits worth of updates!

What's New?

First, a massive update to docs including new UI top-level info tab with access to changelog and wiki, many updates and new articles AND full built-in documentation search capabilities

New integrations

• PuLID: Pure and Lightning ID Customization via Contrastive Alignment
• InstantX InstantIR: Blind Image Restoration with Instant Generative Reference
• nVidia Labs ConsiStory: Consistent Image Generation
• MiaoshouAI PromptGen v2.0 VQA captioning

Workflow Improvements

• Native Docker support
• SD3x & Flux.1: more ControlNets, all-in-one-safetensors, DPM samplers, skip-layer-guidance, etc.
• XYZ grid: benchmarking, video creation, etc.
• Enhanced prompt parsing
• UI improvements
• Installer self-healing venv

And quite a few more improvements and fixes since the last update!
For full list and details see changelog...

README | CHANGELOG | WiKi | Discord

Details for 2024-11-21

• Docs:

  • new top-level info tab with access to changelog and wiki
  • UI built-in changelog search
    since changelog is the best up-to-date source of info
    go to info -> changelog and search/highligh/navigate directly in UI!
  • UI built-in wiki
    go to info -> wiki and search wiki pages directly in UI!
  • major Wiki and Home updates
  • updated API swagger docs for at /docs

• Integrations:

  • PuLID: Pure and Lightning ID Customization via Contrastive Alignment
    • advanced method of face id transfer with better quality as well as control over identity and appearance
      try it out, likely the best quality available for sdxl models
    • select in scripts -> pulid
    • compatible with sdxl for text-to-image, image-to-image, inpaint, refine, detailer workflows
    • can be used in xyz grid
    • note: this module contains several advanced features on top of original implementation
  • InstantIR: Blind Image Restoration with Instant Generative Reference
    • alternative to traditional img2img with more control over restoration process
    • select in image -> scripts -> instantir
    • compatible with sdxl
    • note: after used once it cannot be unloaded without reloading base model
  • ConsiStory: Consistent Image Generation
    • create consistent anchor image and then generate images that are consistent with anchor
    • select in scripts -> consistory
    • compatible with sdxl
    • note: very resource intensive and not compatible with model offloading
    • note: changing default parameters can lead to unexpected results and/or failures
    • note: after used once it cannot be unloaded without reloading base model
  • MiaoshouAI PromptGen v2.0 base and large:
    • in process -> visual query
    • caption modes:
      <GENERATE_TAGS> generate tags
      <CAPTION>, <DETAILED_CAPTION>, <MORE_DETAILED_CAPTION> caption image
      <ANALYZE> image composition
      <MIXED_CAPTION>, <MIXED_CAPTION_PLUS> detailed caption and tags with optional analyze

• Model improvements:

  • SD35: ControlNets:
    • InstantX Canny, Pose, Depth, Tile
    • Alimama Inpainting, SoftEdge
    • note: that just like with FLUX.1 or any large model, ControlNet are also large and can push your system over the limit
      e.g. SD3 controlnets vary from 1GB to over 4GB in size
  • SD35: All-in-one safetensors
    • examples: large, medium
    • note: enable bnb on-the-fly quantization for even bigger gains
  • SD35: skip-layer-guidance
    • enable in scripts -> slg
    • allows for granular strength/start/stop control of guidance for each layer of the model
  • NoobAI XL ControlNets, thanks @lbeltrame

• Workflow improvements:

  • Native Docker support with pre-defined Dockerfile
  • Samplers:
    • FlowMatch samplers:
      • Applicable to SD 3.x and Flux.1 models
      • Complete family: DPM2, DPM2a, DPM2++, DPM2++ 2M, DPM2++ 2S, DPM2++ SDE, DPM2++ 2M SDE, DPM2++ 3M SDE
    • Beta and Exponential sigma method enabled for all samplers
  • XYZ grid:
    • optional time benchmark info to individual images
    • optional add params to individual images
    • create video from generated grid images
      supports all standard video types and interpolation
  • Prompt parser:
    • support for prompt scheduling
    • renamed parser options: native, xhinker, compel, a1111, fixed
    • parser options are available in xyz grid
    • improved caching
  • UI:
    • better gallery and networks sidebar sizing
    • add additional hotkeys
    • add show networks on startup setting
    • better mapping of networks previews
    • optimize networks display load
  • Image2image:
    • integrated refine/upscale/hires workflow

• Other:

  • Installer:
    • Log venv and package search paths
    • Auto-remove invalid packages from venv/site-packages
      e.g. packages starting with ~ which are left-over due to windows access violation
    • Requirements: update
  • Scripts:
    • More verbose descriptions for all scripts
  • Model loader:
    • Report modules included in safetensors when attempting to load a model
  • CLI:
    • refactor command line params
      run webui.sh/webui.bat with --help to see all options
    • added cli/model-metadata.py to display metadata in any safetensors file
    • added cli/model-keys.py to quicky display content of any safetensors file
  • Internal:
    • Auto pipeline switching coveres wrapper classes and nested pipelines
    • Full settings validation on load of config.json
    • Refactor of all params in main processing classes
    • Improve API scripts usage resiliency

• Fixes:

  • custom watermark add alphablending
  • fix xyz grid include images
  • fix xyz skip on interrupted
  • fix vqa models ignoring hfcache folder setting
  • fix network height in standard vs modern ui
  • fix k-diff enum on startup
  • fix text2video scripts
  • multiple xyz-grid fixes
  • dont uninstall flash-attn
  • ui css fixes