Possible EVA clip implementation?

[lhucklen]

Currently, Eva clip does not work in forge. Therefore, pulid is incompatible. I am no pro but I like AI. So I leave it to the coders and professionals. But would the following code alterations be a good start for getting it to work in forge?

After looking at Forge's structure, a good starting point would be modules/core/clip.py as this is where Forge handles its CLIP implementations. Here's a suggested minimal-impact approach:

1. First, examine modules/core/clip.py:

# Add EVA support while maintaining existing structure
from transformers import CLIPProcessor, CLIPModel

class ClipImplementation:
    def __init__(self):
        self.clip_type = None
        self.model = None
        
    def load_eva(self, model_name):
        self.clip_type = 'eva'
        self.model = CLIPModel.from_pretrained(model_name)
        self.processor = CLIPProcessor.from_pretrained(model_name)
        return self

    # This preserves Forge's existing clip handling while adding EVA support
    def encode_text(self, text):
        if self.clip_type == 'eva':
            return self._encode_eva_text(text)
        return self._encode_original_text(text)  # existing method

    def _encode_eva_text(self, text):
        # EVA specific encoding
        inputs = self.processor(text=text, return_tensors="pt", padding=True)
        return self.model.get_text_features(**inputs)

2. Then modify modules/core/implementations.py to add EVA as an option:

class Implementation:
    def clip_model(self):
        if self.clip_type == "eva":
            return ClipImplementation().load_eva(self.model_path)
        return self.original_clip_implementation()  # existing method

This approach:

• Maintains Forge's existing architecture
• Adds EVA support as an additional option
• Minimizes changes to core functionality

Can anyone weigh in?