Custom Node doing something at "queue time"?

[darkpixel]

I wrote a quick custom node to pull in data from a text box and a file and then randomly pick a line based on the seed.
Unfortunately, what I'm seeing is when a job start running, it reads the file and makes the choice...but what I don't see is a way to read the file when the job is queued.

This prevents being able to have a text file where you might queue a job, then while the job is running, make a change to the file, queue it up, make another change, queue it up, etc...

Is there a hook that gets called when a job is queued, not just when it starts running?

[ltdrdata]

You can use onprompt.
https://github.com/ltdrdata/ComfyUI-Inspire-Pack/blob/c8fddd510f569f7ec22f5458659f39127fee8ed0/inspire/inspire_server.py#L357

[darkpixel]

Is there documentation anywhere on the various hooks?
I've been slowly pouring over the source and I'm obviously missing a lot of stuff. :)

[ltdrdata]

Unfortunately, there is no such document, yet.

[darkpixel]

That signal doesn't appear to include any context about the object--so I can't get properties of various fields like "filename" on my widget. Is there a different signal that's fired for that or am I missing something?

[darkpixel]

Sure--but how will I know which instance of the custom node I'm in?

If I place 5 "xyzzy nodes" (made up name) on the screen, and onprompt is calling a function outside of the xyzzy class object and before the defined function inside the xyzzy node is called...I don't see a way of accomplishing it.

[ltdrdata]

'onprompt' is meant to determine the context based on the input values in the widget. What exactly are you trying to achieve?

[darkpixel]

Given one or more xyzzy widgets (that have inputs for a filename, text, and optionally an input from another xyzzy node), I want to resolve all the filenames, read in those files, and "cache" a random line from the file.

It's basically a re-implementation of wildcards. But what I've found with most wildcard formats is that the files are read on init and never again--making changes impossible. If I read the files every time in the node's defined FUNCTION, the reads obviously don't happen until the node is being processed.

What this means is that...let's say you have a file called gender.txt and it contains a line "a man", and you queue up a prompt and it starts executing, it's going to add "a man" to the prompt. Then, while that first prompt is running, you add a line "a woman" and queue up a second prompt (while the first prompt is still running), then delete the line "a woman"...when the second prompt actually starts executing, it's not going to execute based on the state of the file at queue time.

[ltdrdata]

Are you looking to add populated text to the widget, or do you want it to be added to the prompt data of the workflow being handled internally? These two concepts are entirely different.

The values of the widget are what is displayed in the browser and are serialized when the queue button is pressed. 'json_data' is a value only visible to the backend; manipulating this value affects the execution of the workflow but doesn't impact what is displayed in the browser.

'onprompt' operates between the queue send from the browser and the queued state of backend.

[darkpixel]

I guess I'm not entirely certain.
If I add a widget and put in a filename of 'text.txt' and hit the queue button, I want to read 'test.txt' and choose a random line at queue-time, not at run-time. It seems like most of the wildcard-style plugins handle this by reading test.txt in during __init__ of the object and then storing the data in a class variable--then they access that when the defined FUNCTION runs, but they only read in the contents of the file once, and then just keep reading from the class-based variable every time a prompt runs. I'm trying to figure out the value of 'filename' so I can read it and make a random choice at queue time...so it sounds like I need to see if I can figure out an objects definition in json_data at queue time, read in the file data and make a choice and store that for when FUNCTION gets called when the prompt is executed.

I'll dig around and look at json_data to see if I can "find myself" (find the json_data for my class from the function called by onprompt) during queue-time...but I have doubts I can not sure if I can find the instantiated class.

[ltdrdata]

I guess I'm not entirely certain. If I add a widget and put in a filename of 'text.txt' and hit the queue button, I want to read 'test.txt' and choose a random line at queue-time, not at run-time. It seems like most of the wildcard-style plugins handle this by reading test.txt in during __init__ of the object and then storing the data in a class variable--then they access that when the defined FUNCTION runs, but they only read in the contents of the file once, and then just keep reading from the class-based variable every time a prompt runs. I'm trying to figure out the value of 'filename' so I can read it and make a random choice at queue time...so it sounds like I need to see if I can figure out an objects definition in json_data at queue time, read in the file data and make a choice and store that for when FUNCTION gets called when the prompt is executed.

I'll dig around and look at json_data to see if I can "find myself" (find the json_data for my class from the function called by onprompt) during queue-time...but I have doubts I can not sure if I can find the instantiated class.

https://github.com/ltdrdata/ComfyUI-Inspire-Pack/blob/c8fddd510f569f7ec22f5458659f39127fee8ed0/inspire/inspire_server.py#L350
https://github.com/ltdrdata/ComfyUI-Inspire-Pack/blob/c8fddd510f569f7ec22f5458659f39127fee8ed0/inspire/inspire_server.py#L259

In that case, it's essentially no different from the existing wildcard structure. Just read the file each time instead of using the cache.

The reason wildcard processing nodes utilize cached data is because scanning massive wildcards, such as billions of wildcard, can consume significant scan time on its own.