Run (Instant) not assigning new jobs until all workers are done

[ionostrat-cmd]

Due to having 4 very different GPU's running a batch of 4 is not efficient.

I specifically leave the Master with nothing to do.

The solution of activatingload_balance=trueleads to having to click Run every 30 seconds for optimum load balancing

Trying to Run (Instant)

Sadly only assigns one job to a worker and then waits until it is done with the job.

Perhaps a Job can be assigned to any idle worker?
This would eliminate any downtime.

[robertvoy]

Ok, can you let me know the use case for this? This is so I can better understand.

Right now this kind of dynamic load balancing happens with the Distributed upscaler, so it is possible. I just need to understand how you want to use Run (Instant). I've never used that function before.

[ionostrat-cmd]

Run (Instant) issues a new job as soon as master is idle. Master is busy until all workers are done.

The use case for this is to never have a worker be idle. As soon as a worker is idle- they are immediately assigned a job even if the master is 'busy'.

I figure the easiest way to implement this is to show master as idle until all workers are assigned a job.

In my specific use case I have a workflow that cannot use the distributed upscaler.

I have 3 Pc's with a 5060 and one with a 4080TI

5060 needs 90 seconds for one image
4080TI needs 30 seconds for one image

If i click on (1) Run 4 times i have to issue a new job every 30 seconds to keep the 4080TI busy.

Alternatively i click on (16) Run- thus giving the 4080TI 4 jobs which it has done in 2 minutes. Which takes the 5060's 6 Minutes and would leave the 4080TI idle for 4 minutes

So then i click on (16) Run again. Every two minutes. most of those 16 Jobs are assigned to the 4080TI- thus i only need to press (16) Run every 3-4 minutes.

For now i have produced a small autohotkey script that checks if any worker shows green on screen and then clicks on (1) Run.

Lastly i wanted to thank you for the great tool that is ComfyUI-Distributed

[robertvoy]

Ok so run instant is a workaround for now, right? But your goal is to run a large batch with maximum worker utilisation?

So for example you would setup a workflow and then you want to run it 16 times to get 16 results that are distributed dynamically amongst workers.

[ionostrat-cmd]

This is how it works currently:

When i press (16) Run- each worker gets 4 jobs assigned when load balancing=true. That is dynamically distributed.

But the 4080TI takes 2 minutes for the 4 assigned jobs and will be idle for 4 minutes while the others are doing their 4 jobs . The entire process takes 6 Minutes.

16 jobs in 6 minutes

This is how i think it should ideally work:

The master only gives one job to a worker once it is idle.

Thus:
I press (18) Run
4 out of the 18 jobs are given to the 4 workers

after 30 seconds, the 4080TI gets a new job, then another. The 4080TI has done 3 jobs in 1min 30sec. The other three have done one each in that time.

Now there are 12 jobs left

This process repeats.

The 18 jobs are done in 4min 30sec. None of the workers were idle at any time.

18 jobs in 4min 30sec