Support setting minMember when creating workloads

[hwdef]

What is the problem you're trying to solve

Currently, the default value of minMember is 1, which is inappropriate in some scenarios.

volcano/pkg/controllers/podgroup/pg_controller_handler.go

Line 215 in 68fba2c

MinMember: 1,

I hope that minMember can be set when creating a deployment or other non-vcjob workloads.

Describe the solution you'd like

When creating a podgroup, we can read an annotation, such as
scheduling.volcano.sh/minMember: 10
Set minMember based on this value

Additional context

No response

[noobzzw]

Hi, this is a great idea. Besides setting the minMember manually, can we also read the StatefulSet or ReplicaSet in the ownerReferences to obtain the minMember automatically?

[hwdef]

This is probably not a good idea since many deployment functions don't want to be affected by the gang.

[noobzzw]

got it.

[Monokaix]

I think we can support read an annotation key from the workload, and users can set an expected value, or else the minmember is still 1.

[JesseStutler]

scheduling.volcano.sh/minMember or scheduling.volcano.sh/minAvailable? From the perspective of vcjob, is minAvailable, deployment/statefulset/daemonset/job and vcjob are at the same level, so should it be minAvailable?

[Monokaix]

minAvailable is the semantics of vcjob, and for other workloads users may just need to concern the podgroup.
But the user experience will be more unified usingminAvailable.

[Monokaix]

Another question is: The annotation should be set at pod level or workload level, though its more reasonable setting at workload level, the current queue and podgroup annotations are both set at pod level, so it's better to keep consistent.
How do you guys think?

[hwdef]

I think it's more reasonable to set at the workload level.
But we can do compatibility. priority reading the workload annotation. If not exist, then read the pod annotation.

[Monokaix]

I think it's more reasonable to set at the workload level. But we can do compatibility. priority reading the workload annotation. If not exist, then read the pod annotation.

It seems a little confused...

[lowang-bh]

It is more reasonable setting at workload level.

[hwdef]

We also should consider minResource

Ref: #4000

[Monokaix]

/assign @sceneryback

[volcano-sh-bot]

@Monokaix: GitHub didn't allow me to assign the following users: sceneryback.

Note that only volcano-sh members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @sceneryback

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sceneryback]

@hwdef @Monokaix @lowang-bh @JesseStutler Thanks for all your suggestions, have implemented this feature in #4000 and volcano-sh/apis#154, please have a look

[Monokaix]

It is more reasonable setting at workload level.

There is also a question when setting at workload level, there will be an extra API request to get the workload's annotations, which may cause performance at scale.

[Monokaix]

It is more reasonable setting at workload level.

There is also a question when setting at workload level, there will be an extra API request to get the workload's annotations, which may cause performance at scale.

A workaround is that we cache annotations of rs/sts, etc. and get annotations from cache, which not only solve the current problem but also inheritUpperAnnotations can avoid to request APIServer.