opt: add LoadBalancerClass support for TiDB service (#5964)

[ti-chi-bot]

This is an automated cherry-pick of #5964

What problem does this PR solve?

closes #5543

What is changed and how does it work?

Code changes

• Has Go code change
• Has CI related scripts change

Tests

• Unit test
• E2E test
• Manual test
  1. set .service.type: LoadBalancer and .service.loadBalancerClass: service.k8s.aws/nlb for TiDB in TidbCluster CR
  2. apply the TidbCluster CR and wait for the TidbCluster to be ready
  3. check an LB service is created
  4. check the log of tidb-controller-manager, no repeated error log as LoadBalancerClass error #5543
  5. update the replicas of TiDB to +1
  6. wait for the TidbCluster to be ready again
  7. check the log of tidb-controller-manager, no repeated error log as LoadBalancerClass error #5543
• No code

Side effects

• Breaking backward compatibility
• Other side effects:

Related changes

• Need to cherry-pick to the release branch
• Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign charleszheng44 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 50.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 56.98%. Comparing base (c23a794) to head (2a2cf07).
Report is 11 commits behind head on release-1.6.

Additional details and impacted files

@@               Coverage Diff               @@
##           release-1.6    #5965      +/-   ##
===============================================
- Coverage        57.20%   56.98%   -0.22%     
===============================================
  Files              259      260       +1     
  Lines            33233    45030   +11797     
===============================================
+ Hits             19010    25661    +6651     
- Misses           12290    17384    +5094     
- Partials          1933     1985      +52     

Flag	Coverage Δ
unittest	56.98% <50.00%> (-0.22%)	⬇️