Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add rke2 network configuration to installer #886

Merged
merged 3 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,10 @@ type Install struct {
VipHwAddr string `json:"vipHwAddr,omitempty"`
VipMode string `json:"vipMode,omitempty"`

ClusterDNS string `json:"clusterDns,omitempty"`
ClusterPodCIDR string `json:"clusterPodCidr,omitempty"`
ClusterServiceCIDR string `json:"clusterServiceCidr,omitempty"`

ForceEFI bool `json:"forceEfi,omitempty"`
Device string `json:"device,omitempty"`
ConfigURL string `json:"configUrl,omitempty"`
Expand Down
4 changes: 4 additions & 0 deletions pkg/config/templates/rancherd-10-harvester.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,10 @@ resources:
enabled: true
kube-vip-cloud-provider:
enabled: true
promote:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is RKE2 better than promote?

we also have another requirement: customize the RKE2 CNI, some one prefers Calico

Copy link
Contributor Author

@ihcsim ihcsim Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial thought is to communicate that, within the ManagedChart, these settings are only used by the promotion controller, instead of a general mechanism to override RKE2, per harvester/harvester#7156. My current preference leans towards keeping it as promote. LMKWYT.

Re: CNI, can we work on it as a separate issue?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, customized CNI will be in another issue & PR, which needs some investigation.

promotion is also good.

Copy link
Contributor Author

@ihcsim ihcsim Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Custom CNI enhancement request added at harvester/harvester#7197.

clusterPodCIDR: {{ or .ClusterPodCIDR "10.52.0.0/16" }}
clusterServiceCIDR: {{ or .ClusterServiceCIDR "10.53.0.0/16" }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When users customize/default those 3 fields, the value will be writen to harvester managedchart, could you check if we can add webhook to deny any later change on the managedchart related fields? thanks.

If a user tries to change the managedchart on the fly, the result will be unexpected.

btw, do you have a document PR to for those installation configuration options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point on validation webhook. I don't see why not. I'll create a new issue for it. The document PR is still WIP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created harvester/harvester#7196 to track validating webhook implementation.

clusterDNS: {{ or .ClusterDNS "10.53.0.10" }}
- apiVersion: management.cattle.io/v3
kind: ManagedChart
metadata:
Expand Down
6 changes: 3 additions & 3 deletions pkg/config/templates/rke2-90-harvester-server.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
cni: multus,canal
cluster-cidr: 10.52.0.0/16
service-cidr: 10.53.0.0/16
cluster-dns: 10.53.0.10
cluster-cidr: {{ or .ClusterPodCIDR "10.52.0.0/16" }}
bk201 marked this conversation as resolved.
Show resolved Hide resolved
service-cidr: {{ or .ClusterServiceCIDR "10.53.0.0/16" }}
cluster-dns: {{ or .ClusterDNS "10.53.0.10" }}
tls-san:
- {{ .Vip }}
{{- with $args := .GetKubeletArgs }}
Expand Down
12 changes: 12 additions & 0 deletions pkg/console/constant.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,18 @@ const (
vipLabel = "VIP"
askVipMethodLabel = "VIP Mode"

clusterNetworkTitle = "Configure cluster network"
clusterPodCIDRLabel = "Pod CIDR"
clusterServiceCIDRLabel = "Service CIDR"
clusterDNSLabel = "Cluster DNS IP"
clusterPodCIDRPanel = "podCIDRPanel"
clusterServiceCIDRPanel = "serviceCIDRPanel"
clusterDNSPanel = "clusterDNSPanel"
clusterNetworkNotePanel = "clusterNetworkNotePanel"
clusterNetworkDNSNotePanel = "clusterNetworkDNSNotePanel"
clusterNetworkValidatorPanel = "clusterNetworkValidatorPanel"
clusterNetworkNote = "Note: Leave blank to use the default pod CIDR 10.52.0.0/16, service CIDR 10.53.0.0/16 and cluster DNS 10.53.0.10. If the service CIDR is changed, the DNS IP must be updated to be within the service CIDR."

clusterTokenCreateNote = "Note: The token is used for adding nodes to the cluster"
clusterTokenJoinNote = "Note: Input the token of the existing cluster"
serverURLNote = "Note: Input VIP/domain name of the management node"
Expand Down
251 changes: 251 additions & 0 deletions pkg/console/install_panels.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package console
import (
"fmt"
"net"
"net/netip"
"os"
"os/exec"
"strconv"
Expand Down Expand Up @@ -157,6 +158,7 @@ func setPanels(c *Console) error {
addDiskPanel,
addHostnamePanel,
addNetworkPanel,
addClusterNetworkPanel,
addVIPPanel,
addDNSServersPanel,
addNTPServersPanel,
Expand Down Expand Up @@ -1230,6 +1232,9 @@ func addHostnamePanel(c *Console) error {

prev := func(_ *gocui.Gui, _ *gocui.View) error {
c.CloseElements(hostnamePanel, hostnameValidatorPanel)
if c.config.Install.Mode == config.ModeCreate {
return showClusterNetworkPage(c)
}
return showNetworkPage(c)
}

Expand Down Expand Up @@ -1448,6 +1453,9 @@ func addNetworkPanel(c *Console) error {
spinner.Stop(false, "")
g.Update(func(_ *gocui.Gui) error {
closeThisPage()
if c.config.Install.Mode == config.ModeCreate {
return showClusterNetworkPage(c)
}
return showHostnamePage(c)
})
}
Expand Down Expand Up @@ -1744,6 +1752,249 @@ func addNetworkPanel(c *Console) error {
return nil
}

func showClusterNetworkPage(c *Console) error {
return showNext(
c,
clusterServiceCIDRPanel,
clusterDNSPanel,
clusterNetworkNotePanel,
clusterNetworkValidatorPanel,
clusterPodCIDRPanel)
}

func addClusterNetworkPanel(c *Console) error {
// define page navigation
closePage := func() {
c.CloseElements(
clusterPodCIDRPanel,
clusterServiceCIDRPanel,
clusterDNSPanel,
clusterNetworkNotePanel,
clusterNetworkValidatorPanel)
}

prevPage := func(_ *gocui.Gui, _ *gocui.View) error {
closePage()
return showNetworkPage(c)
}

nextPage := func() error {
closePage()
return showHostnamePage(c)
}

setLocation := createVerticalLocator(c)

// set up the pod CIDR input panel
podCIDRInput, err := widgets.NewInput(
c.Gui,
clusterPodCIDRPanel,
clusterPodCIDRLabel,
false)
if err != nil {
return err
}
podCIDRInput.PreShow = func() error {
c.Cursor = true
podCIDRInput.Value = c.config.ClusterPodCIDR

if err := c.setContentByName(
titlePanel,
clusterNetworkTitle); err != nil {
return err
}

if err := c.setContentByName(
clusterNetworkNotePanel,
clusterNetworkNote); err != nil {
return err
}

// reset any previous error in the validator panel before
// showing the rest of the page
return c.setContentByName(clusterNetworkValidatorPanel, "")
}

// set up the service CIDR input panel
serviceCIDRInput, err := widgets.NewInput(
c.Gui,
clusterServiceCIDRPanel,
clusterServiceCIDRLabel,
false)
if err != nil {
return err
}

serviceCIDRInput.PreShow = func() error {
c.Cursor = true
serviceCIDRInput.Value = c.config.ClusterServiceCIDR
return nil
}

// set up the cluster DNS input panel
dnsInput, err := widgets.NewInput(
c.Gui,
clusterDNSPanel,
clusterDNSLabel,
false)
if err != nil {
return err
}

dnsInput.PreShow = func() error {
c.Cursor = true
dnsInput.Value = c.config.ClusterDNS
return nil
}

// define inputs validators
validateCIDR := func(cidr string) error {
cidr = strings.TrimSpace(cidr)
if cidr == "" {
return nil
}

_, err := netip.ParsePrefix(cidr)
return err
}

validateDNSIP := func(ip string) error {
ip = strings.TrimSpace(ip)
serviceCIDR, err := serviceCIDRInput.GetData()
if err != nil {
return err
}
if ip == "" && serviceCIDR == "" {
return nil
}

// the DNS IP address must be well-formed and within the
// service CIDR
ipAddr, err := netip.ParseAddr(ip)
if err != nil {
return fmt.Errorf("Invalid cluster DNS IP: %w", err)
}

svcNet, err := netip.ParsePrefix(serviceCIDR)
if err != nil {
return fmt.Errorf("To override the cluster DNS IP, the service CIDR must be valid: %w", err)
}

if !svcNet.Contains(ipAddr) {
return fmt.Errorf("Invalid cluster DNS IP: %s is not in the service CIDR %s", ip, serviceCIDR)
}

return nil
}

// define input confirm actions
podCIDRConfirm := func(_ *gocui.Gui, _ *gocui.View) error {
podCIDR, err := podCIDRInput.GetData()
if err != nil {
return err
}

if err := validateCIDR(podCIDR); err != nil {
c.setContentByName(
clusterNetworkValidatorPanel,
fmt.Sprintf("Invalid pod CIDR: %s", err))
return nil
}
c.config.ClusterPodCIDR = podCIDR

// reset any previous error in the validator panel before
// moving to the next panel
c.setContentByName(clusterNetworkValidatorPanel, "")
return showNext(c, clusterServiceCIDRPanel)
}

serviceCIDRConfirm := func(_ *gocui.Gui, _ *gocui.View) error {
serviceCIDR, err := serviceCIDRInput.GetData()
if err != nil {
return err
}

if err := validateCIDR(serviceCIDR); err != nil {
c.setContentByName(
clusterNetworkValidatorPanel,
fmt.Sprintf("Invalid service CIDR: %s", err))
return nil
}
c.config.ClusterServiceCIDR = serviceCIDR

// reset any previous error in the validator panel before
// moving to the next panel
c.setContentByName(clusterNetworkValidatorPanel, "")
return showNext(c, clusterDNSPanel)
}

dnsConfirm := func(_ *gocui.Gui, _ *gocui.View) error {
dns, err := dnsInput.GetData()
if err != nil {
return err
}
if err := validateDNSIP(dns); err != nil {
c.setContentByName(clusterNetworkValidatorPanel, err.Error())
return nil
}
c.config.ClusterDNS = dns

// reset the validator panel before moving to the next page
c.setContentByName(clusterNetworkValidatorPanel, "")
return nextPage()
}

// configure key bindings and element locations
podCIDRInput.KeyBindings = map[gocui.Key]func(*gocui.Gui, *gocui.View) error{
gocui.KeyEsc: prevPage,
gocui.KeyArrowUp: prevPage,
gocui.KeyArrowDown: podCIDRConfirm,
gocui.KeyEnter: podCIDRConfirm,
}
setLocation(podCIDRInput, 3)
c.AddElement(clusterPodCIDRPanel, podCIDRInput)

serviceCIDRInput.KeyBindings = map[gocui.Key]func(*gocui.Gui, *gocui.View) error{
gocui.KeyEsc: prevPage,
gocui.KeyArrowUp: func(_ *gocui.Gui, _ *gocui.View) error {
return showNext(c, clusterPodCIDRPanel)
},
gocui.KeyArrowDown: serviceCIDRConfirm,
gocui.KeyEnter: serviceCIDRConfirm,
}
setLocation(serviceCIDRInput, 3)
c.AddElement(clusterServiceCIDRPanel, serviceCIDRInput)

dnsInput.KeyBindings = map[gocui.Key]func(*gocui.Gui, *gocui.View) error{
gocui.KeyEsc: prevPage,
gocui.KeyArrowUp: func(_ *gocui.Gui, _ *gocui.View) error {
return showNext(c, clusterServiceCIDRPanel)
},
gocui.KeyArrowDown: dnsConfirm,
gocui.KeyEnter: dnsConfirm,
}
setLocation(dnsInput, 3)
c.AddElement(clusterDNSPanel, dnsInput)

// set up notes panels
notePanel := widgets.NewPanel(c.Gui, clusterNetworkNotePanel)
notePanel.Focus = false
notePanel.Wrap = true
setLocation(notePanel, 4)
c.AddElement(clusterNetworkNotePanel, notePanel)

// set up validator panel for warning and error messages
validatorPanel := widgets.NewPanel(c.Gui, clusterNetworkValidatorPanel)
validatorPanel.FgColor = gocui.ColorRed
validatorPanel.Focus = false
maxX, _ := c.Gui.Size()
validatorPanel.X1 = maxX / 8 * 6
setLocation(validatorPanel, 3)
c.AddElement(clusterNetworkValidatorPanel, validatorPanel)

return nil
}

func getBondModeOptions() ([]widgets.Option, error) {
return []widgets.Option{
{
Expand Down
Loading