Skip to content

OPNET-787: Add optional top-of-rack BGP speaker for the baremetal network#1929

Open
mkowalski wants to merge 1 commit into
openshift-metal3:masterfrom
mkowalski:bgp-tor-speaker
Open

OPNET-787: Add optional top-of-rack BGP speaker for the baremetal network#1929
mkowalski wants to merge 1 commit into
openshift-metal3:masterfrom
mkowalski:bgp-tor-speaker

Conversation

@mkowalski

Copy link
Copy Markdown
Member

What

ENABLE_BGP_TOR=true deploys an FRR container on the hypervisor host network that acts as a top-of-rack BGP speaker for the baremetal (ostestbm) network:

  • accepts dynamic BGP sessions from cluster nodes anywhere on the external subnet (bgp listen range, so DHCP node addresses need no per-peer config)
  • installs routes learned from the cluster into the host kernel via zebra, so prefixes advertised by the cluster (e.g. VIPs) are reachable from the hypervisor over the BGP paths
  • no bgp ebgp-requires-policy (RFC 8212 relaxation), required for route exchange with FRR ≥ 8 defaults

Deployment happens at the end of 02_configure_host.sh (the network exists by then and the speaker survives ocp_cleanup, matching its ToR role); teardown is wired into host_cleanup.sh. Follows the metallb/ add-on layout. ASNs and the FRR image are configurable, defaults documented in config_example.sh:

export ENABLE_BGP_TOR=true
#export BGP_TOR_ASN=64513         # speaker ASN
#export BGP_CLUSTER_ASN=64512     # expected cluster ASN
#export BGP_TOR_IMAGE=quay.io/frrouting/frr:9.1.0

Listen ranges and address-family activation are derived from EXTERNAL_SUBNET_V4/V6 (dual-stack aware; families without a subnet are omitted).

Why

Primary consumer is BGP-based VIP management (openshift/enhancements#1982, OPNET-595): the cluster advertises API/Ingress VIPs via BGP and needs an external peer playing the ToR role. Also generally useful for any BGP-on-baremetal testing (e.g. MetalLB BGP mode without the e2e harness, OVN-K route advertisements).

Validation

  • Live-tested against a running BGP-VIP cluster: 3 control-plane FRR sessions re-establish within ~1 min of (re)deploying the speaker; advertised VIP /32s are ECMP-installed on the host; full cleanup→configure cycle verified (container, $WORKING_DIR/bgp-tor, firewall port).
  • bash -n + shellcheck clean on the new scripts.
  • Rendered v4-only frr.conf is byte-identical to the config used throughout the enhancement-1982 demo runs.

@openshift-ci
openshift-ci Bot requested review from elfosardo and zaneb July 9, 2026 14:04
@mkowalski mkowalski changed the title OPNET-595: Add optional top-of-rack BGP speaker for the baremetal network OPNET-787: Add optional top-of-rack BGP speaker for the baremetal network Jul 9, 2026

@cybertron cybertron left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/retest-required

Comment thread host_cleanup.sh
sudo rm -rf /etc/exports.d/dev-scripts.exports /opt/dev-scripts/nfsshare

# Remove the optional top-of-rack BGP speaker
bgp/cleanup_bgp_tor.sh || true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This script isn't set -e so the || true isn't needed.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2026
@cybertron

Copy link
Copy Markdown
Contributor

/retest-required

This job has been pretty flaky lately, judging by the job history.

ENABLE_BGP_TOR deploys an FRR container on the host network that accepts
dynamic BGP sessions from cluster nodes on the external subnet and
installs learned routes on the hypervisor. Primary consumer is BGP-based
VIP management (openshift/enhancements#1982): the cluster advertises API
and Ingress VIPs to this speaker and the host reaches them over the BGP
paths. ASNs and the FRR image are configurable; teardown is wired into
host_cleanup.
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 22, 2026
@mkowalski

Copy link
Copy Markdown
Member Author

@cybertron the PR was rebased on top of master to resolve conflicts with the recently merged LLDP ToR change (no other changes introduced). Could you please re-lgtm?

/approve

This message was generated using AI. Please verify before acting on it.

@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mkowalski

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 22, 2026
@cybertron

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 22, 2026
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown

@mkowalski: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-agent-compact-ipv4-iso-no-registry 67d82ad link false /test e2e-agent-compact-ipv4-iso-no-registry
ci/prow/e2e-metal-ipi-ovn-ipv4-bond 67d82ad link false /test e2e-metal-ipi-ovn-ipv4-bond
ci/prow/e2e-metal-ipi-serial-ovn-ipv6 67d82ad link false /test e2e-metal-ipi-serial-ovn-ipv6
ci/prow/e2e-metal-ipi-ovn-ipv4 67d82ad link unknown /test e2e-metal-ipi-ovn-ipv4
ci/prow/e2e-metal-ipi-ovn-ipv6 67d82ad link unknown /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mkowalski

Copy link
Copy Markdown
Member Author

/retest-required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants