1. Setup¶

The VIM server should be able to reach the nodes management interface to be able to perform the introspection and the deployment.

Tip

If you do not have direct access to the git from the lab system you can use sshfs (https://github.com/libfuse/sshfs/releases):

mkdir -p /mnt/ocp
sudo sshfs -o allow_other,default_permissions,IdentityFile=/home/ubuntu/.ssh/id_rsa admin@10.127.64.10:/home/share /mnt/ocp

1.1. Getting the Code¶

The main repository for cluster is in the Stroila Gitlab repository:

https://gitlab.stroila.ca/slg/ocp/cluster

You can get a read-only checkout from there:

$ git clone --recurse-submodules https://gitlab.stroila.ca/slg/ocp/cluster.git

After cloning update the submodules

$ git submodule update --init --force --remote

Edit the env file to fit your setup and generate the ignition files

podman run -i --rm quay.io/coreos/fcct:latest --pretty --strict < ./docs/source/config/automated_install.yaml > automated_install.ign

Make sure you have the meadow and dnsmasq services configured up and running

1.2. ipv6 over ipv4 unicast VXLAN¶

Warning

This section it is an example

In the scenario where multicast is not supported, you need to configure the vxlan layer using unicast addresses. This is done easily using iproute2.

Note

The overlay ipv6 addresses are constructed based on the management interface MAC address and fabric id (ex. for fabricId=666 ipv6 prefix should be fd02:9a01::). The controllers must be connected to the Leaf in order to switch over from the temporary VXLAN network to FCN0 network after OCP and basic fabric deployment.

VXLAN Mesh Topology (SDC - Star of David Configuration)

With the preceding layout, we need the OCP infrastructure nodes to be able to communicate with each other. We cannot use L3 routes because the provider might not route anything that’s not on the svc0 network, so we need to set up our own L2 network layer over which we can establish our L3 routes. For this we’ll use a Virtual Extensible LAN ( VXLAN ).

Add the temporary ipv4 network on the OCP nodes

ifconfig eno1 10.127.64.81/26
route add default gw 10.127.64.65

ifconfig eno1 10.127.64.82/26
route add default gw 10.127.64.65

ifconfig eno1 10.127.64.83/26
route add default gw 10.127.64.65

ifconfig eno1 10.127.64.71/26
route add default gw 10.127.64.65

Create a vxlan interface on the bastion and bootstrap nodes

ip link add vxlan0 type vxlan id 53 dev eth0 dstport 0

This creates the vxlan0 device, attaches it to eth0 listening on the iana default port. This does not assign any endpoints, so we’ll create connections to OCP nodes. Replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs

192.168.200.9

bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.8 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.81 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.82 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.83 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.71 dev vxlan0

Assign an address and bring up the interface

ip -6 addr add fd02:9a01::fa16:3e9d:beb5/64 dev vxlan0
ip link set up dev vxlan0

192.168.200.8

bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.9 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.81 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.82 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.83 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.71 dev vxlan0

Assign an address and bring up the interface

ip -6 addr add fd02:9a01::fa16:3e9d:beb4/64 dev vxlan0
ip link set up dev vxlan0

On the OCP nodes do the same:

10.127.64.83

ip link add vxlan0 type vxlan id 53 dev eno1 dstport 0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.8 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.9 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.71 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.81 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.82 dev vxlan0
ip -6 addr add fd02:9a01::90b1:1c26:6aa2/64 dev vxlan0
ip link set up dev vxlan0

10.127.64.82

ip link add vxlan0 type vxlan id 53 dev eno1 dstport 0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.8 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.9 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.71 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.81 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.83 dev vxlan0
ip -6 addr add fd02:9a01::549f:3509:8eac/64 dev vxlan0
ip link set up dev vxlan0

10.127.64.81

ip link add vxlan0 type vxlan id 53 dev eno1 dstport 0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.8 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.9 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.71 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.82 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.83 dev vxlan0
ip -6 addr add fd02:9a01::1866:dae6:4674/64 dev vxlan0
ip link set up dev vxlan0

10.127.64.71

ip link add vxlan0 type vxlan id 53 dev eno1 dstport 0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.8 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 192.168.200.9 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.81 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.82 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.127.64.83 dev vxlan0
ip -6 addr add fd02:9a01::0090:fb6e:bb02/64 dev vxlan0
ip link set up dev vxlan0