2.1 Infrastructure nodes
Not exactly a day 2-operations task, but one that immensily simplifies operations. The concept of infrastructure nodes was more prominently advertised by Red Hat when OpenShift 3 was the latest and greatest. This evidently changed somehow for OpenShift 4, but with a bit of legwork we’re able to create those, too.
Infra nodes are dedicated worker nodes intended for infrastructure components such as the image registry, routers or the whole monitoring stack. Providing dedicated nodes helps ensuring that there’s always enough resources for infrastructure components.
Task 2.1.1: Create a machine set
The best way to add infra nodes to the cluster is via machine sets. Machine sets are usually only supported on clusters installed with the IPI installation method. They could be compared to deployments: If deployments are responsible for creating replica sets and ultimately pods, machine sets create machines to then add them as nodes to the cluster.
Probably the easiest way to create a machine set for infra nodes is to copy and adapt the existing worker MachineSet in the openshift-machine-api namespace.
First of all, get the yaml definition of the existing worker MachineSet resource inside the openshift-machine-api namespace.
Hints
Check the name of the existing worker MachineSet resource inside the openshift-machine-api namespace:
oc get machinesets -n openshift-machine-api
Using this information, write the existing worker MachineSet resource into a file:
oc get machinesets -n openshift-machine-api -o yaml <machineset name> > machineset_infra.yaml
Now, adapt the file so its content describes an infra MachineSet.
Warning
By far not all occurrences of worker must be changed into infra.
Refer to the documentation if you are not sure what to replace.
Hints
Edit the file using your favorite editor (still vim of course):
- Remove all the unnecessary clutter such as the
.metadata.managedFieldspart orcreationTimestampfields - Replace all occurrences of
workerwithinfraexcept for everything underproviderSpec - Ensure that
replicasis set to 3 - Add the infra role label to the
.spec.template.spec.metadata.labelsfield like so:
spec:
metadata:
creationTimestamp: null
labels:
node-role.kubernetes.io/infra: ""
providerSpec:
When you’re finished, your MachineSet resource file should look similar to this:
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
annotations:
machine.openshift.io/GPU: "0"
machine.openshift.io/memoryMb: "32768"
machine.openshift.io/vCPU: "8"
labels:
machine.openshift.io/cluster-api-cluster: user01-ops-training-75gjz
name: user01-ops-training-75gjz-infra-eu-north-1a
namespace: openshift-machine-api
spec:
replicas: 3
selector:
matchLabels:
machine.openshift.io/cluster-api-cluster: user01-ops-training-75gjz
machine.openshift.io/cluster-api-machineset: user01-ops-training-75gjz-infra-eu-north-1a
template:
metadata:
labels:
machine.openshift.io/cluster-api-cluster: user01-ops-training-75gjz
machine.openshift.io/cluster-api-machine-role: infra
machine.openshift.io/cluster-api-machine-type: infra
machine.openshift.io/cluster-api-machineset: user01-ops-training-75gjz-infra-eu-north-1a
spec:
metadata:
creationTimestamp: null
labels:
node-role.kubernetes.io/infra: ""
providerSpec:
value:
ami:
id: ami-0080eb90a48d9655e
apiVersion: awsproviderconfig.openshift.io/v1beta1
blockDevices:
- ebs:
encrypted: true
iops: 0
kmsKey:
arn: ""
volumeSize: 120
volumeType: gp2
credentialsSecret:
name: aws-cloud-credentials
deviceIndex: 0
iamInstanceProfile:
id: user01-ops-training-75gjz-worker-profile
instanceType: m5.2xlarge
kind: AWSMachineProviderConfig
metadata:
creationTimestamp: null
placement:
availabilityZone: eu-north-1a
region: eu-north-1
securityGroups:
- filters:
- name: tag:Name
values:
- user01-ops-training-75gjz-worker-sg
subnet:
filters:
- name: tag:Name
values:
- user01-ops-training-75gjz-private-eu-north-1a
tags:
- name: kubernetes.io/cluster/user01-ops-training-75gjz
value: owned
- name: acend-training
value: ocp4-ops
- name: customer
value: acend
- name: username
value: user01
userDataSecret:
name: worker-user-data
Warning
You cannot use the sample MachineSet resource definition above without further modification. It’s recommended you use your own worker MachineSet definition and make the described changes.Now, create the MachineSet.
Hints
oc create -f machineset_infra.yaml
Check if the machines were created and that they’re in the Provisioning state:
Hints
oc get machines -n openshift-machine-api
It shouldn’t take too long and you will see new infra nodes popping up.