OpenShift Disconnected Installs – Part 3

Now that we have all the container content we need mirrored, we need to tackle mirroring the ova. If you are using SonaType Nexus (like I am) you can work with your admin and they can have it host the OVA for you. For others, I would recommend spinning up a small webserver and just hosting it behind apache or nginx (your choice).

With that out of the way, let’s actually install a disconnected OpenShift cluster. To start with, you will need the openshift-install command line for the version of OpenShift you are wanting to install. You can download the latest openshift-install for Linux from here. Once you have the openshift-install command on the Linux machine you are using to do your work it is time to create the install-config.yaml file. If you have never created an install-config.yaml file, I highly recommend you go through the process several times to get familiar with it. There is a wizard available when you run openshift-install create install-config --dir <installation_directory> that will give you a basic install-config.yaml file. A word of warning about this file: it will contain credential information for talking to your infrastructure provider, and it will be consumed by the install process. Be sure to keep a backup copy to use for installing multiple clusters, and do not store it in source control due to the credentials required in the file (if there is a way around this, I do not know it). Below is an example install-config.yaml with offline pieces included (I even include operators from OperatorHub even though I don’t really need it, it doesn’t cause any harm as the install process does not use them).

apiVersion: v1
metadata:
  name: demo
baseDomain: dev.company.com
proxy:
  httpProxy: http://proxy.company.com:8080
  httpsProxy: http://proxy.company.com:8080
  noProxy: git.company.com,registry.company.com,.company.com,10.0.0.0/8,172.17.0.0/16,172.18.0.0/16,127.0.0.0/8,192.168.0.0/16
controlPlane:
  name: master
  architecture: amd64
  hyperthreading: Enabled
  replicas: 3
  platform:
    vsphere:
      cpus: 4
      coresPerSocket: 2
      memoryMB: 16384
      osDisk:
        diskSizeGB: 120
compute:
  - name: worker
    hyperthreading: Enabled
    architecture: amd64
    replicas: 3
    platform:
      vsphere:
        cpus: 4
        coresPerSocket: 2
        memoryMB: 16384
        osDisk:
          diskSizeGB: 120
networking:
  networkType: OVNKubernetes
  clusterNetwork:
    - cidr: 10.128.0.0/14
      hostPrefix: 23
  machineNetwork:
    - cidr: 10.0.0.0/16
  serviceNetwork:
    - 172.30.0.0/16
platform:
  vsphere:
    vCenter: vcenter.company.com
    username: server_account@company.com
    password: password
    datacenter: datacenter
    defaultDatastore: datastore
    cluster: cluster
    apiVIP: 1.2.3.4
    ingressVIP: 4.5.6.7
    network: 'kubernetes'
    diskType: thin
    folder: /folder/vm/okd/dev
    clusterOSImage: https://nexus.company.com/repository/openshift-v4-raw-proxy/x86_64/dependencies/rhcos/4.14/latest/rhcos-4.14.15-x86_64-vmware.x86_64.ova
pullSecret: '{"auths":{"registry.company.com:443":{"auth":"base64encodedusernameandpassword"}}}'
sshKey: ssh-ed25519 ssh-public-key user@company.com
additionalTrustBundle: |-
  -----BEGIN CERTIFICATE-----
  MIIEPDCCA...
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  MIIEPDCCA...
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  MIIDKDCCA...
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  MIIE6jCCA...
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  MIIHwjCCB...
  -----END CERTIFICATE-----
imageContentSources:
  - mirrors:
      - registry.company.com:443/docker-hosted/redhat-cop
    source: quay.io/redhat-cop
  - mirrors:
      - registry.company.com:443/docker-hosted/rh-sso-7
    source: registry.redhat.io/rh-sso-7
  - mirrors:
      - registry.company.com:443/docker-hosted/ubi8
    source: registry.redhat.io/ubi8
  - mirrors:
      - registry.company.com:443/docker-hosted/netapp
    source: docker.io/netapp/trident
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift-logging
    source: registry.redhat.io/openshift-logging
  - mirrors:
      - registry.company.com:443/docker-hosted/ansible-automation-platform-24
    source: registry.redhat.io/ansible-automation-platform-24
  - mirrors:
      - registry.company.com:443/docker-hosted/community-operator-pipeline-prod
    source: quay.io/community-operator-pipeline-prod
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift-gitops-1
    source: registry.redhat.io/openshift-gitops-1
  - mirrors:
      - registry.company.com:443/docker-hosted/advanced-cluster-security
    source: registry.redhat.io/advanced-cluster-security
  - mirrors:
      - registry.company.com:443/docker-hosted/rhel8
    source: registry.redhat.io/rhel8
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift4
    source: registry.redhat.io/openshift4
  - mirrors:
      - registry.company.com:443/docker-hosted/ansible-automation-platform
    source: registry.redhat.io/ansible-automation-platform
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift-community-operators
    source: quay.io/openshift-community-operators
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift/release
    source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
  - mirrors:
      - registry.company.com:443/docker-hosted/openshift/release-images
    source: quay.io/openshift-release-dev/ocp-release

You will of course have to generate your own install-config.yaml specific to your environment, but this at least gives you an example of one that works. One thing to keep in mind is the format of install-config.yaml can change between OpenShift versions, so if you have an install config suddenly stop working, check the documentation first.

Now that you have your install-config.yaml you are ready to create your cluster. Issue the command ./openshift-install create cluster --dir /path/to/folder/with/install-config --log-level=debug

Once you run this command, you are at the point where you can take a break while openshift-install does its work. I personally always run openshift-install at level-level debug just in case something goes wrong in the process.

With this step completed, it is time to close out Part 3. In Part 4 (which may very well be the final part) we will cover configuring OpenShift to use your mirror’d operator repository you previously setup.