Moving Kops Aws Kubernetes API from Public to Internal ELB

Published 05-14-2018 00:00:00

Kops made very easy this change, first you need to edit your cluster:

# kops edit cluster --name CLUSTER_NAME

Change the type from Public in to private

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: kops/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: 2018-04-25T04:23:38Z
  name: staging.MY-DOMAIN.ltd
spec:
  api:
    loadBalancer:
      type: Internal      <------- Change from Public to Internal
  authorization:
    alwaysAllow: {}
  channel: stable
  cloudProvider: aws
  configBase: s3://MY-BUCKET-NAME/staging.MY-DOMAIN.ltd
  dnsZone: MY-DOMAIN.ltd
  etcdClusters:
  - etcdMembers:
    - instanceGroup: master-us-west-2a
      name: a
    - instanceGroup: master-us-west-2b
      name: b
    - instanceGroup: master-us-west-2c
      name: c
    name: main
  - etcdMembers:
    - instanceGroup: master-us-west-2a
      name: a
    - instanceGroup: master-us-west-2b
      name: b
    - instanceGroup: master-us-west-2c
      name: c
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
  kubernetesApiAccess:
  - 0.0.0.0/0
  kubernetesVersion: 1.9.3
  masterInternalName: api.internal.staging.MY-DOMAIN.ltd
  masterPublicName: api.staging.MY-DOMAIN.ltd
  networkCIDR: 172.50.0.0/16
  networking:
    weave:
      mtu: 8912
  nonMasqueradeCIDR: 100.64.0.0/10
  sshAccess:
  - 0.0.0.0/0
  subnets:
  - cidr: 172.50.32.0/19
    name: us-west-2a
    type: Private
    zone: us-west-2a
  - cidr: 172.50.64.0/19
    name: us-west-2b
    type: Private
    zone: us-west-2b
  - cidr: 172.50.96.0/19
    name: us-west-2c
    type: Private
    zone: us-west-2c
  - cidr: 172.50.0.0/22
    name: utility-us-west-2a
    type: Utility
    zone: us-west-2a
  - cidr: 172.50.4.0/22
    name: utility-us-west-2b
    type: Utility
    zone: us-west-2b
  - cidr: 172.50.8.0/22
    name: utility-us-west-2c
    type: Utility
    zone: us-west-2c
  topology:
    bastion:
      bastionPublicName: bastion.staging.MY-DOMAIN.ltd
    dns:
      type: Public
    masters: private
    nodes: private

Then the modification will fail beacause you can’t have an ELB without subnets

W0504 17:01:44.233037     889 executor.go:118] error running task "LoadBalancer/api.staging.MY-DOMAIN.ltd" (4m29s remaining to succeed): Error detaching Load Balancer from old subnets: InvalidConfigurationRequest: Requested configuration change for LoadBalancer "api-staging-kb-snpd-io-as0gcq" is invalid because you attempted to detach all the subnets for this LoadBalancer and a LoadBalancer cannot be attached to zero subnets in VPC.

To solve that, remove manually the Public ELB on AWS and the Kops process will continue.