Skip to content

20180503_talk_4

20180503_talk_4

K8s Multicluster without Fed

Overview

Multi-clusters
  • MultiCloud
  • MultiRegion
  • clusters / teams
  • clusters / ....
  • …

Hard to maintain and manage

Federation

Idea:
* Federation API Server
* Above API servers
* …

Issues ?
* Security
* Operations
- FederationApiServer is a full controlplane
- Need HA, but etcd accross wan is not recommanded

=> Complicated

Break up the pb

Expectations

App Owner POV
* CI/CD for my App
* Cluster discovery
* Hope failover between clusters
* Credentials mgmt ?

Infra Admin POV
* Connect and track clusters
* Ensure overall security
* Lock down
* Resources Mgmt

/!\ about muti-tenant clusters
-> BEWARE of resources exhaustion

Existing resources

Cluster Registry (Tectonic)
* kubectl get clusters

-> Is CRD
-> BUT Lives on 1 cluster, Need to mesh them …

AccessControl
* Users
- Apps Owners
- Enginner
- SRE
- Bots
* Stages / Environment

-> Hard with policies

Bring your own worflows

Usage expectations

Apps & Config: same concerns:
* CI / CD
* Deployment policy ?

Apps Owner:
* version
* Config & secrets

SRE
* RBACs

Tectonic Multi-clusters

Based on

ClusterRegistry
* Sync to all clusters
* USe registry as selector for polyc

Sync Policy
* Agent running on cluster
* Focus on RBAC
* CRUD namespaces
* Updated immediately across clusters

kind: ClusterPolicy
Example
Cluster registry
Regions1
---------------------------------
Prod
---------------------------------
Common Roles and Bindings

Better Security:
* No cluster need to access all clusters
* ServiceAccounts can be audited and revoked
* ServiceAccounts only ready Clusters and ClusterPolicy

More specialized software

Least complex
* Bash script
* Jenkins

More Complex
* Operatos
++ Specialized controller for an app
++ Model apps using CRDs
++ Use native k8s objetcs and concepts
– Requires k8s native app

Smarter
* Federationv2
– This is a vast problem space
++ Don’t have to use all parts
++ Polices are modeled
-> Implemented as Aggregated API
-> Secured with ServiceAccounts
-> Will be possible to write custom scheduler