Architecture

general

  • skip the virtual machine layer to avoid noisy neighbor problems
  • get bare-metal performance and reduce costs
  • rely on k8s for application availability
  • a machine belongs to a single cluster
  • each cluster gets its own dedicated firewall machine
  • machine networks do not overlap

networking

  • use a CLOS topology for scaleable, resilient switch cabling
  • deliver switch components continuously to avoid manual administration
  • use open switching hardware to run network control logic
  • follow a layer-3-only approach to avoid big broadcast domains and gain higher isolation (VRF, EVPN)
  • everything, even machines, are BGP speakers to get highly available IPs (Routing-to-the-Host, eBGP unnumbered, ECMP, Anycast)
  • firewalls use the same hardware as regular machines

metal-api

  • manages bare metal objects: machines, machine sizes, partitions, firewalls, switches, networks, IPs, OS images
  • a.k.a. machines can be created and destroyed with this API
  • CLI Client: metal-stack/metalctl
  • Golang Client: https://github.com/metal-stack/metal-go

metal-core

  • runs on every leaf switch and registers the switch at the metal-api
  • sets boot order of machines
  • serves mini OS image location to pixiecore requests
  • configures the switches based on the configuration in the metal-api
  • monitors liveliness of machines

metal-hammer

  • detects and registers machines at the metal-api
  • erases, wipes and partitions disks
  • "hammers" the requested OS onto disk and starts it

metal-images

  • builds OS images based on Dockerfiles

metal-networker

  • configures network interfaces, BGP and nftables for machines and firewalls

machine-controller

  • implements a machine-controller-manager for Gardener
  • manages bare metal machines based on k8s CRDs

metal-ccm

  • implements a cloud controller manager for k8s
  • contains a controller for k8s nodes and services
  • communicates with the metal-api for ip / loadbalancer acquisition

gardener integration

  • extensions needed for the gardener project
  • configures the set of OS images useable