Update README.md
This commit is contained in:
219
README.md
219
README.md
@@ -1,84 +1,87 @@
|
|||||||
# monok8s
|
# monok8s
|
||||||
An Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit
|
|
||||||
https://docs.mono.si/gateway-development-kit/getting-started
|
|
||||||
|
|
||||||
## Features
|
This is an Alpine-based Kubernetes image for Mono's Gateway Development Kit.
|
||||||
* A/B deployment
|
|
||||||
* Read-only RootFS
|
|
||||||
* k8s style OS upgrade (see Upgrading)
|
|
||||||
|
|
||||||
## DISCLAIMER
|
It gives you a ready-to-boot Kubernetes control-plane image so you can get your device running first, then learn and customize from there.
|
||||||
|
|
||||||
* This is not your everyday linux image! It is best suited for users that is already familiar
|
Project/device docs: <https://docs.mono.si/gateway-development-kit/getting-started>
|
||||||
with k8s. For first-timers, you may want to try the default config that gives you a ready-to-use
|
|
||||||
cluster and get yourself started from there
|
|
||||||
|
|
||||||
* USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this.
|
---
|
||||||
|
|
||||||
## Current Status
|
## What you get
|
||||||
|
|
||||||
### Boostrapping
|
The default image boots into a small Kubernetes control-plane environment with:
|
||||||
* [x] initramfs
|
|
||||||
* [x] booting into alpine
|
|
||||||
* [x] k8s control-plane
|
|
||||||
* [ ] k8s worker node
|
|
||||||
|
|
||||||
### Kubernetes
|
- Alpine Linux
|
||||||
* OSUpgrade
|
- Kubernetes initialized through `kubeadm`
|
||||||
* [x] Control Plane - kubeadm upgrade apply
|
- read-only root filesystem layout
|
||||||
* Upgrade chain
|
- A/B rootfs layout for safer OS upgrades
|
||||||
* [x] 1.33.3 -> 1.33.10
|
- a Kubernetes-style OS upgrade path through `OSUpgrade`
|
||||||
* [x] 1.33.10 -> 1.34.6
|
|
||||||
* [x] 1.34.6 -> 1.35.3
|
|
||||||
* [ ] Worker node - kubeadm upgrade node
|
|
||||||
* CNI
|
|
||||||
* [x] default bridge-cni
|
|
||||||
* [ ] Cilium
|
|
||||||
|
|
||||||
### Network Traffics
|
You do **not** need to know Go or understand the internal build system to try the image.
|
||||||
* VPP Pod
|
|
||||||
* [x] fmc - works? But no way to test it yet
|
|
||||||
* [ ] vpp - does not work
|
|
||||||
|
|
||||||
## Table of Contents
|
---
|
||||||
1. Flashing
|
|
||||||
- [USB](docs/flashing-usb.md)
|
|
||||||
- [Over network (tftp)](docs/flashing-network.md)
|
|
||||||
2. [Upgrading](docs/ota.md)
|
|
||||||
3. Getting shell access to the host
|
|
||||||
- UART. The thing you did in first time flashing.
|
|
||||||
- [Install an ssh pod](docs/installing-ssh-pod.md) (Recommended)
|
|
||||||
|
|
||||||
## Build
|
## Before you start
|
||||||
|
|
||||||
Prerequisites
|
You need:
|
||||||
* make
|
|
||||||
* Docker
|
|
||||||
|
|
||||||
|
- a Linux build machine or VM
|
||||||
|
- Docker
|
||||||
|
- `make`
|
||||||
|
- basic command-line comfort
|
||||||
|
|
||||||
|
If you are building on a fresh Debian machine, you can install the usual build dependencies with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
devtools/setup-build-host.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Or install the minimum packages yourself:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y docker.io make qemu-user-static binfmt-support
|
||||||
|
```
|
||||||
|
|
||||||
|
Make sure your user can run Docker, or use `sudo` where needed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Fast path: build an image
|
||||||
|
|
||||||
|
Download the project tarball, extract it, then run:
|
||||||
|
|
||||||
|
```bash
|
||||||
make release
|
make release
|
||||||
```
|
```
|
||||||
|
|
||||||
(You can run `devtools/setup-bulid-host.sh` if you have spin up a new VM and want to install build dependency automatically.)
|
When the build finishes, check the `out/` directory for the generated image artifacts.
|
||||||
|
|
||||||
### Common issues
|
That is the main path most users should try first.
|
||||||
If you have encounter this error during build
|
|
||||||
```
|
|
||||||
chroot: failed to run command '/bin/sh': Exec format error
|
|
||||||
```
|
|
||||||
|
|
||||||
You need to install support for arm64 emulation
|
---
|
||||||
|
|
||||||
|
## Flash the image
|
||||||
|
|
||||||
|
After building, flash the generated image to your device.
|
||||||
|
|
||||||
|
Start with one of these guides:
|
||||||
|
|
||||||
|
- [Flash over USB](docs/flashing-usb.md)
|
||||||
|
- [Flash over network / TFTP](docs/flashing-network.md)
|
||||||
|
|
||||||
|
USB flashing is usually the easiest path when you are setting up the device for the first time.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## First boot
|
||||||
|
|
||||||
|
The default configuration is intended to boot as a first-time Kubernetes control-plane node.
|
||||||
|
|
||||||
|
Default-style control-plane configuration looks like this:
|
||||||
|
|
||||||
For Debian
|
|
||||||
```bash
|
```bash
|
||||||
apt-get install -y qemu-user-static binfmt-support
|
|
||||||
```
|
|
||||||
|
|
||||||
### cluster-config
|
|
||||||
The default configuration will boot as a first time control-plane.
|
|
||||||
|
|
||||||
For control-plane
|
|
||||||
```
|
|
||||||
make cluster-config \
|
make cluster-config \
|
||||||
MKS_HOSTNAME=monok8s-master \
|
MKS_HOSTNAME=monok8s-master \
|
||||||
MKS_CLUSTER_ROLE=control-plane \
|
MKS_CLUSTER_ROLE=control-plane \
|
||||||
@@ -87,18 +90,88 @@ make cluster-config \
|
|||||||
MKS_APISERVER_ADVERTISE_ADDRESS=10.0.0.10
|
MKS_APISERVER_ADVERTISE_ADDRESS=10.0.0.10
|
||||||
```
|
```
|
||||||
|
|
||||||
For worker
|
If you are just trying the image for the first time, start with the default control-plane setup. Worker-node setup is still incomplete.
|
||||||
```
|
|
||||||
make cluster-config \
|
For all available configuration values, see:
|
||||||
MKS_HOSTNAME=monok8s-worker \
|
|
||||||
MKS_CLUSTER_ROLE=worker \
|
- [configs/cluster.env.default](configs/cluster.env.default)
|
||||||
MKS_INIT_CONTROL_PLANE=no \
|
|
||||||
MKS_MGMT_ADDRESS=10.0.0.10/24 \
|
---
|
||||||
MKS_APISERVER_ADVERTISE_ADDRESS=10.0.0.10 \
|
|
||||||
MKS_API_SERVER_ENDPOINT=10.0.0.1:6443 \
|
## Getting shell access
|
||||||
MKS_CNI_PLUGIN=none \
|
|
||||||
MKS_BOOTSTRAP_TOKEN=abcd12.ef3456789abcdef0 \
|
For first-time setup, UART is the most direct option because it is already part of the flashing process.
|
||||||
MKS_DISCOVERY_TOKEN_CA_CERT_HASH=sha256:9f1c2b3a4d5e6f7890abc1234567890abcdef1234567890abcdef1234567890ab
|
|
||||||
|
After the device is running, the recommended path is:
|
||||||
|
|
||||||
|
- [Install an SSH pod](docs/installing-ssh-pod.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Upgrading
|
||||||
|
|
||||||
|
monok8s includes a Kubernetes-style OS upgrade flow using the `OSUpgrade` custom resource.
|
||||||
|
|
||||||
|
See:
|
||||||
|
|
||||||
|
- [OTA upgrade guide](docs/ota.md)
|
||||||
|
|
||||||
|
The currently tested upgrade chain is:
|
||||||
|
|
||||||
|
- `1.33.3 -> 1.33.10`
|
||||||
|
- `1.33.10 -> 1.34.6`
|
||||||
|
- `1.34.6 -> 1.35.3`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current status
|
||||||
|
|
||||||
|
This project is usable for experimenting with a single control-plane device image, but it is still a development project.
|
||||||
|
|
||||||
|
Working today:
|
||||||
|
|
||||||
|
- initramfs boot flow
|
||||||
|
- Alpine boot
|
||||||
|
- Kubernetes control-plane bootstrap
|
||||||
|
- default bridge CNI
|
||||||
|
- control-plane OS upgrade path
|
||||||
|
|
||||||
|
Still in progress:
|
||||||
|
|
||||||
|
- Kubernetes worker-node support
|
||||||
|
- Cilium support
|
||||||
|
- VPP/DPAA networking experiments
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common build issue
|
||||||
|
|
||||||
|
### `chroot: failed to run command '/bin/sh': Exec format error`
|
||||||
|
|
||||||
|
This usually means the build host cannot run ARM64 binaries.
|
||||||
|
|
||||||
|
On Debian, install ARM64 emulation support:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt-get install -y qemu-user-static binfmt-support
|
||||||
```
|
```
|
||||||
|
|
||||||
Check inside [configs/cluster.env.default](configs/cluster.env.default) for configuration details
|
Then run the build again:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make release
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
This is not a general-purpose Linux distribution. It is a device image for experimenting with Kubernetes on Mono's Gateway Development Kit.
|
||||||
|
|
||||||
|
The safest path is:
|
||||||
|
|
||||||
|
1. build the default image,
|
||||||
|
2. flash it,
|
||||||
|
3. boot the control-plane,
|
||||||
|
4. confirm Kubernetes is running,
|
||||||
|
5. customize only after the base image works.
|
||||||
|
|||||||
Reference in New Issue
Block a user