123 lines
3.2 KiB
Markdown
123 lines
3.2 KiB
Markdown
# monok8s
|
|
|
|
Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit
|
|
|
|
https://docs.mono.si/gateway-development-kit/getting-started
|
|
|
|
## DISCLAIMER
|
|
USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this. I am testing them all by myself right now.
|
|
|
|
## IMPORTANT NOTES
|
|
* The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
|
|
* So `ip addr eth0` is your right most port
|
|
* If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.
|
|
|
|
## Build
|
|
|
|
Find the latest package versions and update build.env
|
|
* [kernel](https://github.com/nxp-qoriq/linux/archive/refs/tags/)
|
|
* [busybox](https://github.com/mirror/busybox/archive/refs/tags/)
|
|
* [CRI-O](https://github.com/cri-o/cri-o/releases)
|
|
* [Kubelet](https://kubernetes.io/releases/download/)
|
|
|
|
Then run
|
|
```
|
|
make release
|
|
```
|
|
|
|
Then run, follow the instructions
|
|
```
|
|
make cluster-config
|
|
```
|
|
|
|
## Flashing
|
|
|
|
#### USB Drive
|
|
1. `make release`
|
|
2. Format the USB in vfat
|
|
3. Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
|
|
4. Run
|
|
```
|
|
usb start
|
|
usb tree
|
|
fatls usb 0:1 # For fat
|
|
ext4ls usb 0:1 # For ext4 if you insist
|
|
fatload usb 0 0x80000000 board.itb
|
|
|
|
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
|
bootm 0x80000000
|
|
```
|
|
|
|
5. Inside initramfs, run
|
|
```
|
|
flash-emmc.sh
|
|
```
|
|
|
|
4. If it boots, create the A/B deployment scheme
|
|
- (WORK IN PROGRESS)
|
|
|
|
#### tftp (network is required)
|
|
1. Put out/[RELEASE].img.gz and out/board.itb into your ftp server
|
|
```
|
|
setenv ipaddr 10.0.0.153
|
|
setenv serverip 10.0.0.129
|
|
tftp 0x80000000 board.itb
|
|
|
|
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
|
bootm 0x80000000
|
|
```
|
|
2. Inside initramfs, run
|
|
```
|
|
flash-emmc.sh
|
|
```
|
|
|
|
|
|
### Making sub stages
|
|
```bash
|
|
make build-base # The image the builds kernel and everything
|
|
make kernel # Builds our kernel from NXP
|
|
make initramfs
|
|
make itb # Builds out/board.itb (contains the kernel and the initramfs)
|
|
```
|
|
|
|
## Architecture
|
|
- A/B deployment (So we can just do kubectl apply upgrade.yaml with version jumps. Agent will walk through it automatically)
|
|
- Read-only OS
|
|
|
|
## Upgrade process
|
|
|
|
We use a CRD with an agent to handle this. Our versions follows upstream's.
|
|
|
|
kubectl apply -f upgrade.yaml
|
|
```yaml
|
|
apiVersion: k8s.mono.si/v1alpha1
|
|
kind: OSUpgrade
|
|
metadata:
|
|
name: "my-ugrade"
|
|
spec:
|
|
version: "v1.35.1" # Or just "latest"
|
|
imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
|
|
checksum: "sha256:..."
|
|
```
|
|
|
|
kubectl get osugrades
|
|
```
|
|
NAME TARGET STATUS AGE
|
|
my-upgrade-1 latest pending 1m
|
|
my-upgrade-2 v1.35.3 accepted 1m # latest gets realized into a version number
|
|
their-upgrade v1.33.2 succeeded 1m
|
|
```
|
|
|
|
kubectl get osupgradeprogress
|
|
```
|
|
NODE SOURCE STATUS
|
|
my-node my-upgrade-2 active
|
|
their-node my-upgrade-2 completed
|
|
```
|
|
|
|
## NOTES
|
|
### The device's dts files are located at here
|
|
https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale
|
|
* We need both `mono-gateway-dk-sdk.dts` and `mono-gateway-dk.dts` since the sdk one includes the non-sdk one.
|
|
* The actual dts being used is the `mono-gateway-dk-sdk.dts`
|