# monok8s

Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit

https://docs.mono.si/gateway-development-kit/getting-started

## DISCLAIMER
USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this. I am testing them all by myself right now.

## IMPORTANT NOTES
 * The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
   * So `ip addr eth0` is your right most port
 * If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.

## Build

Find the latest package versions and update build.env
 * [kernel](https://github.com/nxp-qoriq/linux/archive/refs/tags/)
 * [busybox](https://github.com/mirror/busybox/archive/refs/tags/)
 * [CRI-O](https://github.com/cri-o/cri-o/releases)
 * [Kubelet](https://kubernetes.io/releases/download/)

Then run
```
make release
```

Then run, follow the instructions
```
make cluster-config
```

## Flashing

#### USB Drive
1. `make release`
2. Format the USB in vfat
3. Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
4. Run
```
usb start
usb tree
fatls usb 0:1   # For fat
ext4ls usb 0:1  # For ext4 if you insist
fatload usb 0 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000
```

5. Inside initramfs, run
```
flash-emmc.sh
```

4. If it boots, create the A/B deployment scheme
 - (WORK IN PROGRESS)

#### tftp (network is required)
1. Put out/[RELEASE].img.gz and out/board.itb into your ftp server
```
setenv ipaddr 10.0.0.153
setenv serverip 10.0.0.129
tftp 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000
```
2. Inside initramfs, run
```
flash-emmc.sh
```


### Making sub stages
```bash
make build-base # The image the builds kernel and everything
make kernel     # Builds our kernel from NXP
make initramfs
make itb        # Builds out/board.itb (contains the kernel and the initramfs)
```

## Architecture
 - A/B deployment (So we can just do kubectl apply upgrade.yaml with version jumps. Agent will walk through it automatically)
 - Read-only OS

## Upgrade process

We use a CRD with an agent to handle this. Our versions follows upstream's.

kubectl apply -f upgrade.yaml
```yaml
apiVersion: k8s.mono.si/v1alpha1
kind: OSUpgrade
metadata:
  name: "my-ugrade"
spec:
  version: "v1.35.1" # Or just "latest"
  imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
  checksum: "sha256:..."
```

kubectl get osugrades
```
NAME           TARGET     STATUS      AGE
my-upgrade-1   latest     pending     1m
my-upgrade-2   v1.35.3    accepted    1m # latest gets realized into a version number
their-upgrade  v1.33.2    succeeded   1m
```

kubectl get osupgradeprogress
```
NODE        SOURCE         STATUS
my-node     my-upgrade-2   active
their-node  my-upgrade-2   completed
```

## NOTES
### The device's dts files are located at here
https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale
* We need both `mono-gateway-dk-sdk.dts` and `mono-gateway-dk.dts` since the sdk one includes the non-sdk one.
* The actual dts being used is the `mono-gateway-dk-sdk.dts`