Split readme into small files
This commit is contained in:
131
README.md
131
README.md
@@ -1,19 +1,25 @@
|
|||||||
# monok8s
|
# monok8s
|
||||||
|
|
||||||
Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit
|
Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit
|
||||||
|
|
||||||
https://docs.mono.si/gateway-development-kit/getting-started
|
https://docs.mono.si/gateway-development-kit/getting-started
|
||||||
|
|
||||||
## DISCLAIMER
|
## Features
|
||||||
USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this. I am testing them all by myself right now.
|
* A/B deployment
|
||||||
|
* Read-only OS
|
||||||
|
* k8s style OS upgrade (see Upgrading)
|
||||||
|
|
||||||
## IMPORTANT NOTES
|
## IMPORTANT NOTES
|
||||||
* The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
|
* The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
|
||||||
* So `ip addr eth0` is your right most port
|
* So `ip addr eth0` is your right most port
|
||||||
* If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.
|
* If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.
|
||||||
|
|
||||||
## Build
|
### Table of Contents
|
||||||
|
1. Flashing
|
||||||
|
- [USB](docs/flashing-usb.md)
|
||||||
|
- [Over network (tftp)](docs/flashing-network.md)
|
||||||
|
2. [Upgrading](docs/ota.md)
|
||||||
|
|
||||||
|
## Build
|
||||||
Find the latest package versions and update build.env
|
Find the latest package versions and update build.env
|
||||||
* [kernel](https://github.com/nxp-qoriq/linux/archive/refs/tags/)
|
* [kernel](https://github.com/nxp-qoriq/linux/archive/refs/tags/)
|
||||||
* [busybox](https://github.com/mirror/busybox/archive/refs/tags/)
|
* [busybox](https://github.com/mirror/busybox/archive/refs/tags/)
|
||||||
@@ -25,131 +31,24 @@ Then run
|
|||||||
make release
|
make release
|
||||||
```
|
```
|
||||||
|
|
||||||
Then run, follow the instructions
|
Inspect and modify config/cluster.env, then run
|
||||||
```
|
```
|
||||||
make cluster-config
|
make cluster-config
|
||||||
```
|
```
|
||||||
|
|
||||||
## Flashing
|
|
||||||
|
|
||||||
#### USB Drive
|
|
||||||
1. `make release`
|
|
||||||
2. Format the USB in vfat
|
|
||||||
3. Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
|
|
||||||
4. Run
|
|
||||||
```
|
|
||||||
usb start
|
|
||||||
usb tree
|
|
||||||
fatls usb 0:1 # For fat
|
|
||||||
ext4ls usb 0:1 # For ext4 if you insist
|
|
||||||
fatload usb 0 0x80000000 board.itb
|
|
||||||
|
|
||||||
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
|
||||||
bootm 0x80000000
|
|
||||||
```
|
|
||||||
|
|
||||||
5. Inside initramfs, run
|
|
||||||
```
|
|
||||||
flash-emmc.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
4. If it boots, create the A/B deployment scheme
|
|
||||||
- (WORK IN PROGRESS)
|
|
||||||
|
|
||||||
#### tftp (network is required)
|
|
||||||
1. Put out/[RELEASE].img.gz and out/board.itb into your ftp server
|
|
||||||
```
|
|
||||||
setenv ipaddr 10.0.0.153
|
|
||||||
setenv serverip 10.0.0.129
|
|
||||||
tftp 0x80000000 board.itb
|
|
||||||
|
|
||||||
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
|
||||||
bootm 0x80000000
|
|
||||||
```
|
|
||||||
2. Inside initramfs, run
|
|
||||||
```
|
|
||||||
flash-emmc.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### Making sub stages
|
### Making sub stages
|
||||||
```bash
|
```bash
|
||||||
make build-base # The image the builds kernel and everything
|
make build-base # The image that builds the kernel and everything
|
||||||
make kernel # Builds our kernel from NXP
|
make kernel # Builds our kernel from NXP
|
||||||
make initramfs
|
make initramfs
|
||||||
make itb # Builds out/board.itb (contains the kernel and the initramfs)
|
make itb # Builds out/board.itb (contains the kernel and the initramfs)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Architecture
|
|
||||||
- A/B deployment (So we can just do kubectl apply upgrade.yaml with version jumps. Agent will walk through it automatically)
|
|
||||||
- Read-only OS
|
|
||||||
|
|
||||||
## Upgrade process
|
|
||||||
|
|
||||||
We use a CRD with an agent to handle this. Our versions follows upstream's.
|
|
||||||
|
|
||||||
kubectl apply -f upgrade.yaml
|
|
||||||
```yaml
|
|
||||||
apiVersion: monok8s.io/v1alpha1
|
|
||||||
kind: OSUpgrade
|
|
||||||
metadata:
|
|
||||||
name: "my-ugrade-2"
|
|
||||||
spec:
|
|
||||||
version: "v1.35.3"
|
|
||||||
imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
|
|
||||||
checksum: "sha256:..."
|
|
||||||
nodeSelector: {}
|
|
||||||
status:
|
|
||||||
phase: Pending
|
|
||||||
resolvedVersion: "v1.35.3"
|
|
||||||
observedGeneration: 1
|
|
||||||
summary:
|
|
||||||
targetedNodes: 0
|
|
||||||
pendingNodes: 0
|
|
||||||
runningNodes: 0
|
|
||||||
succeededNodes: 0
|
|
||||||
failedNodes: 0
|
|
||||||
```
|
|
||||||
|
|
||||||
kubectl get osugrades
|
|
||||||
```
|
|
||||||
NAME DESIRED RESOLVED PHASE TARGETS OK FAIL AGE
|
|
||||||
my-upgrade-3 stable v1.35.4 RollingOut 3 1 0 1m
|
|
||||||
my-upgrade-2 v1.35.3 v1.35.3 Accepted 2 0 0 1m
|
|
||||||
my-downgrade-1 v1.33.2 v1.33.2 Rejected 2 0 2 1m
|
|
||||||
```
|
|
||||||
|
|
||||||
kubectl get osupgradeprogress
|
|
||||||
```
|
|
||||||
NAME NODE SOURCE CURRENT TARGET STATUS
|
|
||||||
osupgrade-abc123f node-1 my-upgrade-2 v1.34.1 v1.35.3 downloading
|
|
||||||
osupgrade-cde456g node-2 my-upgrade-2 v1.35.3 v1.35.3 completed
|
|
||||||
```
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: monok8s.io/v1alpha1
|
|
||||||
kind: OSUpgradeProgress
|
|
||||||
metadata:
|
|
||||||
name: "osupgrade-abc123f"
|
|
||||||
spec:
|
|
||||||
sourceRef:
|
|
||||||
name: my-upgrade-2
|
|
||||||
nodeName: node-1
|
|
||||||
status:
|
|
||||||
currentVersion: "v1.34.1"
|
|
||||||
targetVersion: "v1.35.3"
|
|
||||||
phase: Downloading
|
|
||||||
startedAt: null
|
|
||||||
completedAt: null
|
|
||||||
lastUpdatedAt: null
|
|
||||||
retryCount: 0
|
|
||||||
inactivePartition: "B"
|
|
||||||
failureReason: ""
|
|
||||||
message: ""
|
|
||||||
```
|
|
||||||
|
|
||||||
## NOTES
|
## NOTES
|
||||||
### The device's dts files are located at here
|
### The device's dts files are located at here
|
||||||
https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale
|
https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale
|
||||||
* We need both `mono-gateway-dk-sdk.dts` and `mono-gateway-dk.dts` since the sdk one includes the non-sdk one.
|
* We need both `mono-gateway-dk-sdk.dts` and `mono-gateway-dk.dts` since the sdk one includes the non-sdk one.
|
||||||
* The actual dts being used is the `mono-gateway-dk-sdk.dts`
|
* The actual dts being used is the `mono-gateway-dk-sdk.dts`
|
||||||
|
|
||||||
|
## DISCLAIMER
|
||||||
|
USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this.
|
||||||
|
|||||||
17
docs/flashing-network.md
Normal file
17
docs/flashing-network.md
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
Mostly just the same as tutorial from mono
|
||||||
|
|
||||||
|
## tftp (network is required)
|
||||||
|
1. Put out/[RELEASE].img.gz and out/board.itb into your ftp server
|
||||||
|
```
|
||||||
|
setenv ipaddr 10.0.0.153
|
||||||
|
setenv serverip 10.0.0.129
|
||||||
|
tftp 0x80000000 board.itb
|
||||||
|
|
||||||
|
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
||||||
|
bootm 0x80000000
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Inside initramfs, run
|
||||||
|
```
|
||||||
|
flash-emmc.sh
|
||||||
|
```
|
||||||
33
docs/flashing-usb.md
Normal file
33
docs/flashing-usb.md
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
We have two type of flashing
|
||||||
|
1. Flash the image directly into USB. And boot using it.
|
||||||
|
2. Copy the files into the USB drive. Then boot into initramfs. Then flash the image into the eMMC.
|
||||||
|
|
||||||
|
## Flashing into eMMC
|
||||||
|
1. `make release`
|
||||||
|
2. Format the USB in vfat
|
||||||
|
3. Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
|
||||||
|
4. Run
|
||||||
|
```
|
||||||
|
usb start
|
||||||
|
usb tree
|
||||||
|
fatls usb 0:1 # For fat
|
||||||
|
ext4ls usb 0:1 # For ext4 if you insist
|
||||||
|
fatload usb 0 0x80000000 board.itb
|
||||||
|
|
||||||
|
setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
|
||||||
|
bootm 0x80000000
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Inside initramfs, run
|
||||||
|
```
|
||||||
|
flash-emmc.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
4. If it boots, create the A/B deployment scheme
|
||||||
|
- (WORK IN PROGRESS)
|
||||||
|
|
||||||
|
## Flashing into USB
|
||||||
|
On MacOS
|
||||||
|
1. ./macos/flashusb.sh
|
||||||
|
|
||||||
|
On Windows (Work In Progress)
|
||||||
117
docs/ota.md
117
docs/ota.md
@@ -1,3 +1,120 @@
|
|||||||
|
## Upgrade process
|
||||||
|
|
||||||
|
We use a CRD with an agent to handle this. Our versions follows upstream's.
|
||||||
|
|
||||||
|
To issue an upgrade. Simply use
|
||||||
|
|
||||||
|
kubectl apply -f upgrade.yaml
|
||||||
|
```yaml
|
||||||
|
apiVersion: monok8s.io/v1alpha1
|
||||||
|
kind: OSUpgrade
|
||||||
|
metadata:
|
||||||
|
name: "my-ugrade-2"
|
||||||
|
spec:
|
||||||
|
version: "v1.35.3"
|
||||||
|
imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
|
||||||
|
checksum: "sha256:..."
|
||||||
|
nodeSelector: {}
|
||||||
|
catalog:
|
||||||
|
inline: |
|
||||||
|
stable: v1.35.1
|
||||||
|
images:
|
||||||
|
- version: v1.34.6
|
||||||
|
url: https://example.invalid/images/monok8s-v1.34.6.img.zst
|
||||||
|
checksum: sha256:abc
|
||||||
|
- version: v1.34.1
|
||||||
|
url: https://example.invalid/images/monok8s-v1.34.1.img.zst
|
||||||
|
checksum: sha256:abc
|
||||||
|
- version: v1.35.0
|
||||||
|
url: https://example.invalid/images/monok8s-v1.35.0.img.zst
|
||||||
|
checksum: sha256:ghi
|
||||||
|
- version: v1.35.4
|
||||||
|
url: https://example.invalid/images/monok8s-v1.35.4.img.zst
|
||||||
|
checksum: sha256:jkl
|
||||||
|
- version: v1.35.1
|
||||||
|
url: http://localhost:8000/rootfs.ext4.zst
|
||||||
|
checksum: sha256:99af82a263deca44ad91d21d684f0fa944d5d0456a1da540f1c644f8aa59b14b
|
||||||
|
size: 1858076672 # expanded image size in bytes, use "zstd -lv image.zst to check"
|
||||||
|
blocked:
|
||||||
|
- v1.34.0
|
||||||
|
```
|
||||||
|
|
||||||
|
catalog accepts URL or ConfigMap
|
||||||
|
```yaml
|
||||||
|
catalog:
|
||||||
|
URL: https://example.com/images.yaml
|
||||||
|
|
||||||
|
catalog:
|
||||||
|
ConfigMap: images-cm
|
||||||
|
```
|
||||||
|
|
||||||
|
Contents should look like this
|
||||||
|
```yaml
|
||||||
|
stable: v1.35.1
|
||||||
|
images:
|
||||||
|
- version: v1.34.6
|
||||||
|
url: https://example.invalid/images/monok8s-v1.34.6.img.zst
|
||||||
|
checksum: sha256:abc
|
||||||
|
- version: v1.34.1
|
||||||
|
url: https://example.invalid/images/monok8s-v1.34.1.img.zst
|
||||||
|
checksum: sha256:abc
|
||||||
|
- version: v1.35.0
|
||||||
|
url: https://example.invalid/images/monok8s-v1.35.0.img.zst
|
||||||
|
checksum: sha256:ghi
|
||||||
|
- version: v1.35.4
|
||||||
|
url: https://example.invalid/images/monok8s-v1.35.4.img.zst
|
||||||
|
checksum: sha256:jkl
|
||||||
|
- version: v1.35.1
|
||||||
|
url: http://localhost:8000/rootfs.ext4.zst
|
||||||
|
checksum: sha256:99af82a263deca44ad91d21d684f0fa944d5d0456a1da540f1c644f8aa59b14b
|
||||||
|
size: 1858076672 # expanded image size in bytes, use "zstd -lv image.zst to check"
|
||||||
|
blocked:
|
||||||
|
- v1.34.0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Monitoring the upgrades
|
||||||
|
|
||||||
|
kubectl get osugrades
|
||||||
|
```
|
||||||
|
NAME DESIRED RESOLVED PHASE TARGETS OK FAIL AGE
|
||||||
|
my-upgrade-3 stable v1.35.4 RollingOut 3 1 0 1m
|
||||||
|
my-upgrade-2 v1.35.3 v1.35.3 Accepted 2 0 0 1m
|
||||||
|
my-downgrade-1 v1.33.2 v1.33.2 Rejected 2 0 2 1m
|
||||||
|
```
|
||||||
|
|
||||||
|
kubectl get osupgradeprogress
|
||||||
|
```
|
||||||
|
NAME NODE SOURCE CURRENT TARGET STATUS
|
||||||
|
osupgrade-abc123f node-1 my-upgrade-2 v1.34.1 v1.35.3 downloading
|
||||||
|
osupgrade-cde456g node-2 my-upgrade-2 v1.35.3 v1.35.3 completed
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
kubectl describe osupgradeprogress osupgrade-abc123f
|
||||||
|
```yaml
|
||||||
|
apiVersion: monok8s.io/v1alpha1
|
||||||
|
kind: OSUpgradeProgress
|
||||||
|
metadata:
|
||||||
|
name: "osupgrade-abc123f"
|
||||||
|
spec:
|
||||||
|
sourceRef:
|
||||||
|
name: my-upgrade-2
|
||||||
|
nodeName: node-1
|
||||||
|
status:
|
||||||
|
currentVersion: "v1.34.1"
|
||||||
|
targetVersion: "v1.35.3"
|
||||||
|
phase: Downloading
|
||||||
|
startedAt: null
|
||||||
|
completedAt: null
|
||||||
|
lastUpdatedAt: null
|
||||||
|
retryCount: 0
|
||||||
|
inactivePartition: "B"
|
||||||
|
failureReason: ""
|
||||||
|
message: ""
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development notes
|
||||||
|
|
||||||
### Simulate OTA
|
### Simulate OTA
|
||||||
|
|
||||||
**Use nmap ncat**. Otherwise we'll have all kinds of fabulous issues sending it.
|
**Use nmap ncat**. Otherwise we'll have all kinds of fabulous issues sending it.
|
||||||
|
|||||||
Reference in New Issue
Block a user