131 lines
3.5 KiB
Markdown
131 lines
3.5 KiB
Markdown
## Upgrade process
|
|
|
|
We use a CRD with an agent to handle this. Our versions follows upstream's.
|
|
|
|
To issue an upgrade. Simply use
|
|
|
|
kubectl apply -f upgrade.yaml
|
|
```yaml
|
|
apiVersion: monok8s.io/v1alpha1
|
|
kind: OSUpgrade
|
|
metadata:
|
|
name: "my-ugrade-2"
|
|
spec:
|
|
version: "v1.35.3"
|
|
imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
|
|
checksum: "sha256:..."
|
|
nodeSelector: {}
|
|
catalog:
|
|
inline: |
|
|
stable: v1.35.1
|
|
images:
|
|
- version: v1.34.6
|
|
url: https://example.invalid/images/monok8s-v1.34.6.img.zst
|
|
checksum: sha256:abc
|
|
- version: v1.34.1
|
|
url: https://example.invalid/images/monok8s-v1.34.1.img.zst
|
|
checksum: sha256:abc
|
|
- version: v1.35.0
|
|
url: https://example.invalid/images/monok8s-v1.35.0.img.zst
|
|
checksum: sha256:ghi
|
|
- version: v1.35.4
|
|
url: https://example.invalid/images/monok8s-v1.35.4.img.zst
|
|
checksum: sha256:jkl
|
|
- version: v1.35.1
|
|
url: http://localhost:8000/rootfs.ext4.zst
|
|
checksum: sha256:99af82a263deca44ad91d21d684f0fa944d5d0456a1da540f1c644f8aa59b14b
|
|
size: 1858076672 # expanded image size in bytes, use "zstd -lv image.zst to check"
|
|
blocked:
|
|
- v1.34.0
|
|
```
|
|
|
|
catalog accepts URL or ConfigMap
|
|
```yaml
|
|
catalog:
|
|
URL: https://example.com/images.yaml
|
|
|
|
catalog:
|
|
ConfigMap: images-cm
|
|
```
|
|
|
|
Contents should look like this
|
|
```yaml
|
|
stable: v1.35.1
|
|
images:
|
|
- version: v1.34.6
|
|
url: https://example.invalid/images/monok8s-v1.34.6.img.zst
|
|
checksum: sha256:abc
|
|
- version: v1.34.1
|
|
url: https://example.invalid/images/monok8s-v1.34.1.img.zst
|
|
checksum: sha256:abc
|
|
- version: v1.35.0
|
|
url: https://example.invalid/images/monok8s-v1.35.0.img.zst
|
|
checksum: sha256:ghi
|
|
- version: v1.35.4
|
|
url: https://example.invalid/images/monok8s-v1.35.4.img.zst
|
|
checksum: sha256:jkl
|
|
- version: v1.35.1
|
|
url: http://localhost:8000/rootfs.ext4.zst
|
|
checksum: sha256:99af82a263deca44ad91d21d684f0fa944d5d0456a1da540f1c644f8aa59b14b
|
|
size: 1858076672 # expanded image size in bytes, use "zstd -lv image.zst to check"
|
|
blocked:
|
|
- v1.34.0
|
|
```
|
|
|
|
### Monitoring the upgrades
|
|
|
|
kubectl get osugrades
|
|
```
|
|
NAME DESIRED RESOLVED PHASE TARGETS OK FAIL AGE
|
|
my-upgrade-3 stable v1.35.4 RollingOut 3 1 0 1m
|
|
my-upgrade-2 v1.35.3 v1.35.3 Accepted 2 0 0 1m
|
|
my-downgrade-1 v1.33.2 v1.33.2 Rejected 2 0 2 1m
|
|
```
|
|
|
|
kubectl get osupgradeprogress
|
|
```
|
|
NAME NODE SOURCE CURRENT TARGET STATUS
|
|
osupgrade-abc123f node-1 my-upgrade-2 v1.34.1 v1.35.3 downloading
|
|
osupgrade-cde456g node-2 my-upgrade-2 v1.35.3 v1.35.3 completed
|
|
```
|
|
|
|
|
|
kubectl describe osupgradeprogress osupgrade-abc123f
|
|
```yaml
|
|
apiVersion: monok8s.io/v1alpha1
|
|
kind: OSUpgradeProgress
|
|
metadata:
|
|
name: "osupgrade-abc123f"
|
|
spec:
|
|
sourceRef:
|
|
name: my-upgrade-2
|
|
nodeName: node-1
|
|
status:
|
|
currentVersion: "v1.34.1"
|
|
targetVersion: "v1.35.3"
|
|
phase: Downloading
|
|
startedAt: null
|
|
completedAt: null
|
|
lastUpdatedAt: null
|
|
retryCount: 0
|
|
inactivePartition: "B"
|
|
failureReason: ""
|
|
message: ""
|
|
```
|
|
|
|
## Development notes
|
|
|
|
### Simulate OTA
|
|
|
|
**Use nmap ncat**. Otherwise we'll have all kinds of fabulous issues sending it.
|
|
|
|
Sending side
|
|
```
|
|
pv "out/rootfs.ext4.zst" | ncat 10.0.0.10 1234 --send-only
|
|
```
|
|
|
|
Receiving side
|
|
```
|
|
ncat -l 1234 --recv-only | zstd -d -c | dd of=/dev/sda3 bs=4M status=progress && sync && echo "SUCCESS"
|
|
```
|