muehe/onefinity-firmware
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite BUILD.md: clean up, add quick start, remove dead weight

Browse Source 
- Quick start section at the top (3 commands)
- Removed inline Bullseye build recipe (moved to 'why not' appendix)
- Added build time estimates
- Cleaner table formatting
- gplan.so contents documented (cbang + camotics)
This commit is contained in:
Henrik Muehe
2026-04-30 16:39:57 +02:00
parent 704bc8d35c
commit 54a15f9d12
1 changed files with 147 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files

331
.pi/BUILD.md
View File

@@ -1,200 +1,129 @@
# Onefinity CNC Firmware — Build, Flash & Backup
## Quick Start
```bash
# 1. Build gplan.so (first time ~25min, then ~1sec)
.pi/build-gplan.sh
# 2. Build firmware package (frontend + AVR + Python, ~1min)
docker run --rm -v "$(pwd):/workspace" -w /workspace onefin-dev \
bash -c 'make all && python3 ./setup.py sdist'
# 3. Flash to controller
curl -X PUT -F "firmware=@dist/bbctrl-1.6.7.tar.bz2" \
-F "password=onefinity" http://10.1.10.55/api/firmware/update
```
## Architecture Overview
The Onefinity controller is a **Raspberry Pi 2/3** (armv7l, Raspbian Stretch, Python 3.5) connected to an **ATxmega192a3u** AVR microcontroller over serial. The Pi runs a Tornado web server (`bbctrl`) that serves the UI and plans G-code motion. The AVR executes realtime step/direction pulses.
The controller is a **Raspberry Pi 2/3** (armv7l, Raspbian Stretch, Python 3.5)
connected to an **ATxmega192a3u** AVR over serial. The Pi runs a Tornado web
server that serves the UI, parses G-code, and plans motion. The AVR executes
realtime step/direction pulses.
```
Browser ←WebSocket→ Pi (Tornado/Python) → GCode Planner → Serial → AVR → Stepper drivers
```
The firmware package (`bbctrl-X.Y.Z.tar.bz2`) contains:
- **Python backend** (`src/py/bbctrl/`) — Tornado web server, planner, state machine
- **Web frontend** (`build/http/`) — Pug/Stylus/Svelte compiled to static HTML/JS/CSS
- **AVR firmware** (`src/avr/bbctrl-avr-firmware.hex`) — realtime motion controller
- **gplan.so** (`src/py/camotics/gplan.so`) — CAMotics G-code planner (native ARM .so)
- **Install scripts** (`scripts/install.sh`, etc.)
| Component | Source | Description |
|---|---|---|
| Python backend | `src/py/bbctrl/` | Tornado web server, state machine, planner bridge |
| Web frontend | `build/http/` | Pug + Stylus + Svelte → static HTML/JS/CSS |
| AVR firmware | `src/avr/bbctrl-avr-firmware.hex` | Realtime motion controller |
| gplan.so | `src/py/camotics/gplan.so` | CAMotics G-code planner (native ARM C++ extension) |
| Install scripts | `scripts/install.sh` | AVR flash, Python install, service restart |
## Prerequisites
- Docker (for the devcontainer build environment)
- The devcontainer image: `docker build -t onefin-dev -f .devcontainer/Dockerfile .devcontainer/`
- SSH access to the Pi: `ssh bbmc@10.1.10.55` (password: `onefinity`)
- **Docker** with QEMU binfmt support (default on Docker Desktop for Mac)
- **devcontainer image**: `docker build -t onefin-dev -f .devcontainer/Dockerfile .devcontainer/`
- **SSH access**: `ssh bbmc@10.1.10.55` (password: `onefinity`)
## Building
### Full build (frontend + AVR + package)
### Step 1: gplan.so
`gplan.so` is the CAMotics G-code planner — a C++ Python extension that must
be a **32-bit ARM binary linked against Python 3.5**. It cannot be built in the
devcontainer (wrong arch + wrong Python + wrong glibc).
**Build from source** (recommended):
```bash
.pi/build-gplan.sh
```
This uses a Raspbian Stretch Docker image (`balenalib/raspberry-pi-debian:stretch`)
with the Pi's exact toolchain: GCC 6.3, Python 3.5, GLIBC 2.24. The image is
built once (~25min under QEMU), then cached — subsequent runs take ~1sec.
The image pre-compiles two C++ dependencies:
- [cbang](https://github.com/CauldronDevelopmentLLC/cbang) @ `18f1e96` — C++ utility library
- [camotics](https://github.com/CauldronDevelopmentLLC/camotics) @ `ec876c8` — G-code planner with S-curve motion planning
To force a full rebuild: `docker rmi onefin-gplan && .pi/build-gplan.sh`
**Alternatives** (if Docker build fails):
```bash
# From official release
curl -sL https://github.com/OneFinityCNC/onefinity-firmware/releases/download/v1.6.6/onefinity-1.6.6.tar.bz2 \
| tar xjf - --include='*/gplan.so' -O > src/py/camotics/gplan.so
# From the running Pi
scp bbmc@10.1.10.55:/usr/local/lib/python3.5/dist-packages/bbctrl-*.egg/camotics/gplan.so src/py/camotics/
```
**Verify** — must show `ELF 32-bit ... ARM ... libpython3.5m`:
```bash
file src/py/camotics/gplan.so
readelf -d src/py/camotics/gplan.so | grep python
```
### Step 2: Firmware package
```bash
docker run --rm -v "$(pwd):/workspace" -w /workspace onefin-dev \
bash -c 'make all && python3 ./setup.py sdist'
```
Produces: `dist/bbctrl-1.6.7.tar.bz2` (~3MB)
This builds inside the devcontainer (arm64 Bullseye — fine for frontend/AVR/Python):
### What `make all` builds
| Component | Command | Output |
| Component | Tool | Time |
|---|---|---|
| Web frontend | `npm install`, pug/stylus/svelte compile | `build/http/` |
| AVR firmware | `make -C src/avr` (avr-g++) | `src/avr/bbctrl-avr-firmware.hex` |
| Bootloader | `make -C src/boot` | `src/boot/bbctrl-avr-boot.hex` |
| Power MCU | `make -C src/pwr` | `src/pwr/bbctrl-pwr-firmware.hex` |
| Jig firmware | `make -C src/jig` | `src/jig/bbctrl-jig-firmware.hex` |
| Node modules | `npm install` | ~30sec |
| Svelte components | `vite build` | ~5sec |
| Pug/Stylus → HTML | `pug-cli`, `stylus` | ~2sec |
| AVR firmware | `avr-g++` (ATxmega192a3u) | ~10sec |
| Boot/Power/Jig MCUs | `avr-gcc` | ~5sec |
| Python sdist | `setup.py sdist` | ~2sec |
### gplan.so — the critical gotcha
Produces: `dist/bbctrl-X.Y.Z.tar.bz2` (~3-4MB)
`gplan.so` is the CAMotics G-code planner compiled as a Python C extension. It **must be a 32-bit ARM binary** linked against **Python 3.5** to run on the Pi.
### bbserial.ko (kernel module — usually skip)
**Do NOT build gplan.so in the devcontainer.** The devcontainer runs arm64/Debian Bullseye with Python 3.9. The resulting `.so` will be the wrong architecture and wrong Python ABI. Cross-compiling with `CXX=arm-linux-gnueabihf-g++` also fails because SCons ignores CC/CXX env vars.
The correct file must be:
```
ELF 32-bit LSB shared object, ARM, EABI5 (linked against libpython3.5m.so.1.0)
```
If you see `ELF 64-bit LSB shared object, ARM aarch64` or `libpython3.9` — wrong.
**Option A: Build from source (recommended)**
Uses a Raspbian Stretch Docker image that exactly matches the Pi's toolchain:
GCC 6.3, Python 3.5, GLIBC 2.24. No cross-compile hacks, no ABI mismatches.
```bash
.pi/build-gplan.sh
```
- First run: ~25min (builds `onefin-gplan` Docker image with pre-compiled cbang + camotics)
- After that: ~1sec (copies cached `gplan.so` from image)
- Force rebuild: `docker rmi onefin-gplan && .pi/build-gplan.sh`
See `.pi/Dockerfile.gplan` for the full build recipe.
**Option B: From official release**
```bash
curl -sL https://github.com/OneFinityCNC/onefinity-firmware/releases/download/v1.6.6/onefinity-1.6.6.tar.bz2 \
| tar xjf - --include='*/gplan.so' -O > src/py/camotics/gplan.so
```
**Option C: From a working Pi**
```bash
scp bbmc@10.1.10.55:/usr/local/lib/python3.5/dist-packages/bbctrl-*.egg/camotics/gplan.so src/py/camotics/
```
**Failed approach: Debian Bullseye armhf (documented for reference)**
Uses Docker's armv7 QEMU emulation on Apple Silicon. Requires a one-time
copy of Python 3.5 headers from the Pi (~1.7MB):
```bash
# One-time: grab Python 3.5 headers + lib from Pi
ssh bbmc@10.1.10.55 'tar czf - /usr/include/python3.5m \
/usr/lib/arm-linux-gnueabihf/libpython3.5m.so*' > /tmp/pi-python35.tar.gz
```
Then build (takes ~30min under QEMU):
```bash
docker run --rm --platform linux/arm/v7 \
-v "$(pwd):/workspace" -w /workspace \
-v /tmp/pi-python35.tar.gz:/tmp/pi-python35.tar.gz \
debian:bullseye bash -c '
set -e
tar xzf /tmp/pi-python35.tar.gz -C /
apt-get update -qq && apt-get install -y -qq build-essential scons git \
ca-certificates python3-dev libssl-dev libexpat1-dev libbz2-dev \
liblz4-dev zlib1g-dev perl file
ln -sf /usr/lib/arm-linux-gnueabihf/libpython3.5m.so.1.0 \
/usr/lib/arm-linux-gnueabihf/libpython3.5m.so
# Clone cbang + camotics at pinned commits
mkdir -p /tmp/cbang && cd /tmp/cbang && git init -q
git remote add origin https://github.com/CauldronDevelopmentLLC/cbang
git fetch --depth 1 -q origin 18f1e963107ef26abe750c023355a5c40dd07853
git reset --hard FETCH_HEAD -q
mkdir -p /tmp/camotics && cd /tmp/camotics && git init -q
git remote add origin https://github.com/CauldronDevelopmentLLC/camotics
git fetch --depth 1 -q origin ec876c80d20fc19837133087cef0c447df5a939d
git reset --hard FETCH_HEAD -q
# Build cbang
cd /tmp/cbang && scons -j2 disable_local="re2 libevent"
export CBANG_HOME="/tmp/cbang"
# Patch camotics (clamp div-by-zero in planner)
P="/tmp/camotics/src/gcode/plan"
mkdir -p /tmp/camotics/build && touch /tmp/camotics/build/version.txt
for F in LineCommand.cpp LinePlanner.cpp; do
for V in maxVel maxJerk maxAccel; do
perl -i -0pe "s/(fabs\\((config\\.$V\\[axis\\]) \\/ unit\\[axis\\]\\));/std::min(\\2, \\1);/gm" $P/$F
done
done
# Compile with Python 3.9 (SCons needs python3-dev to find headers)
cd /tmp/camotics && scons -j2 gplan.so with_gui=0 with_tpl=0
# Relink against Python 3.5m (the Pi target)
g++ -o /workspace/src/py/camotics/gplan.so \
-Wl,--as-needed -Wl,-s -Wl,-x -Wl,--gc-sections -pthread -shared \
build/gplan.os -L/tmp/cbang/lib \
build/libCAMoticsPy.a build/libCAMotics.a build/libDXF.a \
build/libSTL.a build/libGCode.a \
-lstdc++ -lutil -lm -ldl -lz -lcbang -lcbang-boost \
-lssl -lcrypto -llz4 -lexpat -lbz2 -lcrypt -lpthread \
-lpython3.5m build/dxflib/libdxflib.a
'
```
**Why this doesn't actually work:** The `.o` files compile fine and the
Python relink works, but the compiled objects use GLIBC_2.29+ symbols
(from Bullseye's glibc 2.31) and GLIBCXX_3.4.26+ (from GCC 10). The Pi's
Stretch has GLIBC_2.24 / GLIBCXX_3.4.22 max. Even with `-static-libstdc++
-static-libgcc`, glibc symbols like `GLIBC_2.29` leak through the object
files compiled against Bullseye headers.
The solution was to use `balenalib/raspberry-pi-debian:stretch` with
`legacy.raspbian.org` repos — these still work unlike bare `debian:stretch`.
See `.pi/Dockerfile.gplan`.
**Key Pi constraints for native code:**
- GLIBC ≤ 2.24 (Stretch)
- GLIBCXX ≤ 3.4.22 (GCC 6)
- Python 3.5m (`libpython3.5m.so.1.0`)
- ELF 32-bit ARM EABI5
### bbserial.ko — kernel module
The `bbserial.ko` kernel module requires cross-compiling against the Pi's exact kernel headers (4.9.59-v7+). The `make pkg` target tries to build it but it's rarely needed — the Pi already has a matching `.ko` installed. The `install.sh` script skips it gracefully if the file is missing (`cp: cannot stat 'src/bbserial/bbserial.ko': No such file or directory`).
### AVR emulator (for local demo mode)
```bash
docker run --rm -v "$(pwd):/workspace" -w /workspace/src/avr/emu onefin-dev make
```
Produces `src/avr/emu/bbemu` — a native binary that emulates the AVR for demo mode.
Cross-compiles against the Pi's kernel headers (4.9.59-v7+). The Pi already has
a working `bbserial.ko` installed. `install.sh` skips it gracefully if missing.
## Flashing
### Via web API (recommended, machine must be running)
### Via web API (machine running)
```bash
curl -X PUT \
-H "Content-Type: multipart/form-data" \
-F "firmware=@dist/bbctrl-1.6.7.tar.bz2" \
-F "password=onefinity" \
http://10.1.10.55/api/firmware/update
curl -X PUT -F "firmware=@dist/bbctrl-1.6.7.tar.bz2" \
-F "password=onefinity" http://10.1.10.55/api/firmware/update
```
Or: `make update HOST=10.1.10.55 PASSWORD=onefinity`
Or: `make update HOST=10.1.10.55`
### Via SSH (if web UI is down / crash-looping)
### Via SSH (web UI down or crash-looping)
```bash
scp dist/bbctrl-1.6.7.tar.bz2 bbmc@10.1.10.55:/tmp/
ssh bbmc@10.1.10.55 'echo onefinity | sudo -S bash -c "
systemctl stop bbctrl
mkdir -p /var/lib/bbctrl/firmware
@@ -212,21 +141,22 @@ ssh bbmc@10.1.10.55 'echo onefinity | sudo -S bash -c "
- Restarts `bbctrl` systemd service
- May reboot if boot config or kernel module changed
### Recovery if flash breaks the Pi
### Recovery from bad flash
If bbctrl is crash-looping after a bad flash:
1. SSH still works: `ssh bbmc@10.1.10.55`
2. Check the error: `sudo python3 /usr/local/bin/bbctrl 2>&1 | head -20`
3. Common fix: replace gplan.so with correct ARM binary (see above)
4. Nuclear option: restore from SD card backup (see below)
SSH still works even when bbctrl is crash-looping:
1. Check the error: `sudo python3 /usr/local/bin/bbctrl 2>&1 | head -20`
2. Common cause: wrong gplan.so architecture → replace with correct one (see above)
3. Nuclear option: restore SD card from backup
## Running locally (demo mode)
## Running Locally (demo mode)
Run the full stack in Docker with the AVR emulator:
Full stack in Docker with AVR emulator — no Pi needed:
```bash
# Build everything first (make all + bbemu + gplan.so for arm64 devcontainer)
# Then:
# Build bbemu (AVR emulator, native in devcontainer)
docker run --rm -v "$(pwd):/workspace" -w /workspace/src/avr/emu onefin-dev make
# Run demo (needs arm64 gplan.so for the container, not armhf)
docker run --rm -d --name onefin-demo \
-v "$(pwd):/workspace" -w /workspace -p 8765:80 \
onefin-dev bash -c '
@@ -237,14 +167,16 @@ docker run --rm -d --name onefin-demo \
'
```
Note: demo mode needs its own gplan.so matching the container's arch (arm64 + Python 3.9). Build it with the gplan build procedure in the Makefile, or use the one already in `src/py/camotics/` if it matches.
Open http://localhost:8765 — full UI with emulated controller.
Open http://localhost:8765 — full UI with emulated AVR.
Note: demo mode needs a **container-arch** gplan.so (arm64 + Python 3.9), not the
Pi one. The devcontainer build from the Makefile's `gplan` target produces this,
or it can be built following the procedure in `scripts/gplan-build.sh`.
## SD Card Backup & Restore
```bash
# Backup (streams raw dd from Pi, compresses locally with gzip, ~50 min)
# Backup (~50 min, streams raw dd from Pi, compresses locally)
./backup/onefinity-backup.sh backup
# Verify
@@ -252,35 +184,66 @@ Open http://localhost:8765 — full UI with emulated AVR.
# Restore to local SD card
./backup/onefinity-backup.sh restore backup/onefinity-20260430.img.gz /dev/diskN
# Restore back to Pi over SSH
./backup/onefinity-backup.sh restore backup/onefinity-20260430.img.gz
```
See `backup/onefinity-backup.sh` for details. Environment variables: `ONEFINITY_HOST` (default 10.1.10.55), `ONEFINITY_USER` (bbmc), `ONEFINITY_PASS` (onefinity).
Environment: `ONEFINITY_HOST` (default 10.1.10.55), `ONEFINITY_USER` (bbmc),
`ONEFINITY_PASS` (onefinity).
## Python 3.5 Compatibility
The Pi runs Python 3.5.3. Watch out for features added in later versions:
- No `f"strings"` (use `"%s" % var` or `"{}".format(var)`)
- No `subprocess.run(capture_output=True)` (use `stdout=PIPE, stderr=PIPE`)
- No `subprocess.run(text=True)` (use `.decode('utf-8')` on stdout/stderr)
- No `dataclasses`, no `:=` walrus operator
- No `asyncio.run()` (use `loop.run_until_complete()`)
- No `typing` generics like `dict[str, int]` (use `Dict[str, int]` from typing)
The Pi runs Python 3.5.3. Avoid features added in later versions:
| Avoid | Use instead |
|---|---|
| `f"hello {name}"` | `"hello %s" % name` or `"hello {}".format(name)` |
| `subprocess.run(capture_output=True)` | `stdout=subprocess.PIPE, stderr=subprocess.PIPE` |
| `subprocess.run(text=True)` | `.stdout.decode('utf-8')` |
| `dataclasses` | plain classes with `__init__` |
| `:=` walrus operator | separate assignment |
| `asyncio.run()` | `loop.run_until_complete()` |
| `dict[str, int]` | `Dict[str, int]` from `typing` |
## Pi Details
| | |
|---|---|
| Host | `10.1.10.55` |
| SSH user | `bbmc` |
| sudo password | `onefinity` |
| SSH | `bbmc` / `onefinity` |
| OS | Raspbian Stretch (Debian 9) |
| Kernel | 4.9.59-v7+ |
| Python | 3.5.3 |
| Arch | armv7l (32-bit ARM) |
| SD card | 30GB |
| GCC | 6.3.0 |
| GLIBC | 2.24 (max symbol: GLIBC_2.24) |
| GLIBCXX | 3.4.22 |
| Arch | armv7l (32-bit ARM, EABI5) |
| SD card | 30GB (~2.8GB used) |
| Service | `systemctl {start,stop,restart,status} bbctrl` |
| Log | `/var/log/bbctrl.log` or `journalctl -u bbctrl` |
| Config | `/var/lib/bbctrl/config.json` |
| Uploads | `/var/lib/bbctrl/upload/` |
| Web root | `/usr/local/lib/python3.5/dist-packages/bbctrl-*.egg/bbctrl/http/` |
| AVR serial | `/dev/ttyAMA0` at 230400 baud |
## Why Not Build gplan.so on Bullseye?
Documented for reference — we tried two approaches that don't work:
**1. devcontainer (arm64 Bullseye):** Wrong ELF class (64-bit vs 32-bit) and wrong
Python (3.9 vs 3.5). Cross-compiling with `CXX=arm-linux-gnueabihf-g++` fails
because SCons ignores CC/CXX environment variables.
**2. Bullseye armhf container:** Correct architecture, but GCC 10 / glibc 2.31
produce objects requiring GLIBC_2.29+ and GLIBCXX_3.4.26+ symbols. The Pi's
Stretch only has GLIBC_2.24 / GLIBCXX_3.4.22. Even `-static-libstdc++
-static-libgcc` doesn't help — glibc symbols leak through the object files.
Relinking against Python 3.5m works but the GLIBC mismatch remains.
**3. Plain `debian:stretch` armhf:** The archived repos have broken package
metadata — `apt-get install build-essential` fails with unresolvable version
conflicts.
**Solution:** `balenalib/raspberry-pi-debian:stretch` with `legacy.raspbian.org`
repos. See `.pi/Dockerfile.gplan`.