I'm not the best one to answer your question, but I know there's Buildroot and Yocto - the latter being a bit "heavy" in terms of setup. I think I would go for one of those distros that focus on minimalism (Damn Small Linux, Alpine, ...) which also have precompiled packages for your target (note: Debian does have a tool to build custom root filesystems [1]).
From there, get gcc and make for your platform, so you can carry on with compilation on-target, if possible; cross-compilation can be tricky. It should be ok if you don't want the heavier GUI stuff like X11/Wayland, browser etc.
From there, get gcc and make for your platform, so you can carry on with compilation on-target, if possible; cross-compilation can be tricky. It should be ok if you don't want the heavier GUI stuff like X11/Wayland, browser etc.
[1] https://wiki.debian.org/Debootstrap