That's a good idea. I probably could've put a CDN in front of this and rate limited there while keeping things public. That might've been faster than using Ruby to be honest. The downside was that our API already shared the non-CDN URLs, so that would leave the problem open for anyone who already had that data.
The bucket is private though, only accessible through the CDN. The old URLs would cease to function. On AWS this is implemented through OAI/OAC, granting the CloudFront distribution access via its own unique principal. AWS has had a baseline security recommendation for years now to disable S3 public access at the account/org level.
Maybe this breaks things, maybe you need to expire some caches, but (forgive me for being blunt, I can't think of a better way to say it) that's the cost of not doing things correctly to begin with.
My first thought as a security engineer when setting something up to be public has always been "how hard could someone hit this, and how much would it cost/affect availability?"