My guess is that it's much lower. I'm having a hard time finding a LLM output logit visualizer online, but IIRC, around half of tokens are predicted with >90% confidence. There are regularly more difficult tokens that need to be predicted but the >1% probability tokens aren't so many, probably around 10-20 in most cases.
This is of course based on the outputs of actual models that are only so smart, so a tree search that considers all possibly relevant ideas is going to have a larger amount of branches. Considering how many branches would be pruned to maintain grammatical correctness, my guess is that the token-level branching factor would be around 30. It could be up to around 300, but I highly doubt that it's larger than that.
This is of course based on the outputs of actual models that are only so smart, so a tree search that considers all possibly relevant ideas is going to have a larger amount of branches. Considering how many branches would be pruned to maintain grammatical correctness, my guess is that the token-level branching factor would be around 30. It could be up to around 300, but I highly doubt that it's larger than that.