@Bobino, as always I appreciate your ability to dissect an issue and bring up various points to consider.
I know some people are sometimes annoyed with topics derailing. On the other hand, I have noticed that the entropy it creates generates often very interesting and fruitful exchanges, albeit not always in line with the op's question.
I regularly grab chunks of derailing conversations and put them in their own threads. I usually wait to see if it starts having a life on its own or if its just an à propos here and there. Sometimes, it's more of the same or closely related to a topic that was discussed in the past. In that case, if I remember such thread, I will move those stray posts to it. Although it might be weird to revive old threads, I feel it is better than having tons of thread on a similar topic, especially when the thread's tone and the new posts is more of the same bit of "whining" about that given topic. I am sure it is not a perfect strategy, but the one I have on hand at this time.
The other reason why I wait a bit before moving the stray posts is to allow for it to grow enough to show some potential. Sometimes great ideas are stricken dead if we put the breaks too soon. That has the value of generating new interesting content and, because it is eventually moved to continue its life elsewhere, it doesn't rebuke people that consult the original thread later on.
Sometimes, it is not so much that the topic is being derailed but people having a little side chatter about current events and personal stuff like birthdays, sickness, Australian fires, etc.
I find it more difficult to decide what to do with those. Although I feel really uncomfortable deleting people's posts, especially when they are sharing about themselves, I have deleted a few such comments when they were mainly by two people that were starting to use the thread as a chat room. In my opinion, the content is not mine and I would hate for people to decide that what I say has no value. I just discovered yesterday that I can actually flip these chunks into becoming private messages between the involved people, which is what I will probably do in the future, again, not right away when it occurs but after a reasonable time.
The reason I consider it should not be right away is that, as I mentioned elsewhere, the soup plays an important role for peers to mingle and have friendly chatter. I often refer to it as the church front porch. In the past, people would gather there and have social interactions, talking into clusters, yet being able to overhear other conversations, feeling part of the community.
Of course in the future, it becomes noise, in terms of the knowledge base. That is why I "clean" threads once in a while. The discovery of the new tool (converting some posts into private messages) will make this task less challenging ethically for me or other members with moderation or editing rights. Level trust 3 members have that ability.