Publish and subscribe - real time system

I seems to get the feeling that this is not the first time I wrote something about this pattern. Since I could not control my temptation, as always, I decided to pick it up again. It does make perfect sense, to me at least, that there are generally two roles everything plays, producer or consumer. In another word, every entity in the system is either producing something or consuming, or both. 

In a system where the scale is by nature big, there is little hope that you can manage and control the behavior via tight-bound relationships among the entities you are able to indentify. Under this circumstance, what has bigger chance to survive is a loose and distributed system - we are expecting each individual cell is 'good' enough to be self-sufficient, have its own purpose of life ;-).

Topic-based messaging publish and subscribe structure is quite useful and flexible to meet this requirement - a word I am a little reluctant to use right now. There are some good articles descrbing how to implement this concept in WCF and raw socket. Compared with zeromq, I am still yet to try which one offers better flexibility. I think I the easiest simulation is to come up with a set up producing tons of publishers and consumer and see how the time latency and throughput would hold up. When it comes to real time (or near real time) communication, whether leaving the publish service running in each individual entity or centralising the service publishing capabilities in server (entity itself will only issue publish or subscription requests to the centralised service host), where redundancy could be applied is still a bit question mark for me. If each entity hosts the service publishing itself, that means itself will need to maintain all subscribers' information locally, as well as the callbacks for all the published service on the network this entity is interested, or even maybe those publishers' information as well. There will a burst of multi-casting packages whenever new entity joins or leaves the party. While in the centralised server structure, most of the communication will be uni-cast.

Down another level, to use TCP might not be a good idea for the de-centralised structure as the overhead of maintaining the connection, and also the fact TCP packets demanding more load on the network might present a bit of problem for the overall throughput when the number of entity increases significantly. In the real time control scenario, some of the data are timing and quality critical, sometimes 'best-effort' delivery just is not good enough. Field bus is really for this purpose. I am thinking maybe a different priority or type of delivery-assured channel could be available when a service is to be published by certain entity, with default higher QoS level. (I wonder there is an easy way for one multi-casting receiver to determine whether the other receiver has consumed the same packet or not?)