Section 7: Ninety Times Fewer Messages - Deep Dive: The Reaction Pipeline

Executive Summary: You steer straight at the burst, not the average. Four dives: two-stage coalescing with a leading-edge throttle, the fan-out tree and backpressure, abuse and fairness, and the client half that decides what a human sees. The running numbers: Contoso's 1,800 claps arrive at 360 reactions a second, which would be 360K deliveries a second naively and is 4K frames a second after coalescing, 90× fewer. A five-person stand-up pays nothing for any of it.

"Where would you like to go deeper?"

The tempting answer is the average: 50K reactions a second, ~7 Gbps of egress, ~12 Mbps per gateway. That is the boring part, and saying so is the first signal.

You: "I'd like to go deep on the burst. On average this system is idle. The moment that hurts is 10:12 on Thursday, when Contoso's CTO announces the launch and 600 people clap three times each within five seconds. That's 1,800 reactions, 360 a second, into one 1,000-person meeting. Forwarded naively, that's 360K deliveries a second to one meeting. Everything here exists to make that moment cheap and keep it away from anyone's audio or raised hand."

Dive 1: Two-Stage Coalescing

The trick is to merge clicks twice: once where they are born, once where the whole meeting can be seen.

Stage one, the client. Maya's first click goes out immediately. Further clicks within 250 ms fold into a single reaction.send {type, count}, so no participant ever sends more than 4 messages a second however fast they click. This bounds the owner's ingress per person.

Stage two, the meeting owner. Only the owner sees the whole meeting (Section 6's reason it, not the gateway, coalesces). It runs a per-meeting leading-edge throttle: if no frame went out in the last 250 ms, flush now; otherwise accumulate and flush on the next 250 ms boundary.

const WINDOW_MS = 250;

function onReaction(m: Meeting, from: ParticipantId, type: ReactionType, count: number) {
  if (!m.options.reactionsEnabled) return;              // organizer kill switch
  const admitted = m.buckets.get(from).takeUpTo(count);  // 10 per second, burst 10
  if (admitted === 0) return;                            // excess dropped, silently
  m.acc.counts[type] += admitted;
  if (!m.acc.senders.includes(from)) m.acc.senders.push(from); // ring buffer, last 10

  if (m.timer !== null) return;                          // a flush is already scheduled
  const now = clock.now();
  if (now - m.lastFlushAt >= WINDOW_MS) {
    flush(m, now);                                       // leading edge: quiet room, send now
  } else {
    m.timer = schedule(m.lastFlushAt + WINDOW_MS, () => {
      m.timer = null;
      flush(m, clock.now());                             // trailing edge: next boundary
    });
  }
}

function flush(m: Meeting, now: number) {
  const frame = { meetingId: m.id, tick: ++m.tick, epoch: m.epoch, sentAt: now,
                  counts: m.acc.counts, senders: m.acc.senders.toArray() };
  m.acc = emptyAccumulator();
  m.lastFlushAt = now;
  for (const gw of m.gateways) gw.sendEphemeral(frame); // fan-out per gateway, not per socket
}

Two properties fall out. Flushes are at least 250 ms apart, so a meeting emits at most 4 frames a second and fan-out is capped at 4 × n frames a second regardless of clicking. And a sparse reaction never waits: in a five-person meeting almost every clap arrives in a quiet window and leaves on the leading edge. It costs nothing small and caps everything large.

For Contoso: 4 frames a second × 1,000 participants = 4K frames a second (about 1.2 MB/s) instead of 360K deliveries. That is the 90× in this section's title.

Interviewer: "Why 250 ms? Why not 100, or a full second?"

You: "Because it is perceptual. In a crowd effect, batching under a quarter second is invisible. It also gives a clean ceiling of 4 frames a second per meeting. And it fits the latency budget: uplink 50 ms, gateway to owner 10, owner's coalescing wait up to 250, owner to gateway 10, downlink 50, render about 16. That's about 386 ms worst case for a click that leaves on the client's leading edge. A click merged on the client can wait another 250 ms, about 636 ms. The p95 of 500 ms still holds because the two waits are independent and rarely both max out, and later clicks of a burst land in a wave already on screen. If the click-to-render SLI disagrees, I shrink the client window to 100 ms. A one-second window would blow the budget outright."

The rejected alternatives: forwarding every reaction is O(n² × r), and fixed-interval batching without the leading edge would add up to 250 ms to every sparse small-meeting clap for no benefit. The flip trigger is town-hall scale (10,000), where Section 9 adds gateway pre-aggregation.

Quick check before moving on: