Design Top-K System — Section 3: Data Modeling, System Characteristics & API Design

Goal of this section: Define how data is modeled, stored, partitioned, and accessed efficiently for Top-K queries.

1. System Characteristics

Before modeling data, clarify access patterns:

1.1 Read Patterns

• Very high QPS
• Same Top-K lists repeatedly requested
• Hot keys (global, US, popular categories)
• Mostly read-only during serving

1.2 Write Patterns

• Continuous event ingestion
• Frequent aggregate updates
• Append-only raw data
• Occasional recomputation / backfill

1.3 Data Lifecycle

1. Raw events ingested
2. Aggregates updated (streaming)
3. Ranked Top-K lists materialized
4. Served from cache / store
5. Periodic recompute from raw events if needed