BurstGPT Ordered A/B/C RPS Regimes
This is the renamed version with labels in increasing RPS order:
A / low
B / mid-high
C / high
Each 10-minute window is split into `A1/A2`, `B1/B2`, and `C1/C2`.
How These Windows Were Created
The source data is the BurstGPT request trace stored under traces/burstgpt/data/, specifically the cleaned no-failure CSVs BurstGPT_without_fails_1.csv and BurstGPT_without_fails_2.csv. BurstGPT is a real request-arrival workload trace: each row describes one request with an arrival timestamp, model/application label, input token count, output token count, total token count, and log type. It does not include deployment configuration, hardware, engine settings, or observed request duration.
To create benchmark windows, the trace was treated as a continuous arrival stream and scanned with 10-minute candidate windows. For each candidate, requests were counted to estimate average RPS, and the within-window arrival shape was measured with 10-second buckets. The goal was not to pick the highest bursts, but to find three reasonably stable operating regimes that can be split into optimization and held-out halves.
Candidate windows were ranked using:
half_ratio: similarity between the first 5 minutes and second 5 minutes. Values closer to 1.0 are better.
bucket10_cv: coefficient of variation of 10-second RPS buckets. Lower is steadier.
spike_ratio: peak short-window RPS divided by average RPS. Values closer to 1.0 have fewer spikes.
stability_score: a heuristic combination of the imbalance, 10-second variability, and spike penalties. Lower means more stable.
The final labels are ordered by load: A is the low-RPS window, B is the mid-high window around 2 RPS, and C is the high-RPS window around 10 RPS. Each 10-minute window was split into two 5-minute halves (A1/A2, B1/B2, C1/C2). The AIPerf CSVs in traces/burstgpt/aiperf_windows/ preserve the original within-window arrival timing and token lengths, with timestamps rebased to the start of each selected segment.
Regime Mapping
| regime | window | start_s | end_s | rps | use |
|---|---|---|---|---|---|
| low | A | 10,300,200 | 10,300,800 | 0.218 | low-load stable traffic |
| mid-high | B | 5,389,800 | 5,390,400 | 2.002 | moderate/high traffic around 2 RPS |
| high | C | 849,000 | 849,600 | 10.340 | high-load stable traffic |
Selected 10-Minute Windows
| window_id | rps | stability_score | half_ratio | bucket10_cv | spike_ratio |
|---|---|---|---|---|---|
| 17,167 | 0.218 | 0.637 | 0.926 | 0.353 | 1.832 |
| 8,983 | 2.002 | 0.231 | 0.988 | 0.145 | 1.299 |
| 1,415 | 10.340 | 0.139 | 0.954 | 0.066 | 1.103 |
Segment Statistics
| segment | start_s | end_s | n | rps | input_mean | input_p50 | input_p95 | output_mean | output_p50 | output_p95 | total_tps | rps_10s_cv | rps_10s_max | api_log_pct | gpt4_pct |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | 10,300,200 | 10,300,800 | 131 | 0.218 | 289.321 | 306.000 | 351.500 | 2,571.1 | 2,713.0 | 3,856.5 | 624.535 | 0.353 | 0.400 | 100.000 | 12.977 |
| A1 | 10,300,200 | 10,300,500 | 63 | 0.210 | 291.254 | 308.000 | 357.500 | 2,585.7 | 2,664.0 | 3,928.1 | 604.163 | 0.402 | 0.400 | 100.000 | 12.698 |
| A2 | 10,300,500 | 10,300,800 | 68 | 0.227 | 287.529 | 306.000 | 347.650 | 2,557.6 | 2,721.5 | 3,617.6 | 644.907 | 0.305 | 0.400 | 100.000 | 13.235 |
| B | 5,389,800 | 5,390,400 | 1,201 | 2.002 | 218.207 | 213.000 | 257.000 | 10.109 | 7.000 | 7.000 | 457.013 | 0.145 | 2.600 | 98.251 | 0.083 |
| B1 | 5,389,800 | 5,390,100 | 604 | 2.013 | 217.922 | 211.500 | 255.850 | 9.248 | 7.000 | 7.000 | 457.370 | 0.136 | 2.600 | 98.675 | 0.000 |
| B2 | 5,390,100 | 5,390,400 | 597 | 1.990 | 218.496 | 214.000 | 257.400 | 10.980 | 7.000 | 7.000 | 456.657 | 0.155 | 2.400 | 97.822 | 0.168 |
| C | 849,000 | 849,600 | 6,204 | 10.340 | 176.167 | 89.000 | 112.000 | 10.664 | 6.000 | 6.000 | 1,931.8 | 0.066 | 11.400 | 99.742 | 0.113 |
| C1 | 849,000 | 849,300 | 3,029 | 10.097 | 176.364 | 90.000 | 110.000 | 10.751 | 6.000 | 6.000 | 1,889.2 | 0.074 | 11.400 | 99.703 | 0.066 |
| C2 | 849,300 | 849,600 | 3,175 | 10.583 | 175.980 | 89.000 | 113.000 | 10.581 | 6.000 | 6.000 | 1,974.4 | 0.050 | 11.400 | 99.780 | 0.157 |
Suggested Combinations
ID transfer: `A1 -> A2`, `B1 -> B2`, `C1 -> C2`.
OOD transfer: `A1 -> B2`, `B1 -> C2`, `C1 -> A2`.
Three-regime robustness: optimize on `A1+B1+C1`, test on `A2+B2+C2`.
Plots
