How to Benchmark Throughput and Latency

Pix2PGP includes a benchmarking suite under software/scripts/benchmarking that measures the maximum trigger rate an ASIC’s Pix2PGP implementation can sustain as a function of hit occupancy, and the total time it takes for a consolidated frame to be transmitted by the FPGA receiver (measured from the SRO trigger).

The suite is driven by a per-ASIC JSON file (asics/<AsicName>.json) and a top-level VHDL testbench with the BENCHMARKING_G generic set to true.

JSON File Format

Each entry in asics/<AsicName>.json has the following structure:

{
  "occ": 80.0,
  "colHits": 512,
  "allHits": 12288,
  "colBusy": 25367,
  "superBusy": 11459,
  "totalLatency": 25796
}

Field meanings:

occ — occupancy percentage
colHits — hits per column at this occupancy (populated by benchmarking.py)
allHits — total hits per lane (populated by benchmarking.py)
colBusy — clock ticks the Column Manager is busy for (measured by the testbench)
superBusy — clock ticks the Column Supervisor is busy for (measured by the testbench)
totalLatency — clock ticks between SRO and the tLast of the final FPGA-side AXI frame (measured by the testbench)

Clock-tick values are measured against the PGP clock — the period must be provided as a command-line argument when producing plots (default: 5.384 ns).

Step 1 — Populate colHits and allHits

From software/scripts/benchmarking:

$ python benchmarking.py --getHitArray --updateJson \
                         --cols=24 --rows=640 \
                         --asicType=SparkPixS

--cols — columns per Pix2PGP lane (NUM_OF_COL_MANAGERS_C)
--rows — total rows per column
--asicType — ASIC name (must match asics/<AsicName>.json)

The script prints two arrays:

_colHitArray — the number of hits per column corresponding to each occ entry in the JSON. E.g. for 224 rows and 1 % occupancy: 224 * 0.01 = 2.24 → the entry becomes 2
_allHitArray — total hits across the entire lane. This is stored in the JSON as metadata and shown in the resulting plot

Step 2 — Run the VHDL Testbench

In firmware/targets/Pix2Pgp<Asic>Emu/tb/Pix2Pgp<Asic>EmuTb.vhd:

Set BENCHMARKING_G = true (change <Asic> accordingly)
Locate the colHitsArray definition at the top-level testbench. Its entries must match the colHits values in the JSON
Note CLK_PERIOD_PGP_C — the Pix2PGP core clock period
Note CLK_PERIOD_SPARSE_C — the User Logic / matrix clock period
Optionally, first run with BENCHMARKING_G = false and inspect the timing between sof/eof/wrEn strobes. Adjust the U_DummyPixel model if the simulated behavior does not match the in-silicon reality
If the frames grow very large at high occupancy, increase AXIS_FIFO_ADDR_WIDTH_G in the FPGA receiver logic (Pix2PgpAsicStreamRx)
Run the testbench (VCS recommended). Watch for messages such as:
```
[INFO]: occ = 1.500000%  superBusyCnt = 253
```
After the sweep, a summary of colBusyCnt, superBusyCnt, and totalLatencyCnt is printed. Copy the last counter values into the matching positions of asics/<AsicName>.json

Step 3 — Produce Plots and Tables

$ python benchmarking.py --verbose \
                         --pgpClkPeriod=5.384 \
                         --matrixClkPeriod=10.768 \
                         --cols=24 --rows=640 \
                         --asicType=SparkPixS

--pgpClkPeriod — PGP clock period in ns (default 5.384). Must match CLK_PERIOD_PGP_C in the testbench
--matrixClkPeriod — matrix / User Logic clock period. Only affects the plot title
--cols / --rows / --asicType — same as Step 1

Outputs

Occupancy-vs-max-trigger-rate plot
A tabular Markdown file (table.md under software/scripts/benchmarking/) summarizing the same numbers

For a full comparison of measured performance across every deployed ASIC, see ASIC Implementations.