Given a more powerful A$100$ GPU, using FP$16$ tensor core, we run one common workload $--$ a seven billion parameter LLM

Calculate: $1)$ the operations per byte $($ops:byte$)$ ratio

The arithmetic intensity. Arithmetic intensity is the number of compute operations an algorithm takes divided by the number of byte accesses it requires and is a

hardware$-$agnostic measurement.

to make the most of the compute capacity that we're paying for, how many requests do we want to batch a time during inference to fill our KV cache to increase the

throughput?