Is ChatGPT 175 Billion Parameters? Technical Analysis

Analyzing the memory bandwidth of the A100 GPU, it’s evident that the actual speed of inference from the API is much faster than the theoretically-optimal inference speed for a 175 billion dense equivalent parameter model