I started this to learn more about chips and the hardware side of AI, and to try Astra and Opus 5.5 for 3D modeling. I wanted something I could move around and take apart to better understand how the parts fit together.
Compute needs operands → caches reuse them → HBM supplies data → dense connections carry it → power and cooling sustain the work.
Illustrative models, not production CAD. Layouts and proportions are simplified.Model sources
The learning work
Questions behind the model
Mechanisms behind the model: data reuse, dense connections, and the cost of working across chips.
01GPU memory
More compute is not always the answer
A processor can have room to do more calculations and still be waiting for data. That makes memory bandwidth part of the explanation, not just a specification to list beside compute performance.
What explains itArithmetic intensity measures work per byte transferred. Low reuse can leave compute waiting; batching and caching can increase reuse.
How much work can each byte support?
H100 SXM example: 989 TFLOP/s dense BF16 ÷ 3.35 TB/s ≈ 295 FLOP/byte. The bend is where memory bandwidth meets the compute ceiling. Both axes use logarithmic spacing; this is a theoretical bound, not a benchmark.
1 FLOP/byte → at most 3.4 TFLOP/s in this simplified model.
Assumes two-byte weights, two FLOPs per multiply-add and one weight read shared across the batch. Ignores activation/KV traffic, cache effects, communication and launch overhead. Higher batch size is not a latency guarantee.
The interposer connects the compute dies and memory. The substrate connects the package to the wider board. Separating these layers in the model makes their different jobs easier to see.
What explains itSmaller dies can improve die yield and mix process nodes. Crossing die boundaries adds energy and latency, and assembly adds cost and yield risk. MI300X exposes the extra I/O layer in the architecture comparison.
Host software schedules GPU work. Memory holds the working data, networks connect systems, and power and cooling support the task throughout. The walkthrough separates these roles without treating every arrow as a physical wire.
What explains itCalculation and exchange repeat within distributed execution. The frequency depends on tensor, pipeline or other parallelization choices; communication can become the limiting factor.
The models cover a GPU package and a compute tray. I’m using them to look at where memory sits, how the chips connect, and how power and cooling support the system.
Trying Astra and Opus 5.5 in 3D
I wanted to see how well they could work from hardware references, get the details into a model, and make the parts interactive. Still checking the models against the sources and figuring out what needs to be corrected.
What I’ve added so far
Selectable parts.Started with layers and components I could select and inspect.
Notes on what they do.Added explanations of what each part does and why it’s there.
A compute-tray walkthrough.Added a way to follow how the parts work together through an AI request.
Built with React and Three.js. Models and arrows are teaching aids; they do not reproduce processing time or every physical connection.