Inside a Transformer
Follow inputs through attention and MLPs to the next token. Change the numbers and see what happens.
Models, hardware and infrastructure. Hands-on courses and labs you can explore in your browser.
Follow inputs through attention and MLPs to the next token. Change the numbers and see what happens.
Write and optimize kernels in your browser, then move to CUDA and model inference.
Follow power, cooling and networking from the substation to the rack, then learn how to choose a site.
Trace a 5 MW cluster’s water and power from the systems to the plant.
Learn who orders, installs, configures and tests the network, from purchase order to burn-in.
Understand OEMs, ODMs, allocation and the steps to getting racks delivered.
Pull out compute trays, reveal the chips and cooling, and follow a request through the rack.
Wire an inference cluster into a data center. Choose optics, connect handoffs and test redundancy.
Pipe and power an inference cluster, then test how the system handles failures.
Walk the facility, open racks and inspect the hardware, cooling and electrical systems.