Nvidia GPU's
Nvidia GPU's have different hardware and CUDA abstractions. CUDA is the software platform. Hardware abstraction isn't exactly the right word to use – it's really the hardware layout and terminology associated with it.
Hardware Layout and Terminology
- Graphics Processing Clusters
- Streaming Multiprocessors
- CUDA/Tensor Cores
Software
- Grid
- Thread Blocks – assigned to Streaming Multiprocessors – this is the software/hardware interaction bridge.
- Threads - directly run the
Scheduling Terminology
- Warps – groups of 32 threads scheduled by an SM under the SIMT (Single Instruction, Multiple Threads) model – they share a program counter and execute in lockstep.