General Purpose GPUs

Question 1

what 3 things are CPUs limited by

Answer 1

1. small number of threads
2. high overhead per thread
3. power and heat constraints

Question 2

describe GPU

Answer 2

• orginally designed for fast 3D graphics rendering
• now used for Ai, simulation, medical imaging & finance
• massive parallelism - thousands of lightweight threads
• ideal for compute-heavy, data-parallel tasks
• enabled by programmer-friendly APIs

Question 3

what is CUDA

Answer 3

• Compute Unified Device Architecture is NVIDIA’s platform for GPGPU
• provides parallel programming model and Apis to harness GPU’s massive core count

Question 4

what is the program structure in CUDA

Answer 4

• programs run partly on the CPU (host) and partly on the GPU (device)
• C/C++ includes:
  
  • host code
  • device code (kernel)
    
    • data transfer code

Question 5

describe kernel

Answer 5

• function written by the programmer to run on the GPU
• when launched, the kernel runs in parallel across many threads
• each thread executed the same kernel code but on different data
• threads must mostly be independent - branching or divergence leads to slower serial execution

Question 6

how are threads launched

Answer 6

blocks and grids
- threads -> blocks
- blocks -> grids
- threads in blocks can share fast memory & sync
- threads execute in groups of 32 = warps
- threads, blocks & grids = software abstraction

Question 7

how do we chose grid & block dimension

Answer 7

• each block runs of a streaming multiprocessor (SM)
• each SM has a max number of threads per block
• too many blocks = idle SMs = wasted performance
• choose based on:
  
  • problem size
  • hardware constraints
    
    • maximising GPU usage

Question 8

what is CPU optimised for

Answer 8

sequential code
- chip = cache & control logic
- handles complex logic

Question 9

what is a GPU optimised for

Answer 9

data-parallel tasks
- chip = processing logic - SIMD
- minimal control logic & cache
- hides memory latency by oversubscribing threads to cores
- designed to maximise FLOPs - graphics & scientific applications

Question 11

how does memory & thread scheduling work with NVIDIA fermi

Answer 11

• global DRAM interfaces surround the chip & connects SMs to external VRAM
• VRAM is GPU’s dedicated memory
• host interface: connects GPU to CPU through PCIe
• GigaThread scheduler assigns thread blocks to SMs

Question 12

why should we maximise warp throughput in GPUs

Answer 12

• GPUs are most effective when many warps are active, maximising CUDA core utilisation
• fermi architecture: 2 warps every 2 clock cycles
• structural hazards - can block warps
• memory latency - can be hidden by having other ready warps available

Question 13

what is the warp execution flow

Answer 13

• each warp executes one instruction at a time
• threads use different execution units:
  
  • ALU ops
  • Loads/store
    
    • Special ops

Question 14

what are the 4 items in CUDA memory hierachy

Answer 14

1. registers
2. L1 cache
3. L2 cache
4. global memory (VRAM)

Question 15

what are registers used for in CUDA

Answer 15

• private to each thread, extremely fast
• used for temporary variables during execution

Question 16

what is L1 cache used for in CUDA memory hierarchy

Answer 16

• shared among threads in SM
• caches global memory to hide latency

Question 17

what is L2 cache used for in CUDA memory hierarchy

Answer 17

• shared across all SMs
• backup if data misses in L1

Question 18

what is VRAM used for in CUDA

Answer 18

• largest but slowest
• shared across the whole CPU

Question 19

how does CUDA use shared memory for thread communication

Answer 19

• registers are private - inaccessible by others
• shared memory is per block - threads in the same block can share data via shared memory
• avoid write hazards: assign specific threads to write to specific shared memory locations
• synchronise threads: use barriers to ensure all writes finish before reads begin

Question 20

describe Intel’s Gen8 GPU execution unit

Answer 20

• basic compute unit
• SMT with 7 hardware threads
• contains two SIMD floating point units
• includes branch and send units for control flow & memory access

Question 21

describe instruction flow in Intel’s Gen8 GPU

Answer 21

• cycle begins: 1 instruction fetched per active thread
• each thread has its own superscalar pipeline

Question 22

describe subslices in Intel’s Gen8 GPU

Answer 22

• includes up to 8 EU and shared resources
• thread dispatcher assigns threads to EUs for execution
• instruction cache is shared - stores program code for active threads

Question 23

describe slice-level organisation in Intel’s Gen8 GPU

Answer 23

• slices group multiple subslices - 24 EUs
• enables sharing of temporary variables across EUs for efficient processing

Question 24

when does a program benefit from GPU acceleration

Answer 24

• GPUs are SIMD processors with hundreds/thousands of lightweight cores
• best suited for tasks with large-scale parallelism
• ideal for processing large data sets concurrently using many lightweight threads