9 - parallel computing Flashcards
Moore’s Law
Number of transistors doubles every ~2 years.
Observation that is not a law
Why parallel computing?
Single core performance increase each year is no longer significant enough after 2002
Why can frequency not be increased?
Physical limits
- Electrical Leakage
- Hear dissipation
- Extreme size of transistors
- Increased cooling costs
FPGA
Field Programmable Gate Array
HPC system components
Compute nodes
Fast interconnect
Parallel filesystem
Scheduler
HPC Programs
Remote, non-interactive usage.
Submit jobs (no interaction), user gets results on completion
Supercomputer
Cabinets with servers
500 rankings
TOP 500 - FLOPS
green 500 - ordered by FLOPS/watts
Is it assumed that time to communicate is completed in parallel in parallel computations?
NO
Assume that a controller cannot communicate to multiple places at once
Compute to communication ratio, in the context of addition on a stack
Give the symbols and expression
α - time to perform single addition
β - time to communicate a stack of numbers
γ = α / β is the ratio
Amdahl’s Law
Bound for max speedup
(1 core runtime)/(p cores runtime)
1/(s+(1-s)/p)
Speedup bottlenecks
Communication
Sync
Not all parts are parallelisable
Amdahl Law Assumptions
Fixed workload
Serial fraction s cannot be parallelised
Parallelisable fraction 1-s CAN be parallelised
Amdahl’s Law infinite core maximum speedup
1/s
Gustafson’s Law definition (question: given the workload on one core….)
Given the amount of workload on one core, how much more can be achieved on p cores
