Multi-Processing

Question 1

What are Flynn’s Taxonomy of Parallel Machines?

Answer 1

Classified by number of instructions streams and number of data streams.
SISD (uniprocessor)
SIMD (vector processor)
MISD (stream processor?)
MIMD (Multi-processor)

Question 2

What are the limitations of uniprocessors?

Answer 2

• Diminishing returns with increasing issue-width.

- increasing frequency increases the power cubically

Question 3

What are the types of multi-processors?

Answer 3

• Centralized Shared Memory

- Distributed Memory

Question 4

What is centralized shared memory good for and why?

Answer 4

Memory access is slow and can only handle a certain number of accesses per second => good for small machines < 16

Question 5

What is distributed memory?

Answer 5

Each core has its own memory and must communicate with other cores via a NIC. Must rely on message passing

Question 6

Pros/cons of message passing vs shared memory?

Answer 6

• Message passing is difficult to program correctly and be performant, but once it’s correct, it’s usually easy to be performant
• Shared memory is less difficult to program correctly, but much harder to be performant

Question 7

What is SMP?

Answer 7

Shared Multi-Processor. Multiple cores with one unified address space

Question 8

What is hyperthreading?

Answer 8

AKA Simultaneous multithreading (SMT) can execute instructions from multiple threads in the same cycle (requires most hardware support)

Question 9

What are the benefits of multithreading in cores?

Answer 9

• Can avoid overhead of thread scheduling
• Can take advantage of under-utilized hardware / idle time when scheduling just one thread
• More efficient / lower cost than have two cores

Question 10

What hardware changes are required for SMT?

Answer 10

Need multiple PCs and extra logic before RSes (extra RFs, RATs). RSes and ROB stays the same, ROB just has interleaved values from both threads

Question 11

Why is VIVT caching bad for SMT?

Answer 11

Both threads have the same address space!

Question 12

How do we do caching for SMT?

Answer 12

• use PIPT, VIPT

- Make TLB thread-aware

Question 13

What are the pros/cons for SMT w.r.t. caching?

Answer 13

(+) good data sharing between threads (since you will get a cache hit even if your thread hasn’t accessed the data yet)
(-) cache capacity and associativity is shared

Question 14

What is cache thrashing?

Answer 14

when the data required by both threads > cache size => get lots of cache misses