Task Parallelism and Manager-Worker Parallelism

Question 1

What do MPI_ANY_SOURCE and MPI_ANY_TAG do? What can they be used for?

Answer 1

• MPI_ANY_SOURCE allows receiving from an unspecified source
• MPI_ANY_TAG allows receiving from an unspecified tag

In a manager/worker model, we don’t know which worker will request work next

Question 2

Why define a new communicator instead of using MPI_COMM_WORLD?

Answer 2

To limit communication to a subset of processes.

For example all the worker processes (but not the manager process)

Question 3

What are the steps to create a new communicator?

Answer 3

1. Get world group handle (MPI_Comm_group)
2. Create a new group (MPI_Group_incl() or MPI_Group_excl())
3. Create a communicator (MPI_Comm_create())
4. Get rank in new communicator (MPI_Comm_rank())
5. Perform communication
6. Free the group and communicator (MPI_Group_free(), MPI_Comm_free())

Question 4

What does MPI_Group_incl do?

Answer 4

Creates a new group from an existing group including specific processes.

Question 5

What does MPI_Group_excl do?

Answer 5

Creates a new group from an existing group excluding specific processes.

Question 6

What does MPI_Group_union do?

Answer 6

Combines the union two groups members.

Question 7

What does MPI_Group_intersection do?

Answer 7

Creates a group containing only common members.

Question 8

What does MPI_Group_difference do?

Answer 8

Creates a group containing processes in the first group but not in the second.

Question 9

How does the Manager-Worker model work in MPI?

Answer 9

1. The manager assigns work to worker processes
2. Workers request more work when finished

Question 10

How do workers signal completion?

Answer 10

Workers send a request, receive a task, and exit if the received task is a special end flag.

Question 11

What are two OpenMP work-sharing constructs besides parallel for? When is each used?

Answer 11

1. sections - When there are multiple, independent, blocks of code that can execute concurrently
2. single - Ensures that only thread executes a block of code inside a parallel region

Question 12

How does OpenMP handle tasks?

Answer 12

One thread generates tasks, and other threads execute them.

Question 13

What OpenMP directive ensures only one thread generates tasks?

Answer 13

#pragma omp single

Question 14

When using a single construct, which thread executes the code?

Answer 14

It can be any thread