spmdSend

Send data to another worker in spmd block

Since R2022b

Syntax

spmdSend(A,destination)

spmdSend(___,tag)

Description

spmdSend(A,destination) sends data A from the current worker in an spmd block or communicating job to the workers specified by destination.

When you offload computations using parfor and parfeval, only one worker at a time runs each computation. These workers are independent and do not communicate with each other. If you apply spmdSend to these workers, the function has no effect.

To use spmdSend, the number of workers running the current spmd block must be greater than 1. To get the number of workers running the current spmd block, use the spmdSize function.

spmdSend(___,tag) sends data with the tag tag. When you use spmdSend to send data between workers, multiple items of data can wait to be collected. When you send multiple items of data to a worker, add a tag to each item to distinguish between the items.

Examples

Send Data Between Workers in spmd Block

This example shows how to send data between workers in an spmd block or communicating job.

Create a parallel pool with four workers.

parpool(4);

When you execute an spmd block after creating a parallel pool, by default all available workers in the pool run the code inside the spmd block.

Create an spmd block. On the worker whose index is equal to 1, create an array. Use spmdSend to send the array to the worker whose index is equal to 2 and use spmdReceive to collect the data.

spmd
    switch spmdIndex
        case 1
            A = magic(3)
            spmdSend(A,2);
        case 2
            B = spmdReceive
    end
end

Worker 1: 
  
  A =
  
       8     1     6
       3     5     7
       4     9     2
  
Worker 2: 
  
  B =
  
       8     1     6
       3     5     7
       4     9     2
  

Tag and Send Data Between Workers in spmd Block

This example shows how to tag and send data between workers in an spmd block or communicating job.

Create a parallel pool with four workers.

parpool(4);

Create an spmd block. On the worker whose index is equal to 1, create two arrays A1 and A2. Before and between creating arrays, pause the execution using the pause function to simulate some work. Then, use spmdSend to send the matrices to the worker whose index is equal to 2.

Tag each matrix with an integer. On the worker whose index is equal to 2, use spmdReceive to collect the data.

tic
spmd
    switch spmdIndex
        case 1
            pause(5);
            A1 = magic(1)
            pause(5);
            A2 = magic(2)
            spmdSend(A1,2,1);
            spmdSend(A2,2,2);
        case 2
            B1 = spmdReceive('any',1)
            B2 = spmdReceive('any',2)
  
    end
end
toc

Worker 1: 
  
  A1 =
  
       1
  
  
  A2 =
  
       1     3
       4     2
  
Worker 2: 
  
  B1 =
  
       1

  B2 =
  
       1     3
       4     2
  
Elapsed time is 10.061523 seconds.

In some cases, you can improve the performance of your code by moving some work from one worker to another. Move some work from the worker whose index is equal to 1 to the worker whose index is equal to 2. When you use tags, you can easily move calculations from one worker to another without updating code on the receiving worker.

tic
spmd
    switch spmdIndex
        case 1
            pause(5);
            A1 = magic(1)
            spmdSend(A1,2,1);
        case 2
            B2 = spmdReceive('any',2)
            B1 = spmdReceive('any',1)
        case 3
            pause(5);
            A2 = magic(2)
            spmdSend(A2,2,2);
    end
end
toc

Worker 1: 
  
  A1 =
  
       1
  
Worker 2: 
  
  B2 =
  
       1     3
       4     2
  
  
  B1 =
  
       1
  
Worker 3: 
  
  A2 =
  
       1     3
       4     2
  
Elapsed time is 5.117787 seconds.

Input Arguments

A — Data to send

MATLAB variable

Data to send from the current worker, specified as a scalar, vector, matrix, multidimensional array, table, timetable, or any other MATLAB variable.

Example: magic(3)

destination — Indices of workers receiving data

positive integer scalar | vector of positive integers

Indices of the workers receiving data, specified as a positive integer or vector of positive integers. The indices must be less than or equal to the number of workers running the current spmd block or communicating job.

Example: [2 3 4]

tag — Message tag

0 (default) | nonnegative integer

Message tag, specified as nonnegative integer. When you specify this input, spmdSend sends data with this tag. If you receive the data on another worker using spmdReceive, that function returns the tag in its tagOut output.

Example: 314159

Tips

1. Tags have many uses, for example:

   • Use tags to save memory by only loading arrays on workers when you need the data.

   • Use tags to create code that does not depend on the index of the sending worker.

2. A worker that sends data using spmdSend might finish execution before the receiving worker receives the data. When you need synchronized workers in an spmd block or communicating job, such as when you close a shared resource, use spmdBarrier after calling spmdSend and spmdReceive.

Extended Capabilities

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

The spmdSend function fully supports GPU arrays. To run the function on a GPU, specify the input data as a gpuArray. For more information, see Run MATLAB Functions on a GPU.

If data is a gpuArray, data transfer between multiple GPUs in a parallel pool uses fast peer-to-peer communication, including NVLink, if available.

Version History

Introduced in R2022b