## What Is Parallel Computing in Optimization Toolbox?

### Parallel Optimization Functionality

Parallel computing is the technique of using multiple processors on a single problem. The reason to use parallel computing is to speed computations.

The following Optimization Toolbox™ solvers can automatically distribute the numerical estimation of gradients of objective functions and nonlinear constraint functions to multiple processors:

`fmincon`

`fminunc`

`fgoalattain`

`fminimax`

`fsolve`

`lsqcurvefit`

`lsqnonlin`

These solvers use parallel gradient estimation under the following conditions:

You have a license for Parallel Computing Toolbox™ software.

The option

`SpecifyObjectiveGradient`

is set to`false`

, or, if there is a nonlinear constraint function, the option`SpecifyConstraintGradient`

is set to`false`

. Since`false`

is the default value of these options, you don't have to set them; just don't set them both to`true`

.Parallel computing is enabled with

`parpool`

, a Parallel Computing Toolbox function.The option

`UseParallel`

is set to`true`

. The default value of this option is`false`

.

When these conditions hold, the solvers compute estimated gradients in parallel.

**Note**

Even when running in parallel, a solver occasionally calls the objective and nonlinear constraint functions serially on the host machine. Therefore, ensure that your functions have no assumptions about whether they are evaluated in serial or parallel.

### Parallel Estimation of Gradients

One solver subroutine can compute in parallel automatically: the subroutine that
estimates the gradient of the objective function and constraint functions. This
calculation involves computing function values at points near the current location
*x*. Essentially, the calculation is

$$\nabla f(x)\approx \left[\frac{f(x+{\Delta}_{1}{e}_{1})-f(x)}{{\Delta}_{1}},\frac{f(x+{\Delta}_{2}{e}_{2})-f(x)}{{\Delta}_{2}},\dots ,\frac{f(x+{\Delta}_{n}{e}_{n})-f(x)}{{\Delta}_{n}}\right],$$

where

*f*represents objective or constraint functions*e*are the unit direction vectors_{i}Δ

_{i}is the size of a step in the*e*direction_{i}

To estimate ∇*f*(*x*) in parallel, Optimization Toolbox solvers distribute the evaluation of (*f*(*x* + Δ_{i}*e*_{i})
–
*f*(*x*))/Δ_{i} to extra processors.

#### Parallel Central Differences

You can choose to have gradients estimated by central finite differences instead of the default forward finite differences. The basic central finite difference formula is

$$\nabla f(x)\approx \left[\frac{f(x+{\Delta}_{1}{e}_{1})-f(x-{\Delta}_{1}{e}_{1})}{2{\Delta}_{1}},\dots ,\frac{f(x+{\Delta}_{n}{e}_{n})-f(x-{\Delta}_{n}{e}_{n})}{2{\Delta}_{n}}\right].$$

This takes twice as many function evaluations as forward finite differences, but is usually much more accurate. Central finite differences work in parallel exactly the same as forward finite differences.

Enable central finite differences by using `optimoptions`

to set the `FiniteDifferenceType`

option to
`'central'`

. To use forward finite differences, set the
`FiniteDifferenceType`

option to
`'forward'`

.

### Nested Parallel Functions

Solvers employ the Parallel Computing Toolbox function `parfor`

(Parallel Computing Toolbox) to perform parallel
estimation of gradients. `parfor`

does not work in parallel when
called from within another `parfor`

loop. Therefore, you cannot
simultaneously use parallel gradient estimation and parallel functionality within
your objective or constraint functions.

**Note**

The documentation recommends not to use `parfor`

or
`parfeval`

when calling Simulink^{®}; see Using sim Function Within parfor (Simulink). Therefore, you might
encounter issues when optimizing a Simulink simulation in parallel using a solver's built-in parallel
functionality.

Suppose, for example, your objective function `userfcn`

calls
`parfor`

, and you wish to call `fmincon`

in a loop. Suppose also that the conditions for parallel gradient evaluation of
`fmincon`

, as given in Parallel Optimization Functionality, are satisfied. When parfor Runs In Parallel shows three cases:

The outermost loop is

`parfor`

. Only that loop runs in parallel.The outermost

`parfor`

loop is in`fmincon`

. Only`fmincon`

runs in parallel.The outermost

`parfor`

loop is in`userfcn`

.`userfcn`

can use`parfor`

in parallel.

**When parfor Runs In Parallel**

## See Also

Using Parallel Computing in Optimization Toolbox | Improving Performance with Parallel Computing | Minimizing an Expensive Optimization Problem Using Parallel Computing Toolbox