parallelize

R2026b

Class: matlab.compiler.mlspark.SparkContext
Namespace: matlab.compiler.mlspark

(To be removed) Create an RDD from a collection of local MATLAB values

matlab.compiler.mlspark.SparkContext.parallelize will be removed in a future release. There is no replacement functionality. For more information, see Version History.

Syntax

rdd = parallelize(sc,cellArray)
rdd = parallelize(sc,cellArray,numSlices)


Description

rdd = parallelize(sc,cellArray) creates an RDD from a collection of local MATLAB^® values grouped as a cell array.

rdd = parallelize(sc,cellArray,numSlices) creates an RDD with the number of partitions specified by numSlices.

Input Arguments

sc — SparkContext to use

SparkContext object

The SparkContext to use, specified as a SparkContext object.

cellArray — Collection of values

cell array

A collection of values, specified as a MATLAB cell array.

Data Types: cell

numSlices — Number of partitions to create

scalar

Number of partitions to create, specified as a scalar.

Data Types: double

Output Arguments

rdd — Output RDD created from the collection of values

RDD object

An output RDD, created from the collection of values, and returned as an RDD object.

Examples

Create an RDD From MATLAB Values

Create an RDD from local MATLAB values.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% parallelize
x = sc.parallelize({1, 2, 3, 4, 5});
y = x.count()

Version History

Introduced in R2016b

R2026b: To be removed

Support for deploying applications using the MATLAB API for Spark™ will be removed in a future release. There is no replacement functionality.