foldByKey

Class: matlab.compiler.mlspark.RDD
Namespace: matlab.compiler.mlspark

Merge the values for each key using an associative function and a neutral “zero value”

Syntax

result = foldByKey(obj,zeroValue,func,numPartitions)


Description

result = foldByKey(obj,zeroValue,func,numPartitions) merges the values for each key in obj using an associative function func and a neutral “zero value” represented by zeroValue. The input argument numPartitions specifies the number of partitions to create in the resulting RDD.

Input Arguments

obj — Input RDD

RDD object

An input RDD, specified as a RDD object.

zeroValue — Neutral “zero value”

cell array of numbers

A neutral “zero value”, specified as a cell array of numbers.

Data Types: cell

func — Function for folding the values of each key

function handle

Function for folding the values of each key, specified as a function handle.

Data Types: function_handle

numPartitions — Number of partitions to create

scalar value

Number of partitions to create, specified as a scalar value.

Data Types: double

Output Arguments

result — RDD containing the aggregation result

RDD object

A pipelined RDD containing the aggregation result, returned as a RDD object.

Examples

Merge Values for Each Key

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% foldByKey
x = sc.parallelize({ {'a',1}, {'b',1}, {'a',1} });
y = x.foldByKey(5, @(x,y)(x+y));
viewRes = y.collect() % {{'a',7},{'b',6}}

Version History

Introduced in R2016b