Supported Platform: Linux® only.
Before you can interactively debug your applications using the MATLAB® API for Spark™, you must configure your MATLAB environment by adding the location of the Spark assembly jar to the front of MATLAB's static Java class path.
A Spark assembly JAR file includes all the Spark dependencies, including some internal Hadoop® dependencies. This JAR file is created during Spark installation and is specific to a particular combination of
Spark and Hadoop versions. For example, using Spark version 1.4.0 and Hadoop version 2.4.0 creates the file
To add the location of the Spark assembly jar to the front of MATLAB’s static Java® class path, use the following procedure:
Open a Linux terminal.
Navigate to the directory
location where MATLAB is
Set the environment variable
SPARK_HOME to point to the
Spark installation directory. For example:
|zsh / csh / tcsh||
% setenv SPARK_HOME /share/spark
$ export SPARK_HOME=/share/spark
From the directory
execute the shell script
This will create a file called
directory. The contents of a sample
file look like
<before> /share/spark/conf/ /share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jar /share/spark/lib/datanucleus-rdbms-3.2.9.jar /share/spark/lib/datanucleus-api-jdo-3.2.6.jar /share/spark/lib/datanucleus-core-3.2.10.jar
<before> tag ensures that
spark-assembly-1.4.0-hadoop2.4.0.jar is added to the
front of the Java class path.
Start MATLAB from the
You can create the
javaclasspath.txt file in a directory
-o option. The file must be named
$ ./generatesparkjcpath -o /home/user/javaclasspath.txt
You can verify whether
added to the front of the Java class path by typing the following at the MATLAB command
>> spath = javaclasspath('-static') ; >> spath(1:5)
/share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jaron the top of the list.
You need to complete this configuration just once. Subsequent usage only requires
that you start MATLAB from the folder
You may instead place the generated
prefdir. If you do so, MATLAB adds the necessary libraries to the static path at launch.