Event Stream Processing

Read and write events, such as Kafka^® events

When you install the Streaming Data Framework for MATLAB^® Production Server™, you can:

Read and write event streams from an event streaming platform, such as Kafka.
Simulate the production environment for testing your streaming analytic algorithms. (Requires MATLAB Compiler SDK™)
Scale your streaming analytic algorithms by deploying them. (Requires MATLAB Production Server)

Install the Streaming Data Framework for MATLAB Production Server support package from the MATLAB Add-On Explorer. For information about installing add-ons, see Get and Manage Add-Ons (MATLAB).

Functions

expand all

Connect to Event Streams

`kafkaStream`	Create connection to event stream in Kafka topic (Since R2022b)
`inMemoryStream`	Create connection to event stream hosted by MATLAB without schema processing applied (Since R2022b)
`testStream`	Create connection to event stream hosted by MATLAB with schema processing applied (Since R2022b)

Read and Write Events

Basic Import and Export

`readtimetable`	Read timetable from event stream (Since R2022b)
`writetimetable`	Write timetable to event stream (Since R2022b)
`seek`	Set read position in event stream (Since R2022b)
`preview`	Preview subset of events from event stream (Since R2022b)
`identifyingName`	Event stream name (Since R2022b)

Import and Export Options

`detectImportOptions`	Create import options based on event stream content (Since R2022b)
`eventStreamImportOptions`	Create options for importing events from stream into MATLAB (Since R2022b)
`ImportOptions`	Import options for event stream (Since R2022b)
`getvartype`	Data types used to export variables to stream (Since R2022b)
`setvartype`	Set data types used to import and export variables to stream (Since R2022b)
`detectExportOptions`	Create export options based on event stream content (Since R2022b)
`ExportOptions`	Export options for event stream (Since R2022b)

Kafka Stream Operations

`readevents`	Read raw events from Kafka stream without schema processing applied (Since R2022b)
`flush`	Reset read window boundaries (Since R2022b)
`stop`	Stop processing event streams from Kafka topic (Since R2022b)
`createTopic`	Create topic in Kafka cluster (Since R2022b)
`deleteTopic`	Remove topic from Kafka cluster (Since R2022b)
`loggederror`	Error information for Kafka stream operation (Since R2022b)

Kafka Provider Properties

`categoryList`	Kafka stream provider property list (Since R2022b)
`getProviderProperties`	Kafka stream configuration property data (Since R2022b)
`setProviderProperties`	Set properties specific to Kafka configuration (Since R2022b)
`isProperty`	Determine if Kafka stream provider property is set (Since R2022b)

Simulate Production Environment

`eventStreamProcessor`	Apply stream analytic function to event stream (Since R2022b)
`execute`	Execute event stream processing function on specific number of event windows (Since R2022b)
`package`	Package stream processing function into deployable archive configured by `EventStreamProcessor` (Since R2022b)
`seek`	Set position in event stream to begin processing events (Since R2022b)
`start`	Start processing event streams using local test server (Since R2022b)
`startServer`	Start local test server (Since R2022b)
`stop`	Stop processing event streams using local test server (Since R2022b)
`stopServer`	Shut down local test server (Since R2022b)

Deploy Streaming Analytic

streamingDataCompiler Package stream processing function into deployable archive (Since R2022b)

Topics

Process Event Streams

Streaming Data Framework for MATLAB Production Server Basics
Use Streaming Data Framework for MATLAB Production Server to read from and write to event streaming platforms, such as Kafka.
Process Kafka Events Using MATLAB
Process events from a Kafka stream using a streaming analytic function.
Connect to Secure Kafka Cluster
Set Kafka provider properties during stream object creation.

Test and Deploy Streaming Analytic Function

Test Streaming Analytic Function Using Local Test Server
Use the development version of MATLAB Production Server to test a streaming analytic function before deployment to MATLAB Production Server.
Deploy Streaming Analytic Function to MATLAB Production Server
Package a streaming analytic function and deploy to server.

Troubleshooting

Obtain Kafka Event Stream Log Files

Locate event stream log files to help with debugging issues.