Main Content

Preprocessing Data

Data cleaning, smoothing, grouping

Data can require preprocessing techniques to ensure accurate, efficient, or meaningful analysis. Data cleaning refers to methods for finding, removing, and replacing bad or missing data. Detecting local extrema and abrupt changes can help to identify significant data trends. Smoothing and detrending are processes for removing noise and polynomial trends from data, while scaling changes the bounds of the data. Grouping and binning methods identify data characteristics by groups.

Live Editor Tasks

Clean Missing DataFind, fill, or remove missing data in the Live Editor
Clean Outlier DataFind, fill, or remove outliers in the Live Editor
Find Change PointsFind abrupt changes in data in the Live Editor
Find Local ExtremaFind local maxima and minima in the Live Editor
Smooth DataSmooth noisy data in the Live Editor
Remove TrendsRemove polynomial trend from data in the Live Editor
Join TablesCombine two tables using key variables in the Live Editor

Functions

expand all

ismissingFind missing values
rmmissingRemove missing entries
fillmissingFill missing values
missingCreate missing values
standardizeMissingInsert standard missing values
isoutlierFind outliers in data
filloutliersDetect and replace outliers in data
rmoutliersDetect and remove outliers in data
movmadMoving median absolute deviation
ischangeFind abrupt changes in data
islocalminFind local minima
islocalmaxFind local maxima
smoothdataSmooth noisy data
movmeanMoving mean
movmedianMoving median
detrendRemove polynomial trend
normalizeNormalize data
rescaleScale range of array elements
discretizeGroup data into bins or categories
groupcountsNumber of group elements
groupfilterFilter by group
groupsummaryGroup summary computations
grouptransformTransform by group
histcountsHistogram bin counts
histcounts2Bivariate histogram bin counts
findgroupsFind groups and return group numbers
splitapplySplit data into groups and apply function
rowfunApply function to table or timetable rows
varfunApply function to table or timetable variables
accumarrayConstruct array with accumulation

Topics

Clean Messy and Missing Data in Tables

This example shows how to find, clean, and delete table rows with missing data.

Detrending Data

Remove linear trends from data.

Grouping Variables To Split Data

You can use grouping variables to categorize data variables.

Split Data into Groups and Calculate Statistics

This example shows how to group data and apply statistics functions to each group.

Split Table Data Variables and Apply Functions

This example shows how to group data variables and apply functions to each group.

Featured Examples