Code covered by the BSD License  

Highlights from
TIMEIT Benchmarking Function

5.0

5.0 | 7 ratings Rate this file 147 Downloads (last 30 days) File Size: 4.54 KB File ID: #18798

TIMEIT Benchmarking Function

by

 

17 Feb 2008 (Updated )

TIMEIT.M measures the time required to call a user-specified function

Editor's Notes:

This file was selected as MATLAB Central Pick of the Week

| Watch this File

File Information
Description

T = TIMEIT(F) measures the time (in seconds) required to run F, which is a function handle.

TIMEIT handles automatically the usual benchmarking procedures of "warming up" F, figuring out how many times to repeat F in a timing loop, etc. TIMEIT uses a median to form a reasonably robust time estimate.

UPDATED 16-May-2010: New option to specify number of output arguments to call F with. Overhead measurements now cached so most timings run faster.

UPDATED 31-Dec-2008: More accurate when timing very fast functions; warns you when the reported time might be affected by time-measurement overhead; calls F fewer times when F takes more than a few seconds to run.

MATLAB release MATLAB 7.5 (R2007b)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (11)
11 Mar 2013 Jan Motl

It would be great if the function returned also confidence interval.

14 Jul 2011 Steve Eddins

Andy, I don't fully understand your question. What do you mean by "warm in the time," for example? But I would guess that you're looking for something like this:

x = zeros(1024, 1024);
timeit(@() fft2(x))

12 Jul 2011 James

How do i use timeit to calculate iterartion times to run a for loop instead of using tic toc so i don't have the warm in the time? E.g.

x = zeros(1024,1024);
tic
for i = 1:200;
y = fft2(x)
end
toc

21 Oct 2010 Steve Hoelzer  
19 Aug 2010 David Romero-Antequera  
21 Apr 2009 Joao Henriques

This is extremely useful, but since it requires a function definition, you must either use lambdas, which don't allow procedural code (ie, a list of statements), define a local function, which means you have to turn your script into a function file (sometimes not desirable when dealing with workspace variables), or create a new file (lots of throwaway files with simple routines).

I wonder if it would be possible to add the following functionality (either in the same function or through a different one) :

while timeit %returns true when a new measurement is needed
... my procedural code ...
... multiple lines, will run as many times as timeit() sees fit ...
end
result = timeit; %next call: return result. reset state.

As you can see, this sort of usage would be tremendously useful to just profile a piece of code with no extra details -- just wrap it in a loop. Of course, it's only fair if you conclude it's not worth the additional effort :)

03 Mar 2009 Steve Eddins

Paolo, thanks for the suggestion. I'll look into it.

24 Feb 2009 Paolo de Leva

A suggestion to improve accuracy when you measure small execution times. I believe it would be useful to subtract from median(times) the time needed for the inner iterations (“offset”). Here is the code:

[…omissis…]
offsets = zeros(num_outer_iterations, 1);
times = offsets;
for k = 1:num_outer_iterations
t1 = tic;
for p = 1:num_inner_iterations
end
offsets(k) = toc(t1);
end

for k = 1:num_outer_iterations
t1 = tic;
for p = 1:num_inner_iterations
[outputs{:}] = f();
end
times(k) = toc(t1);
end

t = (median(times-offsets)) / num_inner_iterations;
[…omissis…]

This offset is not always negligible, especially when you want to time the execution of very quick commands or functions. Notice that the value of each element of variable “offsets” also include the time needed to execute

t1 = tic; times(k) = toc(t1);

On my system, this is very small (1.7321e-005 s), but not negligible when testing the execution of very quick commands, such as RESHAPE.

Paolo de Leva

30 Oct 2008 Steve Eddins

Anna - thanks. I just updated the file to fix the nargout problem.

19 Apr 2008 Anna Nowak

Small remark - nargout parameter is sometimes less than 0. I had -1 result when there was no output arguments from function.

In case of problems with error "Too many output argument" juz change line :
num_outputs = ~(nargout(f) == 0);
to
num_outputs = ~(nargout(f) <= 0);

Still excellent file!

20 Feb 2008 John D'Errico

Ok, this does nothing exotic. But what it does it does well. Its a simple way to time your code without worrying about some of the extra bookkeeping, like warming up the codes to make sure you get an accurate assessment of the time required, plus taking the median of the time over multiple runs.

The help and the code is as good as you would expect from this source.

Updates
29 Oct 2008

Fixed nargout problem for timing anonymous function handles.

31 Dec 2008

More accurate when timing very fast functions; warns you when the reported time might be affected by time-measurement overhead; calls F fewer times when F takes more than a few seconds to run.

16 May 2010

New option to specify number of output arguments to call F with. Overhead measurements now cached so most timings run faster.

Contact us