what do the AudioDeviceWriter do with a complex inputdata?

when i use ifft() to process and get a frame of audio data but forget to real() the results, it sounds weired.

so i wonder what happened when AudioDeviceWriter play with a complex inputdata?

i try it with a 1kHz pure tone and then plus it with a very little imag part，when played independently(commented the other)，these two sound very different. but when i play the two in sequence with a pause(3)，when the complex one first and real one latter,they sound the same(weired).

furthermore. when i change the order, put the real one first and the complex one latter, an error occured.

i'll be glad to get an answer to the question ,thank you!

here is the code,you can change the order of adw(playbuff2) and adw(playbuff) then test ;

clc;
clear;
fs=48e3;
f0=1e3;
T=1;
N=1:T*fs;
t=N/fs;
wav=sin(2*pi*f0*t).';
adw=audioDeviceWriter('SampleRate',fs);
wav2=wav;
wav2=wav+1e-18*j*ones(size(wav));
playbuff2=[wav2];
playbuff=[wav];

adw(playbuff2);
pause(3);
adw(playbuff);

Answer 1

[William Rose]

@康,

I ran your code and got the same results you describe:

No error if I play the complex file, then the real. Both playbacks sound normal.

If I play the real file, then the complex, th real file sounds normal (but louder than before), and there is an error when the script tries to play the complex file.

Error using ()

Changing the complexity (from real to complex) on input 1 of System object

audioDeviceWriter is not allowed without first calling the release() method.

Error in complexSoundTest (line 18)

adw(playbuff2);

The error message explains why: it is not allowed to change the data type from real to complex unless you release the audio device writer.

Why do you want to play a complex array as a sound file? A sound recording (in the time domain) is not complex.

[William Rose]

@康,

audioDeviceWriter() was never designed to play complex sequences, so the results below are unsurprising, especially in light of @Walter Roberson's point about how complex numbers are stored. The results are also likely to be platform-dependent.

On my Windows notebook PC, the sound amplitude is related in a complicated way to the real and imaginary parts of the complex sequence, and the frequency of the output signal is half of the expected frequency. The code below demonstrates this by generating and playing three complex vectors.

wav1: pure real, magnitude=0.7

wav2: equal real and imaginary, magnitude=1

wav3: pure imaginary, magnitude=0.7

All three signals are designed to be 1 kHz sinusoids. When the input signal to audioDeviceWriter is type single or double, the input is clipped to +-1. The signals above are designed to avoid clipping.

When I play the three signals, wav2 is loudest, and wav3 is least loud.

clear
% Create audioDeviceWriter object
fs=48e3;        % audio output rate (Hz)
adw=audioDeviceWriter('SampleRate',fs);
% Make the signals
T=2;            % signal duration (s)
N=T*fs;         % signal duration (samples)
t=(0:N-1)'/fs;  % time vector (s)
f0=1000;        % sound fundamental frequency (Hz)
wav1=complex(0.7*sin(2*pi*f0*t));
wav2=0.7*(1+1j)*sin(2*pi*f0*t);
wav3=1j*0.7*sin(2*pi*f0*t);
% Play the signals
adw(wav1);
adw(wav2);
adw(wav3);
release(adw)

I recorded the audio output signal on my cell phone while it played through my notebook computer speaker (.m4a file attached).

Analyze the recording of the playback:

y=audioread('complexSoundTest.m4a');
t=(1:length(y))'/48000;  % time vector
figure
subplot(211), plot(t,y,'-b'); 
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-.7,.7])
subplot(234), plot(t(t>=2 & t<2.01),y(t>=2 & t<2.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-.5,.5])
subplot(235), plot(t(t>=4 & t<4.01),y(t>=4 & t<4.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-.5,.5])
subplot(236), plot(t(t>=6 & t<6.01),y(t>=6 & t<6.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-.5,.5])

The code above produces the figure below.

The figure shows that the playback ampltude is larger when the complex amplitude is larger (wav2, middle). But the playback amplitude during the pure real segment (left) is larger than during the pure imaginary segment (right), even though the complex amplitudes of the vectors sent to the audio player object are the same.

The figure shows that the signal durations are 2 seconds each, as designed. But playback signal frequency is 500 Hz, which is half the frequency specified when the signals were created.

I suspect the temporary amplitude overshoot at the transition from wav1 to wav2 is due to nonlinear response of my built-in speaker. It is also clear, expecially in the bottom middle plot, that the playback amplitude is not exactly sinusoidal, which is another indication of nonlinear response of the speaker.

[康]

@William Rose

Thank you for replying!

i download your .m4a files and the waveforms are exactly as you described.

To explore what happened， i get an extern sound card(Creative X-FI)，and linked the input and output ，then run your codes several times . I get a very different wavform but sounds like your recorded .m4a audio. the plot is generated by your code with slightly modified

unlike what we heard ,the amplitude of middle part is smaller than the first and last，but the wave is less noisy and more normal. i wonder if the larger portion of the first an the last part are filtered by the speaker and transducer or masked by each other? the spectrum wav show there are strong noise are the cutoff frequency(open the .wav file with Audition).

i do a lowpass under 16kHz on the record wav;

the filtered wav looks like this

So we can now find the wavs are similar as you recorded.

But what the AudioDeviceWriter do with a complex input to the sound card is not cleard uncovered yet.If the sound card correctly record what it get's as input（if i used it in the right way),maybe the attached record can help to explore this.

@Jimmy Lapierreand @Walter Roberson 's opinion about how complexed numbers stored and played maybe useful to discover the answer, thank you !

BTW,

i crash into this problem just when i process sound frame by frame and applying fft、frequency-spectrum-multiply and ifft. maybe due to the float calculation precision, the input before ifft is not excatly conj symmetry so the output is with a very littter image part but I forget to real() it.>_<.

here is the code

[y,fs]=audioread('complexSoundRecordedBySoundCard.wav');
t=(1:length(y))'/fs;  % time vector
figure
subplot(211), plot(t,y,'-b'); 
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(234), plot(t(t>=24 & t<24.01),y(t>=24 & t<24.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(235), plot(t(t>=26 & t<26.01),y(t>=26 & t<26.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(236), plot(t(t>=28 & t<28.01),y(t>=28 & t<28.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])

figure
z=lowpass(y,16e3,fs);

figure
subplot(211), plot(t,z,'-b'); 
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(234), plot(t(t>=24 & t<24.01),z(t>=24 & t<24.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(235), plot(t(t>=26 & t<26.01),z(t>=26 & t<26.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])
subplot(236), plot(t(t>=28 & t<28.01),z(t>=28 & t<28.01),'-b.')
xlabel('Time (s)'); ylabel('Amplitude'); grid on; ylim([-1,1])

[William Rose]

@康,

You have done a nice careful investigation of the results of sending a complex sequence to the sound output device using audioDeviceWriter. I mentioned in my previous comment that the results may be platform-dependent, since audioDeviceWriter was never designed to handle complex input data, and your research seems to support that.

A sound in the time domain is not complex, and a time domain recording of sound should never be complex, with or without filtering.

[Jimmy Lapierre]

May I recommend ifft's 'symmetric' flag, that's one way to avoid the small complex residual causing issues (the output will always be real). Just make sure there's not a bug in the code causing the data to not be conjugate symmetric.

[William Rose]

@康, I agree with @Jimmy Lapierre that you may want to examine your code carefully for possible errors. The signal complexSoundRecordedBySoundCard, which you uploaded, suggests that mistakes may have occurred. I say this because the original signal power spectrum plot which you shared (copied below) looks odd: it looks like the two-sided spectrum of a signal sampled at 24 kHz, instead of the one-sided spectrum of a signal sampled at 48 kHz.

Answer 2

[Jimmy Lapierre]

This object should probably error out for complex values (which is not possible with any audio device). Instead, it looks like it's interpreting the buffer's complex values as samples to play, like this:

playbuff3=zeros(96e3,1);
playbuff3(1:2:end,:)=real(playbuff2);
playbuff3(2:2:end,:)=imag(playbuff2);
adw(playbuff3(1:48e3,:))

And as William said, you might need to release the object to change data type (or buffer length, etc.).

[Walter Roberson]

Note that complex numbers started to be stored in interweaved format as of R2017b. Before that, real and imaginary portions were stored seperately.