Monte Carlo Simulation/Markov Chains

I was assigned a computer project regarding Markov Chains and simulations. Trouble is, I've never taken a probability course in my life and am having some heavy difficulty starting the project and was wondering if anybody would be able to assist me. I have a forward and reverse transition rate for a simple two state model but am not sure what to do with these equations.

Answer 1

[Ragini Gupta]

Hi all,

Since I am new to MCMC simulation I am facing a similar problem.I have to simulate a smart meter data for a week's time using Markov chain model. Now,I need to run the markov model to generate a new Smart meter value for each day (i.e. 7 readings). However, from the following code I know how to train the model untill using the dhmm_em function.But I do not understand how to use this trained model and generate a new value for the next state.

Here is the code I have so far:

filename = 'C:\Users\profile\Desktop\SmartMeter.xlsx'; Furnace=xlsread(filename,'E:E'); %H1D1 x=length(Furnace); hist(Furnace) edges = linspace(min(Furnace),max(Furnace),10); % generates a row vector of 10 linearly equally spaced points between min % and max furnace values

% edges are the end points of each bin

[counts,bins] = histc(Furnace, edges);

figure bar(edges,counts,'histc') O = 10; % output symbols Q = 5; % number of states

prior0 = bins; T=10; nex=20; t1 = datetime(2017,1,1,24,0,0); t2 = datetime(2017,1,7,24,0,0); t = t1:hours(24):t2; s=length(t);

k=1;

%Generate Sequence of Dates and Time with a step size of 15 minutes fileID = fopen('C:\Users\profile\Desktop\raginiData5.txt','w'); for t = t1:hours(24):t2 transmat0 = mk_stochastic(rand(Q,Q)); obsmat0 = mk_stochastic(rand(Q,O)); %Now we sample nex=20 sequences of length T=10 each from this model, to use as training data. data = dhmm_sample(prior0, transmat0, obsmat0, nex, T); %SAMPLE_DHMM Generate random sequences from a Hidden Markov Model with discrete outputs. %Each row of data is an observation sequence of length T. % initial guess of parameters prior1 =normalise(rand(Q,1)); transmat1 = mk_stochastic(rand(Q,Q)); obsmat1 = mk_stochastic(rand(Q,O));

    % improve guess of parameters using EM algorithm (using the
    % maximum likelihood of parameters in statistical model
    [LL, prior2, transmat2, obsmat2] = dhmm_em(data, prior1, transmat1, obsmat1);

    DateString = datestr(t);
    fprintf(fileID,'%s   %f \n',DateString);

end fclose(fileID);

Please can anyone advise me on the next step.

Thanks in advance.