IE 304 OR PERHAPS III: Stochastic Models

Markov Chains

♣ Introduction

♣ Chapman-Kolmogorov Equations

♣ Category of Declares

♣ Limiting Probabilities

♣Mean Time Put in in Transitive States

Aybek Korugan

FOR EXAMPLE 304 OR PERHAPS III: Stochastic Models

Aybek Korugan

Introduction

A probabilistic event occurs in time. (changes in inventoıy, weather etc . )

We observe the procedure in equivalent and deterministic time time periods A product inventory is discovered at the start of each week:

Xn: level of inventory at each week n, in = zero, 1, 2,...

The level of products on hand in week n+1, Xn+1 depends on past inventory levels.

Assume the inventory is replenished to 10 when no item is remaining in the products on hand and the purchase arrives instantly. Also believe no backorders is allowed.

Then Xn œ H = 0,1,2,...,10

2

FOR EXAMPLE 304 OR III: Stochastic Models

Aybek Korugan

Intro

Then Xn+1 = m, j = 0,1,2,...,10 with probability

P X n +1 = j .

Below

n: period index,

my spouse and i, j: states of the process

S: the state space.

In case the value of Xn+1 will depend on only within the value of Xn then this process is named a Markov Chain.

P X n = i = Pij

Here Pij are called one stage transition possibilities.

3

FOR EXAMPLE 304 OR PERHAPS III: Stochastic Models

Aybek Korugan

Case 1: (Forecasting the Weather) Suppose that the chance of rainfall tomorrow is determined by previous climate only through whether or not it is raining today but not on earlier weather conditions. Suppose also that if it rains today, then it can rain down the road w/ prob. α; of course, if it doesn't rainwater today it will rain another day w/ prob. β.

Answer: If we the process is state zero when it down pours and express 1 when it doesn't, then this preceding is known as a two-state MC with transition probs.

Point out 0: rainwater

State 1: no rainwater

then S = 0,1

Time span: one day

4

IE 304 OR 3: Stochastic Types

Aybek Korugan

Introduction

The probabilities of event are

G X 1 = 0 = α sama dengan P00

P X 0 = 1 sama dengan β sama dengan P10

L X 0 = 0 = 1 − α sama dengan P01

L X 1 = 1 sama dengan 1 − β = P11

Which is often formalized as

P00

P=

P10

P01

α

=

P11

β

1−α

1− β

The P matrix is called one step transition matrix in the Markov Sequence.

5

IE 304 OR PERHAPS III: Stochastic Models

Aybek Korugan

Intro

Since the probabilities will be non-negative and since the process must make a changeover into several state, we have

Pij ≥ 0,

∑P

i, j ≥ 0;

m =0

ij

= you,

i sama dengan 0, you, L

Permit P denote the matrix of one-step transition possibilities Pij, then

K → ∑ l =0 P0 j = 1

M → ∑ ∞ P1 j = 1

j =0

P00

P01

P02

P10

P= M

Pi zero

P11

P12

M

Pi 1

Pi 2

Meters

M

M

L

six

IE 304 OR 3: Stochastic Designs

Aybek Korugan

Introduction

� A MC model starts by defining the random adjustable that identifies the state.

� Next the state of hawaii space (all possible ideals the 3rd there�s r. v. can take) plus the time interval are identified.

� Then the interaction btw. these claims and the excess weight of these connections are identified by the P matrix.

� A good approach to defining these interactions can be achieved by attracting a state change diagram.

� The rainwater example picture:

7