Expected Value of a Random Variable

The mean of a random variable, also known as its expected value, is the weighted average of all the values that a random variable would assume in the long run. The expected value of a random variable can be thought of this way: the random variable is made to assume values according to its probability distribution, all the values are recorded and the mean is computed. If this process is repeated indefinitely, the calculated mean of the values will approach some finite quantity, assuming that the mean of the random variable does exist (i.e., it does not diverge to infinity). This finite value is the mean of the random variable.

The expected value of a random variable X is denoted by E(X). For a discrete random variable, E(X) is calculated as

EX. The random variable X has the following probability distribution:

x	pX(x)
Â	Â
2	1 / 36
3	2 / 36
4	3 / 36
5	4 / 36
6	5 / 36
7	6 / 36
8	5 / 36
9	4 / 36
10	3 / 36
11	2 / 36
12	1 / 36

The random variable X assumes a value equal to the sum of two dice rolls. Its expected value is calculated as

= 2(1/36) + 3(2/36) + 4(3/36) + 5(4/36) + 6(5/36) + 7(6/36) + 8(5/36) + 9(4/36) + 10(3/36) + 11(2/36) + 12(1/36)
= (1/36) (2 + 6 + 12 + 20 + 30 + 42 + 40 + 36 + 30 + 22 + 12)
= (252/36) = 7

In the long run, the average value of two dice rolls using regular dice is 7.

The expected value of the function g(X) of a discrete random variable X is the mean of another random variable Y which assumes the values of g(X) according to the probability distribution of X. Denoted by E[g(X)], it is calculated as

The expected value of a discrete random variable X is actually a special case of E[g(X)], where g(X) = x.

The expected value of the function g(X,Y) of two discrete random variables, with joint probability mass function p_X,Y(x,y), is denoted by E[g(X,Y)] and is calculated as

This definition can be extended to three or more discrete random variables. In general, if a set of random variables X₁ , X₂, .... , X_n has a joint probability function p(x₁ , x₂ , ...., x_n), the expected value of the function g(X₁ , X₂ , .... , X_n ) is given by

EX. The random variable X that assumes the value of a dice roll has the probability mass function p(x) = 1/6 for x ÃŽ {1, 2, 3, 4, 5, 6}. If g(X) = X², then

= (1/6) (1 + 4 + 9 + 16 + 25 + 36)
= 91/6

EX. Two random variables X and Y have the following joint probability distribution

(x,y)	pX,Y(x,y)
(1,1)	1/3
(1,2)	1/8
(2,1)	1/2
(2,2)	1/24

The expected value of the function g(X,Y) = XY is calculated as follows:

= (1)(1)(1/3) + (1)(2)(1/8) + (2)(1)(1/2) + (2)(2)(1/24)
= 1/3 + 1/4 + 1 + 1/6
= 7/4

For any discrete random variable whose expected value exists and is E(X), the expected value of g(X) = aX + b, where a and b are constants, is given by

From this equality,

(1) E[aX] = aE(X)
(2) E[b] = b

The first statement asserts that the expected value of a scalar function of a random variable is the product of the expected value of the random variable and the scalar value. The second statement asserts that the expected value of a constant is the constant itself.

The expected value of the sum of two discrete random variables X and Y is

Similarly, E(X - Y) = E(X) - E(Y).

The mean of the product of two independent discrete random variables is

The mean of the product of two independent random variables is simply the product of the mean of the two independent random variables.

 

Variance of a Random Variable

The variance of a random variable is the variance of all the values that the random variable would assume in the long run. The variance of a random variable can be thought of this way: the random variable is made to assume values according to its probability distribution, all the values are recorded and their variance is computed. If this process is repeated indefinitely, the calculated variance of the values will approach some finite quantity, assuming that the variance of the random variable does exist (i.e., it does not diverge to infinity). This finite value is the variance of the random variable.

The variance of the random variable X is denoted by Var(X). For a discrete random variable, Var(X) is calculated as

Although this formula can be used to derive the variance of X, it is easier to use the following equation:

= E(x²) - 2E(X)E(X) + (E(X))²
= E(X²) - (E(X))²

The variance of the function g(X) of the random variable X is the variance of another random variable Y which assumes the values of g(X) according to the probability distribution of X. Denoted by Var[g(X)], it is calculated as

EX. The random variable X that assumes the value of a dice roll has the probability mass function:
p(x) = 1/6 for x ÃŽ {1, 2, 3, 4, 5, 6}.

=91/6 - 12.25
= 35/12

 

For any random variable X whose variance is Var(X), the variance of X + b, where b is a constant, is given by

Var(X + b) = E [(X + b) - E(X + b)]² = E[X + b - (E(X) + b)]²

= E [(X - E(X)]²
= Var(X)

i.e. the variance of a random variable does not change if a constant is added to all values of the random variable.

For any random variable X whose variance is Var(X), the variance of aX, where a is a constant, is given by

Var(aX) = E [aX - E(aX)]² = E [aX - aE(X)]²

= a²E[(X - E(X)]²
= a²Var(X)

The variance of a scalar function of a random variable is the product of the variance of the random variable and the square of the scalar.

For any two independent random variables X and Y, E(XY) = E(X) E(Y). Thus, the variance of two independent random variables is calculated as follows:

Var(X + Y) = E[(X + Y)²] - [E(X + Y)]²

=E(X² + 2XY + Y²) - [E(X) + E(Y)]²
=E(X²) + 2E(X)E(Y) + E(Y²) - [E(X)² + 2E(X)E(Y) + E(Y)²]
=[E(X²) - E(X)²] + [E(Y²) - E(Y)²]
= Var(X) + Var(Y)

Note that Var(-Y) = Var((-1)(Y)) = (-1)² Var(Y) = Var(Y). Therefore

Var(X - Y) = Var(X + (-Y)) = Var(X) + Var(-Y) = Var(X) + Var(Y)

The variance of the sum or difference of two independent random variables is the sum of the variances of the independent random variables. Similarly, the variance of the sum or difference of a set of independent random variables is simply the sum of the variances of the independent random variables in the set.