Conditional expectation

Prev: Transformations Next: Inequalities and limit theorems

Problems

Exercises marked with s have detailed solutions at http://stat110.net.

Conditional expectation given an event

9.1

Fred wants to travel from Blotchville to Blissville, and is deciding between 3 options (involving different routes or different forms of transportation). The $j$th option would take an average of ${\mu }_j$ hours, with a standard deviation of ${\sigma }_j$ hours. Fred randomly chooses between the 3 options, with equal probabilities. Let $T$ be how long it takes for him to get from Blotchville to Blissville.

(a) Find $E(T)$. Is it simply $({\mu }_1+{\mu }_2+{\mu }_3)/3$, the average of the expectations?

(b) Find $\mathrm{Var}(T)$. Is it simply $({\sigma }_1^2+{\sigma }_2^2+{\sigma }_3^2)/3$, the average of the variances?

---

9.2

While Fred is sleeping one night, $X$ legitimate emails and $Y$ spam emails are sent to him. Suppose that $X$ and $Y$ are independent, with $X\sim \mathrm{Pois}(10)$ and $Y\sim \mathrm{Pois}(40)$. When he wakes up, he observes that he has 30 new emails in his inbox. Given this information, what is the expected value of how many new legitimate emails he has?

---

9.3

A group of 21 women and 14 men are enrolled in a medical study. Each of them has a certain disease with probability $p$, independently. It is then found (through extremely reliable testing) that exactly 5 of the people have the disease. Given this information, what is the expected number of women who have the disease?

---

9.4

A researcher studying crime is interested in how often people have gotten arrested. Let $X\sim \mathrm{Pois}(\lambda )$ be the number of times that a random person got arrested in the last 10 years. However, data from police records are being used for the researcher’s study, and people who were never arrested in the last 10 years do not appear in the records. In other words, the police records have a selection bias: they only contain information on people who have been arrested in the last 10 years.

So averaging the numbers of arrests for people in the police records does not directly estimate $E(X)$; it makes more sense to think of the police records as giving us information about the conditional distribution of how many times a person was arrested, given that the person was arrested at least once in the last 10 years. The conditional distribution of $X$, given that $X\ge 1$, is called a truncated Poisson distribution (see Exercise 14 from Chapter 3 for another example of this distribution).

(a) Find $E(X\mid X\ge 1)$.

(b) Find $\mathrm{Var}(X\mid X\ge 1)$.

---

9.5

A fair 20-sided die is rolled repeatedly, until a gambler decides to stop. The gambler pays $1 per roll, and receives the amount shown on the die when the gambler stops (e.g., if the die is rolled 7 times and the gambler decides to stop then, with an 18 as the value of the last roll, then the net payoff is $18 - $7 = $11). Suppose the gambler uses the following strategy: keep rolling until a value of $m$ or greater is obtained, and then stop (where $m$ is a fixed integer between 1 and 20).

(a) What is the expected net payoff?

Hint: The average of consecutive integers $a,a+1,\dots ,a+n$ is the same as the average of the first and last of these. See the math appendix for more information about series.

(b) Use R or other software to find the optimal value of $m$.

---

9.6

Let $X\sim \mathrm{Expo}(\lambda )$. Find $E(X\mid X<1)$ in two different ways:

(a) by calculus, working with the conditional PDF of $X$ given $X<1$;

(b) without calculus, by expanding $E(X)$ using the law of total expectation.

---

9.7

You are given an opportunity to bid on a mystery box containing a mystery prize. The value of the prize is completely unknown, except that it is worth at least nothing, and at most a million dollars. So the true value $V$ of the prize is considered to be Uniform on $[0,1]$ (measured in millions of dollars).

You can choose to bid any nonnegative amount $b$ (in millions of dollars). If $b<V/4$, then your bid is rejected and nothing is gained or lost. If $b\ge V/4$, then your bid is accepted and your net payoff is $V-b$ (since you pay $b$ to get a prize worth $V$).

Find your expected payoff as a function of $b$ (be sure to specify it for all $b\ge 0$). Then find the optimal bid $b$, to maximize your expected payoff.

---

9.8

Stat110 solution available.

You get to choose between two envelopes, each of which contains a check for some positive amount of money. Unlike in the two-envelope paradox, it is not given that one envelope contains twice as much money as the other envelope. Instead, assume that the two values were generated independently from some distribution on the positive real numbers, with no information given about what that distribution is.

After picking an envelope, you can open it and see how much money is inside (call this value $x$), and then you have the option of switching. As no information has been given about the distribution, it may seem impossible to have better than a 50% chance of picking the better envelope. Intuitively, we may want to switch if $x$ is “small” and not switch if $x$ is “large”, but how do we define “small” and “large” in the grand scheme of all possible distributions? [The last sentence was a rhetorical question.]

Consider the following strategy for deciding whether to switch. Generate a threshold $T\sim \mathrm{Expo}(1)$, and switch envelopes if and only if the observed value $x$ is less than the value of $T$. Show that this strategy succeeds in picking the envelope with more money with probability strictly greater than $1/2$.

Hint: Let $t$ be the value of $T$ (generated by a random draw from the $\mathrm{Expo}(1)$ distribution). First explain why the strategy works very well if $t$ happens to be in between the two envelope values, and does no harm in any case (i.e., there is no case in which the strategy succeeds with probability strictly less than $1/2$).

---

9.9

There are two envelopes, each of which has a check for a $\mathrm{Unif}(0,1)$ amount of money, measured in thousands of dollars. The amounts in the two envelopes are independent. You get to choose an envelope and open it, and then you can either keep that amount or switch to the other envelope and get whatever amount is in that envelope.

Suppose that you use the following strategy: choose an envelope and open it. If you observe $U$, then stick with that envelope with probability $U$, and switch to the other envelope with probability $1-U$.

(a) Find the probability that you get the larger of the two amounts.

(b) Find the expected value of what you will receive.

---

9.10

Suppose $n$ people are bidding on a mystery prize that is up for auction. The bids are to be submitted in secret, and the individual who submits the highest bid wins the prize. The $i$th bidder receives a signal $X_i$, with $X_1,\dots ,X_n$ i.i.d. The value of the prize, $V$, is defined to be the sum of the individual bidders’ signals:

$$V=X_1+\cdots +X_n.$$

This is known in economics as the wallet game: we can imagine that the $n$ people are bidding on the total amount of money in their wallets, and each person’s signal is the amount of money in their own wallet. Of course, the wallet is a metaphor; the game can also be used to model company takeovers, where each of two companies bids to take over the other, and a company knows its own value but not the value of the other company. For this problem, assume the $X_i$ are i.i.d. $\mathrm{Unif}(0,1)$.

(a) Before receiving her signal, what is bidder 1’s unconditional expectation for $V$?

(b) Conditional on receiving the signal $X_1=x_1$, what is bidder 1’s expectation for $V$?

(c) Suppose each bidder submits a bid equal to their conditional expectation for $V$, i.e., bidder $i$ bids $E(V\mid X_i=x_i)$. Conditional on receiving the signal $X_1=x_1$ and winning the auction, what is bidder 1’s expectation for $V$? Explain intuitively why this quantity is always less than the quantity calculated in (b).

---

9.11

Stat110 solution available.

A coin with probability $p$ of Heads is flipped repeatedly. For (a) and (b), suppose that $p$ is a known constant, with $0<p<1$.

(a) What is the expected number of flips until the pattern HT is observed?

(b) What is the expected number of flips until the pattern HH is observed?

(c) Now suppose that $p$ is unknown, and that we use a $\mathrm{Beta}(a,b)$ prior to reflect our uncertainty about $p$ (where $a$ and $b$ are known constants and are greater than 2). In terms of $a$ and $b$, find the corresponding answers to (a) and (b) in this setting.

---

9.12

A coin with probability $p$ of Heads is flipped repeatedly, where $0<p<1$. The sequence of outcomes can be divided into runs (blocks of H’s or blocks of T’s), e.g., HHHTTTTHTTTHH becomes HHH TTTT H TTT HH, which has 5 runs, with lengths $3,4,1,3,2$, respectively. Assume that the coin is flipped at least until the start of the third run.

(a) Find the expected length of the first run.

(b) Find the expected length of the second run.

---

9.13

A fair 6-sided die is rolled once. Find the expected number of additional rolls needed to obtain a value at least as large as that of the first roll.

---

9.14

A fair 6-sided die is rolled repeatedly.

(a) Find the expected number of rolls needed to get a 1 followed right away by a 2.

Hint: Start by conditioning on whether or not the first roll is a 1.

(b) Find the expected number of rolls needed to get two consecutive 1’s.

(c) Let $a_n$ be the expected number of rolls needed to get the same value $n$ times in a row (i.e., to obtain a streak of $n$ consecutive $j$’s for some not-specified-in-advance value of $j$). Find a recursive formula for $a_{n+1}$ in terms of $a_n$.

Hint: Divide the time until there are $n+1$ consecutive appearances of the same value into two pieces: the time until there are $n$ consecutive appearances, and the rest.

(d) Find a simple, explicit formula for $a_n$ for all $n\ge 1$. What is $a_7$ (numerically)?

---

Conditional expectation given a random variable

9.15

Stat110 solution available.

Let $X_1,X_2$ be i.i.d., and let $\bar{X}=\frac{1}{2}(X_1+X_2)$ be the sample mean. In many statistics problems, it is useful or important to obtain a conditional expectation given $\bar{X}$. As an example of this, find $E(w_1{X}_1+w_2{X}_2\mid \bar{X})$, where $w_1,w_2$ are constants with $w_1+w_2=1$.

---

9.16

Let $X_1,X_2,\dots$ be i.i.d. r.v.s with mean 0, and let $S_n=X_1+\cdots +X_n$. As shown in Example 9.3.6, the expected value of the first term given the sum of the first $n$ terms is

$$E(X_1\mid S_n)=\frac{S_n}{n}.$$

Generalize this result by finding $E(S_k\mid S_n)$ for all positive integers $k$ and $n$.

---

9.17

Stat110 solution available.

Consider a group of $n$ roommate pairs at a college (so there are $2n$ students). Each of these $2n$ students independently decides randomly whether to take a certain course, with probability $p$ of success (where “success” is defined as taking the course).

Let $N$ be the number of students among these $2n$ who take the course, and let $X$ be the number of roommate pairs where both roommates in the pair take the course. Find $E(X)$ and $E(X\mid N)$.

---

9.18

Stat110 solution available.

Show that

$$E((Y-E(Y\mid X){)}^2\mid X)=E(Y^2\mid X)-(E(Y\mid X){)}^2,$$

so these two expressions for $\mathrm{Var}(Y\mid X)$ agree.

Hint for the variance: Adding a constant (or something acting as a constant) does not affect variance.

---

9.19

Let $X$ be the height of a randomly chosen adult man, and $Y$ be his father’s height, where $X$ and $Y$ have been standardized to have mean 0 and standard deviation 1. Suppose that $(X,Y)$ is Bivariate Normal, with $X,Y\sim \mathcal{N}(0,1)$ and $\mathrm{Corr}(X,Y)=\rho$.

(a) Let $y=ax+b$ be the equation of the best line for predicting $Y$ from $X$ (in the sense of minimizing the mean squared error), e.g., if we were to observe $X=1.3$ then we would predict that $Y$ is $1.3a+b$. Now suppose that we want to use $Y$ to predict $X$, rather than using $X$ to predict $Y$. Give and explain an intuitive guess for what the slope is of the best line for predicting $X$ from $Y$.

(b) Find a constant $c$ (in terms of $\rho$) and an r.v. $V$ such that $Y=cX+V$, with $V$ independent of $X$.

Hint: Start by finding $c$ such that $\mathrm{Cov}(X,Y-cX)=0$.

(c) Find a constant $d$ (in terms of $\rho$) and an r.v. $W$ such that $X=dY+W$, with $W$ independent of $Y$.

(d) Find $E(Y\mid X)$ and $E(X\mid Y)$.

(e) Reconcile (a) and (d), if your intuitive guess in (a) differed from what the results of (d) implied. Give a clear and correct intuitive explanation of the relationship between the slope of the best line for predicting $Y$ from $X$ and the slope of the best line for predicting $X$ from $Y$.

---

9.20

Let $X\sim {\mathrm{Mult}}_5(n,\mathbf{p})$.

(a) Find $E(X_1\mid X_2)$ and $\mathrm{Var}(X_1\mid X_2)$.

(b) Find $E(X_1\mid X_2+X_3)$.

---

9.21

Let $Y$ be a discrete r.v., $A$ be an event with $0<P(A)<1$, and $I_A$ be the indicator r.v. for $A$.

(a) Explain precisely how the r.v. $E(Y\mid I_A)$ relates to the numbers $E(Y\mid A)$ and $E(Y\mid A^c)$.

(b) Show that $E(Y\mid A)=E(Y{I}_A)/P(A)$, directly from the definitions of expectation and conditional expectation.

Hint: Let $X=Y{I}_A$, and then find an expression for the PMF of $X$.

(c) Use (b) to give a short proof of the fact that

$$E(Y)=E(Y\mid A)P(A)+E(Y\mid A^c)P(A^c).$$

---

9.22

Show that the following version of LOTP, which we encountered in Section 7.1, is also a consequence of Adam’s law: for any event $A$ and continuous r.v. $X$ with PDF $f_X$,

$$P(A)={\int }_{-\infty }^{\infty }P(A\mid X=x)f_X(x)dx.$$

Hint: Consider $E(I(A)\mid X=x)$.

---

9.23

Stat110 solution available.

Let $X$ and $Y$ be random variables with finite variances, and let $W=Y-E(Y\mid X)$. This is a residual: the difference between the true value of $Y$ and the predicted value of $Y$ based on $X$.

(a) Compute $E(W)$ and $E(W\mid X)$.

(b) Compute $\mathrm{Var}(W)$, for the case that $W\mid X\sim \mathcal{N}(0,X^2)$ with $X\sim \mathcal{N}(0,1)$.

---

9.24

Stat110 solution available.

One of two identical-looking coins is picked from a hat randomly, where one coin has probability $p_1$ of Heads and the other has probability $p_2$ of Heads. Let $X$ be the number of Heads after flipping the chosen coin $n$ times. Find the mean and variance of $X$.

---

9.25

Kelly makes a series of $n$ bets, each of which she has probability $p$ of winning, independently. Initially, she has $x_0$ dollars. Let $X_j$ be the amount she has immediately after her $j$th bet is settled. Let $f$ be a constant in $(0,1)$, called the betting fraction. On each bet, Kelly wagers a fraction $f$ of her wealth, and then she either wins or loses that amount. For example, if her current wealth is $100 and $f=0.25$, then she bets $25 and either gains or loses that amount. (A famous choice when $p>1/2$ is $f=2p-1$, which is known as the Kelly criterion.) Find $E(X_n)$ (in terms of $n,p,f,x_0$).

Hint: First find $E(X_{j+1}\mid X_j)$.

---

9.26

Let $N\sim \mathrm{Pois}({\lambda }_1)$ be the number of movies that will be released next year. Suppose that for each movie the number of tickets sold is $\mathrm{Pois}({\lambda }_2)$, independent of other movies and of $N$. Find the mean and variance of the number of movie tickets that will be sold next year.

---

9.27

A party is being held from 8:00 pm to midnight on a certain night, and $N\sim \mathrm{Pois}(\lambda )$ people are going to show up. They will all arrive at uniformly random times while the party is going on, independently of each other and of $N$.

(a) Find the expected time at which the first person arrives, given that at least one person shows up. Give both an exact answer in terms of $\lambda$, measured in minutes after 8:00 pm, and an answer rounded to the nearest minute for $\lambda =20$, expressed in time notation (e.g., 8:20 pm).

(b) Find the expected time at which the last person arrives, given that at least one person shows up. As in (a), give both an exact answer and an answer rounded to the nearest minute for $\lambda =20$.

---

9.28

Stat110 solution available.

We wish to estimate an unknown parameter $\theta$, based on an r.v. $X$ we will get to observe. As in the Bayesian perspective, assume that $X$ and $\theta$ have a joint distribution. Let $\hat{\theta }$ be the estimator (which is a function of $X$). Then $\hat{\theta }$ is said to be unbiased if $E(\hat{\theta }\mid \theta )=\theta$, and $\hat{\theta }$ is said to be the Bayes procedure if $E(\theta \mid X)=\hat{\theta }$.

(a) Let $\hat{\theta }$ be unbiased. Find $E(\hat{\theta }-\theta {)}^2$ (the average squared difference between the estimator and the true value of $\theta$), in terms of marginal moments of $\hat{\theta }$ and $\theta$.

Hint: Condition on $\theta$.

(b) Repeat (a), except in this part suppose that $\hat{\theta }$ is the Bayes procedure rather than assuming that it is unbiased.

Hint: Condition on $X$.

(c) Show that it is impossible for $\hat{\theta }$ to be both the Bayes procedure and unbiased, except in silly problems where we get to know $\theta$ perfectly by observing $X$.

Hint: If $Y$ is a nonnegative r.v. with mean 0, then $P(Y=0)=1$.

---

9.29

Show that if $E(Y\mid X)=c$ is a constant, then $X$ and $Y$ are uncorrelated.

Hint: Use Adam’s law to find $E(Y)$ and $E(XY)$.

---

9.30

Show by example that it is possible to have uncorrelated $X$ and $Y$ such that $E(Y\mid X)$ is not a constant.

Hint: Consider a standard Normal and its square.

---

9.31

Stat110 solution available.

Emails arrive one at a time in an inbox. Let $T_n$ be the time at which the $n$th email arrives (measured on a continuous scale from some starting point in time). Suppose that the waiting times between emails are i.i.d. $\mathrm{Expo}(\lambda )$, i.e., $T_1,T_2-T_1,T_3-T_2,\dots$ are i.i.d. $\mathrm{Expo}(\lambda )$.

Each email is non-spam with probability $p$, and spam with probability $q=1-p$ (independently of the other emails and of the waiting times). Let $X$ be the time at which the first non-spam email arrives (so $X$ is a continuous r.v., with $X=T_1$ if the 1st email is non-spam, $X=T_2$ if the 1st email is spam but the 2nd one isn’t, etc.).

(a) Find the mean and variance of $X$.

(b) Find the MGF of $X$. What famous distribution does this imply that $X$ has (be sure to state its parameter values)?

Hint for both parts: Let $N$ be the number of emails until the first non-spam (including that one), and write $X$ as a sum of $N$ terms; then condition on $N$.

---

9.32

Customers arrive at a store according to a Poisson process of rate $\lambda$ customers per hour. Each makes a purchase with probability $p$, independently. Given that a customer makes a purchase, the amount spent has mean $\mu$ (in dollars) and variance ${\sigma }^2$.

(a) Find the mean and variance of how much a random customer spends (note that the customer may spend nothing).

(b) Find the mean and variance of the revenue the store obtains in an 8-hour time interval, using (a) and results from this chapter.

(c) Find the mean and variance of the revenue the store obtains in an 8-hour time interval, using the chicken-egg story and results from this chapter.

---

9.33

Fred’s beloved computer will last an $\mathrm{Expo}(\lambda )$ amount of time until it has a malfunction. When that happens, Fred will try to get it fixed. With probability $p$, he will be able to get it fixed. If he is able to get it fixed, the computer is good as new again and will last an additional, independent $\mathrm{Expo}(\lambda )$ amount of time until the next malfunction (when again he is able to get it fixed with probability $p$, and so on). If after any malfunction Fred is unable to get it fixed, he will buy a new computer. Find the expected amount of time until Fred buys a new computer. (Assume that the time spent on computer diagnosis, repair, and shopping is negligible.)

---

9.34

A green die is rolled until it lands 1 for the first time. An orange die is rolled until it lands 6 for the first time. The dice are fair, six-sided dice. Let $T_1$ be the sum of the values of the rolls of the green die (including the 1 at the end) and $T_6$ be the sum of the values of the rolls of the orange die (including the 6 at the end). Two students are debating whether $E(T_1)=E(T_6)$ or $E(T_1)<E(T_6)$. They kindly gave permission to quote their arguments here.

Student A: We have $E(T_1)=E(T_6)$. By Adam’s law, the expected sum of the rolls of a die is the expected number of rolls times the expected value of one roll, and each of these factors is the same for the two dice. In more detail, let $N_1$ be the number of rolls of the green die and $N_6$ be the number of rolls of the orange die. By Adam’s law and linearity,

$$E(T_1)=E(E(T_1\mid N_1))=E(3.5N_1)=3.5E(N_1),$$

and the same method applied to the orange die gives $3.5E(N_6)$, which equals $3.5E(N_1)$.

Student B: Actually, $E(T_1)<E(T_6)$. I agree that the expected number of rolls is the same for the two dice, but the key difference is that we know the last roll is a 1 for the green die and a 6 for the orange die. The expected totals are the same for the two dice excluding the last roll of each, and then including the last roll makes $E(T_1)<E(T_6)$.

(a) Discuss in words the extent to which Student A’s argument is convincing and correct.

(b) Discuss in words the extent to which Student B’s argument is convincing and correct.

(c) Give careful derivations of $E(T_1)$ and $E(T_6)$.

---

9.35

Stat110 solution available.

Judit plays in a total of $N\sim \mathrm{Geom}(s)$ chess tournaments in her career. Suppose that in each tournament she has probability $p$ of winning the tournament, independently. Let $T$ be the number of tournaments she wins in her career.

(a) Find the mean and variance of $T$.

(b) Find the MGF of $T$. What is the name of this distribution (with its parameters)?

---

9.36

In Story 8.4.5, we showed (among other things) that if $\lambda \sim \mathrm{Gamma}(r_0,b_0)$ and $Y\mid \lambda \sim \mathrm{Pois}(\lambda )$, then the marginal distribution of $Y$ is $\mathrm{NBin}(r_0,b_0/(b_0+1))$. Derive this result using Adam’s law and MGFs.

Hint: Consider the conditional MGF of $Y\mid \lambda$.

---

9.37

Let $X_1,\dots ,X_n$ be i.i.d. r.v.s with mean $\mu$ and variance ${\sigma }^2$, and $n\ge 2$. A bootstrap sample of $X_1,\dots ,X_n$ is a sample of $n$ r.v.s $X_1^{\ast },\dots ,X_n^{\ast }$ formed from the $X_j$ by sampling with replacement with equal probabilities. Let ${\bar{X}}^{\ast }$ denote the sample mean of the bootstrap sample:

$${\bar{X}}^{\ast }=\frac{1}{n}(X_1^{\ast }+\cdots +X_n^{\ast }).$$

(a) Calculate $E(X_j^{\ast })$ and $\mathrm{Var}(X_j^{\ast })$ for each $j$.

(b) Calculate $E({\bar{X}}^{\ast }\mid X_1,\dots ,X_n)$ and $\mathrm{Var}({\bar{X}}^{\ast }\mid X_1,\dots ,X_n)$.

Hint: Conditional on $X_1,\dots ,X_n$, the $X_j^{\ast }$ are independent, with a PMF that puts probability $1/n$ at each of the points $X_1,\dots ,X_n$. As a check, your answers should be random variables that are functions of $X_1,\dots ,X_n$.

(c) Calculate $E({\bar{X}}^{\ast })$ and $\mathrm{Var}({\bar{X}}^{\ast })$.

(d) Explain intuitively why $\mathrm{Var}(\bar{X})<\mathrm{Var}({\bar{X}}^{\ast })$.

---

9.38

An insurance company covers disasters in two neighboring regions, $R_1$ and $R_2$. Let $I_1$ and $I_2$ be the indicator r.v.s for whether $R_1$ and $R_2$ are hit by the insured disaster, respectively. The indicators $I_1$ and $I_2$ may be dependent. Let $p_j=E(I_j)$ for $j=1,2$, and $p_{12}=E(I_1{I}_2)$.

The company reimburses a total cost of

$$C=I_1{T}_1+I_2{T}_2$$

to these regions, where $T_j$ has mean ${\mu }_j$ and variance ${\sigma }_j^2$. Assume that $T_1$ and $T_2$ are independent of each other and that $(T_1,T_2)$ is independent of $(I_1,I_2)$.

(a) Find $E(C)$.

(b) Find $\mathrm{Var}(C)$.

---

9.39

Stat110 solution available.

A certain stock has low volatility on some days and high volatility on other days. Suppose that the probability of a low volatility day is $p$ and of a high volatility day is $q=1-p$, and that on low volatility days the percent change in the stock price is $\mathcal{N}(0,{\sigma }_1^2)$, while on high volatility days the percent change is $\mathcal{N}(0,{\sigma }_2^2)$, with ${\sigma }_1<{\sigma }_2$. Let $X$ be the percent change of the stock on a certain day. The distribution is said to be a mixture of two Normal distributions, and a convenient way to represent $X$ is as $X=I_1{X}_1+I_2{X}_2$ where $I_1$ is the indicator r.v. of having a low volatility day, $I_2=1-I_1$, $X_j\sim \mathcal{N}(0,{\sigma }_j^2)$, and $I_1,X_1,X_2$ are independent.

(a) Find $\mathrm{Var}(X)$ in two ways: using Eve’s law, and by using properties of covariance to calculate $\mathrm{Cov}(I_1{X}_1+I_2{X}_2,I_1{X}_1+I_2{X}_2)$.

(b) Recall from Chapter 6 that the kurtosis of an r.v. $Y$ with mean $\mu$ and standard deviation $\sigma$ is defined by

$$\mathrm{Kurt}(Y)=\frac{E(Y-\mu {)}^4}{{\sigma }^4}-3.$$

Find the kurtosis of $X$ (in terms of $p,q,{\sigma }_1^2,{\sigma }_2^2$, fully simplified). The result will show that even though the kurtosis of any Normal distribution is 0, the kurtosis of $X$ is positive and in fact can be very large depending on the parameter values.

---

9.40

Let $X_1$, $X_2$, and $Y$ be random variables, such that $Y$ has finite variance. Let

$$A=E(Y\mid X_1)\text{and}B=E(Y\mid X_1,X_2).$$

Show that

$$\mathrm{Var}(A)\le \mathrm{Var}(B).$$

Also, check that this makes sense in the extreme cases where $Y$ is independent of $X_1$ and where $Y=h(X_2)$ for some function $h$.

Hint: Use Eve’s law on $B$.

---

9.41

Show that for any r.v.s $X$ and $Y$,

$$E(Y\mid E(Y\mid X))=E(Y\mid X).$$

This has a nice intuitive interpretation if we think of $E(Y\mid X)$ as the prediction we would make for $Y$ based on $X$: given the prediction we would use for predicting $Y$ from $X$, we no longer need to know $X$ to predict $Y$; we can just use the prediction we have. For example, letting $E(Y\mid X)=g(X)$, if we observe $g(X)=7$, then we may or may not know what $X$ is (since $g$ may not be one-to-one). But even without knowing $X$, we know that the prediction for $Y$ based on $X$ is 7.

Hint: Use Adam’s law with extra conditioning.

---

9.42

A researcher wishes to know whether a new treatment for the disease conditionitis is more effective than the standard treatment. It is unfortunately not feasible to do a randomized experiment, but the researcher does have the medical records of patients who received the new treatment and those who received the standard treatment. She is worried, though, that doctors tend to give the new treatment to younger, healthier patients. If this is the case, then naively comparing the outcomes of patients in the two groups would be like comparing apples and oranges.

Suppose each patient has background variables $X$, which might be age, height and weight, and measurements relating to previous health status. Let $Z$ be the indicator of receiving the new treatment. The researcher fears that $Z$ is dependent on $X$, i.e., that the distribution of $X$ given $Z=1$ is different from the distribution of $X$ given $Z=0$.

In order to compare apples to apples, the researcher wants to match every patient who received the new treatment to a patient with similar background variables who received the standard treatment. But $X$ could be a high-dimensional random vector, which often makes it very difficult to find a match with a similar value of $X$.

The propensity score reduces the possibly high-dimensional vector of background variables down to a single number (then it is much easier to match someone to a person with a similar propensity score than to match someone to a person with a similar value of $X$). The propensity score of a person with background characteristics $X$ is defined as

$$S=E(Z\mid X).$$

By the fundamental bridge, a person’s propensity score is their probability of receiving the treatment, given their background characteristics. Show that conditional on $S$, the treatment indicator $Z$ is independent of the background variables $X$.

Hint: This problem relates to the previous one. Show that $P(Z=1\mid S,X)=P(Z=1\mid S)$, which is equivalent to showing $E(Z\mid S,X)=E(Z\mid S)$.

---

9.43

This exercise develops a useful identity for covariance, similar in spirit to Adam’s law for expectation and Eve’s law for variance. First define conditional covariance in a manner analogous to how we defined conditional variance:

$$\mathrm{Cov}(X,Y\mid Z)=E((X-E(X\mid Z))(Y-E(Y\mid Z))\mid Z).$$

(a) Show that

$$\mathrm{Cov}(X,Y\mid Z)=E(XY\mid Z)-E(X\mid Z)E(Y\mid Z).$$

This should be true since it is the conditional version of the fact that

$$\mathrm{Cov}(X,Y)=E(XY)-E(X)E(Y),$$

and conditional probabilities are probabilities, but for this problem you should prove it directly using properties of expectation and conditional expectation.

(b) ECCE, or the law of total covariance, says that

$$\mathrm{Cov}(X,Y)=E(\mathrm{Cov}(X,Y\mid Z))+\mathrm{Cov}(E(X\mid Z),E(Y\mid Z)).$$

That is, the covariance of $X$ and $Y$ is the expected value of their conditional covariance plus the covariance of their conditional expectations, where all these conditional quantities are conditional on $Z$. Prove this identity.

Hint: We can assume without loss of generality that $E(X)=E(Y)=0$, since adding a constant to an r.v. has no effect on its covariance with any r.v. Then expand out the covariances on the right-hand side of the identity and apply Adam’s law.

---

Mixed practice

9.44

A group of $n$ friends often go out for dinner together. At their dinners, they play “credit card roulette” to decide who pays the bill. This means that at each dinner, one person is chosen uniformly at random to pay the entire bill (independently of what happens at the other dinners).

(a) Find the probability that in $k$ dinners, no one will have to pay the bill more than once (do not simplify for the case $k\le n$, but do simplify fully for the case $k>n$).

(b) Find the expected number of dinners it takes in order for everyone to have paid at least once (you can leave your answer as a finite sum of simple-looking terms).

(c) Alice and Bob are two of the friends. Find the covariance between how many times Alice pays and how many times Bob pays in $k$ dinners.

---

9.45

As in the previous problem, a group of $n$ friends play “credit card roulette” at their dinners. In this problem, let the number of dinners be a $\mathrm{Pois}(\lambda )$ r.v.

(a) Alice is one of the friends. Find the correlation between how many dinners Alice pays for and how many free dinners Alice gets.

(b) The costs of the dinners are i.i.d. $\mathrm{Gamma}(a,b)$ r.v.s, independent of the number of dinners. Find the mean and variance of the total cost.

---

9.46

Joe will read $N\sim \mathrm{Pois}(\lambda )$ books next year. Each book has a $\mathrm{Pois}(\mu )$ number of pages, with book lengths independent of each other and independent of $N$.

(a) Find the expected number of book pages that Joe will read next year.

(b) Find the variance of the number of book pages that Joe will read next year.

(c) For each of the $N$ books, Joe likes it with probability $p$ and dislikes it with probability $1-p$, independently. Find the conditional distribution of how many of the $N$ books Joe likes, given that he dislikes exactly $d$ of the books.

---

9.47

Buses arrive at a certain bus stop according to a Poisson process of rate $\lambda$. Each bus has $n$ seats and, at the instant when it arrives at the stop, has a $\mathrm{Bin}(n,p)$ number of passengers. Assume that the numbers of passengers on different buses are independent of each other, and independent of the arrival times of the buses.

Let $N_t$ be the number of buses that arrive in the time interval $[0,t]$, and $X_t$ be the total number of passengers on the buses that arrive in the time interval $[0,t]$.

(a) Find the mean and variance of $N_t$.

(b) Find the mean and variance of $X_t$.

(c) A bus is full if it has exactly $n$ passengers when it arrives at the stop. Find the probability that exactly $a+b$ buses arrive in $[0,t]$, of which $a$ are full and $b$ are not full.

---

9.48

Paul and $n$ other runners compete in a marathon. Their times are independent continuous r.v.s with CDF $F$.

(a) For $j=1,2,\dots ,n$, let $A_j$ be the event that anonymous runner $j$ completes the race faster than Paul. Explain whether the events $A_j$ are independent, and whether they are conditionally independent given Paul’s time to finish the race.

(b) For the rest of this problem, let $N$ be the number of runners who finish faster than Paul. Find $E(N)$. (Your answer should depend only on $n$, since Paul’s time is an r.v.)

(c) Find the conditional distribution of $N$, given that Paul’s time to finish the marathon is $t$.

(d) Find $\mathrm{Var}(N)$. (Your answer should depend only on $n$, since Paul’s time is an r.v.)

Hint: Let $T$ be Paul’s time, and use Eve’s law to condition on $T$. Alternatively, use indicator r.v.s.

---

9.49

Emails arrive in an inbox according to a Poisson process of rate $\lambda$ emails per hour.

(a) Find the name and parameters of the conditional distribution of the number of emails that arrive in the first 2 hours of an 8-hour time period, given that exactly $n$ emails arrive in that time period.

(b) Each email is legitimate with probability $p$ and spam with probability $q=1-p$, independently. Find the name and parameters of the conditional distribution of the number of legitimate emails that arrive in an 8-hour time period, given that exactly $s$ spams arrived in that time period.

(c) Reading an email takes a random amount of time, with mean $\mu$ hours and standard deviation $\sigma$ hours. These reading times are i.i.d. and independent of the email arrival process. Find the (unconditional) mean and variance of the total time it takes to read all the emails that arrive in an 8-hour time period.

---

9.50

An actuary wishes to estimate various quantities related to the number of insurance claims and the dollar amounts of those claims for someone named Fred. Suppose that Fred will make $N$ claims next year, where $N\mid \lambda \sim \mathrm{Pois}(\lambda )$. But $\lambda$ is unknown, so the actuary, taking a Bayesian approach, gives $\lambda$ a prior distribution based on past experience. Specifically, the prior is $\lambda \sim \mathrm{Expo}(1)$. The dollar amount of a claim is Log-Normal with parameters $\mu$ and ${\sigma }^2$ (here $\mu$ and ${\sigma }^2$ are the mean and variance of the underlying Normal), with $\mu$ and ${\sigma }^2$ known. The dollar amounts of the claims are i.i.d. and independent of $N$.

(a) Find $E(N)$ and $\mathrm{Var}(N)$ using properties of conditional expectation (your answers should not depend on $\lambda$, since $\lambda$ is unknown and being treated as an r.v.).

(b) Find the mean and variance of the total dollar amount of all the claims.

(c) Find the distribution of $N$. If it is a named distribution we have studied, give its name and parameters.

(d) Find the posterior distribution of $\lambda$, given that it is observed that Fred makes $N=n$ claims next year. If it is a named distribution we have studied, give its name and parameters.

---

9.51

Stat110 solution available.

Let $X_1,X_2,X_3$ be independent with $X_i\sim \mathrm{Expo}({\lambda }_i)$ (so with possibly different rates). Recall from Chapter 7 that

$$P(X_1<X_2)=\frac{{\lambda }_1}{{\lambda }_1+{\lambda }_2}.$$

(a) Find $E(X_1+X_2+X_3\mid X_1>1,X_2>2,X_3>3)$ in terms of ${\lambda }_1,{\lambda }_2,{\lambda }_3$.

(b) Find $P(X_1=\min (X_1,X_2,X_3))$, the probability that the first of the three Exponentials is the smallest.

Hint: Restate this in terms of $X_1$ and $\min (X_2,X_3)$.

(c) For the case ${\lambda }_1={\lambda }_2={\lambda }_3=1$, find the PDF of $\max (X_1,X_2,X_3)$. Is this one of the important distributions we have studied?

---

9.52

Stat110 solution available.

A task is randomly assigned to one of two people (with probability $1/2$ for each person). If assigned to the first person, the task takes an $\mathrm{Expo}({\lambda }_1)$ length of time to complete (measured in hours), while if assigned to the second person it takes an $\mathrm{Expo}({\lambda }_2)$ length of time to complete (independent of how long the first person would have taken). Let $T$ be the time taken to complete the task.

(a) Find the mean and variance of $T$.

(b) Suppose instead that the task is assigned to both people, and let $X$ be the time taken to complete it (by whoever completes it first, with the two people working independently). It is observed that after 24 hours, the task has not yet been completed. Conditional on this information, what is the expected value of $X$?

---

9.53

Suppose for this problem that “true IQ” is a meaningful concept rather than a reified social construct. Suppose that in the U.S. population, the distribution of true IQs is Normal with mean 100 and SD 15. A person is chosen at random from this population to take an IQ test. The test is a noisy measure of true ability: it’s correct on average but has a Normal measurement error with SD 5.

Let $\mu$ be the person’s true IQ, viewed as a random variable, and let $Y$ be her score on the IQ test. Then we have

$$Y\mid \mu \sim \mathcal{N}(\mu ,5^2)$$

and

$$\mu \sim \mathcal{N}(100,15^2).$$

(a) Find the unconditional mean and variance of $Y$.

(b) Find the marginal distribution of $Y$. One way is via the MGF.

(c) Find $\mathrm{Cov}(\mu ,Y)$.

---

9.54

Stat110 solution available.

A certain genetic characteristic is of interest. It can be measured numerically. Let $X_1$ and $X_2$ be the values of the genetic characteristic for two twin boys. Given that they are identical twins, $X_1=X_2$ and $X_1$ has mean 0 and variance ${\sigma }^2$; given that they are fraternal twins, $X_1$ and $X_2$ have mean 0, variance ${\sigma }^2$, and correlation $\rho$. The probability that the twins are identical is $1/2$. Find $\mathrm{Cov}(X_1,X_2)$ in terms of $\rho ,{\sigma }^2$.

---

9.55

Stat110 solution available.

The Mass Cash lottery randomly chooses 5 of the numbers from $1,2,\dots ,35$ each day (without repetitions within the choice of 5 numbers). Suppose that we want to know how long it will take until all numbers have been chosen. Let $a_j$ be the average number of additional days needed if we are missing $j$ numbers (so $a_0=0$ and $a_{35}$ is the average number of days needed to collect all 35 numbers). Find a recursive formula for the $a_j$.

---

9.56

Two chess players, Vishy and Magnus, play a series of games. Given $p$, the game results are i.i.d. with probability $p$ of Vishy winning, and probability $q=1-p$ of Magnus winning (assume that each game ends in a win for one of the two players). But $p$ is unknown, so we will treat it as an r.v. To reflect our uncertainty about $p$, we use the prior $p\sim \mathrm{Beta}(a,b)$, where $a$ and $b$ are known positive integers and $a\ge 2$.

(a) Find the expected number of games needed in order for Vishy to win a game (including the win). Simplify fully; your final answer should not use factorials or $\Gamma$.

(b) Explain in terms of independence vs. conditional independence the direction of the inequality between the answer to (a) and $1+E(G)$ for $G\sim \mathrm{Geom}\left(\frac{a}{a+b}\right)$.

(c) Find the conditional distribution of $p$ given that Vishy wins exactly 7 out of the first 10 games.

---

9.57

Laplace’s law of succession says that if $X_1,X_2,\dots ,X_{n+1}$ are conditionally independent $\mathrm{Bern}(p)$ r.v.s given $p$, but $p$ is given a $\mathrm{Unif}(0,1)$ prior to reflect ignorance about its value, then

$$P(X_{n+1}=1\mid X_1+\cdots +X_n=k)=\frac{k+1}{n+2}.$$

As an example, Laplace discussed the problem of predicting whether the sun will rise tomorrow, given that the sun did rise every time for all $n$ days of recorded history; the above formula then gives $(n+1)/(n+2)$ as the probability of the sun rising tomorrow (of course, assuming independent trials with $p$ unchanging over time may be a very unreasonable model for the sunrise problem).

(a) Find the posterior distribution of $p$ given $X_1=x_1,X_2=x_2,\dots ,X_n=x_n$, and show that it only depends on the sum of the $x_j$ (so we only need the one-dimensional quantity $x_1+x_2+\cdots +x_n$ to obtain the posterior distribution, rather than needing all $n$ data points).

(b) Prove Laplace’s law of succession, using a form of the law of total probability to find $P(X_{n+1}=1\mid X_1+\cdots +X_n=k)$ by conditioning on $p$. (The next exercise, which is closely related, involves an equivalent Adam’s law proof.)

---

9.58

Two basketball teams, A and B, play an $n$ game match. Let $X_j$ be the indicator of team A winning the $j$th game. Given $p$, the r.v.s $X_1,\dots ,X_n$ are i.i.d. with $X_j\mid p\sim \mathrm{Bern}(p)$. But $p$ is unknown, so we will treat it as an r.v. Let the prior distribution be $p\sim \mathrm{Unif}(0,1)$, and let $X$ be the number of wins for team A.

(a) Find $E(X)$ and $\mathrm{Var}(X)$.

(b) Use Adam’s law to find the probability that team A will win game $j+1$, given that they win exactly $a$ of the first $j$ games. (The previous exercise, which is closely related, involves an equivalent LOTP proof.)

Hint: Letting $C$ be the event that team A wins exactly $a$ of the first $j$ games,

$$P(X_{j+1}=1\mid C)=E(X_{j+1}\mid C)=E(E(X_{j+1}\mid C,p)\mid C)=E(p\mid C).$$

(c) Find the PMF of $X$. (There are various ways to do this, including a very fast way to see it based on results from earlier chapters.)

(d) The Putnam exam from 2002 posed the following problem:

Shanille O’Keal shoots free throws on a basketball court. She hits the first and misses the second, and thereafter the [conditional] probability that she hits the next shot is equal to the proportion of shots she has hit so far. What is the probability she hits exactly 50 of her first 100 shots?

Solve this Putnam problem by applying the result of Part (c). Be sure to explain why it is valid to apply that result, despite the fact that the Putnam problem does not seem to be using the same model, e.g., it does not mention a prior distribution, let alone mention a $\mathrm{Unif}(0,1)$ prior.

---

9.59

Let $X\mid p\sim \mathrm{Bin}(n,p)$, with $p\sim \mathrm{Beta}(a,b)$. So $X$ has a Beta-Binomial distribution, as mentioned in Story 8.3.3 and Example 8.5.3. Find $E(X)$ and $\mathrm{Var}(X)$.

---

9.60

An election is being held. There are two candidates, A and B, and there are $n$ voters. The probability of voting for Candidate A varies by city. There are $m$ cities, labeled $1,2,\dots ,m$. The $j$th city has $n_j$ voters, so $n_1+n_2+\cdots +n_m=n$. Let $X_j$ be the number of people in the $j$th city who vote for Candidate A, with $X_j\mid p_j\sim \mathrm{Bin}(n_j,p_j)$. To reflect our uncertainty about the probability of voting in each city, we treat $p_1,\dots ,p_m$ as r.v.s, with prior distribution asserting that they are i.i.d. $\mathrm{Unif}(0,1)$. Assume that $X_1,\dots ,X_m$ are independent, both unconditionally and conditional on $p_1,\dots ,p_m$. Let $X$ be the total number of votes for Candidate A.

(a) Find the marginal distribution of $X_1$ and the posterior distribution of $p_1\mid (X_1=k_1)$.

(b) Find $E(X)$ and $\mathrm{Var}(X)$ in terms of $n$ and $s$, where $s=n_1^2+n_2^2+\cdots +n_m^2$.