Problems

1. To Begin or Not to Begin?

An urn contains k black balls and a single red ball. Peter and Paula draw without replacement balls from this urn, alternating after each draw until the red ball is drawn. The game is won by the player who happens to draw the single red ball. Peter is a gentleman and offers Paula the choice of whether she wants to start or not. Paula has a hunch that she might be better off if she starts; after all, she might succeed in the first draw. On the other hand, if her first draw yields a black ball, then Peter’s chances to draw the red ball in his first draw are increased, because then one black ball is already removed from the urn. How should Paula decide in order to maximize her probability of winning?

Let’s solve this by looking at values of k. If k is 0, Going first has a 100% chance of winning, and second has a 0% chance. If k is 1, Going first has a 50% chance of winning, and second has a 50% chance of winning. If k is 2, Going first means you can take 2/3 of the balls vs 1/3 if you go second, so going first is better. If k is 3, Going first or second means you take 2/4 balls, so this is also 50/50.

Thus, even and odd numbers of k are the only difference, where if k is odd, then you have a 50/50 chance of winning as either first or second, and if k is even, you have an edge, so you should always go first.

2. A Tournament Problem

Ten players participate in the first round of a tennis tournament: 2 females and 8 males. Five single matches are fixed at random by successively drawing, without replacement, the names of all 10 players from an urn: the player drawn first plays against the one whose name comes up second, the third against the fourth, etc.

a. What is the probability that there will not be a single match involving two female players? Is this probability smaller, equal to, or larger than the corresponding probability with 20 females and 80 males?

b. Try to answer the general case in which there are 2n players, of whom 2 ≤ k ≤ n are female. What is the probability p(k, n) that among the n matches there will not be a single one involving two female players

• For the chance that there will not be a single match involving two female players, we can calculate the total combinations of pairs (10c2), or 45, and the amount of pairs where there is a woman vs a woman (5, since the pair can be in any of the 5 games).
• Thus the chance is 40/45 or 8/9 for there to be no woman vs woman game.
• For the case with 20 females and 80 males, it drops to about 9%.

3. Mean Waiting Time for 1 − 1 vs. 1 − 2

Peter and Paula play a simple game of dice, as follows. Peter keeps throwing the (unbiased) die until he obtains the sequence 1 − 1 in two successive throws. For Paula, the rules are similar, but she throws the die until she obtains the sequence 1 − 2 in two successive throws.

a. On average, will both have to throw the die the same number of times? If not, whose expected waiting time is shorter (no explicit calculations are required)?

b. Derive the actual expected waiting times for Peter and Paula

a. The mean waiting time for Peter is longer than for Paula, because Peter has to roll 1-1, and if he rolls anything other than a 1 on a roll, he has to restart. For Paula, if she rolls anything other than a 1 into a 1, she can continue on that way.

We can model this using a game tree for Paul:

• With a probability p ($\frac{1}{6}$), Paul will roll a 1.
  • With a probability p ($\frac{1}{6}$) Paul will again roll a 1.
  • With a probability 1 - p ($\frac{1}{6}$), Paul won’t roll a 1 and restart.
• With a probability 1 - p, ($\frac{5}{6}$), Paul won’t roll a 1 and restart.

Thus, there are three outcomes:

The outcome that ends the game, 1-1 has a chance of $p^2$ or ($\frac{1}{36}$. The outcome that he rolls a 1 and then anything else is $p\ast (1-p)$ or $\frac{5}{36}$. The outcome that restarts is $1-p$ or $\frac{5}{6}$.

The first two outcomes take 2 rounds and the last one takes 1 round. Thus, the total waiting time is the time taken * the probability they occur:

$$W=(1+W)\ast (1-p)+(2+W)\ast p(1-p)+2p^2$$

Simplifying:

$$W=\frac{1+p}{p^2}$$

When $p=\frac{1}{6}$, W = 42.

For Paula’s case, we can model this as a markov chain:

There are three states, the starting state $S_0$, a state for when the last throw was a 1 $S_1$, and the final state of 1-2, $S_2$.

$$\left(\begin{array}{ccc}\frac{5}{6}& \frac{1}{6}& -\\ \frac{2}{3}& \frac{1}{6}& \frac{1}{6}\\ -& -& 1\end{array}\right)$$

Thus, we can find the expected waiting time for both $S_0$ and $S_1$ as follows:

$$W_0=1+\frac{5}{6}{W}_0+\frac{1}{6}{W}_1$$ $$W_1=1+\frac{2}{3}{W}_0+\frac{1}{6}{W}_1+\frac{1}{6}{W}_2$$

This gives us W = 36, solving for $W_0$.

4. How to Divide up Gains in Interrupted Games

Peter and Paula play a game of chance that consists of several rounds. Each individual round is won, with equal probabilities of $\frac{1}{2}$, by either Peter or Paula; the winner then receives one point. Successive rounds are independent. Each has staked $50 for a total of $100, and they agree that the game ends as soon as one of them has won a total of 5 points; this player then receives the $100. After they have completed four rounds, of which Peter has won three and Paula only one, a fire breaks out so that they cannot continue their game.

a. How should the $100 be divided between Peter and Paula?

b. How should the $100 be divided in the general case, when Peter needs to win a more rounds and Paula needs to win b more rounds?

a. In the case where Peter has 3 wins and Paula has 1, Peter needs to win 2 times in the next 5 rounds, whereas Paula needs to win 4 times in the 5 rounds.

We can enumerate all the cases, or we can just play 5 rounds and count the times where Peter wins 0 or 1 times out of the 5, to find out Paula’s winning chance:

$$\left(\genfrac{}{}{0ex}{}{5}{0}\right){\left(\frac{1}{2}\right)}^5+\left(\genfrac{}{}{0ex}{}{5}{1}\right){\left(\frac{1}{2}\right)}^5=\frac{6}{32}$$

This is $\frac{3}{16}$, or Peter should receive $\frac{13}{16}$ of the portion, and Paula should receive $\frac{3}{16}$.

b. In the general case, we can apply the same technique:

The number of rounds that can be played is $a+b-1$.

We can apply the same formula in the general case, where we count $[1,a)$ and multiply it by ${\frac{1}{2}}^{a+b-1}$.

5. How Often Do Head and Tail Occur Equally Often?

According to many people’s intuition, when two events, such as head and tail in coin tossing, are equally likely then the probability that these events will occur equally often increases with the number of trials. This expectation reflects the intuitive notion that in the long run, asymmetries of the frequencies of head and tail will “balance out” and cancel. To find the basis of this intuition, consider that $2n$ fair and independent coins are thrown at a time.

a. What is the probability of an even $n:n$ split for head and tail when $2n=20$?

b. Consider the same question for $2n=200$ and $2n=2000$.

a. To count the even n : n split, we can count the number of possible outcomes and subtract out the non n : n splits to get the chance of an n : n split.

In the case of 20 coin flips, there are $2^{2}0$ or about 1 million ways to flip them in a way where order doesn’t matter.

To count the number of times where there are exactly 10 heads and 10 tails:

$$\left(\genfrac{}{}{0ex}{}{20}{10}\right)=\frac{10!\cdot 10!}{20!}$$

Or about 184,756. Dividing this by $2^{2}0$, we get 17.62%.

b. This approaches the normal distribution as $n$ increases.

We can approximate this probability as the following, trying to create a rectangle of the area below the curve:

$$\frac{1}{\sqrt{n\pi }}$$

6. Sample Size vs. Signal Strength

An urn contains six balls — three red and three blue. One of these balls — let us call it ball A — is selected at random and permanently removed from the urn without the color of this ball being shown to an observer. This observer may now draw successively — at random and with replacement — a number of individual balls (one at a time) from among the five remaining balls, so as to form a noisy impression about the ratio of red vs. blue balls that remained in the urn after A was removed. Peter may draw a ball six times, and each time the ball he draws turns out to be red. Paula may draw a ball 600 times; 303 times she draws a red ball, and 297 times a blue ball. Clearly, both will tend to predict that ball A was probably blue. Which of them — if either — has the stronger empirical evidence for his/her prediction?

They both have the same amount of empirical evidence. The only thing that matters is the difference between the red and blue balls being drawn, which is 6 for both.

To calculate the probability that the ball drawn was blue for Peter, we can use Bayes theorem, calculating the probability that the ball drawn was blue or red:

$$P(A=b\mid ⟨rrrrrr⟩)=\frac{1}{1+{\left(\frac{2}{3}\right)}^6}=0.919$$

For Paula, we get the same result:

If we pull blue:

$$P(303r\wedge 297b\mid A=b)=\left(\genfrac{}{}{0ex}{}{600}{303}\right){\left(\frac{3}{5}\right)}^{303}{\left(\frac{2}{5}\right)}^{297}$$

If we pull red:

$$P(303r\wedge 297b\mid A=r)=\left(\genfrac{}{}{0ex}{}{600}{303}\right){\left(\frac{2}{5}\right)}^{303}{\left(\frac{3}{5}\right)}^{297}$$

The chance of blue out of both blue and red:

$$P(A=b\mid 303r\wedge 297b)=\frac{1}{1+{\left(\frac{2}{3}\right)}^6}$$

Which is the same as above.

7. Birthday Holidays

The following problem is described in Cacoullos (1989, pp. 35–36). A worker’s legal code specifies as a holiday any day during which at least one worker in a certain factory has a birthday. All other days are working days. How many workers (n) must the factory employ so that the expected number of working man-days is maximized during the year?

Each worker can work up to 365 days a year. However, if they have the same birthday, everyones workdays go down by 1. Thus, we want to find the point at which

8. Random Areas

Peter and Paula both want to cut out a rectangular piece of paper. Because they are both probabilists they determine the exact form of the rectangle by using realizations of a positive rv, say $U$, as follows. Peter is lazy and generates just a single realization of this rv; he then cuts out a square that has length and width equal to this value. Paula likes diversity and generates two independent realizations of $U$. She then cuts out a rectangle with width equal to the first realization and length equal to the second realization.

a. Will the areas cut out by Peter and Paula differ in expectation?

b. If they do, is Peter’s or Paula’s rectangle expected to be larger?

9. Maximize Your Gain

A nonnegative rv $U$ has DF $F$ and density $f=F^{\prime }$; its mean $\mu$ and variance ${\sigma }^2$ are both finite. A game is offered, as follows: you may choose a nonnegative number $c$; if $U>c$ then you win the amount $c$, otherwise you win nothing.

As an example, suppose $U$ is the height (measured in cm) of the next person entering a specific public train station. If you choose $c=100$ then you will almost surely win that amount. A value of $c=200$ would double your amount if you win, but of course drastically reduce your winning probability.

a. Find an equation to characterize the value of $c$ that maximizes the expected gain.

b. Give a characterization of the optimal value of $c$ in terms of the hazard function of $U$ (see page 2 for the definition of the hazard function).

c. Derive $c$ explicitly for an exponential rv with rate $\lambda$ (see page 1 for a definition). How large is the maximum expected gain?

10. Maximize Your Gain When Losses Are Possible

Under the same assumptions as in Problem 9, the rule of the game is changed, as follows: if $U>c$ an amount of $c$ is won, but otherwise the amount $c$ is lost.

a. Characterize the value of $c$ that maximizes the expected gain, $G(c)$.

b. Can the game be unfavorable, i.e., can even the maximum expected gain become negative?

c. How large is the maximum expected gain for an exponential rv with rate $\lambda$? Compare this to the solution of the previous problem; explain the difference.

11. The Optimal Level of Supply

A man offers milk to the spectators of the weekly baseball matches. Before each match, he orders $c$ units of milk at a price of $b>0$ per unit; during the match he sells each unit for a price of $s>b$. Because the milk cannot be kept fresh for a week, each unsold unit imposes a loss of $b$. The actual demand $D$ of milk varies from week to week, according to a strictly increasing DF $F$ with density $f$. Assume that $F(0)=0$ and that $D$ is a continuous rv with finite mean.

a. For given prices $b$, $s$, and a given demand structure $F$, the milkman can choose among different values of $c$. Consider $c$ as a continuous variable, and denote as $G(c)$ the expected net gain as a function of $c$. Discuss the shape and qualitative properties of $G$.

b. What is the optimal amount of milk, $c_{\mathrm{opt}}$, the man should stock in order to maximize his expected net gain?

12. Mixing RVs vs. Mixing Their Distributions

The concept of a “mixture distribution” is used in probability and its applications in at least two different ways that have quite different meanings.

Let $X$ and $Y$ be two independent normal rvs, with means ${\mu }_x=50$ and ${\mu }_y=150$ and standard deviations of ${\sigma }_x={\sigma }_y=10$. Consider the rv $A$ defined by

$$A=\frac{1}{2}(X+Y)$$

The idea here is that in each realization both a value of $X$ and a value of $Y$ are generated, and half of each is added to produce the value of $A$. Thus, in a fairly direct sense each individual realization of $A$ contains one part of $X$ and one part of $Y$. In this sense, $A$ is a $50/50$ mixture of $X$ and $Y$, much like one mixes half a pound of butter and half a pound of flour.

Next, consider an rv $B$ that comes, in each realization, with probability $1/2$ from a normal distribution (namely, that of $X$) with mean $\mu =50$ and standard deviation $10$ and with probability $1/2$ from a normal distribution (namely, that of $Y$) with mean $\mu =150$ and standard deviation $10$. Thus, its density is equal to

$$f_B(b)=\frac{1}{2}\left[n(b\mid \mu =50,\sigma =10)+n(b\mid \mu =150,\sigma =10)\right],$$

where $n(\cdot \mid \mu ,\sigma )$ is the normal density with mean $\mu$ and standard deviation $\sigma$. The idea here is that in each realization either a value of $X$ or a value of $Y$ is generated (but not of both) with equal probability, and that this value then determines that of $B$. However, across many realizations, the density of $B$ will still represent a $50/50$ mixture of $X$ and $Y$.

a. Sketch the densities of $A$ and $B$. Is $A$ normally distributed? Is $B$?

b. Derive the means and variances of $A$ and $B$. Compare and explain.

c. Let $X,Y$ be two arbitrary but independent rvs with densities $f_X,f_Y$; means ${\mu }_x,{\mu }_y$; and standard deviations ${\sigma }_x,{\sigma }_y$. Let $0\le p\le 1$ be a proportion mixing either the rvs themselves:

$$A=p\cdot X+(1-p)\cdot Y$$

or their densities

$$f_B(b)=p\cdot f_X(b)+(1-p)\cdot f_Y(b).$$

Derive for this more general case the means and variances of $A$ and $B$.

13. Throwing the Same vs. Different Dice

The standard binomial sampling scheme assumes $n$ independent trials with a constant probability of success. Suppose, for example, that we are given a single fair die, and that the success event consists in throwing, say, a $5$. Thus, each single throw results in a success with probability $p=1/6$. Within $n=144$ independent trials, we would thus expect about $n\cdot p=24$ successes, and the variance of the number of successes would be equal to $n\cdot p\cdot (1-p)=20$.

a. Suppose we are given two biased dice. Die A will show a $5$ with probability $p=1/4$, and die B shows a $5$ with probability $p=1/12$. With these two dice 144 trials are conducted, as follows. For the first 72 trials we use die A, and during the last 72 trials we use die B. Given that the average success probability across both dice is equal to $(1/4+1/12)/2=1/6$, we would again expect 24 successes. Is the variance of the number of successes larger than, equal to, or smaller than with the standard binomial sampling scheme?

b. Again we are given the two dice A and B described in a. This time, however, we select at random one die and then throw this selected die 144 times, observing again the number of throws yielding a $5$. Is the variance of the number of successes larger than, equal to, or smaller than with the standard binomial sampling scheme?

c. Once again we are given the two dice A and B. This time, however, in each of 144 trials we start by choosing at random one of the two dice, then throw it and observe whether or not the trial yields a $5$. Is the variance of the number of successes larger than, equal to, or smaller than with the standard binomial sampling scheme?

14. Random Ranks

Peter draws $n=100$ independent realizations of a continuous rv and ranks them in increasing order from 1 to 100. Subsequently, Paula draws a single value from the same population and inserts this value into the rank order created earlier by Peter. For example, if her value is such that 50 of Peter’s draws are smaller and 50 are larger, then the rank associated with her draw would be 51 - that is, overall, her value would be the 51st in increasing order. Or, if her value is smaller than all 100 of Peter’s, then the rank 1 would be associated with it.

a. Is it more likely that Paula’s value will occupy rank 51 than rank 1?

b. Derive for general $n$ the probability that Paula’s value will occupy rank $k$, where $1\le k\le n+1$.

15. Ups and Downs

Ups and downs occur in probability just as in real life. An elementary probability version with the real-life property that ups are more often followed by downs is as follows.

Let $(A,B,C)$ be three independent and identically distributed continuous rvs that are realized sequentially: first $A$, then $B$, and finally $C$. Let us say that an increment occurs with $B$ if $B>A$, and a decrement otherwise. Similarly, an increment occurs with $C$ if $C>B$, and a decrement otherwise.

a. Suppose you are told only that $B$ has led to an increment, but not the actual value of $B$. Argue that conditional on this information $C$ is twice as likely to yield a decrement than an increment, even though $B$ and $C$ are independent.

b. Random variables are said to be exchangeable if their joint density is the same for any permutation of its arguments. Thus, if $(A,B,C)$ are exchangeable, then their density, say $f(a,b,c)$, is the same for any permutation of $(a,b,c)$. Note that exchangeable rvs may still be dependent. Does the property described in a. still hold if $(A,B,C)$ are exchangeable?

c. Suppose that the three rvs have the same marginal distribution with mean $\mu$ and variance ${\sigma }^2$, and let them have the common pairwise correlation $\rho$. Show that the correlation of the rvs $B-A$ and $C-B$ is generally equal to $-1/2$.

16. Is 2X the Same as X1 + X2?

Let $X$ be a continuous rv with density $f$, and let $X_1,X_2$ be two independent rvs, both distributed as is $X$. It is then not usually the case that the rv $2X$ is distributed as is $X_1+X_2$. However, the Cauchy density whose standardized form is given by

$$f(x)=\frac{1}{\pi }\cdot \frac{1}{1+x^2}$$

possesses this property: $X_1+X_2$ has the same distribution as the rv $2X$. It is illustrated and compared to the standard normal distribution in Figure 1.1.

a. Based on the property described earlier, argue without any explicit calculation that the variance of the Cauchy distribution is necessarily infinite.

b. Give an inductive argument for the rather unintuitive feature that for the Cauchy distribution the arithmetic mean from a sample of $n=2^k$ independent realizations of $X$ has exactly the same distribution as each contributing summand itself.

17. How Many Donors Needed?

To organize a charity event that costs $100, an organization raises funds. Independent of each other, one donor after another gives some amount of money (considered as a continuous quantity here) that is exponentially distributed (for definition, see page 1), with a mean of $20. The process is stopped as soon as $100 or more has been collected. Find the distribution, mean, and variance of the number of successive donors needed until at least $100 dollars has been collected.

18. Large Gaps

Denote as $\{N(t),t\ge 0\}$ a Poisson process with rate $\lambda$. Let $W$ be the waiting time until for the first time no event in $\{N(t)\}$ has occurred for the last $\tau$ time units. Derive $E[W]$. For example, $\{N(t)\}$ might stand for the sequence of cars passing at a crosswalk. A pedestrian needs $\tau$ seconds to cross the road. How long does it on average take before he has reached the other side of the road?

19. Small Gaps

Related to the preceding problem, let $U$ be the waiting time until two events occur within $\tau$ time units. Derive $E[U]$.

In some applications, the event that has latency $U$ is called a “coincidence.” For example, a volume of biological tissue could be permanently destroyed when two damaging particles are absorbed within $\tau$ (or less) time units. The idea here is that following the first absorption the tissue needs to recover for $\tau$ time units; this opens a window of vulnerability during which a further (second) particle has a lethal effect.

20. Random Powers of Random Variables

Let $M,N$ be independent Poisson rvs (for definition, see page 1), with parameters (means) $\alpha ,\beta \ge 0$, respectively. Define a rv $Q\equiv M^N$.

a. Find the expectation $E[Q]$.

b. Does $E[Q]$ generally increase as a function of $\alpha$ and $\beta$? Explain.

21. How Many Bugs Are Left?

Peter and Paula are given copies of the same text for independent proofreading. Peter finds 20 errors, and Paula finds 15 errors, of which 10 were found by Peter as well. Estimate the number of errors remaining in the text that have not been detected by either Peter or Paula.

22. ML Estimation with the Geometric Distribution

Let the discrete rv $K$ have a geometric distribution (see page 1),

$$P(K=k)=p\cdot (1-p{)}^{k-1},k=1,2,\dots$$

and let $\{k_i;i=1,\dots ,n\}$ be $n$ independent realizations from it.

a. Find the maximum likelihood (ML) estimate (for definition, see page 2), $\hat{p}$, of $p$. Give a simple argument why $\hat{p}$ overestimates $p$.

b. Determine the asymptotic standard error, $a.s.e.(\hat{p})$, of this estimate as a function of $p$. Consider the asymptotic standard error as $p\to 0$ and as $p\to 1$. For which value of $p$ is $a.s.e.(\hat{p})$ maximal?

c. Explain the apparent paradox that $\hat{p}$ is, on the one hand, the reciprocal of a sum of rvs (which in turn will tend to normality), and on the other hand that it is itself asymptotically normally distributed.

23. How Many Twins Are Homozygotic?

Twin pairs are either homozygotic (with probability $\alpha$) or dizygotic (probability $1-\alpha$). In the first case, their common sex is genetically determined only once, for both twins together, so that they must necessarily be of the same sex. In the second case, the twins’ sexes are determined independently and therefore could potentially be different. Suppose that generally in any genetic sex determination in a given population the outcome will be “male” ($m$) with probability $\beta$ and “female” ($f$) with probability $1-\beta$.

Obviously, the sex-related classification of any given twin pair (i.e., $mm$ vs. $ff$ vs. $mf$) is readily apparent, whereas the determination of their homo- vs. dizygotic status requires elaborate genetic diagnostics. Specifically, if the two twins are of opposite sexes (i.e., the pair is $mf$) then they must necessarily be dizygotic, but if they are not (i.e., an $mm$ or $ff$ pair) then they may be either homo- or dizygotic.

a. L. v. Bortkiewicz (1920; for a summary, see von Mises, 1931, p. 407) has collected the sex-related classification for a total of $n=17798$ twin pairs born in Berlin from 1879 to 1911. The frequencies were $n(mm)=5844$, $n(ff)=5612$, and $n(mf)=6342$. From these data, how could one estimate the latent probability $\alpha$ of a twin pair to be homozygotic?

b. Is the estimate of $\beta$ significantly different from $1/2$?

24. The Lady Tasting Tea

In 1935, R. A. Fisher presented the famous “lady tasting tea” problem. A lady claims she can tell whether in a tea-plus-milk infusion the tea (type $T$ cup) or the milk (type $M$ cup) was poured in first. To test this claim, the lady is presented with $n$ cups of type $T$ and $n$ cups of type $M$; her task is to tell these two sets, which differ in nothing other than the order in which tea and milk were poured in, apart. More specifically, after she has been informed that there is an equal number of $T$ and $M$ cups, she first tastes the content of each of the $2n$ cups and then singles out those $n$ of them that she thinks are, say, of type $T$; this implies that she considers the remaining $n$ cups to be of type $M$.

As a simple model of this situation, suppose the lady can identify the true state of any given cup with probability $\theta$, $0\le \theta \le 1$; with probability $1-\theta$, she can only guess. Of course, the skeptic’s claim would be that $\theta =0$. Judgments about successive cups are considered to be independent.

a. Define a “hit” to occur when a cup is correctly classified. Explain why the number of hits is necessarily the same for both the set of $n$ cups the lady designates as type $T$ and the remaining set of $n$ cups she considers to be of type $M$. Therefore, if we know the number of hits in the first set, the number of hits in the second set is redundant.

b. What is the probability $p_n(k\mid \theta )$ that among the $n$ cups she selects as being of type $T$, there will be exactly $k$, $0\le k\le n$, hits, i.e., cups that are in fact of type $T$?

c. Suppose the lady’s ability to tell $T$ and $M$ apart corresponds to $\theta =0.50$ in the preceding model. Thus, while not perfect, she clearly does much better than guessing. Using the largest possible significance level below the conventional $\alpha =0.05$, how large is her probability $\beta (n)$ to demonstrate her ability in an experiment involving $n=4,10$, and $20$ cups of each type? A computer is required to do these calculations.

25. How to Aggregate Significance Levels

Suppose a one-sided statistical test is based on a statistic $\Lambda$ that under the $H_0$ has the distribution function $F$, the rejection region being formed by large values of $\Lambda$. The test is applied to a first set of data and the statistic $\Lambda ={\lambda }_1$ is observed, with an associated observed p-value of $p_1=P(\Lambda >{\lambda }_1)=0.09$. On a second independent set of data the same test is applied, and this time a statistic $\Lambda ={\lambda }_2$ is observed, with an associated observed p-value of $p_2=P(\Lambda >{\lambda }_2)=0.07$.

a. Argue that the observed p-value is a rv that is uniformly distributed on $[0,1]$ if $H_0$ is true.

b. From a., derive a way to aggregate $p_1,p_2$ into a single overall p-value.

c. With the procedure from b., describe the set of $\{(p_1,p_2)\}$ that is judged overall-significant at some given theoretical significance level $\alpha$, such as $\alpha =0.05$.

26. Approximately How Tall Is the Tallest?

Let $U$ be a continuous rv with strictly increasing DF $F$, and let $M_n$ be the largest of $n$ independent realizations of $U$.

a. Determine the probability of the event $\{M_n\le F^{-1}(1-x/n)\}$, where $F^{-1}$ is the inverse function of $F$.

b. From a., derive an approximation, valid for medium and large $n$, for the DF, say $G_n(m)$, of $M_n$.

c. Show that if $U$ is exponentially distributed, then $M_n$ tends to the double exponential distribution.

d. Suppose the height (measured in cm) of males in a country is normally distributed, with mean $\mu =176$ cm and standard deviation $\sigma =8$ cm. Sketch the approximate DF of the height of the tallest male, if there are (i) 4 million males (as in a small country) or (ii) 120 million males (as in a large country). Compare these graphs to the corresponding figures for females, assuming that their height is normally distributed, with mean $\mu =170$ cm and standard deviation $\sigma =7$ cm.

27. The Range in Samples of Exponential RVs

Let $T$ be an exponential rv with density $\lambda e^{-\lambda t}$; the parameter $\lambda$ is called the rate of the density (cf. page 1). It is not difficult to show that $E[T]=1/\lambda$. Exponential rvs have two interesting properties.

(i) The minimum of independent exponential rvs, which may have different rates, is again exponentially distributed, with a rate equal to the sum of the individual rates. For example, the minimum of two independent exponential rvs with rates ${\lambda }_1$ and ${\lambda }_2$ is again an exponential rv, with rate ${\lambda }_1+{\lambda }_2$.

(ii) If an exponential rv $T$ is larger than some other positive independent rv, let us call it $Q$, then the excess $T-Q$ is again exponentially distributed with rate $\lambda$. This characteristic feature holds also, as a special case, when $Q$ is a constant, say $q$. It is sometimes called the “lack-of-memory” property: given that $T$ exceeds $Q$, then its additional lifetime, from there on, has exactly the same distribution as the original lifetime, just as if the process had “no memory” for the already-elapsed period of time.

We draw a sample of $n$ i.i.d. realizations of $T$ (for the meaning of i.i.d., see page 1). Let the rv $T_{(i)}$, $i=1,\dots ,n$, be the $i$th smallest value in this sample. The positive rv $R_n=T_{(n)}-T_{(1)}$ is the largest minus the smallest value; it is called the range of the sample. Of course, the value of $R_n$ will vary from sample to sample, according to some distribution that depends on $\lambda$ and, of course, on $n$.

a. Use the two properties described earlier heuristically to derive an expression for the expectation of $R_n$.

b. Similarly, reason heuristically to find the distribution function of $R_n$ without explicit calculations.

28. The Median in Samples of Exponential RVs

As in Problem 27, let $T$ be an exponential rv with density $\lambda e^{-\lambda t}$. We draw a sample of size $n=2k-1$ ($k$ a positive integer) i.i.d. realizations of $T$, and estimate the median of $T$ by the $k$th order statistic of this sample, i.e., the midmost element of the sample that is both larger and smaller than $k-1$ other realizations. Call this median estimate $M$.

a. Derive the density of $M$.

b. Find an expression for the expectation of $M$, and show that for small $k$ it is severely biased.

29. Breaking the Record

Consider a process in which, in a chronological (sequential) order, the i.i.d. realizations of an rv $X$ with parent density $f$ and DF $F$ are generated, yielding the sequence $\{X_i;i=1,2,\dots \}$. From time to time, it will then be the case that a realization $X_i$ occurs that is larger than the largest value that has been seen so far. In accordance with the everyday meaning of this term, such a realization is called a record. By this definition, $X_1$ is necessarily a record. Also, the second record is the first realization of $X$ that is larger than $X_1$ was. In general, the $r$th record is the first realization of $X$ that is larger than the $(r-1)$th record. Denote the density of the $r$th record as $g_r(x)$.

a. Consider a sequence of $n=100$ realizations of $X$. How many records could there be minimally and maximally? Give a simple recursive argument, relating the cases $n-1$ and $n$, to find how many records one would on average expect to see in $n=100$ realizations. Explain why the number of records is independent of the parent distribution $f$.

b. Derive (in terms of $f,F$) the density of the second record, $g_2(x)$.

c. Give an inductive argument to establish the general result that the density

$$g_r(x)=f(x)\cdot \frac{1}{(r-1)!}\cdot \{-\ln [1-F(x)]{\}}^{r-1}.$$

d. Explain that in the exponential case of $f(x)=\lambda e^{-\lambda x}$ the result derived in c. could be anticipated without explicit calculations from the lack-of-memory property described in the statement of Problem 27.

30. Paradoxical Contribution

Two large predatory birds, A and B, feed on the same habitat. Bird A’s daily prey (in grams) is normally distributed with mean ${\mu }_A=60$ g and ${\sigma }_A=5$; for bird B, ${\mu }_B=40$ g and ${\sigma }_B=10$. Thus, on average, bird A is a more successful predator; also, its amount of daily prey is less variable than that of bird B. Clearly, on average, the daily overall prey of both birds together equals 100 g. However, as both birds feed on the same habitat, the correlation, $\rho$, of their daily prey is negative across days, say $\rho =-0.8$: any given animal in the habitat that has fallen prey to A is no longer available to B, and vice versa. Let us assume that the situation is adequately described by a bivariate normal distribution.

On an unusually successful day, A and B together manage to reap 175 g.

a. Try to estimate from the information given how much, in expectation, bird A’s share was on days in which A and B together have reaped 175 g.

b. Try to answer the question in a. if the birds feed on different habitats so that $\rho =0$.

31. Attracting Mediocrity

Peter has an IQ of 90 whereas the IQ of Paula is 110. However, due to unsystematic biological or psychological day-to-day variation that is unrelated to the IQ per se, any single measurement of either IQ is distorted by an independent additive measurement error that has a zero-mean normal distribution with variance ${\sigma }^2$. For example, if Paula’s IQ were measured repeatedly, the outcomes would be normally distributed with a mean of 110 (her “true” IQ) and a standard deviation of $\sigma$.

a. Suppose that either Peter or Paula is selected at random ($p=1/2$), and his/her IQ is measured. You do not know who was selected, but you are told that the result of this first measurement is 105. Now the same person, whose identity is unknown to you, is measured a second time. What is your prediction for the outcome of this second measurement if $\sigma =3$?

b. Answer the same question if $\sigma =20$.

c. Suppose that instead of just contemplating Peter and Paula we now deal with a large population of individuals whose true IQs (an rv that we may call $T$) is normally distributed with mean ${\mu }_T$ and variance ${\sigma }_T^2$. As before, each individual measurement is distorted by an independent and additive normally distributed error $E$ that has zero-mean and variance ${\sigma }^2$. A single person is drawn at random from this large population, his/her IQ is measured, and the outcome is some above-average value $x>{\mu }_T$. Show that when the IQ of that same person is measured a second time, the expected outcome of the second (“repeated”) measurement is larger than ${\mu }_T$ but smaller than $x$.

32. Discrete Variables with Continuous Error

Sometimes an underlying latent (i.e., not directly observable) rv $N$ of theoretical or practical interest is generically integer-valued, such as a binomial, Poisson, or geometrical rv. However, measurements $S$ of it are distorted by a random error $E$ that is continuous. For example, a random integer number of coins of one type (e.g., dimes) is inserted into a vending machine with a control balance that registers the total weight, say in units of the nominal weight of a single dime. In this example, there will be an integer number of coins, but due to minting imperfections, wear-off, and soiling, each coin will differ somewhat from its ideal nominal weight. Summing the individual deviations from the nominal weight across all coins we get the overall measurement error $E$.

If we assume that the measurement error $E$ (thought to be independent of $N$) has a normal distribution with zero mean and standard deviation $\sigma >0$, then the actually observed measurements are in fact realizations of the rv $S=N+E$. Let $f$ be the density of $S$.

a. Characterize in words or with a figure the changes of the density $f$ as $\sigma$ varies from a near-zero value to larger values. When, approximately, will $f$ change from multi- to unimodality? To fix ideas, consider the example of a geometric distribution, $P(N=n)=p\cdot (1-p{)}^{n-1}$, $p=0.2$.

b. Argue that $f$ can also be interpreted as a discrete mixture of normal densities with means $\mu =n$ and common standard deviation $\sigma$, the mixture weights being given by $P(N=n)$.

33. The High-Resolution and the Black-White View

Ideally, many properties and processes of the real world show extremely fine gradations and variations. However, in order to measure and to process this information, we are often forced to simplify and reduce it. For example, many measurement devices digitize quantities (such as force, voltage, or time) that in theory are conceptualized as continuous variables. An extreme form of data compression is thresholding, which reduces a continuous input into a simple binary output. Two important questions then are: how should this threshold be set, and how much information do we lose along the way? The following problem deals with these two questions in a simple but exemplary context.

Consider a sample of $n$ independent realizations $\{t_i,i=1,\dots ,n\}$ of an exponential rv with density $f(t\mid \mu )=(1/\mu )\exp (-t/\mu )$. Researchers A and B both seek to estimate the unknown mean, $\mu$. A observes the $n$ original raw data and can make use of the full information to estimate $\mu$. B on the other hand obtains the data only after they have passed a digital filter with a threshold-type mechanism, such that each realization is classified as 0 if it is $\le c$, and classified as 1 if it is $>c$. Suppose $k$, $0<k<n$, of the $n$ realizations turned out to be $\le c$.

a. Derive the ML estimate of $\mu$ and its asymptotic standard error (a.s.e.) from the raw data as seen by researcher A.

b. For a given threshold value $c$, derive the ML estimate of $\mu$ from the digitized data as seen by researcher B. Does this estimate correspond to your intuition?

c. Find the a.s.e. of this estimate from the data as seen by researcher B.

d. For a given value of $\mu$ what is the “best” threshold value of $c$, i.e., the one that minimizes the a.s.e. determined in part c.? How large, relative to the estimate in a., is this a.s.e.?

34. The Bivariate Lognormal

The standard parametric model for simple correlation and regression contexts is the bivariate normal distribution. It is quite interesting to see how important characteristics regarding, e.g., the correlation coefficient change under a different bivariate distribution model, such as the lognormal distribution.

Let $X$ be a normal rv with mean ${\mu }_x$ and standard deviation ${\sigma }_x$. Then the rv $U=\exp (X)$ has a univariate lognormal distribution. A basic result about the rv $U$ is that its expectation is equal to

$$E[U]=\exp \left({\mu }_x+\frac{1}{2}{\sigma }_x^2\right).$$

Similarly, let the rvs $(X,Y)$ have the bivariate normal distribution with parameters $({\mu }_x,{\mu }_y,{\sigma }_x,{\sigma }_y,\rho )$, where ${\mu }_y,{\sigma }_y$ are the mean and standard deviation of $Y$, and $\rho$ is the correlation of $X$ and $Y$. Then the pair $(U=\exp (X),V=\exp (Y))$ has a bivariate lognormal distribution. Note that $(U,V)$ are a pair of nonnegative rvs.

a. Based on the result given earlier concerning the expectation of $U=\exp (X)$, derive without new explicit calculations the variance of $U$.

b. Assume that $(X,Y)$ are uncorrelated, $\rho =0$. In this case, are the rvs $(U,V)$ uncorrelated, too?

c. Explain why for $\rho =+1$ the $(U,V)$ are perfectly correlated, too, if ${\sigma }_x={\sigma }_y$. Why does this result not extend to $\rho =-1$ as well, even though in the bivariate normal model the cases $\rho =+1$ and $\rho =-1$ are perfectly symmetric?

d. Using the results from a. and b., show that the correlation of $U$ and $V$ equals

$$\mathrm{corr}(U,V)=\frac{\exp (\rho {\sigma }_x{\sigma }_y)-1}{\sqrt{(\exp ({\sigma }_x^2)-1)(\exp ({\sigma }_y^2)-1)}}.$$

35. The arcsin(sqrt(p)) Transform

Let the rv $N$ be the number of successes in $n$ independent trials, each with success probability $p$. Clearly, $N$ is binomially distributed with parameters $n$ and $p$, and $\hat{p}=N/n$ is the usual estimate of $p$. It is unbiased, that is, $E[\hat{p}]=p$, and its variance is $\mathrm{Var}[\hat{p}]=p(1-p)/n$, which varies as a quadratic function of $p$. However, many applications, especially in analysis of variance and regression contexts, require variables that may differ in mean but not in variance across conditions. To circumvent this problem, it is customary to analyze the strictly increasing transformation $\arcsin (\sqrt{\hat{p}})$ rather than $\hat{p}$ itself.

a. Let $g(p)$ be a differentiable function of $p$. Argue heuristically that if $n$ is large, then

$$g(\hat{p})=g(p+\Delta p)\approx g(p)+\Delta p\cdot g^{\prime }(p),$$

where $\Delta p$ is the sampling error $\hat{p}-p$ of the estimate $\hat{p}$. From this representation, derive the approximate mean and variance of the rv $g(\hat{p})$.

b. At first glance, the choice of the transform $p\mapsto g(p)=\arcsin (\sqrt{p})$ seems rather exotic. Give a principled rationale for this particular choice.

36. Binomial Trials Depending on a Latent Variable

In many situations, the outcome of binomial trials is modeled through an underlying latent, that is, not directly observable, rv $X$ with strictly increasing DF $\Phi$, such that each of the binomial trials yields a success if and only if $X\le c$. For example, in a population of individuals the susceptibility to flu may follow a particular population distribution. A flu is successfully avoided by an individual $i$ if his or her (standardized) susceptibility $X_i$ does not exceed some critical, but unknown, flu threshold $c$. From observations of the relative flu frequency in a random sample taken from this population, one would then like to estimate the threshold parameter, $c$.

Suppose $n$ binomial trials yield $k$ successes, leading to the usual estimate $\hat{p}=k/n$ for the true success probability $p=\Phi (c)$. Thus, on equating $\hat{p}=\Phi (\hat{c})$, the natural estimate, $\hat{c}$, of the model parameter $c$ is ${\Phi }^{-1}(\hat{p})$, where ${\Phi }^{-1}$ is the inverse function of $\Phi$.

Of course, from sample to sample our estimate $\hat{c}$ will vary, just as the relative success probability, $\hat{p}$, of which it is a function. Determine the approximate standard error (i.e., the standard deviation) of the estimate $\hat{c}$ based on a sample of $n$ trials if $\Phi$ is

a. the standard logistic DF, $\Phi (x)=1/[1+\exp (-x)]$,

b. the standard normal DF.

37. The Delta Technique with One Variable

Let $X$ be an rv with expectation $\mu$ and variance ${\sigma }^2$, and let $f(X)$ be a given, known function of $X$. In general then

$$E[f(X)]\ne f[E(X)].$$

a. Find a series-based approximation to $E[f(X)]$ when $\sigma$ is small so that $X$ is concentrated in the neighborhood of $\mu$.

b. Apply this result to the case of $f(x)=e^x$ when $X$ is a normal rv with mean $\mu$ and variance ${\sigma }^2$. Compare this approximation to the exact result for this case, $E[\exp (X)]=\exp (\mu +\frac{1}{2}{\sigma }^2)$, as given in Problem 34. When will the approximation be acceptable?

c. A machine is constructed to throw a ball vertically with an initial velocity of 120 [m/s]. However, due to imperfections of the machine, the actual angle varies from throw to throw according to a normal distribution with a mean of $90^{\circ }$ and a standard deviation of $10^{\circ }$. What is, approximately, the expected maximum height that the ball will reach, neglecting the air resistance?

38. The Delta Technique with Two Variables

Problem 37 can be generalized to functions of more than one rv. Thus, let $X,Y$ be two rvs with expectations ${\mu }_x,{\mu }_y$, variances ${\sigma }_x^2,{\sigma }_y^2$, and correlation $\rho$. Also, let $f(X,Y)$ be a given function of these rvs.

a. Find a series-based approximation to $E[f(X,Y)]$ that holds when $X$ and $Y$ are concentrated in a region around the point $({\mu }_x,{\mu }_y)$.

b. Apply this result to the case of the ratio of two independent rvs, i.e., $f(x,y)=x/y$ and $\rho =0$.

c. Explain how the solution in part b. may also be obtained from the solution of Problem 37, related to functions of a single rv.

d. Consider the special case of an rv defined as

$$F_{m,n}=\frac{U/m}{V/n}$$

where $U,V$ are independent ${\chi }^2$-rvs, with $m,n$ degrees of freedom, respectively. Compare for this case the approximation to the exact result, namely the mean of the $F$-distribution, which is $n/(n-2)$.

e. Suppose that in question c. of Problem 37 in addition to, and independent of, the variation of the angle $\alpha$, the initial velocity $v_0$ of the ball varies according to a normal distribution with ${\mu }_{v_0}=120$ [m/s] and ${\sigma }_{v_0}=10$ [m/s]. Under these conditions, what is, approximately, the expected maximum height the ball will reach?

39. How Many Trials Produced a Given Maximum?

Let $N$ be a positive discrete rv with probability distribution $p(n)$ and associated probability generating function $g(z)=E[z^N]$ (cf. page 2). Also, let $X_i$ be continuous i.i.d. rvs, with a common DF $F$. The rv defined by

$$M=\underset{i=1,\dots ,N}{\max }\{X_i\}$$

is then a random maximum, the largest of a random number (namely, $N$) of rvs (namely, the $X_i$). It is intuitively clear that $M$ and $N$ are positively related. If $N$ is large, then $M$ will, on average, tend to be larger, because it is then the maximum of a larger number of i.i.d. realizations.

a. Consider the simple case that $N$ equals (with probability $1/2$) either 1 or 2. This implies that $E[N]=1.5$. Assume the $X_i$ to be uniform rvs on $[0,1]$. This means that the random maximum $M$ is (with equal probability) either simply $X_1$ (namely, if $N=1$) or the maximum of $X_1$ and $X_2$ (i.e., if $N=2$). Given that $M=0.9$, what is the conditional expectation of $N$?

b. Show that in general

$$r(x)=E[N\mid \max (X_1,\dots ,X_N)=x]=1+F(x)\cdot \frac{g^{\prime \prime }[F(x)]}{g^{\prime }[F(x)]}.$$

c. Consider $r(x)$ for the case that $N$ has a geometric distribution, so that $P(N=n)=p\cdot (1-p{)}^{n-1}$.

d. Consider $r(x)$ the special case in which $N$ equals either 1 or $k>1$, both with probability $1/2$. For a uniform rv, $F(x)=x$, $0\le x\le 1$, the case $k=2$ corresponds to part a.

40. Waiting for Success

The probability $p$ of a certain event is usually estimated by looking at how often it occurs in $n$ independent trials. If this frequency is $k$, then the usual estimate of $p$ is $k/n$. In this procedure, $n$ is fixed in advance, independent of the outcome of the individual trials.

An alternative way to estimate $p$ is to look at how long (i.e., how many trials) it takes to achieve a preset number $r$ of successes. With this procedure, the total number of trials required is an rv, $N_r$. Intuitively, the larger $N_r$ is, the smaller will be our estimate of $p$.

a. Use $E[N_1]$ to derive a moment estimate, say $\hat{p}$, for $p$. In Problem 22 we already saw that $\hat{p}$ is also the ML estimate of $p$ derived from $N_1$. Find the expectation of this estimate. Is $\hat{p}$ biased?

b. Argue that $E[N_r]=r\cdot E[N_1]$, and derive a moment estimate for $p$ based on $N_r$. Use the results from Problem 37 to derive its approximate expectation. When will this approximation be valid?

c. Haldane proposed the improved estimate $\hat{p}=(r-1)/(N_r-1)$ for $r>1$. What is the bias of Haldane’s estimate?