Techniques For finding The Distribution Of A Transformation .

TRANSFORMATIONS OF RANDOM VARIABLES1. I NTRODUCTION1.1. Definition. We are often interested in the probability distributions or densities of functions ofone or more random variables. Suppose we have a set of random variables, X1, X2 , X3 , . . . Xn , witha known joint probability and/or density function. We may want to know the distribution of somefunction of these random variables Y φ(X1, X2, X3, . . . Xn). Realized values of y will be related torealized values of the X’s as followsy Φ (x1, x2, x3 , · · · , xn)(1)A simple example might be a single random variable x with transformationy Φ (x) log (x)(2)1.2. Techniques for finding the distribution of a transformation of random variables.1.2.1. Distribution function technique. We find the region in x1 , x2 , x3 , . . . xn space such that Φ(x1 ,x2 , . . . xn ) φ. We can then find the probability that Φ(x1 , x2 , . . . xn ) φ, i.e., P[ Φ(x1 , x2 , . . . xn ) φ ] by integrating the density function f(x1, x2 , . . . xn ) over this region. Of course, FΦ (φ) is justP[ Φ φ ]. Once we have FΦ (φ), we can find the density by integration.1.2.2. Method of transformations (inverse mappings). Suppose we know the density function of x. Alsosuppose that the function y Φ (x) is differentiable and monotonic for values within its range forwhich the density f(x) 6 0. This means that we can solve the equation y Φ (x) for x as a functionof y. We can then use this inverse mapping to find the density function of y. We can do a similarthing when there is more than one variable X and then there is more than one mapping Φ.1.2.3. Method of moment generating functions. There is a theorem (Casella [2, p. 65] ) stating thatif two random variables have identical moment generating functions, then they possess the sameprobability distribution. The procedure is to find the moment generating function for Φ and thencompare it to any and all known ones to see if there is a match. This is most commonly done to seeif a distribution approaches the normal distribution as the sample size goes to infinity. The theoremis presented here for completeness.Theorem 1. Let FX (x) and FY (y) be two cummulative distribution functions all of whose moments exist.Thena: If X and Y hae bounded support, then FX (u) and FY (u) for all u if an donly if E Xr E Yr for allintegers r 0,1,2, . . . .b: If the moment generating functions exist and MX (t) MY (t) for all t in some neighborhood of 0,then FX (u) FY (u) for all u.For further discussion, see Billingsley [1, ch. 21-22] .Date: August 9, 2004.1

2TRANSFORMATIONS OF RANDOM VARIABLES2. D ISTRIBUTION F UNCTION T ECHNIQUE2.1. Procedure for using the Distribution Function Technique. As stated earlier, we find the region in the x1 , x2 , x3 , . . . xn space such that Φ(x1 , x2 , . . . xn ) φ. We can then find the probabilitythat Φ(x1 , x2 , . . . xn ) φ, i.e., P[ Φ(x1, x2 , . . . xn ) φ ] by integrating the density function f(x1, x2 ,. . . xn ) over this region. Of course, FΦ (φ) is just P[ Φ φ ]. Once we have FΦ(φ), we can find thedensity by integration.2.2. Example 1. Let the probability density function of X be given byf (x) 6 x (1 x), 0 x 10 otherwise(3)Now find the probability density of Y X3 .Let G(y) denote the value of the distribution function of Y at y and writeG( y ) P ( Y y ) P ( X3 y ) P X y1/3Z y1/3 6x (1 x)dx Z(4)0y 1/3 6 x 6 x2 d x0 1/33 x2 2 x3 y0 3 y2/3 2yNow differentiate G(y) to obtain the density function g(y)d G (y)dy d 2/3 2y3ydyg(y) 2 y 1/3 2 2(y 1/3(5) 1), 0 y 12.3. Example 2. Let the probability density function of x1 and of x2 be given byf ( x 1 , x2 ) 2 e x1 2 x2 , x1 0 , x2 00 otherwise(6)Now find the probability density of Y X1 X2. Given that Y is a linear function of X1 and X2,we can easily find F(y) as follows.Let FY (y) denote the value of the distribution function of Y at y and write

TRANSFORMATIONS OF RANDOM VARIABLESFY (y) P (Y y)Z y Z y x2 2 e x1 2 x2 d x1 d x2Z0 y 0 2 e x1 2 x2 y0 x2 d x20Z y 2 e y x2 2 x2 2 e 2 x2d x2Z0 y 2 e y x2 2 e 2 x2 d x20Z y 2 e 2 x2 2 e y x2 d x23(7)0Now integrate with respect to x2 as followsF (y) P (Y y)Z y 2 e 2 x2 2 e y x2 d x20 e 2 x2 2 e y x2 y0 e 2 y 2 e y y e0 2 e y(8) e 2 y 2 e y 1Now differentiate FY (y) to obtain the density function f(y)d F (y)dy d e 2 y 2 e y 1dy 2 e 2 y 2 e yFY (y) (9) 2 e 2 y ( 1 e y )2.4. Example 3. Let the probability density function of X be given by2 1x µ1 · e 2 ( σ ) x σ 2π 1( x µ )2 x · exp 2 σ22 π σ2fX ( x ) (10)Now let Y Φ(X) eX . We can then find the distribution of Y by integrating the density functionof X over the appropriate area that is defined as a function of y. Let FY (y) denote the value of thedistribution function of Y at y and write

4TRANSFORMATIONS OF RANDOM VARIABLESFY (y) P ( Y y)eX y PZ P ( X ln y) , y 0 1( x µ )2 d x, y 0· exp 2 σ22 π σ2ln y (11)Now differentiate FY (y) to obtain the density function f(y). In this case we will need the rules fordifferentiating under the integral sign. They are given by theorem 2 which we state below withoutproof.Theorem 2. Suppose that f and f xare continuous in the rectangleR { (x, t) : a x b , c t d}and suppose that u0 (x) and u1 (x) are continuously differentiable for a x b with the range of u0 (x) andu1 (x) in (c, d). If ψ is given byψ (x) Zu1 (x)(12)f(x, t) d tu0 (x)thendψ dx xZu1 (x)(13)f(x, t) d tu0 (x) f ( x, u1 (x) )d u1 (x)d u0 f (x, u0 (x)) dxdxZu1 (x)u0 (x) f (x, t)dt xIf one of the bounds of integration does not depend on x, then the term involving its derivative will be zero.For a proof of theorem 2 see (Protter [3, p. 425] ). Applying this to equation 11 we obtainih2R ln yFY (y) 1 2 · exp ( x2 σ2µ ) d x , y 02 π σi h( ln y µ )21 1FY0 (y) fY (y) ·exp 22σy2 π σ2 R lny ddy 1y 2 π σ2 12 π σ2· exph · exph ( ln y µ )22 σ2( x µ )22 σ2i i dx(14)

TRANSFORMATIONS OF RANDOM VARIABLES5TABLE 1. Outcomes, Probabilities and Number of Heads from Tossing a CoinFour Times.Element of sample space 5TTTH24/625TTTT16/6253. M ETHODValue of random variable X(x)4333322222211110OF TRANSFORMATIONS ( SINGLE VARIABLE )3.1. Discrete examples of the method of transformations.3.1.1. One-to-one function. Find a formula for the probability distribution of the total number ofheads obtained in four tosses of a coin where the probability of a head is 0.60.The sample space, probabilities and the value of the random variable are given in table 1.From the table we can determine the probabilities as169621621681, P (X 1) , P (X 2) , P (X 3) , P (X 4) 625625625625625The probability of 3 heads and one tail for all possible combinations isP (X 0) 33325555or 3 132.55Similarly the probability of one head and three tails for all possible combinations is 32225555or

6TRANSFORMATIONS OF RANDOM VARIABLES 1 33255There is one way to obtain four heads, four ways to obtain three heads, six ways to obtain twoheads, four ways to obtain one head and one way to obtain zero heads. These five numbers are 1,4, 6, 4, 1 which is a set of binomial coefficients. We can then write the probability mass function asf(x) 4x x 4 x32for x 0, 1, 2, 3, 455(15)This, of course, is the binomial distribution. The probabilities of the various possible random variables are contained in table 2.TABLE 2. Probability of Number of Heads from Tossing a Coin Four TimesNumber of 2581/625Now consider a transformation of X in the form Y 2X2 X. There are five possible outcomesfor Y, i.e., 0, 3, 10, 21, 36. Given that the function is one-to-one, we can make up a table describingthe probability distribution for Y.TABLE 3. Probability of a Function of the Number of Heads from Tossing a CoinFour Times.Y 2 * (# heads)2 # of headsProbabilityNumber of Headsf(x)xyg(y)016/6250 16/625196/6253 96/6252216/62510 216/6253216/62521 216/625481/62536 81/6253.1.2. Case where the transformation is not one-to-one. Now let the transformation of X be given by Z (6 - 2X)2. The possible values for Z are 0, 4, 16, 36. When X 2 and when X 4, Y 4. We canfind the probability of Z by adding the probabilities for cases when X gives more than one value asshown in table 4.

TRANSFORMATIONS OF RANDOM VARIABLES7TABLE 4. Probability of a Function of the Number of Heads from Tossing a CoinFour Times (not one-to-one).Y (6 - (# heads))2Number of Headsxyg(y)03216/62542, 4216/625 81/625 29716196/62536016/6253.2. Intuitive Idea of the Method of Transformations. The idea of a transformation is to considerthe function that maps the random variable X into the random variable Y. The idea is that if we candetermine the values of X that lead to any particular value of Y, we can obtain the probability of Yby summing the probabilities of those values of X that mapped into Y. In the continuous case, tofind the distribution function, we want to integrate the density of X over the portion of its spacethat is mapped into the portion of Y in which we are interested. Suppose for example that both Xand Y are defined on the real line with 0 X 1 and 0 Y 10. If we want to know G(5), we needto integrate the density of X over all values of x leading to a value of y less than five, where G(y) isthe probability that Y is less than five.3.3. General formula when the random variable is discrete. Consider a transformation definedby y Φ(x). The function Φ defines a mapping from the sample space of the variable X, to a samplespace for the random variable Y.If X is discrete with frequency function pX , then Φ(X) is discrete and has frequency functionp Φ(X) (t) XpX (x)(16)x : Φ (x) t XpX (x)x Φ 1 (t)The process is simple in this case. One identifies g 1(t) for each t in the sample space of therandom variable Y, and then sums the probabilities.3.4. General change of variable or transformation formula.Theorem 3. Let fX (x) be the value of the probability density of the continuous random variable X at x. Ifthe function y Φ(x) is differentiable and either increasing or decreasing (monotonic) for all values withinthe range of X for which fX (x) 6 0, then for these values of x, the equation y Φ(x) can be uniquely solvedfor x to give x Φ 1(y) w(y) where w(·) Φ 1(·). Then for the corresponding values of y, the probabilitydensity of Y Φ(X) is given by 1 fX Φ 1 ( y ) · d Φd y (y) (y)dΦ(x)fX [ w (y) ] · d w6 0g (y) fY (y) (17)dydx 0 fX [ w (y) ] · w (y) 0 otherwise

8TRANSFORMATIONS OF RANDOM VARIABLESProof. Consider the digram in figure 1.F IGURE 1. y Φ(x) is an increasing function.As can be seen from in figure 1, each point on the y axis maps into a point on the x axis, thatis, X must take on a value between Φ 1 (a) and Φ 1 (b) when Y takes on a value between a and b.Therefore P (a Y b) P Φ 1 (a) X Φ 1 (b)R Φ 1 (b)(18) Φ 1 (a) fX (x) d xWhat we would like to do is replace x in the second line with y, and Φ 1 (a) and Φ 1 (b) with aand b. To do so we need to make a change of variable. Consider how we make a u substitutionwhen we perform integration or use the chain rule for differentiation. For example if u h(x) thendu h0 (x) dx. So if x Φ 1(y), thendx Then we can writed Φ 1 (y )dy.dy