joint distributions

Question 1

joint distributions and marginals

Answer 1

P(X=xi, Y=yi) = p_ij; note this may be notated with P(xi,yi), and reads: probability of both events occurring

marginal distribution: eg P(X=xi) = sum_j P(X=xi,Y=yj)

Question 2

independent random variables

Answer 2

X and Y are independent if their joint probability mass function is equal to the product of their two marginal distributions

P(X=xi,Y=yj) = P(X=xi)P(Y=yj) (for all i,j) (similarly for continuous distributions)

Question 3

covariance

Answer 3

• cov(X,Y) = E[ (X-E(X))(Y-E(Y)) ] = E(XY) - E(X)E(Y)
• independent variables have a covariance of 0
• the covariance matrix of random variables {X1,X2,…,Xn} is a symmetric matrix with each entry cov(Xi,Xj)

Question 4

correlation

Answer 4

• Pearson’s correlation
  * a kind of normalized covariance
  * assesses linear relationships between X and Y
  * corr(X,Y) = cov(X,Y) / sqrt(var(X)var(Y))
• Spearman’s
  * for X,Y, the Pearson correlation between the rank values of X and Y
  * assesses monotonic relationships between X and Y

Question 5

linear combinations of random variables

Answer 5

• for Y = a1X1 + … + anXn
• E(Y) = a1E(X1)+…+anE(Xn)
• if the variables are independent:
  
  • var(a1X1+…+anXn) = a1^2var(X1)+…+an^2var(Xn)
  • if Xi are normal, then Y is also normal
• if the variables are not independent:
  var(a1X1+a2X2) = a1^2var(X1)+2a1a2cov(X1,X2)+a2^2var(X2)
• if the Xi are “jointly normal,” then Y is also normal, by definition

Question 6

functions (compositions) of random variables / variable transformations

Answer 6

• given random variable X, we want distribution of random variable Y, Y=g(X)
• assuming f_X is the pdf of X, this can be viewed as a rescaling of the domain of f_X
• the conversion is performed via the CDF:
  
  • F_Y(y) = F_X(g^{-1}(y))
  • ie under the new distribution, p(Y<=y) is given in terms of known functions, F_X and the inverse of g
• note statistics (eg between 2 distributions, such as p-value for difference in means) are not necessarily preserved under random variable transformations