BIG DATA ANALYSIS

Question 1

Descriptive analysis

Answer 1

What has happened?
- Access and manipulate past datas
- Inform decision-making

Question 2

Predictive analytics

Answer 2

Predictive analytics

What could happen in the future ?
- Use historical data to make future decision
- Estimation of variables

Question 3

prescriptive analytics

Answer 3

What should we do ?
- Optimization and simulation to provide advice
- Explore several possible actions and suggest course of action.
- Build models

Question 4

5 Vs of data

Answer 4

Volume, Velocity, Variety, Veracity, Value

Question 5

3 Statistical data types :

Answer 5

3 Statistical data types :

Cross-sectional data, Times-series data, Panel data

Question 6

Cross-sectional data

Answer 6

A given sort of entity for a single period of time

Question 7

Times-series data

Answer 7

For a single entity for multiple periods of time

Question 8

Panel data

Answer 8

Panel data
Multiples entities for multiple periods of time

Question 9

3 types of variables

Answer 9

numerical, categorical dummy

Question 10

numerical variables

Answer 10

Data that represent quantities or measurements.
Ex : age

Question 11

categorical variables

Answer 11

Data that represent distinct categories or groups. It
attributes a Number for each category.

Question 12

dummy variables

Answer 12

Data that represent categorical data in a set of binaries
(0 and 1) - La fonction indicatrice en Mathématiques -

Question 13

The Linear Regression Model: Definition

Answer 13

modeling method that postulates a relationship between dependent variable (𝒚) and one or more independent variables (𝑥1, 𝑥2, … , 𝑥𝑘)

y dep de 𝑥1, 𝑥2, … , 𝑥𝑘

Question 14

simple linear regression model

Answer 14

x1 only independent variable, y dep
𝑦 = β0 + β1 𝑥1 + ϵ

Question 15

Beta 0

Answer 15

C’est l’intercept (ordonnée à l’origine), c’est la valeur de y lorsque x=0.

Question 16

Beta 1

Answer 16

unknown slope : coefficient directeur

Question 17

β0 + β1 𝑥1

Answer 17

deterministic component of the linear model

Question 18

Positive
negative
no linear relationship

Answer 18

positive, pente vers le haut Beta1 positif negative, pente vers le bas Beta1 négatif
no relation : pente droite, B1 = 0

Question 19

For a multiple linear regression model
what is ∀𝑖 ∈ [1; 𝑘] β𝑖

Answer 19

unknown pop parameter associated with variable xi

Question 19

The multiple linear regression model:

Answer 19

Given 𝑦 the dependent variable, { x𝑖 | 𝑖 ∈ { 1, 2, 3, … , k } the dependent variables : 𝑦 = β0 + ∑ β𝑖𝑥𝑖
𝑘
𝑖=1
+ ϵ = β0 + β1x1 + β2x2 + ⋯ + βkxk + ϵ

Question 20

residual error

Answer 20

e = y - y^

Question 21

Ordinary least squares (OLS) formula:

Answer 21

SSE = ∑ 𝑒^2

Question 22

def OLS

Answer 22

used to find the minimum when summarize the squared errors between the observed data points and the values predicted by the linear model.

Question 23

Given 𝑌 1, 𝑌 2, … , 𝑌 𝑁 N dependent variables, using Matrix :
{ 𝑌 1 = β0 + β1𝑋1 + ϵ1
𝑌 2 = β0 + β1𝑋2 + ϵ2
⋮
𝑌𝑁 = β0 + β1𝑋𝑁 + ϵ𝑁

What is the value of Y?

Answer 23

Y (Y1
Y2
YN)
<=> Y = XBeta + epsilon

Question 24

SSE = ?

Answer 24

//Y-XBeta//^2

Question 25

The Linear Regression Model - Derivation

Answer 25

Beta^= (tXX)^-1* tXY

Question 26

Definition: The variance of the estimator 𝛃:

Answer 26

Measures the uncertainty of the estimated parameter. The smaller the variance is, the more confident we are when predicting the parameter.

Question 27

Definition: The variance of the estimator 𝛃^:

Answer 27

Measures the correlation of the two variables. A positive covariance indicates that the two variables are positively correlated, they increase together, and vis-versa.

Question 28

Defintion: The 𝛔𝟐̂:

Answer 28

Represents the residual variance - La variance des erreurs -, Measures the dispersion of observations around the regression model. (we don't know the exact 𝛔𝟐̂ because 𝛜 is uncertain)

Question 29

𝑉(β̂)

Answer 29

sigma carré (tXX)^-1

Question 30

𝑉(β̂) Matrix:

Answer 30

(var (Bo^). cov(Bo^, B1^) cov(B1^, B0^) var (B1^))

Question 31

𝑉𝑎𝑟(β0̂)

Answer 31

somme des xi carré / n* somme des (xi-Xbarre)^2

Question 32

𝑉𝑎𝑟(𝛽1̂)

Answer 32

sigma carré / n* somme des (xi-Xbarre)^2

Question 33

sigma carré

Answer 33

1 / n - (k-1) * somme des ei^2

Question 34

se

Answer 34

standard error of the estimate

Question 35

R ^2

Answer 35

coefficient of determination

Question 36

diff entre R^2 et R^2 ajusté

Answer 36

-R2 (coefficient de détermination) mesure proportion de la variance de la variable dépendante expliquée par le modèle, mais augmente toujours quand on ajoute des variables explicatives, même inutiles. -R2 ajusté corrige cette limitation en pénalisant l’ajout de variables non pertinentes, en tenant compte du nombre de prédicteurs et de la taille de l’échantillon ( il peut diminuer si l'ajout d'une variable n'a rien à voir avec les autres).

Question 37

Definitions: The sample variance 𝒔𝒆^2

Answer 37

measures the average squared deviation b/w th observed / predicted values

Question 38

standard error of the estimate se

Answer 38

standard deviation of the error of the estimation.

Question 39

formule se

Answer 39

racine de (SSE / n-k-1)

Question 40

def R^2

Answer 40

quantifies the sample variation in the dependent variable that is explained by the sample regression equation. ratio of the explained variation of the dependent variable to its total variation. compris entre 0 et 1

Question 41

SST def formule

Answer 41

somme des (yi - ybarre)^2

Question 42

SST

Answer 42

SSR + SSE

Question 43

SSR def formule

Answer 43

somme des (yi^- ybarre)^2

Question 44

R^2

Answer 44

SSR/SST=1-(SSE/SST)

Question 45

Definition: Adjusted R^2:

Answer 45

Definition: Adjusted R^2: - We cannot use 𝑅2 for model comparison when the competing models do not include the same number of independent variables (although dependent variable is the same). - 𝑅2 never decreases as we add more variables. It accounts for the sample size n and the number of independent variables k. - Imposes a penalty for any additional independent variables. - The higher the adjusted 𝑅2 the better model. When comparing models with the same dependent variable, the model with the higher adjusted 𝑹𝟐 is preferred. - Note that as n increases, adjusted 𝑹𝟐 gets closer to 𝑹𝟐 I.E. 𝑙𝑖𝑚 𝑛→+∞ 𝑎𝑗𝑢𝑠𝑡𝑒𝑑 𝑅2 → 𝑅2

Question 46

R^2 adjusted=

Answer 46

1 - (1- R^2) * (n-1 / n-k-1)

Question 47

Test of joint significance

Answer 47

Évalue si plusieurs coefficients (ou tous) sont significatifs ensemble. Utilise généralement un test F pour tester l’hypothèse H0:β1=β2=⋯=βk=0 Répond à la question : L’ensemble des variables explicatives améliore-t-il significativement le modèle ?

Question 48

F test formule

Answer 48

SSR/k / SSE/ n-k-1 MSR / MSE R^2 n-k-1 / 1-R^2 k

Question 49

p value for a F test

Answer 49

if p value = 0 reject the null hypo

Question 50

individual test of significance

Answer 50

->Évalue si un seul coefficient d’une régression (βi​) est significatif. ->Répond à la question : Cette variable a-t-elle un effet significatif sur la variable dépendante ?

Question 51

Ho et H1 for an individual test of significance for a two tailed test

Answer 51

Ho : Bi = 0 H1 : Bi =/= 0

Question 52

test statistic for a test of individual significance

Answer 52

bj- BjO / se(bj)

Question 53

p value for a test of individual significance

Answer 53

p value less than alpha reject h nutt

Question 54

CAPM equation

Answer 54

𝑅 − 𝑅𝑓 = α + β(𝑅𝑀− 𝑅𝑓) + ϵ * R : The rate of return on a stock or portfolio * RM : market return * Rf : risk-free interest rate * B values : measures how sensitive the stock's return is to changes in the market. 𝛽 =1 : change in the market = same change in the stock. 𝛽 >1 : stock is more aggressive or riskier than the market. 𝛽 <1 : stock is conservative or less risky than the market * alpha values : - 𝛼 : predicted to be zero, thus nonzero values indicate abnormal returns. Called the stock's alpha. 𝛼 > 0 : Positive abnormal returns 𝛼 < 0 : Negative abnormal returns

Question 55

t value

Answer 55

coefficient erreur type du coefficient

Question 56

MSR mean square regression

Answer 56

SSR / Df

Question 57

MSE

Answer 57

SSE / Df

Question 58

MST

Answer 58

SST / Df

Question 59

3 hypo de la regression OLS

Answer 59

Linéarité entre les variables indépendantes et dépendante (relation X / Y doit ê linéaire) Homoscédasticité: variance des résidus doit être constante à tous les niveaux de X amplitude des erreurs (la taille des résidus) à peu près la même pour tout X. Les variables explicatives ne doivent pas être parfaitement corrélées

Question 59

Etapes pour savoir si une variable est significant ou non

Answer 59

1) on calcule la t stat ( B1/se(B1)) 2)on regarde dans la t table à l'intersection entre la ligne de df( n-k-1) et la colonne du niveau de risque ( en général 5%-> donc 0,025 car il y a deux queues) 3) Si t stat > valeur critique du tableau: on rejette H0 et donc la variable sera significant

Question 60

In an ANOVA table F stat p-value t-stat

Answer 60

*F stat : determine whether independent variables collectively explain the variation in the dependent variable? *p-value : explique la significance individual des prédicateurs *t-stat : explique la significance d'un coefficient en particulier