Time series

Question 1

define a time series {r_t}

Answer 1

A sequence of random variables over time

Question 2

what does the models of AR, MA, and ARMA sort of “do”?

Answer 2

They attempt to make use of whatever information they have prior to some point, and the goal is to make a prediction by utilizing the linear relationship between the prior information and the prediction point.

Question 3

what is the ultimate focus on linear time series analysos?

Answer 3

the aspect of correlation. In time series, it is referred to as autocorrelation.

Question 4

What is the foundation of time series analysis?

Answer 4

A concept called stationarity.

Question 5

Define stationarity

Answer 5

The book differ between strict and weak stationarity.

Strict stationarity: A time series {r_t} is said to be strictly stationariy if the joint distribution (r_t1, r_t2, …, r_tk) and the joint distribution that has the same variables, but shifted by t, are the same. Same distribution, same mean, same variance all identical.

Weak stationarity: A time series {r_t} is said to be weakly stationary if both the mean of r_t and the covariance of r_t and r_{t-l} is time invariant. This means that E[r_t] = mu, and cov(r_t, r_{t-l}) = gamm_l, which depend only on l.
Make no mistake: This notaiton entail that each r_t in the time series {r_t} has the same mean. this makes sure that the trend is flat around this mean value.

Also take time to consider the covariance requirement. It say that the covaraince between r_t, which is just a random variable in the time series, and its lag-l corresponding random variable, is the same regardless of what r_t is. all that matters, is the lag-l. Thus, the covariance between each consecutive random variable in the time series must be the same, and the same goes for each second consecutive etc and so on. This is basically saying that the same relationship among all variables must remain true through the entire time series. Note that it is still very plausible that the covariance is close to 0 for many of the cases, which is totally fine. This is ultimately a part of what we want to figure out with time series analysis.

From the requirement of cov() = constant, we also get that the variance of each random variable in the time series must be the same.

Note as well what the cov() requirement does to seasonality. It enforce that there is no seasonality because of for instance how lag-1 covariance being constant for all consecutive random variables means that we cant get those seasonal swings.

Question 6

what is lag-l autocovariance

Answer 6

Facny word for the correlation between two random variables separated by a lag of “l” in a time series.

Question 7

what are we actually considering with {r_t}+

Answer 7

I believe a “true” time series with infinite random variables. I believe this is the case because of how the book describe samples of a time series as {r_t}^T_{t=0}

Question 8

how do we denote the lag-l autocorrelation of r_t?

Answer 8

p_l

Question 9

What is portmanteau and Ljung-box used for?

Answer 9

testing to see whether multiple differnet lag-l autocorrelations are 0 or nto

Question 10

define white noise

Answer 10

White noise is a time series consisting of iid random variables with finite mean and variance.

It is typical to use white noise with mean zero and variance equal to sigma^2, along with normal distribution. Then it is called gaussian white noise.

Question 11

what can we say about the ACF’s of white noise series+

Answer 11

all are zero because of how they are iid.

Question 12

what is actually being defined here?

Answer 12

Here we define what it would entail for the UNDERLYING PROCESS that governs the time series data generation to be linear. It is not related to a linear model.

it simply say that IF the time series is inherently linear, its data generation process will follow that formula.

This means that if we assume that some time series is linear, we basically assume that its data generation process can be described using that formula.

On the specifics, the assumption is that a linear time series follow a sum of weights and white noise. This indicate that the next data point is a function of all the previous movements multiplied by some weight.
Also, do not neglect the mu. This is the mean. The white noise series is more related to representing some kind of relationship regarding “what happens next if the shock was this”. we know that the mean is mu on aggregated metrics, but our goal is to relate the shocks to future movement in order to make our predicitons. We’re essentially looking to analyze how the time series react to certain shocks.

The more interesting part is how past values are not included. We’re only using the mean and how the time series react to shocks.

Question 13

AR models depend on r_(t-i) and not just the white noise. Can it then be defined as linear?

Answer 13

Yes, because we can rewrite the model (using the expected value for mu etc) to get:

r_t - mu = ø1 (r_{t-1} - mu) + a_t

and if we repeat the substitutions, we get that the result is linear, where the weights are powered increasingly. Since the ø’s must be less than 1 in absolute value, this sum will diminish.

Question 14

in the book, they do this for all models. They say that “for a weakly stationary AR(1) model….”. what exactly are they referring to? What is weakly stationary?

Answer 14

The model in its abstract form represent a process of data generation. The idea is that if the underlying process that we wish to describe with a model is stationary, we need our model to inhibit the same stationary features. Because of this, it is important to us that we know our model do this.

Stationarity is a baseline for time series because without stationarity in the data process that generate the time series in the first place, there is nothing to predict. THere will be no way to use a time series model to forecast. Because of this, we place great emphasis on the fact that our time sereis are at least weakly stationary.
And due to the fact that our time series are weakly stationary, we need a model that capture this. And the model should capture it with the same values as well, but this enters into the world of estimation.

Question 15

what is meant by “autocovariance”

Answer 15

Autocovariance determine how some random variable relate to its previous values. Covariance of the same random variable.

Question 16

what is the interpretation of autocovariance/covariance?

Answer 16

Nothing really. This is why we use correlation

Question 17

express corrlation using only gamma

Answer 17

correlation is defined as covariance divided by the product of standard deviation. Since standard deviation is the same regardless of which random variable we look at in the time series, we know that this is just the variance. thus we get “covariance/variance” and for some specific lag we simply take the covariance (auto) and divide on the variance. gamma_l / gamma_0.

Question 18

what is the autocorrelation function?

Answer 18

the graph we get from plottign the correlation values for each lag.

Question 19

what is the key regarding white noise series

Answer 19

the covariance (autocovariance) between different variables in the white noise series is always 0 unless we take the covariance with itself, which is the variance, which is sigma^2.

Question 20

what does th autocorrelation function of white noise look like?

Answer 20

A single peak of 1 at the lag 0, all 0 otherwise

Question 21

what is the distribution of a white noise autocorrelation function?

Answer 21

Note that this applies only for the sample autocorrelation function of a gaussian white noise series.

Normal with mean 0 and variance 1/T.

It will approximate zero variance for larger time series.

This means that while we should expect a value of 0 for hte autocorrelation, it will swing/fluctuate with a variance of 1/T.

We use this in hypothesis testing to see if something is basically white noise or not. If we use a t-ratio test, we’re cehcking the hypothesis that the mean is indeed 0. If it turns our that the observed values are CRITICALLY EXTREME for the t-ratio statistic, we know that the observed autocorrelation is likely not a result of a white noise series, because it indicates that there is significant autocorrelation in our sample for the specific lag we’re checking (autocorrelation is specified on a par-lag basis).

Question 22

how can we easily test if something is white noise? Say we have an autocorrelation funciton, and we want to establish bands indicating signifance?

Answer 22

Easiest ir pobably to set up confidence interval like this:

+- 1.96 x 1/sqrt(T)

this gives us the limits for 95% confidence.

Question 23

for MA models, we typically neglect the constant mu term. Why?

Answer 23

It considerably eases computaitons, and at no loss of generality. This is because we can obtain a zero-mean series by simply subtracting the mean, shifting all values/observations.

Question 24

how does a q’th order MA model look like+

Answer 24

Question 25

what is the mean of an MA(q) series?

Answer 25

mu

Question 26

what is the variance of an MA(q) series?

Answer 26

variance of mu is zero, so it removes. so we have the variance of a sum of white noise variables multiplied by their own theta value. The variance of theta part becomes a squared contribution, and the variance of the white noise random variables is sigma^2. So we get: var(MA(q)) = (1+theta^2_1 + theta^2_2 ...+theta^2_q)sigma^2

Question 27

what can we say about the covariance of MA(q)?

Answer 27

for s>q, the covariance is 0. up to q, the covariance may or may not be zero.

Question 28

An MA process produce some variable value based on shocks only. But the shocks are random and ocnsidered white noise, how can we predict anything based on this?

Answer 28

First of all, the models doesnt only rely on the white noise. We also have the mean component. As a result, a simple MA models is basically trying to see if there is a systematic way the time series move around its mean based on the specific shocks. For a true MA process, the time series will fluctuate around a mean, and these fluctuations would be completely determined by the shock. The shocks are considered random, but the reaction to it is not. The key is that the assumption when using MA models is that the variable we're intersted in will behave in a certain way as a reaction to the shocks. By using MA we're trying to learn the reaction. Also, it helps to consider the MA as a process. A process where a series of white noise together influence the reaction of the time series variable of interest. If a time series is a true MA process, we can capture it fully by an MA model.

Question 29

why is acf 0 for MA for lags s>q when the MA is MA(q)?

Answer 29

Here we are considfering a TRUE MA process. A true MA process use exactly q random white noise variables to produce the output. he reason why the autocorrelation function of MA(q) models cuts of at s>q lags is because an MA process use (by definition) exactly q variables to produce the reaction/output variable value. If the underlying process only use the latest q white noise shocks to determine the next step in our time series, then there is obviously no correlation to earlier lags. If there happened to be correlation to earlier lags, then it would simply not be a true MA(q) process, and'd have to increase q to compensate for this.

Question 30

define an AR(p) process

Answer 30

autoregressive process where the time series variable is determined completely by the p latest variable values.

Question 31

define an AR(p) model

Answer 31

A model that builds on the assumption that the past p variable values determine the next variable value, along with a single white noise term.

Question 32

Does an AR(1) model depend on more shocks than a_t?

Answer 32

if we consider an AR(1) process: it can be written as r_t = ø_0 + ø_1r_(t-1) + a_t Then we can apply recursively and get: r_t = ø_0 + ø_1(ø_0+ø_1r_(t-2)+a_(t-1))+a_t and so on, right? this creates a scenario where actually, modeling r_t as a process that only use 1 lag entail that we can consider it as a function of the shocks and the coefficients ø_0 and ø_1, which will create an infinite sequence that ultimately converge to a fixed point given that certain requirements for ø_0 and ø_1 are met. So yes, all AR models depend on all previous shocks. The key is really "convergence" of the sums that emerge.

Question 33

elaborate on the back operator

Answer 33

L, lag operator. L represent a single lag. L^x represent x lags. L is used in conjunction with r_t, y_t. For isntance: Ly_t is actually just y_{t-1}, one lag. L^10 y_t is simply y_{t-10}

Question 34

elaborate on this formula

Answer 34

We have the mean, which is alos typically used as ø_0. then we have a sum of p terms, one per order of the AR(p) model we are considering. Each term in the sum consist of the corresponding parameter ø_i, and the lagged variable. The lagged variable is simply represented as L^k y_t. this would be y_{t-k}. The benefit of using the lag operator is that we can separate y_t from the equation. Then we remain with L^k which is a specific function.

Question 35

elaborate on this formula

Answer 35

This formula is the simple AR(p) model, but it is heavily compact. We have defiend a function ø(L), which is a function of ø_i params and back operators/lag operators. ø(L) = (1 - ø_1L - ø_2L^2 - .... - ø_p L^p) The formula have simply moved the orgiginal RHS sum to the left hand side, and extracted the common y_t using the power of the lag variables. Of course, separating y_t allow us to isolate ø(L), which we refer to as the characteristic equaiton.

Question 36

elaborate on finding the characteristic equation

Answer 36

We start by the dense form of the AR(p) model: ø(L) y_t = mu + u_t Setting my = 0 by performing a shift in all the variables we observe (so that we dont lose generality), we get: ø(L) y_t = u_t this means, we also have: y_t = (ø(L))^(-1) u_t y_t is equal to the inverse of the function ø(L) multiplied by u_t. And, we know that the mean must be zero, so we naturally want y_t to approximate zero. Therefore, the inverse of ø(L) must converge to zero. This will happen if the impact that earlier variables have on the current variable will diminish as we go further and further back.

Question 37

important card. elaborate on values for phi in AR models

Answer 37

Again it is important to understadn the process. If some process is AR(1) and phi_1 is larger than 1, then it will simply explode. It will explode because the process will generate new numbers that are bigger and bigger and basically just grow to infinity. The idea is that if we let the time series continue, what can we say about its heading. For stationarity, we want it to remain flat around some mean.

Question 38

what is the mean of an AR model?

Answer 38

Not the first phi value at least. we need to consider the convergence. the key is to understand that the mean is constant for a stationary process, which means that we can use the property that the mean of y_t is the same as the mean of Ly_t which is the same as the mea nof L^2 y_t and so on. This allows us to separate/isolate an expression for the expectation.

Question 39

why cant we just check the phi values of the AR(p) model to see whehter it will converge or not?

Answer 39

Because for orders higher than 1, it is a recurrence relation that consist of more than 1 phi, and then the relationship is not that straight forward. The recurrence can converge without both phis being less than 1. Stationarity depends on the combination.

Question 40

how do we check whether AR model is stationary or not

Answer 40

solve characteristic polynomial. Roots must be outside of unity. Behind this is math surrounding concepts: - Recurrence relations - eigenvalues

Question 41

is the process y_t = y_{t-1}+u_t stationariy?

Answer 41

No, it is actually known as a random walk. formally, we can check it: y_t (1-øL) = u_t 1/(1-øL) is the inverse 1-øL = 0is the characteristic eq. 1 = øL L = 1/ø L = 1/1 = 1 Root is 1. 1 is not outside unity, which means that it is non-stationary.

Question 42

what is yule walker?

Answer 42

Yule-Walker is a set of equations that can be solved as a system of equations. It basically solve all the autocorrelations.

Question 43

define ARMA process

Answer 43

ARMA process is a time series where the next value is determined based on a combination of a some past values and linear combination of past noise. It combines AR with MA. We basically just add them together. Since we add them together, ARMA will have a geometrically declining ACF. Therefore, the first step of identifying ARMA is probably to notice the declining ACF. The same happens for the MA step and PACF. ARMA has geometrically declining ACF and PACF. the effects are additive.

Question 44

elaborate on intuition behind ACF of MA process

Answer 44

We're asking "is there a correlation between current y_t and y_{t-k}". And for a true MA process of the k'th order, we know that y_{t-k} was determined based on whatever the noise at that time was. And since we also include this noise in the determination of hte current point, there is a correlation between them. At the same time, this highlights why the ACF for moving averages suddenly drop. The true MA process simply doesnt rely on any common information when the time backwards is larger than the order.

Question 45

Define PACF

Answer 45

PACF is the correlation between lags after accounting for the effects that intermediary lags have.

Question 46

what can cause alternating ACF/PACF

Answer 46

negative values in the MA

Question 47

what do we mean by information criteria?

Answer 47

a two term thing. It will have some term related to the RSS, and one term related to punishing having more parameters. When we add a parameter to a model, the RSS will likely drop (can never increase) but the penalty term will increase. These are competing effects that determine whether we consider it a good choice or not. When we use informaiton criteria as the basis for selection, we want to minimize it.

Question 48

name the most popular information criteria, and give their forms

Answer 48

AIC, SBIC, HQIC. They are based on the same things. sigma^2 is the residual sum of squares, the RSS AIC = ln(sigma^2) + 2k/T SBIC = ln(sigma^2) + (k/T) lnT HQIC = ln(sigma^2) + (2k/T)ln(ln(T))

Question 49

Recall the expression that any true linear time series process must follow. Why does it not include any past values of the random variable we're looking at?

Answer 49

r_t = mu + ∑w_i a_{t-i} [i=0, infinity] It doesnt include any past variables r_{t-l} because these can be recursively expanded. The recursion ultimately bottoms out (in the infinite) and we will have only shocks and weights.

Question 50

why do we need the mean to be constant for stationarity?

Answer 50

If it is not constant, the values we use in the autocorrelation function will be differnet depending on where we are along the time series, which makes our predicitons bad.

Question 51

we know that white noise autocorrelation is typically assumed to be normally distributed. The sample distribution is normal with mean 0 and vairance 1/T. why do we give af?

Answer 51

We can use this to test whether a time series is white noise (just random mess) or not. If we have a large time series sample, the variance is extremely low. This means that if we use the t-ratio test to test whether the correlation for a certain lag is zero or not, we obtain really good values.

Question 52

why does the MA model definition not multiply a paramter theta with the a_t noise term?

Answer 52

because of how we defined linear models. That term would be index 0 in the sum which would cause theta to be power of 0, which gives 1. Helps to consider the AR collapsed version which creates powers of the weights.

Question 53

for MA models, what is key in computing ACF etc?

Answer 53

We need to remember that crossproducts are 0, because of how white noise series is iid. This is in the context of taking the expected value of a cross product. Since independent normal variables allow us to do E[XY]=E[X]E[Y], we can make use of the fact that the expected value of each random variable in the white noise series is zero. This is why we can neglect the cross products.