Theoretically best linear estimator is crucial in definition of Partial Autocorrelation in time series. It provides an estimation with correlated variable considered.
L ( X τ ∣ X 1 , X 2 , … , X n ) = β 0 + X 1 β 1 + X 2 β 2 + … + X n β n = β 0 + X ′ β { β 0 , β } = arg min β 0 , β E X τ , X [ ( X τ − L ( X τ ∣ X 1 , X 2 , … , X n ) ) 2 ] \begin{align}
L({X}_\tau|X_1,X_2,\ldots,X_n)=&\beta _0+X_{1}\beta _1+X_2\beta _2+\ldots+X_n\beta _n=\beta _0+X'\beta \\
\{\beta _0,\beta \}=&\mathop{\arg\min}\limits_{\beta _0,\beta } \mathbb{E}_{X_\tau,X}\left[ \left( X_\tau-L({X}_\tau|X_1,X_2,\ldots,X_n) \right)^2\right] \\
\end{align} L ( X τ ∣ X 1 , X 2 , … , X n ) = { β 0 , β } = β 0 + X 1 β 1 + X 2 β 2 + … + X n β n = β 0 + X ′ β β 0 , β arg min E X τ , X [ ( X τ − L ( X τ ∣ X 1 , X 2 , … , X n ) ) 2 ] Express equation ( 2 ) (2) ( 2 ) E \mathbb{E} E Σ ⋅ , ⋅ \Sigma _{\cdot ,\cdot } Σ ⋅ , ⋅
E X τ , X [ ( X τ − L ( X τ ∣ X ) ) 2 ] = E ( X τ 2 ) − 2 E ( X τ ( β 0 + X ′ β ) ) + E ( ( β 0 + X ′ β ) 2 ) = Σ X τ + E ( X τ ) 2 − 2 β 0 E ( X τ ) − 2 ( Σ X , X τ + E ( X τ ) E ( X ) ) ′ β + β 0 2 + 2 β 0 E ( X ) ′ β + β ′ ( Σ X + E ( X ) E ( X ) ′ ) \begin{align}
\mathbb{E}_{X_\tau,X}\left[ \left( X_\tau- L(X_\tau|X) \right)^2 \right]=&\mathbb{E}\left( X_\tau^2 \right) -2\mathbb{E}\left( X_\tau(\beta _0+ X'\beta ) \right) +\mathbb{E}\left( (\beta _0+X'\beta )^2 \right) \\
=&\Sigma _{X_\tau}+\mathbb{E}\left( X_\tau \right)^2\\
&-2\beta _0\mathbb{E}\left( X_\tau \right) -2\left( \Sigma_{X,X_\tau}+\mathbb{E}\left( X_\tau \right) \mathbb{E}\left( X \right) \right)'\beta \\
&+\beta _0^2+2\beta _0\mathbb{E}\left( X \right)'\beta +\beta '\left( \Sigma _X+\mathbb{E}\left( X \right) \mathbb{E}\left( X \right) ' \right)\\
\end{align} E X τ , X [ ( X τ − L ( X τ ∣ X ) ) 2 ] = = E ( X τ 2 ) − 2 E ( X τ ( β 0 + X ′ β ) ) + E ( ( β 0 + X ′ β ) 2 ) Σ X τ + E ( X τ ) 2 − 2 β 0 E ( X τ ) − 2 ( Σ X , X τ + E ( X τ ) E ( X ) ) ′ β + β 0 2 + 2 β 0 E ( X ) ′ β + β ′ ( Σ X + E ( X ) E ( X ) ′ ) where Σ X , X τ = c o v ( X , X τ ) \Sigma _{X,X_\tau}=cov(X,X_\tau) Σ X , X τ = co v ( X , X τ )
Its minimun w.r.t. { β 0 , β } \{\beta _0,\beta \} { β 0 , β }
0 = { ∂ ∂ β 0 = − 2 E ( X τ ) + 2 β 0 + 2 E ( X ) ′ β ∂ ∂ β = − 2 ( Σ X , X τ + E ( X τ ) E ( X ) ) + 2 β 0 E ( X ) + 2 ( Σ X + E ( X ) E ( X ) ′ ) ⇒ { β 0 = E ( X τ ) − E ( X ) ′ β β = Σ X − 1 Σ X , X τ \begin{align}
0=&\begin{cases}
\dfrac{\partial^{} }{\partial \beta _0^{}}=-2\mathbb{E}\left( X_\tau \right) +2\beta _0+2\mathbb{E}\left( X \right) '\beta \\
\dfrac{\partial^{} }{\partial \beta ^{}}=-2(\Sigma _{X,X_\tau}+\mathbb{E}\left( X_\tau \right) \mathbb{E}\left( X \right) )+2\beta _0\mathbb{E}\left( X \right) +2(\Sigma _{X}+\mathbb{E}\left( X \right) \mathbb{E}\left( X \right) ')
\end{cases}\\
\Rightarrow &\begin{cases}
\beta _0=\mathbb{E}\left( X_\tau \right) -\mathbb{E}\left( X \right) '\beta \\
\beta =\Sigma _{X}^{-1}\Sigma _{X,X_\tau}
\end{cases}
\end{align} 0 = ⇒ ⎩ ⎨ ⎧ ∂ β 0 ∂ = − 2 E ( X τ ) + 2 β 0 + 2 E ( X ) ′ β ∂ β ∂ = − 2 ( Σ X , X τ + E ( X τ ) E ( X ) ) + 2 β 0 E ( X ) + 2 ( Σ X + E ( X ) E ( X ) ′ ) { β 0 = E ( X τ ) − E ( X ) ′ β β = Σ X − 1 Σ X , X τ i.e. Best linear estimator
X ^ τ = L ( X τ ∣ X 1 , X 2 , … , X n ) = E ( X τ ) + ( X − E ( X ) ) ′ Σ X Σ X , X τ \begin{align}
\hat{X}_\tau=L(X_\tau|X_1,X_2,\ldots,X_n)=\mathbb{E}\left( X_\tau \right) +(X-\mathbb{E}\left( X \right) )'\Sigma _X\Sigma _{X,X\tau}
\end{align} X ^ τ = L ( X τ ∣ X 1 , X 2 , … , X n ) = E ( X τ ) + ( X − E ( X ) ) ′ Σ X Σ X , X τ in weak stationary time series with E [ X t ] = 0 , γ k , Γ k \mathbb{E}\left[ X_t \right]=0,\,\gamma _k,\Gamma _k E [ X t ] = 0 , γ k , Γ k
L ( X t + k ∣ X t , X t + 1 , … , X t + k − 1 ) = X t + k − 1 : t ′ Γ k − 1 − 1 γ k − 1 \begin{align}
L(X_{t+k}|X_{t},X_{t+1},\ldots,X_{t+k-1})=X_{t+k-1:t}'\Gamma _{k-1}^{-1}\gamma _{k-1}
\end{align} L ( X t + k ∣ X t , X t + 1 , … , X t + k − 1 ) = X t + k − 1 : t ′ Γ k − 1 − 1 γ k − 1