Chapter 6

Question 1

Definition: dot products

Answer 1

For vectors u, v in R^n the inner product or dot product of u and v is given by u*v = u^T * v, where u and v and viewed as n x 1 matrices and is a scalar.

Question 2

Theorem 6.1 (Properties of the Dot Product)

Answer 2

Let vectors u, v, w be elements of R^n, c is an element of the real numbers. Then:

a) u dot v = v dot u
b) (u+v) dot w = u dot w + v dot w
c) (cu) dot v = c(u dot v) = u dot (cv)
d) u dot u >=0, and u dot u = 0 if and only if u = zero vector

Question 3

Definition: distance between u and v

Answer 3

For vectors u, v are elements of R^n, the distance between u and v, dist(u,v), is given by dist(u,v) = ||u-v||

Question 4

Definition: orthogonal vectors

Answer 4

Two vectors u and v are called orthogonal (u is perpendicular to v) if u dot v = 0

Question 5

Theorem 6.2 (The Pythagorean Theorem)

Answer 5

Two vectors u and v are orthogonal if and only if

|| u + v ||^2 = ||u||^2 + ||v||^2

Question 6

Orthogonal Complement

Answer 6

Let W be a subspace of R^n and let z be an element of R^n. The vector z is orthogonal to W if Z is orthogonal to every vector in W. The orthogonal complement of W, W perp is the set of all vectors orthogonal to the subspace W.

Question 7

Theorem 6.3

Answer 7

For any m x n matrix A, (Row A) perp = Nul A and (Col A) perp = Nul A^T

Question 8

Other facts about W perp

Answer 8

For a subspace W of R^n:

1: x is an element of W perp if and only if x is orthogonal to every element in a basis for W
2: W perp is a subspace of R^n

Question 9

Definition of Orthogonality

Answer 9

A set {u1, … , up} in R^n is orthogonal if ui * uj = o whenever i is not equal to j.
* An orthogonal set of unit vectors is called an orthonormal set.

Question 10

Theorem 6.4

Answer 10

If a set S = {u1, … , up} is an orthogonal set of nonzero vectors in R^n, the S is linearly independent.

Question 11

Orthogonal basis

Answer 11

An orthogonal basis for a subspace W of R^n is a basis that is also an orthogonal set.

Question 12

Theorem 6.5

Answer 12

Let {u1, … , up} be an orthogonal basis for a subspace W of R^n. For every vector y is an element of W, if
y = c1u1+ … + cpup, then cj = (yuj)/(ujuj) for j = 1, … , p

Question 13

Theorem 6.6

Answer 13

An m x n matrix U has orthonormal columns if and only if U^T * U = I

Question 14

Theorem 6.8 The Orthogonal Decomposition Theorem

Answer 14

Let W be a subset of R^n. Then each vector y is an element of R^n can be written uniquely as y = y hat + x, where y hat is an element of W and z is an element of W perp. In fact, if {u1, … , up} is an orthogonal basis for W, then

1: y hat= projection of y onto W = (yu1)/(u1u1)u1 + … + (yup)/(upup)up and
2: z = y - y hat

Question 15

Theorem 6.9 The Best Approximation Theorem

Answer 15

Let W be a subspace of R^n, a vector y is an element of R^n, y hat = projection of y onto W. Then y hat is the closest point in W to y, in the sense that:
|| y - y hat || < || y - v ||
for all v in W such that v is not equal to y hat.

Question 16

Theorem 6.10

Answer 16

If {u1, … , up} is an orthonormal basis for a subspace W of R^n and a vector y is an element of R^n, then
the projection of y onto W = (yu1)u1 + … + (yup)up = U*U^T y,
where U = [ u1 … up].

Question 17

Theorem 6.11 The Gram-Schmidt Process

Answer 17

Given a basis {x1, … , xp} for a subspace W of R^n,
v1 = x1
v2 = x2 - ((x2v1)/(v1v1))v1
v3 = x3 - ((x3v1)/(v1v1))v1 - ((x3v2)/(v2v2))v2
…
vp = xp - ((xpv1)/(v1v1))v1 - ((p3v2)/(v2v2))v2 - … -((xpvp)/(vpvp))v2p
Then {v1, … , vp} is an orthogonal basis for W

Question 18

Least Squares Theorem

Answer 18

The set of least squares solutions of Ax = b coincides with the nonempty set of solutions of the normal equations A^T Ax = A^T b. If the columns of A are linearly independent, then A^T A is invertible and there is a unique least - squares solution, namely
x hat = (A^T A)^-1 A^T b