Date & time: Monday, November 18, 1:30-2:20

Place: Classroom

Coverage: Penney chapter 4, class notes: Orthogonality

Be prepared to do problems and use knowledge from the following areas.

• Coordinates
  • The "point" X = Q X' where X' is the coordinate vector with respect to the basis (the "natural" coordinates of which are) given by the columns of Q. Therefore, X' = Q^-1 X.
  • Perpendicular, orthogonal, orthonormal
  • Dot product theorem
  • Calculating coordinates with respect to an orthogonal basis
• Projections and the Gram-Schmidt process
  • Projections
  • The Fourier Theorem gives the projection of a vector onto a subspace, given an orthogonal basis. (The first "B₀" in line 2 of the theorem should be "B".)
  • The Gram-Schmidt process
  • QR and QN (or GN) factorizations (pp. 224-225/class notes)
  • "S perp": see Exercises 4.2(10), (12).
• Orthogonal matrices
  • The unintuitive definition of an orthogonal transformation (See the Rigid Motions problem) and an orthogonal matrix (p. 246: |AX| = |X| for all X).
  • Properties of orthogonal matrices/transformations: preservation of lengths, distances, and angles
  • Theorem: A square matrix A is orthogonal iff its columns are orthonormal iff its rows are orthonormal iff A^tA = I.
• Least squares
  • Linear models/regression models: y = b₀ x₀ + ... + b_k x_k + error.
    For i^th datum, y_i = b₀ x_i,0 + ... + b_k x_i,k + error_i.
  • Matrix version (stats version): Y = X b + error where I've written b instead of beta, because this html doesn't know Greek.
  • Normal equations (stats version): X^tX b = X^t Y.
    If the columns of X are independent, then the solution is b = (X^tX) ^-1X^tY. The parameters in the b vector then give the model that best fits the given data in the "least squares" sense: the length squared of the vector of deviations is minimized.
  • Orthogonal projection formula: If W is the subspace spanned by the columns of the data matrix X, "predicted/fitted" vector Y = proj_W Y = X b where b is a solution of the normal equations, and so proj_W Y = X(X^tX) ^-1X^tY. (This is Theorem 2 of Section 4.5.)
• "The Fundamental Theorem of Linear Algebra"--especially the orthogonal complement relationships between fundamental subspaces
  (But some people might say that the theorem on page 223 is the fundamental theorem of linear algebra (over the real numbers) because it characterizes all finite-dimensional vector spaces (with the scalars being the real numbers): "Any k-dimensional vector space is isomorphic with [to] R^k.")
  • Be able to determine bases for the four fundamental subspaces: the nullspace, the row space, the column space, and the left nullspace.
  • Know the orthogonal complementarity relations between the subspaces, and be able to draw two-dimensional pictures showing them.