Linear Algebra Theory Flashcards
1
Q
What is a vector - mathematical interpretation
A
A vector is an ordered list of numbers
2
Q
Number of elements in a vector is called
A
Dimensionality
3
Q
What is a vector - geometric interpretation
A
A vector is a straight line with some length and some direction.
4
Q
Vector and coordinate position overlap at
A
Standard position
5
Q
What are the rules for vector addition or subtraction
A
The vectors to be added or subtracted must have same dimensions
6
Q
What scales a vector up and down?
A
Scalar
7
Q
Scalar is denoted by
A
greek letters - lambda, beta, alpha
8
Q
Scalar stretches a vector when
A
the scalar value is greater than 1
9
Q
Scalar flips a vector when
A
the scalar value is less than 1
10
Q
when scalar value is 0
A
The vector is at origin
11
Q
When a scalar is multiplied to a vector - it doesn’t change two things __________ and isn’t that notion contrary to what we know about when the scalar is negative?
A
- Direction and angle of the vector.
- Usually it is assumed the vector lies on a 1-D subspace that stretches in infinite direction.
12
Q
What are the several ways of performing vector-vector multiplication?
A
- Hadamard multiplication
- Dot product
- Cross product
- Outer product
13
Q
What is dot product?
A
Dot product provides a single number that provides the relationship information between two vectors.
14
Q
Dot product is also called as
A
Scalar product
15
Q
Mathematical notations for dot product
A
alpha = <a,b> = aTb = summation i = 1 to n ai*bi
16
Q
Rules to perform dot product on two vectors
A
Both the vectors must have same dimensions.
17
Q
What are the properties to which dot product is true
A
- Commutative
- Distributive
18
Q
What are the operations in which the dimensionality of the two vectors must be equal
A
- Vector addition
- Vector subtraction
- Vector dot product
- Hadamard multiplication
19
Q
What is commutative property in terms dot product?
A
a.b = b.a
20
Q
What is distributive property in terms of dot product
A
a.(b+c) = (a.b)+(a.c)
21
Q
For which properties is the dot product not true?
A
Associative property
22
Q
What is associative property in terms of dot product?
A
a.(b.c) != (a.b).c
23
Q
Definition of vector norm
A
Square root of the sum of each component squared.
24
Q
Formula for vector norm
A
sqrt(vT. V)
25
What are the other names for vector norm
Vector length or vector magnitude.
26
Dot product - geometric interpretation
alpha = |a|.|b|.cos(theta(a,b))
27
How to get the angle between two vectors
theta = acos((a.b) or alpha/|a|.|b|)
28
Cosine is bound by
-1 and 1
29
If theta < 90 degrees then alpha or dot product is
> 90 degrees
30
If theta > 90 then alpha or dot product is
< 90 degrees
31
If theta is 90 degrees then alpha is ____________ and that vector is also called as _____________
0; orthogonal vector
32
If theta is 0 degrees then alpha is
|a||b|
33
If theta is 180 degrees then alpha is
-|a||b|
34
What is cauchy schwarz inequality?
the absolute value of the dot product between two vectors is less than or equal to the product of magnitudes of the two vectors
|aTb| <= ||a||*||b||
35
Is the dot product sign affected by scalar multiplication
Yes
36
Hadamard multiplication is also called as
Element wise multiplication
37
What is the rule for Hadamard or element-wise multiplication
Both the vectors must be of same dimension
38
Formula for outer product
outer(v,w) = v.wT
39
What is the rule for outer product
The vectors don't have to be of same dimension.
40
The output of outer product is
matrix
41
What is the rule for vector cross product
Vector cross product can only be performed on 2 3-Dimensional vectors.
42
What is the output of vector product - dimensionality
3D Vector
43
Formula for cross product
[1;2;3] cross [a;b;c] = [2c-3b; 3a-1c; 1b-2a]
44
What does cross product represent
If two vectors v1 and v2 are used to represent a plane then the output vector v3 represent a vector perpendicular to that plane. (do fact check)
45
Complex numbers have angle with respect to
Positive real axis
46
Complex number format
a + bi
47
i^2 =
-1
48
i in complex number value is
sqrt(-1)
49
Complex numbers have ________ or _________ away from the origin of the complex plane
distance or magnitude
50
Hermitian transpose is also called as
Conjugate transpose
51
Hermitian transpose is performed for ________________
Complex vectors and complex matrices
52
The complex conjugate of a+bi
a-bi
53
Dot product between two complex vectors - v,w
vH. w or v*.W
54
Dot product between two complex vectors - v,w
vH. w or v*.W
55
What is a unit vector
A vector of length 1 is called unit vector.
56
How to normalize a vector?
mu = 1/norm(v)
For normalization
mu.v = normalized vector
norm(normalized vector) = 1
57
The magnitude of the dot product is _____ value of the dot product
absolute value of the dot product
58
What is dimensions
Dimension is the number of elements
59
What are fields
Field is a set of numbers on which addition, subtraction, multiplication and division are valid operations.
60
Example of fields -
Real numbers, complex numbers
61
Example of non-fields
Integers/counting numbers.
62
What is a subspace
A subspace is defined as the set of all vectors that can be created by taking the linear combination of some vector or set of vectors.
63
Linear combination is
1. The multiplication of a vector by a scalar
2. Addition of two vectors.
64
The scalar in scalar-vector multiplication can be ______
any real valued number
65
What is the formal definition for subspace
A vector subspace must be closed under vector addition, scalar multiplication and must contain zero vector.
66
Different vector from _____________ planes
Different
67
All the planes formed from different vectors converge at
origin
68
Ambient 3D space contains
1. 0D Subspace
2. infinite 1D subspace
3. Infinite 2D subspace
4. One 3D subspace
69
Why does having 2 vectors doesn't mean they form a plane
The formation of the plane depends on if one vector is dependent or independent on the other vector
70
What is a subset?
Subset is a set of some points that satisfies some conditions.
71
What are the conditions for a subset?
1. Doesn't need to include origin
2. Doesn't need to be closed
3. Can have boundaries.
72
What is span?
A region of space that you can reach by the linear combination of given vectors. The vectors 'span' that subspace.
73
What is thumb rule for dependent vectors
Any set of M > N vectors in RN - dependent vectors.
74
What is the thumb rule for independent vectors
Any set of M <= N vectors in RN - 'COULD BE' independent vectors.
75
Rules for basis
1. Mutually orthogonal - 90 degrees
2. Basis must be independent vectors
3. We can have different basis
76
Matrix terminology
1. Diagonal elements
2. Off diagonal elements
77
Rectangular matrices or
Non-square matrices
78
Matrix size - format
rows X columns
79
What is a symmetric matrix
Square Matrix elements are mirrored across the diagonal.
80
Square matrices
1. Symmetric matrix,
2. Skew symmetric matrix
3. Identity matrix
81
What is a skew symmetric matrix
Square Matrix elements are mirrored across the diagonal with flipped signs - with 0's as its diagonal.
82
What values fit for a skew-symmetrical matrix diagonal?
0's
83
Identity matrix
A square matrix with 1's as diagonal elements and 0's as off-diagonal elements.
84
Identity matrix is also a
Symmetric matrix
85
Multiplication property of identity matrix
A * I = A
86
Zero matrix
A matrix with all diagonal and off diagonal elements as 0
87
Diagonal matrix
A square or non-square matrix with zero or non-zero diagonal elements and zero off-diagonal elements is called as diagonal matrix
88
Relationship between diagonal matrix and identity matrix
D.I = D
89
Types of triangular matrices
1. Upper triangular matrix
2. Lower triangular matrix
90
What is upper triangular matrix?
1. Elements above the diagonal can be zero or non-zero
2. Elements below the diagonal are all zeros.
91
What is Lower triangular matrix?
1. Elements below the diagonal can be zero or non-zero.
2. Elements above the diagonal must be all zeros
92
What is the rule for concatenating matrices
The two concatenating matrices must have same number of rows.
93
Rule for matrix addition/subtraction
Must have same dimensions
94
Matrix addition/subtraction properties
1. Commutative - A+B = B+A
2. Associative - (A+B)+C = A+(B+C)
95
What is shifting a matrix?
Adding a scaled version of identity matrix
96
Properties of matrix shifting
1. Doesn't change off-diagonal elements.
2. Changes diagonal elements
3. Inflates a matrix
97
What is a linear operation?
Closed under matrix addition and scalar-matrix multiplication.
98
Is matrix-scalar multiplication a linear operation?
Yes
1. l(A+B)= lA + lB
2. la - al??
99
Transpose-Transpose of a matrix A
A.T.T = A
100
Transpose op vs symmetric matrix
For a symmetric matrix A - A.T = A
101
Transpose op vs skew-symmetric matrix
For a skew symmetric matrix A- A.T = -A
102
Hermitian transpose
Swaps signs for complex numbers
103
Matrix diagonal operation
Extracts the diagonal elements into a vector
104
Trace of a matrix
Sum of the diagonal elements of a matrix
105
Matrix types that diagonal and trace operations work on
1. Diagonal operation - square and non-square matrix
2. Trace operation- Only square matrices
106
Is trace a linear operator
Yes
1. trace(A+B) = tr(A) + tr(B)
2. trace(l * A) = l * trace(A)
107
What is broadcasting matrix arithmetic?
Addition of matrix and vector is invalid in traditional linear algebra. But via broadcasting the operation can be performed in modern machines. The row or column vectors are expanded as per the size of the matrix and then the operation is performed.
108
Matrix multiplication is valid when -
MXN * N*M = M*M matrix
109
5x7 * 5x2 matrix multiplication is
invalid
110
Why vector vector dot product is scalar
v1, v2 = 5 x 1
v1.T*v2 = 1x5 * 5*1 = 1x1
111
Ways to think about matrix multiplication
1. Ordered collection of dot products
2. Layered perspective
3. Column perspective
4. Row perspective
112
What is ordered collection of dot products perspective?
1. Similar to vector-vector dot product
(pin on right column, iterate on left rows - For each iteration/row analysis - element-wise and add both)
2. First row * first column - element-wise
[0 1] * a
c
= 0*a + 1.c
3. Second row * first column- element-wise
* a
[2 3] c
= 2 * a + 3.c
4. First row * third column
5. Second row * third column
113
What is layered perspective
1 left column to 1 right row - no element wise addition - forms a matrix
2nd left column to 2nd right row
and add both
1. First column * first row - no element wise addition
0 * [a b]
2
= 0a 0b
2a 2b
2. Second column * second row
1 * [c d]
3
= 1c d
3c 3d
0a 0b. +. 1c 1d
2a. 2b. 3c 3d
addition
1c 1d
2a+3c. 2b+3d
114
What is column perspective
1. Linear weighted combinations of the columns on left matrix
(pin on right column, for each element in right column, multiply it with respective right column)
first element of the left column * first right column +
second element of the left column * second right column
2. First left column * first element of the first column (scalar vector multiplication) +
second left column * second element of the first column
3. first left column * first element of the first column + second left column * second element of second column
115
What is row perspective
first element of the left row * first right row +
second element of the right row * second right row
116
Order of operation
(EVIL).T = L.T * I.T * V.T * E.T
117
Matrix vector multiplication - op is
vector
118
Matrix vector multiplication - vector decides ______
orientation of output vector
119
Matrix vector multiplication - matrix decides ____________
the size of the output vector
120
For a rotation matrix
1. No stretching
2. Vector maintains its length
3. Vector rotates by some angle
121
What is pure rotation matrix
Only rotation, no stretching
122
Impure rotation matrix
Rotation and stretching
123
What is Eigen vector and Eigen value
2 1. *. 1
2 3. 2
4
= 8
This can be expressed as 4 * 1
2
There is nothing special about the matrix or vector, but there is something unique about the combination of the matrix and vector.
When you can express the matrix-vector multiplication as some scalar-vector multiplication - the vector 1
2
is called Eigen vector and the scalar 4 is called the eigen value of the matrix.
124
What is the fundamental eigen value equation
AV = scalar.v
125
What are the two matrix identities
1. Additive identity - A + 0 matrix = A
2. Multiplicative identity - A * I = A
126
Create symmetric matrix from square non-symmetric matrix
S = (A+A.T)
or
S = (A + A.T)/2
127
Create symmetric matrix from non-square non-symmetric matrix
S = A * A.T
S = A.T * A
128
Sum of two symmetric matrices is
Symmetric matrix
129
Hadamard multiplication of symmetric matrices is
Symmetric matrix
130
Matrix multiplication of symmetric matrices
Non-symmetric matrix
131
How to get symmetric matrix from the multiplication of two symmetric matrices?
By adding constraints - a=d; e=g
a b. e f
c d f g
132
Does adding constraints for getting symmetric matrix from the multiplication of two symmetric matrices work for all matrix sizes?
No. only works for 2X2. Test - doesn't work for 10X10
133
For diagonal matrix D, what is the result of D*D, D.*D (multiplication and hadamard multiplication respectively)
They both result in same result
134
How to compute Frobenius dot product via vectorization?
1. Vectorize the matrices => vectors
2. Compute the dot product of the vectors.
135
How to compute Frobenius dot product via transpose-trace?
1. trace((A.T).B)
136
What is the best computational way to compute the Frobenius product?
Via transpose-trace matrix
137
What is Frobenius norm
Frobenius for product with itself is called Frobenius norm
138
Frobenius norm is also called as
Euclidean norm
139
Frobenius norm formula
norm(A) = √F = √tr((A.T).A)
140
What is P-norm formula
||A||p = sup ||Ax||p/||x||p;
x != 0
141
What is p-norm
1. P-norm gives the effect of matrix A on some vector x
2. We care if Ax is longer or shorter than x
142
What is 2-norm
If we replace p with 2 in p-norm, it called 2-norm
143
When is ||Ax||/||x|| = 1
If A is a pure-rotation matrix or orthogonal matrix, it rotates x but keeps the length of the vector same.
144
Schatten P-norm definition
Sum of all singular values of a matrix.
145
What are singular values?
1. Set of numbers
2. It is a set of scalars thats associated with every matrix
146
How to identify singular values?
Goal of SVD is to identify the singular values.
147
Schatten P-norm formula
||A||p = ( (summation i to r) sigma i to the power p) whole power 1/p
148
If p = 1,
Ignore 1/p and called schatten 1-norm
149
What is a self-adjoint matrix?
A linear transformation that is represented by a matrix is said to be self-adjoint or self-adjoint operator if the following statement is true
=
150
Rules for self-adjoint matrix
1. A is square matrix
2. A is symmetric matrix
3. v and w are of same size (m X 1)
151
What is matrix asymmetry index?
Computes a value/index on how symmetric a matrix is
152
Matrix asymmetry index formula
ai = ||A~|| / ||A|| => Ratio of norms
||A|| = √tr(A.A) = √tr((A.T).A)
A~ = (A-A.T)/2 (Asymmetric part of A)
153
Matrix asymmetry index for symmetry matrix
0 since (A-A.T) = 0
154
Matrix asymmetry index for asymmetric matrix
With A-A.t we are actually doubling the matrix and then dividing by 2. This means ||A~|| == ||A|| = > the index is 1
155
Matrix symmetry index
1 - (||A~||/||A||)
