Chapter 4 Subspaces (Zijing Huang).ipynb

Kernel: SageMath 10.1

# import toEquations from Snigdha's code

#functions
var('x y z a c d e')
xVect = vector([x, y, z, a, c, d, e])
# Takes in a transposed matrix (matrix[[col][col]].transpose()) and a vector b
# Throws an exception if b's elements is not equal to A's rows
# returns a list of equations
def toEquations (A, b):
    if (len(A.rows()) != len(b)):
        raise Exception("A and b don't have the correct number of elements")
    print(A, "x = ", b)
    print()

    numOfrows = len(A.rows());
    numOfcols = len(A.columns());
    equations = []
    for i in range(numOfrows):
        equation = ""
        for j in range(numOfcols):
            equation = equation + " + " + str(A[i][j] * xVect[j])

        equation = equation[2:] + " == " + str(b[i])
        equations.append(equation)
    return equations

Chapter 4

Introduction to Subspaces

Subspace

A subset S of $\mathbb{R}^3$ is a subspace if S satisfies the following three conditions:

(a) S contains 0, the zero vector.

(b) If u and v are in S, then u + v is also in S.

(c) If r is a real number and u is in S, then ru is also in S.

A subset of $\mathbb{R}^n$ that satisfies condition (b) above is said to be closed under addition, and if it satisfies condition (c), then it is closed under scalar multiplication. Closure under addition and scalar multiplication ensures that arithmetic performed on vectors in a subspace produce other vectors in the subspace.

If 0 is not in a subset S, then S is not a subspace.

# Define a 3-dimensional vector space
V = VectorSpace(RR, 3)

# Define a subspace by providing basis
S = V.subspace([V([-3, 1, 0]),V([1, -5, 0])])

# Check definition 1: The subspace must contain the zero vector
has_zero_vector = zero_vector(V.dimension()) in S
print("Definition 1 - contains zero vectors:", has_zero_vector)

# Check definition 2: The subspace is closed under addition
u = vector(QQ, [1, 2, 0])
v = vector(QQ, [2, 0, 0])
print("Whether u in S:", u in S)
print("Whether v in S:", v in S)
print("Definition 2 - closed under addition:", (u + v) in S)

# Check definition 3: The subspace is closed under scalarxxxx
c = 2.2
print("Definition 3 - closed under multiplicity:", (c * u) in S)

# Whether the output is a subspace 
print("Whether the output is a subspace：", has_zero_vector and (u + v) in S and (c * u) in S)

# Example of a vector w that does is **not** in S.
w = vector(QQ, [2, 2, 1])
print("Whether w in S:", w in S)

Definition 1 - contains zero vectors: True
Whether u in S: True
Whether v in S: True
Definition 2 - closed under addition: True
Definition 3 - closed under multiplicity: True
Whether the output is a subspace： True
Whether w in S: False

Null space

If A is an n × m matrix, then the set of solutions to the homogeneous linear system Ax = 0 forms a subspace of $R_{m}$.

If A is an n × m matrix, then the set of solutions to Ax = 0 is called the null space of A and is denoted by null(A).

Let T : $R_{m}$ → $R_{n}$ be a linear transformation. Then the kernel of T is a subspace of the domain $R_{m}$ and the range of T is a subspace of the codomain $R_{n}$.

ker(T) = null(A)

# Define a new matrix A
A = Matrix(QQ, [[3, -1, 7, -6], 
                [4, -1, 9, -7], 
                [-2, 1, -5, 5]])

# Augment A with the zero vector
augmented_matrix = A.augment(Matrix(QQ, 3, 1, [0, 0, 0]))

# Perform Gaussian elimination
reduced_row_echelon_form = augmented_matrix.echelon_form()

# Extract the null space basis vectors
null_space_basis = []
for row in reduced_row_echelon_form.rows():
    null_space_basis.append(row[:-1])  # Exclude the last column (the zero vector)

print("Null Space Basis Vectors:", null_space_basis[0:-1])

Null Space Basis Vectors: [(1, 0, 2, -1), (0, 1, -1, 3)]

Ex. Suppose that T : $\mathbb{R_2}$ → $\mathbb{R_3}$ is defined by $$\begin{align*}T\begin{pmatrix}\begin{bmatrix} x_1\\x_2 \end{bmatrix}\end{pmatrix}=\begin{bmatrix}x_1-2x_2\\-3x_1+6x_2\\2x_1-4x_2\end{bmatrix}\end{align*}$$ Find ker(T) and range(T).

Ex. Suppose that T : $R_{2}$ → $R_{3}$ is defined by

Find ker(T) and range(T).

# T(x) = Ax
A = Matrix(QQ, [[1, -2], 
                [-3, 6], 
                [2, -4]])
# ker(T) = null(A)
null_space_basis = A.right_kernel().basis()
# The range of T is equal to the span of the columns of A
# So, we need to find the echelon_form and find the span of the columns of A
echelon_form = A.rref()
linearly_indep_cols = echelon_form.pivots()
range = []
for i in linearly_indep_cols:
    range.append(A.transpose()[0])
print("null space:")
print(null_space_basis)
print("range:", range)

null space:
[
(1, 1/2)
]
range: [(1, -3, 2)]

Basis

A set B = {u1, ... , um} is a basis for a subspace S if

(a) B spans S.

(b) B is linearly independent.

Let B = {$u_1$, ..., $u_m$} be a basis for a subspace S. For every vector s in S there exists a unique set of scalars $s_1$, ..., $s_m$ such that

S = $s_1$$u_1$ + ... + $s_m$$u_m$

Suppose that U = [$u_1$ … $u_m$] and V = [$v_1$ … $v_m$] are two equivalent matrices. Then any linear dependence that exists among the vectors $u_1$, … , $u_m$ also exists among the vectors $v_1$, … , $v_m$.

Let A and B be equivalent matrices. Then the subspace spanned by the rows of A is the same as the subspace spanned by the rows of B.

# Display a matrix
# Here u1, u2, u3 are the rows of A.
A = matrix(QQ,[[-1, -6, 4],
               [2, 7, -3],
               [3, 5, 1],
               [1, 2, 0]])
# Form the rows of a matrix
A_row = A.transpose()
# Find Echelon Form
B = A_row.rref()
# Find the basis
row_pivot = B.pivot_rows()
print(row_pivot)

# We can find which rows are linearly independent from row_pivot, which is 0, 1
basis_0 = B[row_pivot[0], :]
basis_1 = B[row_pivot[1], :]
print("basis:")
print(basis_0)
print(basis_1)

(0, 1)
basis:
[   1    0 11/5  3/5]
[   0    1 13/5  4/5]

Summarizing this method: To find a basis for S = span{$u_1$, ...., $u_m$},

(a) Use the vectors $u_1$, ...., $u_m$ to form the rows of a matrix A.

(b) Transform A to echelon form B.

(c) The nonzero rows of B give a basis for S.

# Display a matrix
# Here u1, u2, u3 are the columns of A
A = matrix(QQ,[[-1, -6, 4],
               [2, 7, -3],
               [3, 5, 1],
               [1, 2, 0]])
# Find Echelon Form
B = A.echelon_form()
# Find the index of pivots
linearly_indep_cols = B.pivots()
# We can find which cols are linearly independent from linearly_indep_cols, which is 0, 1
basis_0 = A.transpose()[row_pivot[0], :]
basis_1 = A.transpose()[row_pivot[1], :]
print("basis:")
print(basis_0)
print(basis_1)

basis:
[-1  2  3  1]
[-6  7  5  2]

Summarizing this method: To find a basis for S = span{$u_1$, ..., $u_m$},

(a) Use the vectors $u_1$, ..., $u_m$ to form the rows of a matrix A.

(b) Transform A to echelon form B.

(c) The nonzero rows of B give a basis for S.

Dimension

If S is a subspace of $R_n$, then every basis of S has the same number of vectors.

Let S be a subspace of $R_n$. Then the dimension of S is the number of vectors in any basis of S.

# Display a matrix
A = matrix(QQ,[[-1, 3, -3, 5],
               [2, -6, 8, -3],
               [5, -15, 19, -11],
               [-1, 3, -5, -2],
               [-4, 12, -18, -1]])
# Find Echelon Form
echelon_form = A.rref()
# Find the index of pivots
linearly_indep_cols = echelon_form.pivots()
print("The dimension of this matrix:", len(linearly_indep_cols))

the dimension of this matrix: 2

Let 𝒰 = {$u_1$, ...,$u_m$} be a set of vectors in a subspace S ≠ {0} of $R^n$.

(a) If 𝒰 is linearly independent, then either 𝒰 is a basis for S or additional vectors can be added to 𝒰 to form a basis for S.

(b) If 𝒰 spans S, then either 𝒰 is a basis for S or vectors can be removed from 𝒰 to form a basis for S.

Ex. Expand the set to a basis for $R^3$.

# Since R^3 has dimension 3, we know that 𝒰 does not have enough vectors to be a basis. We can see that the two vectors in 𝒰 are linearly independent, we can expand 𝒰 to a basis of R3. 
A = matrix(QQ,[[1, 3, 1, 0, 0],
               [1, 2, 0, 1, 0],
               [-2, -4, 0, 0, 1]])
# Find Echelon Form
B = A.echelon_form()
linearly_indep_cols = B.pivots()
# We can find which cols are linearly independent from linearly_indep_cols, which is 0, 1, 3
basis_0 = A.transpose()[linearly_indep_cols[0], :]
basis_1 = A.transpose()[linearly_indep_cols[1], :]
basis_2 = A.transpose()[linearly_indep_cols[2], :]
print("basis:")
print(basis_0)
print(basis_1)
print(basis_2)

basis:
[ 1  1 -2]
[ 3  2 -4]
[0 1 0]

Let 𝒰 = {$u_1$, ..., $u_m$} be a set of m vectors in a subspace S of dimension m. If 𝒰 is either linearly independent or spans S, then 𝒰 is a basis for S.

Ex. Suppose that S is a subspace of $\mathbb{R}^3$ of dimension 2 containing the vectors in the set

Show that 𝒰 is a basis for S.

# All we need to do to show that 𝒰 is a basis for S is verify that 𝒰 is linearly independent or spans S
A = matrix(QQ,[[1, 3],
               [2, 7],
               [0, 1]])
B = A.echelon_form()
print(B)
# Since the two vectors are not multiples of each other, 𝒰 is a linearly independent set. Hence we can conclude that 𝒰 is a basis for S.

[1 0]
[0 1]
[0 0]

Suppose that $S_1$ and $S_2$ are both subspaces of $\mathbb{R}^3$ and that $S_1$ is a subset of $S_2$. Then dim($S_1$) ≤ dim($S_2$), and dim($S_1$) = dim($S_2$) only if $S_1$ = $S_2$.

# Define three-dimensional Euclidean space
V = VectorSpace(RR, 3)

# Define subspace V1, representing the xy-plane
V1 = V3.subspace([V([-3, 1, 0]),V([1, -5, 0])])

# Define subspace V2, representing the entire three-dimensional space
V2 = V

# Verify the conditions
dim_V1 = V1.dimension()
dim_V2 = V2.dimension()

print(f"dim(V1) = {dim_V1}, dim(V2) = {dim_V2}")

# Check if dim(V1) <= dim(V2) holds
assert dim_V1 <= dim_V2, "dim(V1) is not less than or equal to dim(V2)"

# Check if dim(V1) = dim(V2) holds
if dim_V1 == dim_V2:
    # Check if V1 is equal to V2
    assert V1 == V2, "V1 is not equal to V2"

print("Whether dim_V1 = dim_V2:", dim_V1 == dim_V2)

dim(V1) = 2, dim(V2) = 3
Whether dim_V1 = dim_V2: False

Let 𝒰 = {$u_1$, ..., $u_m$} be a set of vectors in a subspace S of dimension k.

(a) If m < k, then 𝒰 does not span S.

(b) If m > k, then 𝒰 is not linearly independent.

# Define symbolic variables
k, m = var('k m')

# Given values
k_value = 4
m_value = 2  # Number of vectors is fewer than dimension

# Define the vector space
V = VectorSpace(QQ, k_value)

# Create a set of vectors U with m vectors
U = [vector(QQ, [i+1 for i in range(k_value)]) for _ in range(m_value)]

# Check if U spans S
spans_S = V.span(U) == V

# Display results
result = f"If k = {k_value} and m = {m_value}, does U span S? {'Yes' if spans_S else 'No'}"

print(result)

If k = 4 and m = 2, does U span S? No

# Define symbolic variables
k, m = var('k m')

# Given values
k_value = 3
m_value = 4  # Number of vectors is more than dimension

# Create a set of vectors U with m vectors
U = [vector(QQ, [i+1 for i in range(k_value)]) for _ in range(m_value)]

# Create a matrix using the vectors
matrix_U = Matrix(U)

# Check if the vectors in U are linearly independent
linearly_independent = matrix_U.rank() == len(U)

# Display results
result = f"If k = {k_value} and m = {m_value}, is U linearly independent? {'Yes' if linearly_independent else 'No'}"

print(result)

If k = 3 and m = 4, is U linearly independent? No

Row and Column spaces

Let A be an n x m matrix.

(a) The row space of A is the subspace of $\mathbb{R}^m$ spanned by the row vectors of A and is denoted by row(A).

(b) The column space of A is the subspace of $\mathbb{R}^n$ spanned by the column vectors of A and is denoted by col(A).

Let A be a matrix and B an echelon form of A.

(a) The nonzero rows of B form a basis for row(A).

(b) The columns of A corresponding to the pivot columns of B form a basis for col(A).

A = matrix(QQ,[[1, -2, 7, 5],
               [-2, -1, -9, -7],
               [1, 13, -8, -4]])
# Find the echelon form of the matrix
B = A.rref()
row_pivot = B.pivot_rows()
# We know that a basis for the row space of A is given by the nonzero rows of B
# We can find which rows are linearly independent from row_pivot, which is 0, 1
basis_0 = B[0, :]
basis_1 = B[1, :]

linearly_indep_cols = B.pivots()
# We know that a basis for the column space of A is given by the columns of A corresponding to the pivot columns of B
# We can find which cols are linearly independent from linearly_indep_cols, which is 0, 1
basis_00 = A.transpose()[row_pivot[0], :]
basis_11 = A.transpose()[row_pivot[1], :]

print("basis for the row space:")
print(basis_0)
print(basis_1)
print("basis for the column space:")
print(basis_00)
print(basis_11)

basis for the row space:
[   1    0    5 19/5]
[   0    1   -1 -3/5]
basis for the column space:
[ 1 -2  1]
[-2 -1 13]

For any matrix A, the dimension of the row space equals the dimension of the column space.

The rank of a matrix A is the dimension of the row (or column) space of A, and is denoted by rank(A).

# Display a matrix
A = matrix(QQ,[[1, -2, 3, 0, -1],
               [2, -4, 7, -3, 3],
               [3, -6, 8, 3, -8]])
# Find the echelon form of the matrix
echelon_form = A.rref()
linearly_indep_cols = echelon_form.pivots()
# Find the number of cols that have pivots
pivot_cols = 0
for i in linearly_indep_cols:
    pivot_cols += 1
# Find the rank
rank = pivot_cols
# m of this matrix is 5
m = 5
# Find the nullity of the matrix
nullity = m - rank
# nullity = A.right_nullity()
print("rank:", rank)
print("nullity:", nullity)

# show the quick way
nullity_quick = A.kernel()
print("nullity_quick:", nullity_quick)

rank: 2
nullity: 3
nullity_quick: Vector space of degree 3 and dimension 1 over Rational Field
Basis matrix:
[   1 -1/5 -1/5]

rank(A) + nullity(A) = m.

Let A be an n × m matrix and b a vector in $\mathbb{R}^n$.

(a) The system Ax = b is consistent if and only if b is in the column space of A.

(b) The system Ax = b has a unique solution if and only if b is in the column space of A and the columns of A are linearly independent.

Change Basis

Let x be expressed with respect to the standard basis, and let B ={u1,... , un} be any basis for Rn.

(a) x = U$[x]_B$

(b) $[x]_B$ = $U^{-1}$x

# Define a 3-dimensional vector space
V = VectorSpace(QQ, 3)

# Define the vector x
x = V([3, 4, 4])

# Define the new basis vectors u1, u2, u3
u1 = V([1, 1, 2])
u2 = V([0, -3, 1])
u3 = V([1, 0, 2])

# Letting U be the matrix with columns given by the vectors in ℬ
U = matrix(QQ,[[1, 1, 2],
               [0, -3, 1],
               [1, 0, 2]])
U_inverse = U.inverse()
x_B = U_inverse * x
print(x_B)

(2, -1, 1)

Let $ℬ_1$ = {$u_1$,… ,$u_m$} and $ℬ_2 =$ {$v_1$,… ,$v_m$} be bases for $\mathbb{R}^n$. If U = [$u_1$ … $u_m$] and V = [$v_1$ … $v_m$], then

(a) $[x]_{ℬ_2}$ = $V^{−1}$U$[x]_{ℬ_1}$

(b) $[x]_{ℬ_1}$ = $U^{−1}$V$[x]_{ℬ_2}$

Suppose that $\mathscr{B}_1 = \left\{ \begin{bmatrix}1 \\ 3 \end{bmatrix}, \begin{bmatrix}2 \\ 7\end{bmatrix} \right\}$ and $\mathscr{B}_2 = \left\{ \begin{bmatrix}3 \\ 5 \end{bmatrix}, \begin{bmatrix}2 \\ 3\end{bmatrix} \right\}$.

Find $[x]_{\mathscr{B}_2}$ if $[x]_{\mathscr{B}_1} = \left\{ \begin{bmatrix}-1 \\ 4 \end{bmatrix} \right\}$.

# We start by setting U and V
U = matrix(QQ,[[1, 2],
               [3, 7]])
V = matrix(QQ,[[3, 2],
               [5, 3]])
x_B1 = matrix(QQ, [[-1],
                   [4]])
# The change of basis matrix from ℬ_1 to ℬ_2 is V^-1*U
V_inverse = V.inverse()
change_of_basis = V_inverse * U

# Use V^-1*U*x_B1 to findx_B2
change_of_basis_matrix = matrix(QQ,[[3, 8],
                                    [-4, -11]])
x_B2 = change_of_basis_matrix * x_B1
print(x_B2)

# Check
# U*x_B1
U_xB1 = U * x_B1
# V*x_B2
V_xB2 = V * x_B2
print("Check:", U_xB1==V_xB2)

[ 29]
[-40]
Check: True

Change of Basis in Subspaces

Let S be a subspace of $\mathbb{R}^n$ with bases $\mathscr{B}_1$ = {$u_1$, ..., $u_m$} and $\mathscr{B}_2$ = {$v_1$, ..., $v_m$}.

If $C =[ [u_1]_{\mathscr{B}_2}... [u_k]_{\mathscr{B}_2}]$

then $[x]_{\mathscr{B}_2}= C[x]_{\mathscr{B}_1}$

# Define a 3-dimensional vector space
V = VectorSpace(RR, 3)

B_1 = matrix(QQ,[[1, -5, 8], [3, -8, 3]]).transpose()
B_2 = matrix(QQ,[[1, -3, 2], [-1, 2, 1]]).transpose()
# Find the change of basis matrix from B_1 to B_2 and find x_B2 if x_B1 = [3
#                                                                          -1]
x_B1 = vector([3, -1])
B1_col1 = vector(B_1[:, 0])
#toEquations takes in a transposed matrix and a vectors and returns a list of corresponding linear equations
equations = toEquations(B_2, B1_col1); print("Generated Equations: ", equations, "\n")
s = solve([eval(equations[0]), eval(equations[1]), eval(equations[2])], x, y, solution_dict=True)
c11 = s[0][x]
c12 = s[0][y]
print("s = ", s)
print("c11 = ", c11)
print("c12 = ", c12)


B1_col2 = vector(B_1[:, 1])
equations = toEquations(B_2, B1_col2); print("Generated Equations: ", equations, "\n")
s = solve([eval(equations[0]), eval(equations[1]), eval(equations[2])], x, y, solution_dict=True)
c21 = s[0][x]
c22 = s[0][y]
print("s = ", s)
print("c21 = ", c21)
print("c22 = ", c22)
print(s)

C = matrix(QQ,[[c11, c12], [c21, c22]]).transpose()
print(C)

x_B2 = C * x_B1
print("x_B2:")
print(x_B2)

[ 1 -1]
[-3  2]
[ 2  1] x =  (1, -5, 8)

Generated Equations:  [' x + -y == 1', ' -3*x + 2*y == -5', ' 2*x + y == 8'] 

s =  [{x: 3, y: 2}]
c11 =  3
c12 =  2
[ 1 -1]
[-3  2]
[ 2  1] x =  (3, -8, 3)

Generated Equations:  [' x + -y == 3', ' -3*x + 2*y == -8', ' 2*x + y == 3'] 

s =  [{x: 2, y: -1}]
c21 =  2
c22 =  -1
[{x: 2, y: -1}]
[ 3  2]
[ 2 -1]
x_B2:
(7, 7)

Thank you!