1
Mathematical Foundation

Afroz* and Basharat Hussain

Department of Mathematics, Maulana Azad National Urdu University, Hyderabad, India

Abstract

The aim of this chapter is to provide the reader an overview of basics of linear algebra and introductory lecture on calculus. We will discuss concept of real vector spaces, basis, span, and subspaces. The idea of solving the system of equations using matrix approach will be discuss. Linear transformation by means of which we can pass from one vector space to another, inverse linear transformation, and transformation matrix will be explain with detail examples. Definition of eigenvectors, eigenvalues, and eigendecomposition along with thorough examples will be provided. Moreover, definition of function, limit, continuity, and differentiability of function with illustrative examples will be included.

Keywords: Vector spaces, basis, linear transformation, transformation matrix, eigenvalue, eigenvector, eigen decomposition, continuous functions, differentiation

1.1 Concept of Linear Algebra

1.1.1 Introduction

Basics problem of linear algebra is to solve n linear equations in n unknowns.

For example,

The above system is two dimensional (n = 2), i.e., two equations with two unknowns. The solution of the above system is the values of unknowns x, y, satisfying the above linear system. One can easily verify that x = 1, y = 2 satisfy the above linear system.

Geometrically, each of the above equation represents a line in R2-plane. We have two lines in same plane and if they do intersect (it is possible that they may not intersect as parallel line don't intersect) on same plane their point of intersection will be the solution of system as illustrated in Figure 1.1.

The matrix (2D-array) representation of above system will be

The matrix is coefficient matrix, and vector is the column vector of unknowns. The values on the right-hand side of the equations form the column vector b.

Figure 1.1 Point of intersection.

Matrix A has two vectors in its column (2, -1) and (-1, 2). The product:

For any input vector x the output of the operation "multiplication by A" is some vector b. A deeper question is to start with a vector b and ask "for what vectors x does Ax = b?" [1]. In our example, this means solving two equations in two unknowns. Solving:

is equivalent to solving:

The only difference between the three-dimensional matrix picture and two-dimensional one is change in size of the vectors and matrices.

In general, a system of linear equations can easily be transformed into the matrix equation Ax = b. The solution of which, if it exists, can easily be find using computer software. Like method of elimination can be used provided that matrix A is non-singular (det (A)? 0), i.e., A is invertible [2, 3].

1.1.2 Vector Spaces

Definition. [4, 5] "A vector space (or linear space) consists of following:

1. A field F of scalars;
2. A set of vectors V;
3. Two rules (or operations) are defined:
   1. Vector addition, i.e., vectors can be added together, and the resulting vector belongs to V.
   2. Scalar multiplication, i.e., a vector from V can be multiplied by an element of F, and resulting element will be a vector belongs to V.
4. Under these two operation V must satisfy
   1. There exists an additive identity (symbol 0) in V such that x + 0 = x for all x belongs to V.
   2. For each x ? V, there exists an additive inverse (written -x) such that x + (-x) = 0.
   3. There exists a multiplicative identity (written 1) in F such that 1x = x for all x ? V.
   4. Commutativity: x + y = y + x for all x, y ? V
   5. Associativity: (x + y) + z = x + (y + z) and a(ßx) = (aß)x for all x, y, z ? Vand a, ß ? F
   6. Distributivity: a(x + y) = ax + ay and (a + ß)x = ax + ßy for all x, y ?V and a, ß ?F."

When there is no chance of confusion, we may simply refer to the vector space as V, or when it is desirable to specify the field, we shall say V is a vector space over the field F, denoted by V(F). One can take field F = R (set of real number) to avoid the unnecessary diversion into abstract algebra. We will take field of real number throughout the book, unless otherwise stated.

Example 1. "R (R), R2 (R), R3 (R) are vector spaces over the field of real numbers R"

Example 2. Mm,n (R) is a vector space of matrices of order m × n over the field of real numbers R where Mm,n (R)={[aij]}m×n,aij ? R}=set of all m × n matrices whose entries are from field R"

Example 3. Pn(R) is a vector space of polynomials of degree at most n over the fields of real numbers R.

Where, Pn (R) = {a0 + a1 x + . + anxn:ai ? R and n is any non-negative integer} = set of all polynomials whose cofficients from R."

1.1.3 Linear Combination

Definition. [4] "A vector y in V is said to be a linear combination of the vectors x1, x2,.xn in V provided there exist scalars a1, a2,.,an in F such that

1.1.4 Linearly Dependent and Independent Vectors

Two (or n) vectors are said to be linearly dependent if they lie on same straight line, and if they lie on different straight line, they are linearly independent.

Clearly, vectors v1, v2 are linearly dependent and u1, u2 are linearly independent as shown in Figure 1.2 and Figure 1.3.

Definition. "A non-empty set S of V containing n vectors x1, x2,., xn are linearly dependent if and only if there exists scalars (belongs to R) a1, a2,., an, not all zero, such that

If no such scalars exist, then the vectors are said to be linearly independent [4]."

Example 4. Consider {(1, 0), (-5, 0)} subset of R2 these two vectors are linearly dependent because there exist 5, 1 (? 0) in R such that 5(1, 0) + 1(-5, 0) = 0.

Figure 1.2 Linearly dependent.

Figure 1.3 Linearly Independent.

Before moving further, let's take a simple case of two linearly independent vectors say u1 = (1, 0) and u2 = (0, 1) belongs to R2 and take all possible linear combinations of these two vectors. What will happen? These linear combinations fill up the whole plane or space R2. Mathematically, we say these vectors u1, u2 span R2. So, basically these vectors u1, u2 work as a basis for the plane R2.

1.1.5 Linear Span, Basis and Subspace

Definition. [4] "If S is nonempty subset of the vector space V, then L(S), the linear span of S, is the set of all linear combinations of finite sets of elements of S. Consider v1,v2,.,vn n vectors ? S ?V, then span {v1,v2,.,vn} is L(S) = {v ? V:?a1,a2,., an such that a1v1+a2v2 + . + anvn = v}."

Definition. [4] "A non-empty set B of n vectors act as a basis for a space V if

1. Set B is linearly independent.
2. L(B) = V, i.e., linear span of B is V (i.e., all possible linear combinations of vectors in B generate whole space V)."

Example 5. (i). {(1, 0), (0, 1)} is the standard basis for vector space R2. (ii). {(1, 0, 0), (0, 1, 0), (0, 0, 1)} is the standard basis for vector space R3.

Number of vectors in B gives the dimension of the space V. Both finite-and infinite-dimensional spaces exists. Our scope of study is limited to finite dimensional space. R2, R3 are conceivable examples of finite dimensional spaces of dimension 2 (two vectors in basis) and 3 (three vectors in basis set), respectively. Can we remove any single vector from basis set? Absolutely not! If we drop any vector from basis set the span will be different. Think of simple case of R2, what will happen if we remove one vector from the basis set?

Note: The vector space V is said to finite-dimensional (over F) if there is a finite subset S in V such that V = L(S). Our focus of study will be n-dimensional Euclidean space, i.e., Rn.

Definition. [4, 6] "A subset...