Process of understanding:
First, let us try to understand the concept in 2d geometry, and let's try to implement that knowledge in linear algebra thereby let's try to understand in multidimensions.
POINT/VECTOR:
The first thing that we need to learn regarding linear algebra is how to represent a point.
A point in a 2d coordinate system is represented using 2 values, that is one value is the distance of the point from the x-axis and the other is the distance of the point from the y-axis.
image reference: https://www.math.net/img/a/geometry/coordinate-plane/coordinates/x-and-y-coordinates-examples.png
whereas while coming to 3d the point can be represented by 3 values that is the distance from the x,y, and z-axis.
image reference:https://www.intmath.com/vectors/img/235-3D.gif
Similarly from this, we can consider that for an n-dimensional point there will be n values.
DISTANCE BETWEEN TWO POINTS:
We can calculate the distance between two points in 2d by using the following formula
image reference: https://java2blog.com/wp-content/uploads/2019/10/distanceFormula.png
Here (x1,y1) and (x2,y2) are two 2d points between which we are calculating the distance.
NOTE:
In the above figure, we can see the formula for distance, which is derived by using Pythagoras's theorem. let's try to see how it is done visually
from the image, we can see that the green line indicates the distance between two points(P and Q). We can also see that we have extended the points to form a right-angle triangle. which in turn produces the point T. From the point, T corresponding distances are mentioned to points P and Q.
Now let's try to apply the Pythagoras theorem to obtain the distance between P and Q.
Here on the left, we have the Pythagoras theorem and on the right, we have implemented the theorem on the triangle produced, resulting in the distance formula between two points.
In this way, for all dimensional points, the distance between two points is derived.
Consider the following example
image reference: http://passyworldofmathematics.com/Images/pwmImagesSeven/DistanceBetweenPointsTen550x388JPG.jpg
from the above image, we can see how the distance is calculated between two points(A and B).
Similarly, for the 3d points, the distance between two points can be calculated using the same formula with a small change for the 3rd value which is
Distance of a point from origin:
Here is 2d, it is calculated with the same formula but the only difference is, that one of the points in the 2 points((x1,y1),(x2,y2)) is (0,0), that is origin. The formula can be modified to
Distance between two points in nd:
Now the distance between the two points in n-dimensions is the summation of squared differences between the values in a point. That can be seen as
here xi1 indicates the ith value of the first point and xi2 indicates the ith value of the second point.
ROW VECTOR AND COLUMN VECTOR:
The next important thing that we need to learn is vector notations. There are 2 kinds of vector notations
- row vector
- column vector
ROW VECTOR:
The vector whose row value is 1(dimension) and column value is n(dimensions). This kind of vector notation is called a row vector.
COLUMN VECTOR:
The vector which has 1(dimension) column and n(dimensions) rows, is called a column vector.
NOTE:
The row vector can be obtained by transposing the column vector and vice versa.
From the above image, we can see that x is a column vector, on transposing x we got a row vector.
On the right, we can see that we have applied transpose over transpose which means that first the x is transposed into a row vector and once you apply another transpose it is turned into the column vector again resulting in no change.
A brief talk about transpose:
Transposing a vector or a matrix is nothing but converting the rows into columns and vice versa.
image credit goes to: http://www4.hcmut.edu.vn/
If you want to know further about transpose click here.
DOT PRODUCT:
This is the most important topic in linear algebra. The dot product between two vectors is given by multiple formulas.
Geometric Representation of vector:
Before going to understand the concept of dot product let's try to understand the geometrical representation of a vector
from the above image, we can see that the vector is nothing but a line drawn from origin to a point which makes angle teta with the corresponding axis.
The formula for the dot product is from 2 perspectives:1. Geometrical perspective: In this, the formula is represented by using a cosine angle and the lengths of two vectors.
image reference: https://www.onlinemath4all.com/images/dotproductoftwovectors.png
In the above image, |a| indicates the length of vector a, and |b| indicates the length of vector b. And teta is the angle between the two vectors. This geometric intuition allows us to calculate the dot product of two vectors if we know the lengths and angle between them without actually knowing the vector values.
We can also get the angle between the two vectors using the above formula
image reference: https://www.onlinemath4all.com/images/anglebetweentwovectors.png
How to calculate the length of a vector:
It is simply the distance of the vector from its origin. The concept of calculating distance from the origin is discussed previously.
2. Vector Representation: In this, the dot product is calculated by considering the actual values of a vector or a point and then multiplying each value in a vector with the corresponding value in another vector in order to obtain a summation of those products that is
Here a1,a2 and a3 indicate the corresponding values of the vector a. Similarly, b1,b2, and b3 are corresponding values of b.
COMBINING BOTH:
Now by combining both we can get the following formula
NOTE:
All the things that we have learned till now are in 2d the same thing can be applied to 3d or nd. The only change is that we are going to have n values for a vector which can be seen bellow
The equation of the line can be represented in two ways. One way is to represent it geometrically and another way is to represent it in an algebraic equation.
The geometrical representation of a line can be expressed in the terms of slope and intercept as bellow
while coming to the algebraic form the equation of a line can be represented in the terms of x and y-axis values as given bellow
The algebraic equation can be related to the geometric equation as bellow
The equation of the line for 3d can be just obtained by making a small improvisation to the 2d equation of a line. That is for this we will have x,y, and z-axis as well.
The line equation for nd:
The equation of line n-d is similar to the 2d and 3d, the only difference is we will have n number of axis. It can be represented as follows
if the line passes through the origin then the equation will be like the below. The equation that is here is similar to the equation of dot product between two vectors.
When you say the angle between the vector w and plane x is 90 degrees, which means that the vector w is perpendicular to every point on the plane x.(here we assumed x as a hyperplane and w, as a vector to that plane)
DISTANCE BETWEEN POINT AND PLANE/LINE:
This can be calculated by the given formula below, which includes a unit vector perpendicular to the plane, along with its length and a point p on the plane.
The key point to observe here is not the distance, the main thing to observe here is the sign of the distance. We all know that distance is only a magnitude value that does not have any sign, but here what we are talking about is the d value which is present in the above formula.
In the above formula, the d value can be negative or positive, but for distance, we only consider magnitude irrespective of the sign. But here our main focus should be on signs because this provides a way big application for working of Logistic Regression(an ML algo) which we will discuss later in the process.
Here the first point to keep in mind is that the direction of the plane is nothing but the direction of the unit vector w. If a point p lies on the same side of the direction of the plane then it results in a positive distance, whereas if a point is opposite to the direction of the plane then it results in a negative distance.
To understand this application let me give you an example of how can we separate the below red and green points. just by drawing a simple line, we can separate the two types of points.
NOTE:
here it tries to make certain assumptions that are
- It fixes one color to a positive distance
- Let us assume positive is assigned to green
- Negative is assigned to red
Now for the points to be correctly separated, all green and positive distance points to be on one side
and all the red and negative points to be on another side. So it tries to find a line that can best separate most of the points correctly. So in order to do the task, it is necessary for it to use the concept of signed distances.
NOTE:
UNIT VECTOR:
a vector A^ is called a unit vector if the vector A^ is in the same direction as vector A, and whose length is 1, then it is called a unit vector.
image credit goes to: https://mathworld.wolfram.com/UnitVector.html
In the below image i,j, and k are unit vectors of axis x,y,z.
image reference: http://mathonline.wdfiles.com/local--files/standard-unit-vectors/Screen%20Shot%202014-06-13%20at%2012.49.19%20PM.png
PROJECTION:
projection is nothing but a taking a vector A and allowing it to fall on vector B perpendicularly, such that the distance that vector A covers on B is called the projection of vector A on B.
The projection is defined in the terms of cosine value and length of vector A as shown bellow
image reference: https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/Dot_Product.svg/1200px-Dot_Product.svg.png
This can be related to the concept of the dot product like bellow
CIRCLE:
We all have learned the concepts regarding circles in our schooling, though let's try to revisit those things. The radius of a circle is nothing but the distance between the center of the circle and all other points on its circumference or border. Now let's define the equation of the circle.
There is one important thing that we need to consider here, which is how can we find whether a point is inside or outside the circle. which helps in separating data in the below scenario.
Now let's try to see how can we find whether a point is inside the circle or outside the circle. The following equations will help us to do so.
For a circle in n-dimensions, the concept will be applied as it is but the only difference is the formula as shown bellow
ELLIPSE:
Similar to the circle the concept is also the same for the ellipse. But the difference is that for an ellipse we don't have a radius, we only have a major axis and a minor axis.
The equation of an ellipse is as below.
Here major axis is nothing but the axis whose length is greater than the other axis. The next important thing is to know whether it is on the circumference or inside the ellipse or outside the ellipse.
SQUARES AND RECTANGLES:
The key point here that is necessary to know is whether a point is inside the square or outside.
This can be achieved by doing the following
Let (X1,Y1),(X2,Y2), (X3,Y3), (X4,Y4) are the 4 vertices
How can we find whether the point is inside or not? The solution can be seen below
Similarly, for an n-dimensional cube or cuboid, the same thing will apply.
Finally, these are some initial basic algebraic or geometric things that we need to learn in order to start our journey with ml. If needed we will include it further in future blogs.
Comments
Post a Comment