Deep Learning 3: Tensors
Scalars, Vectors, Matrices, Tensors, Key Attributes
In general, all current machine learning systems use tensors as their basic data structure. Tensors are fundamental to the field - so fundamental that TensorFlow was named after them.
At its core, a tensor is a container for data - usually numerical data. So, it’s a container for numbers. You may be already familiar with matrices, which are rank-2 tensors: tensors are a generalization of matrices to an arbitrary number of dimensions.
1 - Scalars (Rank 0 Tensors)
A tensor that contains only 1 number is called a scalar (rank 0 tensor). In Numpy, a float32 or float 64 number is a scalar tensor. You can display the number of axes of a NumPy tensor via the ndim attribute; a scalar tensor has 0 axes (ndim ==0). The number of axes of a tensor is also called its rank. Here’s a NumPy scalar:
ndim lets us see the rank of the data point.
2 - Vectors (Rank 1 Tensors)
An array of numbers is called a vector (rank 1 tensor). A rank-1 tensor is said to have exactly 1 axis. Following is a NumPy vector:
This vector has 5 entries, and so is called a 5-dimensional vector. Don’t confuse a 5D vector with a Rank 5 Tensor. A 5D vector has only 1 axis and has 5 dimensions along its axis, whereas a 5D tensor has five axes. Dimensionality can denote either the number of entries along a specific axis or the number of axes in a tensor, which can be confusing at times.
3 - Matrices (Rank 2 Tensors)
An array of vectors is a matrix, or a rank 2 tensor. A matrix has two axes (often referred to as rows & columns). You can visually interpret a matrix as a rectangular grid of numbers. This is a NumPy matrix:
it has the ndim of 2:
The entries from the first axis are called the rows, and the entries from the second axis are called the columns. In the previous example, [5,6,7,8,9] is the first row of x.
And, [5,10,15] is the first column.
4 - Rank 3 Tensors
If you pack such matrices in a new array, you obtain a rank-3 tensor, which you can visually interpret as a cube of numbers. Following is a NumPy rank 3 tensor.
and here is it’s dimensions
By packing rank 3 tensors in an array, you can create a rank 4 tensor, and so on. In deep learning, you’ll generally manipulate tensors with ranks 0 to 4, although you may go up to 5 if you process video data.
5 - Key Attributes
Every tensor is defined by 3 key attributues:
Rank (Number of axes): A rank 3 tensor has 3 axes, and a matrix has 2 axes. This is also called the tensor’s ndim in Python libraries such as NumPy or TensorFlow.
Shape: This is a tuple of integers that describes how many dimensions the tensor has along each axis. For example, the previous matrix example has the shape (3,3,5). A vector has a shape with a single element, such as a (5, ), whereas a scalar has an empty shape, ().
Data Type: This is the type of the data contained in the tensor; for instance, a tensor’s type could be float 16, float 32, float 64, uint8, and so on. In TensorFlow, you are also likely to come across string tensors.
5.1 Shapes
Scalars:
Vectors:
Matrices:
Tensors:
