PyTorch Tensor Module from Beginning to Abandonment

Tensor (Tensor) is the most basic object that PyTorch operates on. In the geometric definition, tensor is based on the concept of scalar, vector and matrix eyes. Commonly understood, scalar can be regarded as a 0-dimensional tensor, vector as a 1-dimensional tensor, and matrix as a 2-dimensional tensor. In the field of deep learning, the tensor can be regarded as a bucket of data, when only one drop of water is put in the bucket is a 0-dimensional tensor, multiple drops of water in a row is a 1-dimensional tensor, the joint rows of surfaces is a 2-dimensional tensor, and so on, extending to n-dimensional vectors.

Data types of tensor

There are two ways to create a tensor in PyTorch, one is through a Python array and the other is from a list. The code to create a tensor is as follows:

import torch
# Build from python array
a = [[1, 2, 3],[4, 5, 6]]
x = (a)
print(a, x)
# Construct a tensor from a list
x = ([[1, 2]])
print(x)

() function

() function is used as (\*sizes,out=None)->Tensor, returns a tensor, obeying a uniform distribution in the interval [0,1), with a shape defined by the parameter sizes, with the following code:

tensor1 = (4)
tensor2 = (2, 3)
print(tensor1, tensor2)

() function

() function is used as (*size,out=None)->Tensor, returns a tensor (Gaussian white noise) obeying a standard normal distribution with mean 0 and variance 1, with shape defined by the parameter sizes, with the following code:

tensor1 = (5)
tensor2 = (2, 4)
print(tensor1, tensor2)

() function

The first use of the () function is (means, std, out=None).

If there is no size parameter, returns a tensor obeying a normal distribution with specified meanmeans and standard deviationstd. meanmeans is a tensor containing the normally distributed mean associated with each output element. The standard deviation std is a tensor containing the standard deviation of the normal distribution associated with each output element. The shapes of the mean and standard deviation need not match, but the elements of each tensor must be the same.

tensor = (mean=(1., 11.), std= (1, 0, -0.1))
print(tensor)

(1, 0, -0.1)

tensor = (mean=0.5, std=(1., 6.))
print(tensor)

tensor = (mean=(1., 6.), std=1.0)
print(tensor)

The second use of () is (mean, std, size*, out=None), which shares the mean and the equation

tensor = (2, 3, size=(1, 4))
print(tensor)

All the numbers generated in this code follow a normal distribution with mean 2 and variance 3, of the shape (1,4)

() function

The () function is used as (start,end,step=100,out=None)->Tensor, which returns a 1-dimensional tensor containing steps of points evenly spaced on the intervals start and end.

tensor = (1, 10, steps=5)
print(tensor)

torch.manual_seed() function

The torch.manual_seed() function is used to fix the random seed; the sequence of random numbers generated below are all identical and act only on the closest random number generating statement.

torch.manual_seed(1)
temp1 = (5)
print(temp1) 
torch.manual_seed(1)
temp2 = (5)
print(temp2)  
temp3 = (5)
print(temp3)  

temp1 and temp2 are the same, and temp3 is different because seed acts only on the closest random number generating statement. the () function in torch has no arguments and is used to set the seed of a random number to a random number, and is not normally used.

()、()、()

The () function is used to produce all-1 arrays, the () function is used to produce all-0 arrays, and the () function is used to produce two-dimensional arrays with a diagonal of 1 and the rest of the array all-0.

tensor1 = (2, 3)
tensor2 = (2, 3)
tensor3 = (3)
print(tensor1, tensor2, tensor3)

The default data type in PyTorch is 32-bit floating-point (). If you want to specify the data type of a tensor, you can either specify the type at creation time or convert the data type of the tensor after creation.

# First method: specify the data type when creating the tensor
x = ((2, 3, 4), dtype=torch.int64) # generate all-1 array
print(x)
# Second method: after the tensor is created, convert the data type
x = (2, 3, 4) # generate all-1 array
x = (torch.int64)
print(x)

Basic operations on tensor

In PyTorch, tensor operations are divided into structural operations and mathematical operations. Structural operations are to change the structure of the tensor itself, and mathematical operations are to perform mathematical operations on the values of the elements of the tensor. Commonly used structure operations include changing the shape of the tensor, adding and removing dimensions, swapping dimensions, splicing and separating, stacking and decomposing, indexing and slicing, and mathematical operations include basic mathematical operations, vector operations, and matrix operations.

() function can change the shape of the Tensor, but must ensure that the total number of elements before and after the same.

x = (3, 2)
print()  # ([3, 2])
y = (2, 3)
print()  # ([6])

Adding and removing dimensions

The unsqueeze() function expands the data dimensions by adding a dimension of dimension 1 to the specified position. Usage: (a,N), or (N), adds a dimension of dimension 1 to the specified position N in a.

# Add dimensions
a = (3, 4)
b = (a, 0)
c = (0)
print()
print()

The squeeze() function can dimensionally compress a tensor, removing dimensions with dimension 1. Usage: (a) removes all dimensions in a with dimension 1, or (1) is to remove dimensions in a with the specified dimension 1.

# Delete the dimension
a = (1, 1, 3, 4)
b = (a)
c = (1)
print()
print()

(math.) the exchange dimension

The problem of exchanging dimensions is often encountered in the process of applying various models.There are two ways of exchanging dimensions in PyTorch. The () function is used to exchange two dimensions, and the () function is free to exchange any position.

a = (1, 3, 28, 32) # ([1, 3, 28, 32]
# The first method
b = (1, 3).transpose(1, 2) # ([1, 28, 32, 3])
print()

# The second method
c = (0, 2, 3, 1)
print() # ([1, 28, 32, 3])

Splicing and Splitting

Multiple tensors can be merged using the () function, which is a link and does not add dimensions; the () function can also be used to split a tensor into multiple tensors. The () function can be viewed as the inverse of the () function. split() function works by splitting the tensor into multiple blocks, each of which is a view of the original tensor.

The code for the # () splice method looks like this:
a = (1, 2)
b = (1, 2)
c = (1, 2)
output1 = ([a, b, c], dim=0) # dim=0 for columnwise splicing
print() # ([3, 2])
output2 = ([a, b, c], dim=1) # dim=1 for rowwise stitching
print() # ([1, 6])

The code for the # () split method looks like this:
a = (3, 4)
output1 = (a, 2)
print(output1)
output2 = (a, [2, 2], dim=1)
print(output2)

Stacking and Decomposition

Multiple tensors can be merged using the () function, which is slightly different from the () function, which is used to perform a stacking operation that adds a dimension. The () function can be seen as the inverse of the () function. What the () function does is to split the Tensor into chunks of Tensor blocks by dim (rows or columns) and returns a tuple.

# () Stacking method
a = (1, 2)
b = (1, 2)
c = (1, 2)
output1 = ([a, b, c], dim=0) # dim=0 for columnwise stitching
print()
output2 = ([a, b, c], dim=1) # dim=1 for rowwise stitching
print()

The code for the # () decomposition method is as follows:
a = (3, 4)
output1 = (a, 2, dim=0)
print(output1)
output2 = (a, 2, dim=1)
print(output2)

Indexing and Slicing

The indexing of the Torch tensor and the slicing rules are basically the same as Numpy, which is simpler.

x = (2, 3, 4)
print(x[1].shape)  
y = x[1, 0:2, :]
print()  
z = x[:, 0, ::2]
print() 

basic mathematical operations

The basic mathematical operations of tensors include tensor summation, product of tensor elements, and finding the mean, variance, and extreme value for a tensor.

Elemental Summation

The first method of summing by elements is (input)->float, which returns the sum of all the elements in the input vector input.

# First method of summing elements
a = (4, 3)
b = (a)
print(b) # tensor(6.4069)

The second method of summation by elements is (input,dim,keepdim=False,out=None)->Tensor, where input is the input tensor, dim is the specified dimension, keep(bool) indicates whether the output tensor is kept to have the same number of dimensions as the input tensor, and the value of ruokeepdim is True The value of keep(bool) indicates whether the output tensor has the same number of dimensions as the input tensor. Otherwise, the dim dimension is equivalent to the executed() dimension compression, resulting in the disappearance of this dimension, and the final output tensor will have one less dimension than the input tensor.

# Second method of summing elements
a = (4, 3)
b = (a, dim=1, keepdim=False)
print(b, )

Summing by index

Summing by index means adding the elements of the parameter tensor Tensor to the elements of the tensor that executes this method, one by one, in the order determined in the index parameter index. The dimensions of the parameter Tensor must strictly match the tensor that executes the method, or an error will occur. The method for summing by index is .index_add_(dim,index,tensor)->Tensor, dim is the dimension pointed to by the index index, index is the tensor containing the number of indices, and Tensor is the tensor containing the elements of the sum.

# Sum by index, not very common
x = ([[1, 2],[3, 4]])
y = ([[3, 4],[5, 6]])
index = ([0, 1])
output = x.index_add_(0, index, y)
print(output)

the product of two elements

The first method for the product of elements is (input)->float, which returns the product of all elements of the input tensor input.

# Elemental product first method
a = (4, 3)
b = (a)
print(b)

The second method for the product of elements is (input,dim,keepdim=False,out=None)->Tensor, where input is the input tensor, dim is the specified dimension, and keepdim(bool) indicates whether or not the output tensor is kept to have the same number of dimensions as the input tensor, and if the keepdim value is True If keepdim value is True, then the dimensions in the output tensor are the same as the input tensor input, except that the value of the dim dimension being operated on is reduced to 1. Otherwise, the dim dimension is equivalent to the dimension compression operation performed by the execute() function, causing this dimension to disappear and the final output tensor evades the input tensor by one less dimension.

# Elemental product second method
a = (4, 3)
b = (a, 1, True)
print(b, )

average

The first method for finding the average is (input), which returns the average of each element of the input tensor input.

# The first way to find the mean
a = (4, 3)
b = (a)
print(b)

The second way to find the average is (input,dim,keepdim=False,out=None), where the meaning of the parameters is the same as that of the elemental summation and the elemental product.

# A second way to find the mean
a = (4, 3)
b = (a, 1, True)
print(b, )

find the variance (math.)

The first method for finding the variance is (input,unbiased=True)->float, which returns the variance of all elements in the input vector input, with unbiased(bool) indicating whether or not to use unbiased estimation based on the modified Bessel function.

# The first way to find the variance
a = (4, 3)
b = (a)
print(b)

The second method for finding the variance is (input,dim,keepdim=False,unbiased=True,out=None)->Tensor,unbiased(bool) indicates whether or not to use the unbiased estimation based on the modified Bessel function, and the rest of the parameter meanings are the same as those of elemental summation and elemental product.

# The second way to find the variance
a = (4, 3)
b = (a, 1, True)
print(b, )

maximize

The first way to find the maximum value is (input)->float, which returns the maximum value of all elements of the input tensor.

# The first way to find the maximum value
a = (4, 3)
b = (a)
print(b)

The second method for finding the maximum is torch(input,dim,keepdim=False,out=None)->(Tensor,LongTensor), which returns the maximum value of each row in the specified dimension dim in the new tensor input, along with the positional index of each maximum.

# The second way to find the maximum value
a = (4, 3)
b = (a, 1, True)
print(b)

find the minimum value

The first way to find the minimum value is (input)->float, which returns the minimum value of the input tensor.

# The first way to find the minimum value
a = (4,3)
b = (a)
print(b)

The second method for finding the minimum is (input,dim,keepdim=False,out=None)->(Tensor,LongTensor), similar to the second method for finding the maximum.

# The second way to find the minimum value
a = (4, 3)
b = (a, 1, True)
print(b)

Vector and Matrix Operations

Linear algebra operations on vectors include dot product (dot(a,b)), inner product (inner(a,b)), and fork product (matmul(a,b)).

Dot product of vectors

The dot product of vectors is also known as the inner product of vectors or the product of quantities. The dot product of two vectors is the operation of summing two vectors after multiplying their corresponding bits one by one.

# dot product of vectors, a and b must be one-dimensional
a = ([1, 2, 3])
b = ([1, 1, 1])
output = (a, b)
print(output) # Equivalent to 1*1+2*1+3*1, tensor(6.)

Fork multiplication of vectors

The outer product of two vectors, also known as the cross product and cross product vector product, results in a vector rather than a scalar.

# Fork multiplication of vectors
a = ([1, 2, 3])
b = ([1, 1, 1])
output = (a, b)
print(output)

inner product of matrices

Two matrices a and b of the same dimension, the inner product of the a and b matrices is the inner product of vectors in the same position.

# Inner product of matrices
a = ([1, 2, 3])
b = ([1, 1, 1])
output = (a, b)
print(output)

The outer product of a matrix

The outer product of a matrix is the general matrix product in the usual sense. The general matrix product is the most important method of multiplying matrices, and it only makes sense if the first matrix has the same number of columns (column) as the second matrix has the same number of rows (row).

# Outer product of matrices: matrix multiplication
a = ([[1, 2, 3], [4, 5, 6]])
b = ([[1, 1], [2, 2], [3, 3]])
output = (a, b)
print(output)

# Multiply by batch
a = (10, 3, 4)
b = (10, 4, 5)
output = (a, b)
print()
# tensor([[14., 14.], # [32., 32.]]))
# [32., 32.]])

(): Einstein Summing Conventions is an amazing function that claims to fulfill all your needs! Powerful.

/p/44954540

Tensor and Numpy arrays

Due to the use of Numpy ndarray data processing is very convenient, often will be tensor and Numpy array for mutual conversion, so mastery of the conversion method between the two is necessary. But need to pay attention to one point: the tensor and the Numpy array generated after the conversion share the same memory (so the conversion between them is very fast), change one of them when the other will also change. To convert a tensor to a Numpy array use the () method, and to convert a Numpy array to a tensor use the torch.from_numpy(ndarry) method.

Tensor to Numpy Array

Use the numpy() function to convert the tensor to a Numpy array.

a = (1, 2)
b = () # Convert
print(a, b)
a += 2
print(a, b)
b += 2 # After a has changed, b has also changed
print(a, b)

Numpy array to tensor

Use the from_numpy() function to convert a Numpy array into a tensor.

import numpy as np
a = ([1, 2])
b = torch.from_numpy(a) # do the conversion
print(a, b)
a += 2
print(a, b)
b += 2 # After a has changed, b has also changed
print(a, b)

Cuda tensor vs. CPU tensor

GPUs can play a role in accelerating the deep learning process.Tensors in PyTorch are stored in CPU devices by default, and can be moved to GPUs if they are available.There are two methods for converting CPU tensors to Cuda tensors. In general, you can use the .cuda method to move the Tensor to the GPU; in the case of multiple GPUs, you can use the to method to determine which device to use. The .cpu method can also be used to move the Tensor to the CPU.

x = (2, 4)
print() # cpu
# First method
x = ()
print() # cuda:0
# Second method
device = ("cuda" if .is_available() else "cpu")
if .is_available(): x = (device)
    x = (device)
    print() #cuda:0
# Convert to cpu
x = ()
print() # cpu

Welcome to the public number: Foolish Life Shallow End.