Getting Started with Deep Learning in C#: Pytorch Basics

Tutorial name: Getting Started with Deep Learning in C#

Author: Kouryou the Idiot

Address:

1.2 Pytorch Basics

This article introduces the basic API of Pytorcn, mainly the creation of arrays and arithmetic, because the content is similar to Numpy, and Numpy type can be turned , so the reader interested in Numpy can refer to the author's other articles:

Getting Started with the Numpy Framework for Python

/archives/21461

/whuanle/p/

Tip: learning this article, if you have enough understanding of linear algebra, the learning effect is better, no linear algebra foundation is okay, will learn later. This article will use both Python and C# to write examples, so that readers can compare the differences, in the subsequent chapters of learning, basically only use C# to write examples.

Basic use

Since many of the values in neural networks exist in the form of vectors or arrays, etc., and are not as simple as the numerical types in everyday programming, printing numerical information is a means of learning to understand or debugging a program, and below we observe how the program prints complex data types in Pytorch.

printable

The following uses Pytorch to create an array from 0..9 and then prints the array.

Python：

import torch
x = (10)
print(x)

tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

The C# version does not use()Instead, use the officially provided libraries.

using TorchSharp;

var x = (10);
(style:);
(style:);
(style:);
(style:);
(style:);

[10], type = Int64, device = cpu 0 1 2 3 4 5 6 7 8 9
[0, 1, 2, ... 7, 8, 9]
[10], type = Int64, device = cpu
[10], type = Int64, device = cpu 0 1 2 3 4 5 6 7 8 9
[10], type = Int64, device = cpu, value = long [] {0L, 1L, 2L, ... 7L, 8L, 9L}

Python prints results that are easier to understand, and C# prints them by default in a way that is harder to read, so in general, visualizations use the Enumeration.

C# prints values with astring? fltFormat = "g5", which means precision, i.e., the number of decimal places to print.

A number of extension methods are provided in the package, and the reader can use thex.print_numpy() The extension directly prints information about the corresponding style.

For later chapters, the torch package name for Python and the TorchSharp namespace for C# are introduced by default; subsequent code examples may omit the introduction code and leave it up to the reader.

fundamental data type

Pytorch's data types are not quite the same as the basic types in our programming language, so the reader should be aware of the difference.

Refer to the link for detailed official documentation:

/docs/stable/tensor_attributes.html

Pytorch creates data types that begin with said. is the underlying structure used to process a variety of data in machine learning models, including scalars, vectors, matrices, and higher dimensional tensors.If the author understands correctly, the Tensor object created in Pytorch is called a tensor. Developers can create tensors in Pytorch with various forms of data。

The data types created by Pytorch are represented using Tensor objects.

For an understanding of this statement, it is recommended to read this article and go back to it.

PyTorch has twelve different data types, listed below:

data type	dtype
32-bit floating point number	`torch.float32` maybe
64-bit floating point number	`torch.float64` maybe
64-bit plural	`torch.complex64` maybe
128-bit complex number	`torch.complex128` maybe
16-bit Floating Point	`torch.float16` maybe
16-bit Floating Point	`torch.bfloat16`
8-bit integer (unsigned)	`torch.uint8`
8-bit integer (signed)	`torch.int8`
16-bit integer (signed)	`torch.int16` maybe
32-bit integer (signed)	`torch.int32` maybe
64-bit integer (signed)	`torch.int64` maybe
boolean

The following demonstrates setting the type of an array when creating an array of all 1 values.

Python：

float_tensor = (1, dtype=)
double_tensor = (1, dtype=)
complex_float_tensor = (1, dtype=torch.complex64)
complex_double_tensor = (1, dtype=torch.complex128)
int_tensor = (1, dtype=)
long_tensor = (1, dtype=)
uint_tensor = (1, dtype=torch.uint8)

C#：

var float_tensor = (1, dtype: torch.float32);
var double_tensor = (1, dtype: torch.float64);
var complex_float_tensor = (1, dtype: torch.complex64);
var complex_double_tensor = (1, dtype: torch.complex128);
var int_tensor = (1, dtype: torch.int32); ;
var long_tensor = (1, dtype: torch.int64);
var uint_tensor = (1, dtype: torch.uint8);

In C#, an enumeration represents the data type of Pytorch, so you can specify the data type in two ways.

Example:

var arr = (3,3,3, .Float32);
arr.print_numpy();

Or:

var arr = (3,3,3, torch.float32);
arr.print_numpy();

CPU or GPU Computing

As we know, AI models can be run on CPUs or GPUs, and Pytorch data can be done the same way, by setting the bound device when creating the data type, and the corresponding device will be used in the computation.

General usecpu indicates the CPU, usecuda maybecuda:{video card serial number} Indicates the GPU.

The following code is written to determine whether Pytorch is running on the GPU or the CPU.

Python：

print(torch.get_default_device())

C#：

 (torch.get_default_device())

If the current device supports GPUs, Pytorch is started using the GPU, otherwise Pytorch is started using the CPU, which can be accessed via the('cuda')、('cuda:0') Specifies the GPU to use, and which GPU to use.

Python：

if .is_available():: print("The current device supports GPUs")
    print("Current device supports GPUs")
    device = ('cuda')
    # Boot with GPU
    torch.set_default_device(device)
    current_device = .current_device()
    print(f "Bound GPU is: {current_device}")
print(f "The bound GPU is: {current_device}")
    # GPU is not supported, use CPU to boot
    device = ('cpu')
    torch.set_default_device(device)

default_device = torch.get_default_device()
print(f "Currently using {default_device}")

C#：

if (.is_available())
{
    ("The current device supports GPUs"); var device = ("cuda",index:0); {
    var device = ("cuda",index:0); // Start up with the GPU.
    // Start with the GPU
    torch.set_default_device(device); }
}
else
var device = ("cpu",index:0); // Start with GPU.
    var device = ("cpu"); // Start with the CPU.
    // Start with the CPU
    torch.set_default_device(device); var device = ("cpu"); // Start with CPU.
    ("Currently using CPU"); }
}

var default_device = torch.get_default_device();
($"Currently using {default_device}"); } var default_device = torch.get_default_device(); }

C# no..current_device() For this method, it is recommended to set which GPU is used by default, i.e. set the index parameter.

Alternatively you can use the.device_count() Getting how many graphics cards the device has will not be covered here.

Pytorch also supports the use of the CPU or GPU for individual data types, as well as the ability to mix the two, which I won't go into here.

Tensor Type

In Pytorch, you can convert types such as scalars and arrays to Tensor types, and the data structure represented by a Tensor is called a tensor.

x = (3.0);

basic array

Pytorch usesasarray() function converts the obj value to an array, which is defined as follows:

(obj, *, dtype=None, device=None, copy=None, requires_grad=False) → Tensor

Official API documentation:/docs/stable/generated/#torch-asarray

obj It can be one of the following:

a tensor
a NumPy array or a NumPy scalar
a DLPack capsule
an object that implements Python’s buffer protocol
a scalar
a sequence of scalars

What the author won't or can't be used in this article will not be translated.

For example, pass in a usual array type and convert it to an array type in Pytorch.

Python：

arr = ([1,2,3,4,5,6], dtype=)
print(arr)

C#：

var arr = torch.from_array(new float[] { 1, 2, 3, 4, 5 });
(style: );

Please note that the versions in the two languages differ somewhat.

As mentioned earlier, you can set whether to use the CPU or the GPU for individual data types.

device = ("cuda",index=0)
arr = (obj=[1,2,3,4,5,6], dtype=, device=device)
print(arr)

Converts the data type to use a CPU device:

device = ("cuda",index=0)
arr = (obj=[1,2,3,4,5,6], dtype=, device=device)
arr = ()
print(arr)

But converting data between GPU and CPU consumes some performance.

Generating arrays

It is used to create an array with all 0 elements, and you can specify the size of the array, which is defined as follows:

(*size, *, out=None, dtype=None, layout=, device=None, requires_grad=False) → Tensor

Python：

arr = (10, dtype=)
print(arr)

C#：

var arr = (10);
(style: );

Alternatively, you can specify the dimension of the generated array, e.g., the following specifies the generation of the2*3 of a multidimensional array.

var arr = (2,3, torch.float32);
(style: );

The code is in C#.

Print:

[[0, 0, 0] [0, 0, 0]]

We can also generate arrays of multiple dimensions, such as the following to generate a3*3*3 of the array:

var arr = (3,3,3, torch.float32);
(style: );

The printout is formatted below for ease of understanding.

[
[[0, 0, 0]  [0, 0, 0]  [0, 0, 0]]
[[0, 0, 0]  [0, 0, 0]  [0, 0, 0]]
[[0, 0, 0]  [0, 0, 0]  [0, 0, 0]]
]

Creating an array filled with all 1's is exactly the same as using it, so I won't repeat it here.

Creating an uninitialized array is exactly the same as using it, so we won't repeat it here.

Since it does not initialize the memory, the memory area will have residual data and the value of the element is indeterminate.

copy function

In addition, the three functions above have corresponding prototype copy functions:

torch.ones_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor

torch.zeros_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor

torch.empty_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor

They work by copying an identical structure, depending on the type of the array, and then populating it with the corresponding values.

The following example copies the same structure of the array, but fills it with the value 1.

var arr = torch.ones_like((3, 3, 3));
(style: );

The code language is C#.

[
[[1, 1, 1]  [1, 1, 1]  [1, 1, 1]]
[[1, 1, 1]  [1, 1, 1]  [1, 1, 1]]
[[1, 1, 1]  [1, 1, 1]  [1, 1, 1]]
]

A tensor will be generated and the array will be populated with the data from the[0,1) uniformly distributed random numbers on the interval.

The function definitions are as follows:

(*size, *, generator=None, out=None, dtype=None, layout=, device=None, requires_grad=False, pin_memory=False) → Tensor

For example, generating2*3 size, ranging from[0,1) Random numbers in intervals, coded in C#:

var arr = (2,3);
(style: );

[[0.60446, 0.058962, 0.65601] [0.58197, 0.76914, 0.16542]]

Since C#'s library for plotting graphs is not as easy to use as Python matplotlib, the reader can refer to the , packages for quick conversion of Pytorch types and generation of plotting windows. The following demonstrates the use of writing code to plot uniformly distributed random numbers for displaying graphs using Python matplotlib and the framework.

Python：

import torch
import  as plt

arr = (100, dtype=)

print(arr)

x = ()
y = x
(x,y)
()

C#：

using ;
using ;
using TorchSharp;

var x = (100);

(style: );

 myPlot = new();
(x, x);
var form = (400, 300);

From the figure, it can be seen that the generated random numbers are uniformly scattered in the[0,1) Interval.

Generate a random sample with a standard normal distribution (mean 0, variance 1) with a given shape. The random sample takes values in the range [0,1).

Definitions are as follows:

(*size, *, generator=None, out=None, dtype=None, layout=, device=None, requires_grad=False, pin_memory=False) → Tensor

Official Documentation:/docs/stable/generated/#

Since C# is not good for drawing, the examples are written in Python:

import torch
import  as plt

arr = (100, dtype=)

print(arr)

x = ()
y = x
(x, bins=30, alpha=0.5, color='b', edgecolor='black')

()

The x-axis is the value and the y-axis is the number of occurrences.

Generate random numbers in some interval.

Definitions are as follows:

(low=0, high, size, \*, generator=None, out=None, dtype=None, layout=, device=None, requires_grad=False) → Tensor

For example, to generate 10 elements in the range 0-100, install the5*2 It is composed of a structure and written in C# code.

var arr = (low: 0, high: 100, size: new int[] { 5, 2 });
(style: );

[[17, 46] [89, 52] [10, 89] [80, 91] [52, 91]]

If you want to generate a floating-point number in a certain interval, you can use the But because The generation range is[0,1), so you need to multiply the multiplier yourself. For example to generate[0,100) of random numbers.

var arr = (size: 100, dtype: .Float32) * 100;
(style: );

Specify the interval as well as the step size, and extract the elements uniformly to generate an array.

Definitions are as follows:

(start=0, end, step=1, *, out=None, dtype=None, layout=, device=None, requires_grad=False) → Tensor

For example, it is necessary to generate[0,1,2,3,4,5,6,7,8,9] Such an array can be used:

var arr = (start: 0, stop: 10, step: 1);
(style: );

If you change the step size to 0.5.

var arr = (start: 0, stop: 10, step: 0.5);

[0.0000, 0.5000, 1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000, 4.0000,
        4.5000, 5.0000, 5.5000, 6.0000, 6.5000, 7.0000, 7.5000, 8.0000, 8.5000,
        9.0000, 9.5000]

Array manipulation and computation

axle

In Pytorch, the dim(dimension) parameter is often used to represent the axes, which are the layers of the tensor.

There are the following arrays:

[[ 1, 2, 3 ], { 4, 5, 6 ]]

If thea = [1,2,3]，b = [4,5,6]Then:

[a,b]

Then when we want to geta whendim(a) = 0，dim(b) = 1。

var arr = torch.from_array(new[,] { { 1, 2, 3 }, { 4, 5, 6 } });

(style: );

// Print the dimensions
();

var a = arr[0];
();

();

[[1, 2, 3] [4, 5, 6]]
[2, 3]
[3], type = Int32, device = cpu 1 2 3
[3], type = Int32, device = cpu 4 5 6

Here we understand it in two steps, since the array is2*3 arrays, you can use the.() Print out.

Since the first layer has two elements, you can use theTensor[i] Get the i-th element of the first layer, wherei<2。

Similarly, since the next layer of a and b both have 3 elements, the second layer ofn<3。

For example to set the array's3、6 Two elements taken out.

You can write this in C#, but you can't select it when printing, otherwise it won't print.

var arr = torch.from_array(new[,] { { 1, 2, 3 }, { 4, 5, 6 } });

var a = arr[0, 2];
(style: );
var b = arr[1, 2];
(style: );

Similarly, if the array has three levels, you can get it like this3、6 Two elements

var arr = torch.from_array(new[, ,] { { { 1, 2, 3 } }, { { 4, 5, 6 } } });

var a = arr[0, 0, 2];
(style: );
var b = arr[1, 0, 2];
(style: );

If you want to take out a part of an element, TorchCsharp can use thea[i..j] syntax interception, an example of which is shown below.

var arr = torch.from_array(new int[] { 1, 2, 3 });
arr = arr[0..2];
(style: );

[1, 2]

Array Sorting

There are a number of sorting functions in Pytorch:

sort : the value along a given dimension in ascending order on theinput The elements of the tensor are sorted.

argsort: It is an indirect ordering along the specified axis, which is not explained in this article.

msort: by the value of theinput The tensor is sorted in ascending order along its first dimension.(t) be equivalent to(t, dim=0)。

sort It can be in descending or ascending order, and the parameters are described below:

(input, dim=-1, descending=False, stable=False, *, out=None)

input (Tensor) - Enter the tensor.
dim (int, optional) - dimension to sort by
descending (bool, optional) - control sort order (ascending or descending)
stable (boo, optional) - Stabilizes the sort routine so that the order of equivalent elements is preserved.

Example:

var arr = (start: 0, stop: 10, step: 1);

// Or use (arr, descending: true)
( Values, Indices) a1 = (descending: true);

(style: );

[9, 8, 7, ... 2, 1, 0]

Values is the sorted result and Indices is the sorting rule.

If the array structure is complex, only the innermost array is sorted when no parameters are set by default. There are two layers of arrays as shown in the following code.

var arr = torch.from_array(new[,] { { 4, 6, 5 }, { 8, 9, 7 }, { 3, 2, 1 } });

( Values,  Indices) a1 = ();

(style: );
(style: );

[[4, 5, 6] [7, 8, 9] [1, 2, 3]]
[[0, 2, 1] [2, 0, 1] [2, 1, 0]]

Indices records the previous sort position of the current element.

When setting the(dim: 0); when sorted according to the first layer.

[[3, 2, 1] [4, 6, 5] [8, 9, 7]]
[[2, 2, 2] [0, 0, 0] [1, 1, 1]]

When setting the(dim: 1); When only the inside layer is sorted.

[[4, 5, 6] [7, 8, 9] [1, 2, 3]]
[[0, 2, 1] [2, 0, 1] [2, 1, 0]]

When the dimensions of a tensor are large, we can order them layer by layer like this.

var arr = torch.from_array(new[, ,] { { { 4, 6, 5 }, { 8, 9, 7 }, { 3, 2, 1 } } });

var dimCount = ;
for (int i = dimCount - 1; i >= 0; i--)
{
    ( Values,  Indices) a1 = (dim: i);
    arr = ;
    (style: );
}

[[[1, 2, 3]  [4, 5, 6]  [7, 8, 9]]]

C# multidimensional arrays are not as convenient as Python, which requires that the number of elements at each level be the same.

For example, the following array declaration is correct:

var array = new int[, , ]
{
    {
        { 10, 12, 11},{ 14, 15, 11 }
    },
    {
        { 4, 6, 5 }, { 8, 9, 7 }
    }
};

An error is reported if the number of layer elements is inconsistent:

Also note that C# has multidimensional arrays and interleaved arrays, and the following is how interleaved arrays are declared, which is not supported by TorchSharp. Multidimensional arrays are arrays, and interleaved arrays are arrays of arrays, or arrays of arrays of arrays, so be careful to distinguish.

var array = new int[][][]
{
    new int[][]
    {
        new int[] { 1, 2, 3 },
        new int[] { 4, 5, 6 },
        new int[] { 7, 8, 9 }
    },
    new int[][]
    {
        new int[] { 10, 12, 11 },
        new int[] { 14, 15, 13 }
    }
};

array operator

In PyTorch, tensors support a number of operators, some of which are described below:

arithmetic operator

+: Addition, e.g.a + b
-: Subtraction, e.g.a - b
*: Element-level multiplication, e.g.a * b
/: Element-level division, e.g.a / b
//: Element-level division, e.g.a // b , TorchCsharp is not supported.
%: modulo operations such asa % b
**: power operations such asa ** bTorchCsharp does not support this, use the.pow(x) Substitute.

logical operator

==: Element-level equivalence comparisons, such asa == b
!=: The element level is not equal to the comparison, e.g.a != b
>: The element level is greater than the comparison, e.g.a > b
<: The element level is less than the comparison, e.g.a < b
>=: Element level is greater than or equal to the comparison, e.g.a >= b
<=: The element level is less than or equal to the comparison, e.g.a <= b

bitwise operator (computing)

&: Per-bit and arithmetic operations, such asa & b
|: Bitwise or operations such asa | b
^: Per-bit arithmetic operations such asa ^ b
~: Bitwise inverse operations such as~a
<<: shifted bitwise left, e.g.a << b
>>: Shift right by bit, e.g.a >> b

Indexing and Slicing

[i]: indexing operators such asa[i]
[i:j]: Slicing operators such asa[i:j] TorchCsharp usesa[i..j] Grammar.
[i, j]: Multidimensional indexing operators such asa[i, j]

For example the value of each element of the tensor*10。

var arr = torch.from_array(new int[] { 1, 2, 3 });
arr = arr * 10;
(style: );

[10, 20, 30]

In addition, there are many more Pytorch functions, which you will gradually learn in later chapters.