Location>code7788 >text

js Precision of numerical calculations

Popularity:348 ℃/2024-11-11 16:05:48

〇、js numerical calculations have imprecise results

Recently contacted the financial system, there will be some simple calculations on the page, it was found that in fact very simple calculations, but the results calculated by js is not the expected value, may bring a large series of 0 or 9, resulting in the calculation of the results of the error, this article to a simple summary of how to deal with, and how to deal with to avoid this problem.

Let's start by looking at what all the inaccuracies can be.

// The four basic operations of addition, subtraction, multiplication and division.
// Accuracy issues are not really present in all of them, such as in the following test:
let num1 = 0.1;

let result = num1 + num2.
("0.1+0.1 = " + result);
// 0.1+0.1 = 0.2
// 0.1+0.2 = 0.30000000000000004
// 1.1*1.2 = 1.32
// 1.1*1.1 = 1.2100000000000002
// In fact, all four basic arithmetic operations are similar, so here's a brief list of the two categories

// [Numerical comparisons
("'0.1+0.2 = 0.3' Comparison result: ", 0.1 + 0.2==0.3); // '0.1+0.2=0.3' = 0.3); // '0.1+0.2=0.3' = 0.3
// '0.1 + 0.2 = 0.3' comparison result: false

I. Brief description of causes

This is because information within a computer is represented in binary, i.e., various codes made up of 0s and 1s, but since theCertain Floating Point NumbersThere's no way to represent it accurately in binary, which brings up a host of precision issues. Of course, this is not a problem unique to js.

Suggest an article, if you're interested in digging deeper: /p/33333351 The talk is supposed to be very detailed, but there are some complexities bloggers don't quite understand even after reading through it.

Here's the general process sorted out.

  • Convert decimals to binary

Multiply the fractional part by 2. Take the integer part, and if the fractional part is not 0 then continue to multiply by 2 until the fractional part is 0. Then put the removed integer digits in positive order. Divide the integer part by 2.

0.1 * 2 = 0.2 // get0
0.2 * 2 = 0.4 // get0
0.4 * 2 = 0.8 // get0
0.8 * 2 = 1.6 // get1
0.6 * 2 = 1.2 // get1
0.2 * 2 = 0.4 // get0
......

  The binary representation of 0.1 is: 0.000110011..... .0011..... (0011 infinite loop)
  The binary representation of 0.2 is: 0.00110011..... .0011...... (0011 infinite loop)

Although it is of infinite length, there still has to be a standard way of displaying it, and that is the IEEE 754 standard.

  • IEEE 754 standardized method of expressing values

The IEEE 754 standard is the standard number of the IEEE Standard for Floating-Point Arithmetic.The IEEE 754 standard specifies the interchange, arithmetic format, and methodology for binary and decimal floating-point number descriptions in a computer programming environment.

The detailed labeling method is skipped, so let's just look at the results.

js has only one numeric type, number, and number uses theIEEE 754 double precision floating point formatThe highest bit is a sign bit (0 plus 1 minus). The highest bit is a sign bit (0 positive and 1 negative), the next 11 bits are the exponent, and the remaining 52 bits are the mantissa (significant digit).

// The binary storage format for 0.1 in js is:
// Significant bits are separated by commas, exponent bits are separated by semicolons
0.1: 0,01111111011;1001100110011001100110011001100110011001100110011010
0.2: 0,01111111100;10011001100110011001100110011001100110011001100110011010
  • Compute the sum of two binary numbers
// Result in binary form
sum = 0.01001100110011001100110011001100110011001100110100
// And finally in decimal form
sum = 2^2 + 2^5 + 2^6 + ... + 2^52 = 0.30000000000000004440892098500626
// Approximate result: 0.30000000000000004

Details are available:/p/33333351

II. Solutions

2.1 () function, multiply and divide by 10 to the nth power

Depending on the precision required, theFirst multiply 10 to the nth power, via the () functionroundLater.Divide by10 to the nth power.

function numToString(num){
    let factor = (10, 4); // keep up to 4 decimal places
    // by multiplying by a factor (e.g. 4 times 10) and rounding up to the nearest whole number
    // Finally, divide by the same factor to get precise control over a specific number of decimal places
    
    return (); }
}

For rules on rounding (), see the following test results:

("(4.2)     ", (4.2)  );
("(4.5)     ", (4.5)  );
("(4.7)     ", (4.7)  );
("(-4.2)    ", (-4.2) );
("(-4.5)    ", (-4.5) );
("(-4.7)    ", (-4.7) );
("(1.5)     ", (1.5)  );
("(2.5)     ", (2.5)  );
("(3.5)     ", (3.5)  );
("(-1.5)    ", (-1.5) );
("(-2.5)    ", (-2.5) );
("(-3.5)    ", (-3.5) );

It can be seen thatPositive numbers with 5 decimal places are rounded to 1; negative numbers with 5 decimal places are discarded

2.2 toFixed() method, take the fixed number of decimal places directly.

This method is a directSpecify the number of decimal places to be retained; if there are fewer decimal places, they will be completed with 0

The rounding rules for toFixed() are not strictly based on the number of decimal places to be retained.

If the number of decimal places to be used as a judgment is 5 and there is no number greater than 0 after it, it is rounded to the nearest odd number; if there is a non-zero value after 5, it is rounded to 1.

For example, 4.55 with 1 decimal place would be 4.5, but 4.5500001 would result in 4.6.

For example, -4.55 with 1 decimal place is -4.5, but -4.5500001 results in -4.6.

The following example is for reference:

let num = 4.22;
("(1):4.22)     ", (1));
num = 4.55;
("(1):4.55)     ", (1));
num = 4.551;
("(1):4.551)    ", (1));
num = 4.65;
("(1):4.65)     ", (1));
num = 4.77;
("(1):4.77)     ", (1));
num = -4.22;
("(1):-4.22)    ", (1));
num = -4.55;
("(1):-4.55)    ", (1));
num = -4.551;
("(1):-4.551)   ", (1));
num = -4.65;
("(1):-4.65)    ", (1));
num = -4.77;
("(1):-4.77)    ", (1));
num = -4.77;
("(1):-4.77)    ", (4));

Attention:The result of toFixed() is a string type.If you want to end up with a number type, you need to convert it with the Number() function.

2.3 Rounding off excess decimal places by regular expressions

You must convert the numeric type to a string before using the match() method.

let num = 3.14959;
let numStr = (); // must be a string
let fixedNumStr = (/^-? \d+(\. \d{0,2})? /)[0]; // Regular expression for two decimals.
let fixedNum = parseFloat(fixedNumStr); (fixedNum); // Output regular expression for two decimals.
(fixedNum); // Output: 3.14