[Solved] Why the result of ‘0.3 * 3’ is 0.89999999999999 in scala language? [duplicate]

Floating point calculation is reasonably complex subject. This has to do with the binary representation of a floating point number, that doesn’t guarantee every possible number (obviously), to have an exact representation, which can lead to errors in operations, and yes, these errors can propagate. Here is a link on the subject, although it isn’t … Read more

[Solved] why java Math.pow arguments double?

The results of pow are often irrational, fractional or too large to store as a long, If you want to use powers of integers you can use BigInteger bi = BigInteger.valueOf(100); BigInteger bi2 = bi.pow(20); // 10^40 Another reason maybe that Math has many function which are taken from C, and also from common CPU … Read more

[Solved] How do I truncate the significand of a floating point number to an arbitrary precision in Java? [duplicate]

Suppose x is the number you wish to reduce the precision of and bits is the number of significant bits you wish to retain. When bits is sufficiently large and the order of magnitude of x is sufficiently close to 0, then x * (1L << (bits – Math.getExponent(x))) will scale x so that the … Read more

[Solved] how to round off float after two place decimal in c or c++

Something like as follows. Hope that its understandable. Output:https://www.ideone.com/EnP40j #include <iostream> #include <iomanip> #include <cmath> int main() { float num1 = 95.345f; float num2 = 95.344f; num1 = roundf(num1 * 100) / 100; //rounding two decimals num2 = roundf(num2 * 100) / 100; //rounding two decimals std::cout << std::setprecision(2) << std::fixed << num1 << ” … Read more

[Solved] Problematic understanding of IEEE 754 [closed]

What is precision? It refers to how closely a binary floating point representation can represent a real value. Real values have infinite precision and infinite range. Digital values have finite range and precision. In practice a single-precision IEEE-754 can represent real values of a precision of 6 significant figures (decimal), while double-precision is good for … Read more

[Solved] Error: invalid types ‘int [200][float]’ for array subscript

So from the comments: if col is float col[H][W];, your trying to index vx/vy via a float. You would have to cast to int again: int vx2 = vx[static_cast<int>(col[iposy][iposx])]; int vy2 = vy[static_cast<int>(col[iposy][iposx])]; Be careful: There is no implicit index checking, so if your floats are out of range (negative or > WIDTH/HEIGHT), you most … Read more

[Solved] Overloaded C++ function float parameters error [duplicate]

The function call func(1.14, 3.33) is ambiguous because 1.14 and 3.33 are doubles and both can be converted to either int or float. Therefore the compiler does not know which function to call. You can fix this by explicitly specifying the type of the constants func(float(1.14), float(3.33)) or by changing the overload from func(float, float) … Read more

[Solved] Blatant floating point error in C++ program

80-bit long double (not sure about its size in MSVS) can store around 18 significant decimal digits without loss of precision. 1300010000000000000144.5700788999 has 32 significant decimal digits and cannot be stored exactly as long double. Read Number of Digits Required For Round-Trip Conversions for more details. 8 solved Blatant floating point error in C++ program