Edit Node

Shealladh

Have a strange EDIT issue which I seem to see from time to time.

Using the edit command, I modify a Node with 18888.62110,4725.00000

Does not matter if I edit same node, or others, entering the number above, it always has the node as 18888.62109,4725.00000 instead. I tried all other nodes around the node I am editing, it always has same entry each time. Does not matter the type of entity either, WHY?

Have tried modifying the Decimal places too from 5 to 6, still same result.

Monsen

That's a limitation in how computers store numbers.

Basically, because computers use binary internally to represent numbers, there will always be errors in precision for decimal numbers. Computers store these numbers smartly, so if the number before the decimal point is small, you get more precision, but since you have a somewhat large number (5 digits), it means that you loose out on precision on the decimal part. (Float can store 7 digits precicely)

There are different data types that can be used that offer different degrees of precision. CC3+ is using a float type to store these numbers, which is less precise than a double, but also uses less memory. Float was selected for the code back in the day, because it gave the best memory performance compared to the needs for very high precision.

In short, this is just how computers work.

If you are interested, here's a code example that shows exactly what happens:

#include <iomanip>
#include <iostream>
using namespace std;

int main() 
{
 float f = 18888.62110;
 double d = 18888.62110;

 cout << endl << "18888.62110" << endl;
 cout << "Float: " << setprecision(10) << f << endl;
 cout << "Double: "<< setprecision(10) << d << endl;

 f = 18.62110;
 d = 18.62110;

 cout << endl << "18.62110" << endl;
 cout << "Float: " << setprecision(7) << f << endl;
 cout << "Double: "<< setprecision(7) << d << endl;
}

Output:

18888.62110
Float: 18888.62109
Double: 18888.6211

18.62110
Float: 18.6211
Double: 18.6211

Shealladh

Thanks for the reply and an example with breakdown of the why's.

Was afraid this might have been the case, sadly.