Significant digits
by Svet on 02/13/04 at 04:45:45

Hello.

Firsty - a great site, went into my favorites on first visit. I've got a question for the devs - what is the accuracy (in significant digits) of your conversion factors?

For most purposes 7 dps default (single data type?) you have is great, but for sensitive calculations I need about 15 (double). I've cheated so far by putting in a large value to convert and extracting all the significant digits I need - but that is limited by the conversion factors.

Could you recommend someplace with high-accuracy conversion factors perchance?

Thank you and my apologies if the info exists elsewhere.

Svet.

Re: Significant digits
by Robert Fogt on 02/13/04 at 19:54:55

Well JavaScript is capable of many more significant digits, but the problem is the rounding errors.

Computers do not work well with Real numbers. So a very good math function library is needed for 7+ significant digit accuracy.

You can find math libraries such as that in C/C++, Pascal, VB, etc, but not in JavaScript.

Lots of times with JavaScript I get results such as 15.9999999999999 instead of 16. So limiting to 7 accurate digits seemed the best idea instead of the alternative.

Though if I do find a decent math function library in JavaScript, rest assured I would snatch that up quick as can be. I too would love to see the conversions into 15+ digits.

Go Back | Archive Index