Float rounding

This topic contains 2 replies, has 2 voices, and was last updated by  DruggedBunny 2 weeks, 2 days ago.

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #16152

    jondecker76
    Participant

    Hello

    I’m having some issues with floats in MX2.    I’m working with currency (2 decimal places).  However, when I work with floats everything always comes out with 8 or more decimal digits with slight rounding errors.

    The above example yields

    This is making it very difficult to work with currency, and I can’t find any useful rounding functions in the docs.

    Any ideas on how I can handle this?  Normally it wouldn’t be an issue, but working with currency it’s extremely important that the math comes out correct!

    #16156

    jondecker76
    Participant

    For anyone else having a similar issue, I just made a simple currency class to ensure decimal rounding is handled as expected.  It’s nothing fancy, but does what I need it to do anyways

     

    #16157

    DruggedBunny
    Participant

    That looks much more appropriate — you should never use floats for anything that requires accuracy, in any language!

    They’re inaccurate by design as they have to work within a set number of bits.

    Try this simple, obvious division:

    Here, it prints 0.10000000000000001, but the result may vary on different platforms/targets, and this is to be expected for floating point numbers — they simply can’t represent all decimal numbers accurately.

    Anyway, you’ve figured out a solution that looks good for currency.

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.