Decimal Math Precision

Using v13768, haven’t tested with earlier.
qty DECIMAL(3.0)
amt DECIMAL(9.2)
qty = 1.0
amt = 9.99
message(format(qty * amt,@n9.2))

Anyone expecting to see 10.00?

That’s not what I see in Build 13768 or in Clarion 9.1 or Clarion 6.X or Clarion 2


I get 9.99 in 13768.

You’re expecting 10, or seeing 10?

Thanks for confirming my insanity.

It seems it comes down to a typo:
DECIMAL(9,2) vs DECIMAL(9.2) <-notice comma vs period

Why I now get a compile error on a new clw & project (ie expression must be a constant) and do not on the old clw & project is very curious. The project settings certainly look the same.

A bit curious. Writing the code as Graham did BUT with my typo declared decimal as a global variable gives a compiler error - Expression must be constant. However, putting that same code into a procedure does NOT yield a compile error and will result in the “rounding” problem when run.

Yes you’re right - it seems to not properly check the declaration when it’s in a procedure




testProc procedure()
qty decimal(3,0)
amt decimal(9.2)

    qty = 1.0
    amt = 9.99
    message('qty ' & qty & | 
        '|amt ' & amt)

    message(format(qty * amt,@N9.2))

Interesting. I thought it was just a typo.

Yeah, the more typos I have, the better chance of finding something interesting.