Skip to content

How to properly set precision of whole numbers and decimals? #285

Answered by RobertZenz
Andre601 asked this question in Q&A
Discussion options

You must be logged in to vote

No, precision/scale of a BigDecimal works by setting the significant number of digits, not decimal places. So if you have the value "123456798" and you set the precision to "3" you get "12300000". Same goes if you have "1.23456789" which becomes "1.23".

What you seem to want to do is to set a "high enough" precision to do your calculation, and then round the result.

Replies: 1 comment 9 replies

Comment options

You must be logged in to vote
9 replies
@RobertZenz
Comment options

@Andre601
Comment options

@RobertZenz
Comment options

@Andre601
Comment options

@RobertZenz
Comment options

Answer selected by Andre601
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants