Ubuntu
openoffice.org package

Using Precision as Shown rounds calculated fields to nearest multiple of 10

Asked by Paul Smith

Cut and paste from spreadsheet:
1 0
2 0
3 0
4 0
5 10
6 10
7 10
8 10
9 10
10 10
Column 2 cells have the formulae =A1, =A2, etc. This is the display with Precision as Shown selected.

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu openoffice.org Edit question
Assignee:
No assignee Edit question
Solved by:
Tony Pursell
Solved:
Last query:
Last reply:
Revision history for this message
marcus aurelius (adbiz) said : 		#1

works fine on my calc.
use reveal formula to see what is there.
i doubt it's =a1, =a2.
looks more like you rounded the numbers up.

Revision history for this message
Paul Smith (paul-smith1066) said : 		#2

Marcus,

I can assure you that column 2 does contain the formulae as given. Placing the cursor in the relevant cell displays the formula in the input box. On my system flipping the "precision as shown" flag off changes column 2 to 1, 2, 3, etc. Flipping it back on again changes column 2 back to 0, 0, 0, etc. There's nothing else up my sleeve.

Revision history for this message
Best Tony Pursell (ajpursell) said : 		#3

Paul

Does the above example need "precision as shown" set? I have to admit that I have never had to set it myself.

There is obviously a problem which is acknowledged by bug #691813. Does anything more need to be done? Have you a further point to make? If not, please mark this question as Solved.

Thanks

Tony

Revision history for this message
Paul Smith (paul-smith1066) said : 		#4

Tony,

The example I gave is a trivial sheet which I put together to demonstrate the precision problem - obviously "precision as shown" is not needed here. "Precision as shown" is essential when using currency to 2 decimal places. Some sums which may look as if they add up correctly may be unequal by a small fraction and fail a comparison in an IF function, for example.

Sorry, I entered this query before I noticed bug #691813 and am quite happy to mark it solved.

Thanks for your help,

Paul

Revision history for this message
Paul Smith (paul-smith1066) said : 		#5

Thanks Tony Pursell, that solved my question.