| Author |
 Topic  |
|
|
jdf726
78 Posts |
 Posted - 01/21/2019 : 4:02:57 PM
|
Origin Ver. and Service Release (Select Help-->About Origin): 2018 pro 64bit
Operating System: win 7
I am deducing a step size in from some imported text data in originc and then writing this to a cell, which in turn is substituted into a plot text label.
SOmetimes the text label reads something like 4.999999999999e-3 instead of 5E-3, which is irritating as it pushed lots of useful info off the page!
The case I was just looking at should have been ((-0.55)-(-0.6))/100 = 0.05/100 = 5e-4.
The answer goes into a declared 'double' and is then written using SetCell.
I think in the past I have put formatting into my text labels to tame the number of decimal places, but here it should not be necessary because the experimental increments are normally 'round numbers' (this is even a cross check of whether the import has been done correctly - a weird increment is a symptom of a missing file).
Any ideas?
J. |
|
|
Castiel
343 Posts |
Posted - 01/21/2019 : 5:21:36 PM
|
quote:
Originally posted by jdf726
Origin Ver. and Service Release (Select Help-->About Origin): 2018 pro 64bit
Operating System: win 7
I am deducing a step size in from some imported text data in originc and then writing this to a cell, which in turn is substituted into a plot text label.
SOmetimes the text label reads something like 4.999999999999e-3 instead of 5E-3, which is irritating as it pushed lots of useful info off the page!
The case I was just looking at should have been ((-0.55)-(-0.6))/100 = 0.05/100 = 5e-4.
The answer goes into a declared 'double' and is then written using SetCell.
I think in the past I have put formatting into my text labels to tame the number of decimal places, but here it should not be necessary because the experimental increments are normally 'round numbers' (this is even a cross check of whether the import has been done correctly - a weird increment is a symptom of a missing file).
Any ideas?
J.
How do you compare two floating point values? It is in some cases dangerous to compare them directly. https://stackoverflow.com/questions/10334688/how-dangerous-is-it-to-compare-floating-point-values
&&&&&&&&&
&&&
&&
& _____ ___________
II__|[] | | I I |
| |_|_ I I _|
< OO----OOO OO---OO
**********************************************************
|
 |
|
|
jdf726
78 Posts |
Posted - 01/21/2019 : 6:07:01 PM
|
Hi Castiel,
It isn't a comparison, just a basic calculation and cell value assignment.
I vaguely remember that in some languages (python?) they are really 'honest' with you about what is stored internally (i.e. it shows you what the number stored in memory really evaluates to given the finite number of bits in the representation).
Most languages are not like that though... and would have thought that 0.05/100 = 0.5e-4 (which is what origin's console tells me). The problem is also intermittent, so sometimes it evaluates to what I expect.
I was wondering whether the format of the columns was related.
J.
quote:
Originally posted by Castiel
quote:
Originally posted by jdf726
Origin Ver. and Service Release (Select Help-->About Origin): 2018 pro 64bit
Operating System: win 7
I am deducing a step size in from some imported text data in originc and then writing this to a cell, which in turn is substituted into a plot text label.
SOmetimes the text label reads something like 4.999999999999e-3 instead of 5E-3, which is irritating as it pushed lots of useful info off the page!
The case I was just looking at should have been ((-0.55)-(-0.6))/100 = 0.05/100 = 5e-4.
The answer goes into a declared 'double' and is then written using SetCell.
I think in the past I have put formatting into my text labels to tame the number of decimal places, but here it should not be necessary because the experimental increments are normally 'round numbers' (this is even a cross check of whether the import has been done correctly - a weird increment is a symptom of a missing file).
Any ideas?
J.
How do you compare two floating point values? It is in some cases dangerous to compare them directly. https://stackoverflow.com/questions/10334688/how-dangerous-is-it-to-compare-floating-point-values
&&&&&&&&&
&&&
&&
& _____ ___________
II__|[] | | I I |
| |_|_ I I _|
< OO----OOO OO---OO
**********************************************************
|
|
|
|
yuki_wu
896 Posts |
Posted - 01/21/2019 : 10:14:36 PM
|
Hi J,
Would you mind send us the opj file that you mentioned in the first post? You mentioned that the plot text label reads a wrong data. You could send that file via tech@originlab.com with subject line: Attn: Yuki.
Thanks, Yuki
OriginLab
|
|
|
|
Castiel
343 Posts |
Posted - 01/21/2019 : 10:15:41 PM
|
quote:
Originally posted by jdf726
Hi Castiel,
It isn't a comparison, just a basic calculation and cell value assignment.
I vaguely remember that in some languages (python?) they are really 'honest' with you about what is stored internally (i.e. it shows you what the number stored in memory really evaluates to given the finite number of bits in the representation).
Most languages are not like that though... and would have thought that 0.05/100 = 0.5e-4 (which is what origin's console tells me). The problem is also intermittent, so sometimes it evaluates to what I expect.
I was wondering whether the format of the columns was related.
J.
The problem seems to be ((-0.55)-(-0.6))/100 = 0.05/100 = 5e-4
In OriginC, as "double" values, (-0.55) is stored as 0xbfe199999999999a, (-0.6) as 0xbfe3333333333333. However, the result of ((-0.55)-(-0.6)) is stored as 0x3fa999999999999a which is about 0.04999999999999993. The Script/Command window may have further process to correct the error.
Python (Python 3.6.3 [MSC v.1900 64 bit (AMD 64)] on win32) gives the same result.
>>> ((-0.55)-(-0.6))
0.04999999999999993
This suggests that you may not calculate the increment like ((-0.55)-(-0.6))/100. Or you may not compare the calculated increment with the experimental one directly.
The former one seems less applicable in OriginC currently. (-0.55), (-0.6) and (0.05), none of them can be stored as "double" exactly. Take (-0.55) as an example,
0xbfe199999999999a
b f e 1 9 9 9 9 9 9 9 9 9 9 9 a
1011 1111 1110 0001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1010
1 01111111110 0001100110011001100110011001100110011001100110011010
sign exponent mantissa
-1 1022 1.0001100110011001100110011001100110011001100110011010 (binary)
-1 * 2^(1022 - 1023) * 1.1 (not exactly 1.1)
-1 * 0.5 * 1.1 (not exactly 1.1)
-0.55 (not exactly -0.55)
For the latter one, you can use is_equal() to compare the calculated increment and the experimental increment.
https://www.originlab.com/doc/OriginC/ref/is_equal
Or You can call round() in OriginC before SetCell().
https://www.originlab.com/doc/OriginC/ref/round
&&&&&&&&&
&&&
&&
& _____ ___________
II__|[] | | I I |
| |_|_ I I _|
< OO----OOO OO---OO
**********************************************************
|
Edited by - Castiel on 01/21/2019 10:19:18 PM |
|
|
|
jdf726
78 Posts |
Posted - 01/22/2019 : 02:48:12 AM
|
Yes thanks... this looks much more like a 'representation' type error.
I can probably put a 'round' in with a large number of decimal places and still check that the correct number of files have been imported (if one in 1000 files is missing, the increment will appear wrong by a 0.1%, which is still much larger than this fractional 'error' (i know it is not really an 'error')
I would be interested to know whether this can be controlled in a less 'manual' way.
J. |
|
|
|
Castiel
343 Posts |
Posted - 01/22/2019 : 03:23:40 AM
|
quote:
Originally posted by jdf726
Yes thanks... this looks much more like a 'representation' type error.
I can probably put a 'round' in with a large number of decimal places and still check that the correct number of files have been imported (if one in 1000 files is missing, the increment will appear wrong by a 0.1%, which is still much larger than this fractional 'error' (i know it is not really an 'error')
I would be interested to know whether this can be controlled in a less 'manual' way.
J.
I prefer is_equal() to round(). That is, change the way of comparing two values of type 'double' instead of rounding the results.
For round(), what about round(x, 14-log10(abs(x)))?
&&&&&&&&&
&&&
&&
& _____ ___________
II__|[] | | I I |
| |_|_ I I _|
< OO----OOO OO---OO
**********************************************************
|
|
|
|
jdf726
78 Posts |
Posted - 01/22/2019 : 12:36:37 PM
|
Done. It works fine now. Thanks!
J. |
|
|
| |
Topic |
|
|
|
Rounding of numbers using setcell