The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
Username:
Password:
Save Password
Forgot your Password? | Admin Options

 All Forums
 Origin Forum for Programming
 LabTalk Forum
 Incomplete array data import from database
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic

deepakdinakaran

India
14 Posts

Posted - 02/07/2019 :  10:37:42 AM  Show Profile  Edit Topic  Reply with Quote  View user's IP address  Delete Topic
Origin Ver. and Service Release (Select Help-->About Origin): Origin 2018, 64 bit, SR 1
Operating System:Windows 8.1

Hello,

I use origin to import data from database (Postgresql). 1D array are stored in the database in the following format {{1,2,3,4,5}}. When I try to access similar array data, origin does not give the complete data points, rather it gives incomplete data import like {{1,2,3

I noticed this behavior for large datasets. When I import around 60 datapoints (ex.{{1,2,3…60}}), origin exactly imports as {{1,2,3…60}}. But when I try to import more than 150 datapoints (ex. {{1,2,3..200}}, it results in incomplete data import as mentioned before.

This is my first issue, and the second issue is the imported data has more numbers after decimal point, which is not expected. The data in the database has only 6 decimal places but origin imports data with more than 10 decimal places. This is observed only while importing array data.

Please refer to the attached picture for a sample import. It would be great to have suggestions.

Thank you and Best regards,
Deepak


cpyang

USA
1406 Posts

Posted - 02/07/2019 :  11:29:22 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
We need a little bit more information since the screen showed data imported as text in column C.

CP
Go to Top of Page

Castiel

343 Posts

Posted - 02/08/2019 :  12:46:55 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by deepakdinakaran

Origin Ver. and Service Release (Select Help-->About Origin): Origin 2018, 64 bit, SR 1
Operating System:Windows 8.1

Hello,

I use origin to import data from database (Postgresql). 1D array are stored in the database in the following format {{1,2,3,4,5}}. When I try to access similar array data, origin does not give the complete data points, rather it gives incomplete data import like {{1,2,3

I noticed this behavior for large datasets. When I import around 60 datapoints (ex.{{1,2,3…60}}), origin exactly imports as {{1,2,3…60}}. But when I try to import more than 150 datapoints (ex. {{1,2,3..200}}, it results in incomplete data import as mentioned before.

This is my first issue, and the second issue is the imported data has more numbers after decimal point, which is not expected. The data in the database has only 6 decimal places but origin imports data with more than 10 decimal places. This is observed only while importing array data.

Please refer to the attached picture for a sample import. It would be great to have suggestions.

Thank you and Best regards,
Deepak






You should have called unnest() in SQL to expand the array.


                                          &&&&&&&&&
                                        &&&
                                       &&
                                      &  _____ ___________
                                     II__|[] | |   I I   |
                                    |        |_|_  I I  _|
                                   < OO----OOO   OO---OO
**********************************************************
Go to Top of Page

deepakdinakaran

India
14 Posts

Posted - 02/12/2019 :  11:00:49 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by cpyang

We need a little bit more information since the screen showed data imported as text in column C.

CP




Thank you for your suggestion.
Go to Top of Page

deepakdinakaran

India
14 Posts

Posted - 02/13/2019 :  03:42:02 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
[quote]Originally posted by Castiel

I used your suggestion.

I was able to find a solution for the issues by modifying the sql query.
There seems to be a maximum limit for characters a cell can handle in Origin, so I had to round off the data to specific decimal places through sql. This solved both issue 1 and issue 2.

Thanks.

Best regards,
Deepak

Edited by - deepakdinakaran on 02/13/2019 03:43:39 AM
Go to Top of Page

Chris D

428 Posts

Posted - 02/13/2019 :  10:10:03 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi,
By testing via copy and paste of long strings, Origin reports that the maximum total length of string data in a cell is 3799 characters and after that, Origin will truncate data after that length.

The additional decimal places are the result of the inability to perfectly represent exact floating point values that is not limited to Origin but is a general issue with most modern computer systems. If you are curious, the following link explains it quite well. It's definitely a good read for the intellectually curious.
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Perhaps you can truncate the numeric at the server as convert to string/text type there. Experiment.



Thanks,
Chris Drozdowski
Originlab Technical Support
Go to Top of Page

deepakdinakaran

India
14 Posts

Posted - 02/20/2019 :  08:25:07 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by Chris D

Hi,
By testing via copy and paste of long strings, Origin reports that the maximum total length of string data in a cell is 3799 characters and after that, Origin will truncate data after that length.

The additional decimal places are the result of the inability to perfectly represent exact floating point values that is not limited to Origin but is a general issue with most modern computer systems. If you are curious, the following link explains it quite well. It's definitely a good read for the intellectually curious.
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Perhaps you can truncate the numeric at the server as convert to string/text type there. Experiment.



Thanks,
Chris Drozdowski
Originlab Technical Support





Hi,

Good to know. Thanks for replying in detail.

Best regards,
Deepak
Go to Top of Page
  Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000