| Author |
 Topic  |
|
|
ticofiz1
6 Posts |
 Posted - 05/19/2016 : 8:13:00 PM
|
Origin Ver. and Service Release (OriginLab 2016 SR0):
Operating System:Windows 7/64
Dear all,
I have to import multiple files into Origin Workbook and each file has 500 columns and 1024 rows. First imported file takes about five minutes to be imported but the next one is about 10 minutes, the third one about 20 minutes and so on. Any idea why is so slow? Things are more complicated when I start analyzing the data as it takes ages to get the results. In Matlab the whole process takes about 30-40 seconds to import and analyze each file.
This problem is encountered in any version of Originlab I have tried, no matter if it was on 32 or 64 bit.
Many thanks,
Alexander |
|
|
SeanMao
China
288 Posts |
Posted - 05/20/2016 : 02:16:50 AM
|
Hi,
Can you send your files to tech@originlab.com with detailed requirements about what you want to do so that we can try on our side?
Thanks!
Sean
OriginLab Tech. |
 |
|
|
ticofiz1
6 Posts |
Posted - 05/20/2016 : 02:38:00 AM
|
I have sent you some typical data that I try to import in Originlab.
Please let me know if you need more information.
Many thanks,
Alexander |
|
|
|
ticofiz1
6 Posts |
Posted - 05/20/2016 : 04:03:30 AM
|
I think that one of the issues is the fact that OriginLab is not designed to work on multicore processors. I have 16 threads available on my computer but no matter what the OriginLab uses one thread all the time. I have disabled hyperthreading and now OriginLab is a bit faster (maybe by 30%). But still it takes ages to import the 100's of files that I have. Maybe OriginLab team has in mind some multicore version of Origin.
Many thanks,
Alexander |
|
|
|
SeanMao
China
288 Posts |
Posted - 05/20/2016 : 04:31:42 AM
|
Hi,
I have replied to you in email, please check!
I will bring up your suggestion for discussion and see what we can get from it.
Thanks!
Regards!
Sean
OriginLab Tech. |
|
|
|
Hideo Fujii
USA
1582 Posts |
Posted - 05/20/2016 : 5:34:43 PM
|
Hi Alexander,
> But still it takes ages to import the 100's of files that I have.
If your job is to import bunch of files, which is more I/O-intensive task rather than CPU-intensive,
you can run multiple Origin instances, probably not much overhead.
For example, I have a sample ascii file with 1,000,000 rows with 2 columns, and when I import
these files serially in a single Origin instance, it took around 36 seconds (=18x2). When I run
two Origin instances, and import separately at each instance, they completed importing almost
the same time by 18 seconds. Multi-core may be more effective for numerical computation on a
matrix or image processing.
Hope this hints something.
--Hideo Fujii
Originlab
|
|
|
|
ticofiz1
6 Posts |
Posted - 05/20/2016 : 9:43:38 PM
|
Hi Hideo,
Thank you for your suggestion. Unfortunately it does not apply to my case as I would have to open 100's of Origin files to import all my data. Each bunch of data (501 columns and 1024 rows) it takes around +8 minutes to import. Plus the data analysis it takes another +8 minutes. For the same steps it takes around 45 seconds on Matlab because instead of using one core it uses all the cores available. OriginLab is a fantastic software, everything available within one mouse click, but it is way too slow to be used when big files are to be analyzed.
Best wishes,
Constantin |
|
|
|
muh
France
14 Posts |
Posted - 01/18/2017 : 11:06:31 AM
|
I have similar problem. However, I do not see why the number of processors is important. First,how is it related to the increase of the import time as number of already imported files increases?
Secondly,I do not see why importing 500x1024 table should take 20 minutes. Or 5. Or anything more than 5 seconds. This is clearly problem in the import bloatware. Just for comparison - Excel imported 4000x1000 file in less then 20 seconds. Origin is working on it for some 20 minutes already. |
|
|
|
Hideo Fujii
USA
1582 Posts |
Posted - 01/18/2017 : 3:40:20 PM
|
Hi muh,
I have tried to import an ASCII file with 500 columns and 1024 rows into my Origin 2017. and
it took only 1.5 seconds. I suspect that some extra factor involves, such as network traffic, etc.
though not sure exactly. (All your Origin installation is local, not remote?)
Sorry for not be able to give you better insight.
--Hideo Fujii
OriginLab
|
|
|
|
muh
France
14 Posts |
Posted - 01/22/2018 : 1:42:57 PM
|
20 minute import time was given by the topicstarter, so it is unrelated to my installation.
However my experience is similar. As stated even in the title of the topic - import time grows as more and more data are added to the project. Eventually the time becomes quite unresonable. It applies to other operations as well. It seems that things become worse when there are many columns in each table (as opposed to tables with fewer column but more rows). |
|
|
| |
Topic |
|
Origin slower and slower as more data is imported