| T O P I C R E V I E W |
| marcel1992 |
Posted - 11/11/2020 : 03:55:58 AM
Origin Ver. and Service Release (Select Help-->About Origin): OriginPro 2020 9.7.0.188
Operating System: Windows 10, 2004 Build 19041.572
I load a tdms file into Origin via labtalk scrip with following script:
- dlgPath for the folder
- findFiles for every file in the folder
- impNITMD for loading first tmds file into Origin
end finish the sequence with
- expNITDM for export the data
- win -cwd Book* to delete the complete file out of the RAM
Following strange situation happens:
- If the destination folder is on the external HDD (8Tb WD Gold, 150Mb/sec read/write) the script loads the data with 20Mb/sec in the Ram/in Origin
- If the destination folder is on the external SSD (500Gb Samsung Evo 860 Pro, 500Mb/sec read/write) the script loads the data with 100Mb/sec in the Ram/in Origin
- If the destination folder is on the internal M.2 storage (250Gb, WD, 1,5Gb/sec) the script loads the data with 120Mb/sec in the Ram/in Origin
In general -> higher basic speed -> higher load speed into the RAM but why only 20Mb/sec if the storage can perform 8 times faster?
Is there a "hack" to increase the load speed because I have have some TB data and this will take weeks to finish. |
| 3 L A T E S T R E P L I E S (Newest First) |
| snowli |
Posted - 11/13/2020 : 10:05:56 AM
Hi Marcel,
We have added script for data connector. It may not be available in your version.
E.g. in Origin 2021, the following script works.
newbook;wbook.dc.add("tdms");
wks.dc.source$ = "C:\temp\xxxx.tdms";
wks.dc.sel$="TDMS"; //the top node
wks.dc.Import();
https://www.originlab.com/doc/LabTalk/ref/Wks-DC-obj
Thanks, Snow
|
| marcel1992 |
Posted - 11/13/2020 : 03:56:49 AM
I benchmarked the tdms data connector and wow, from 170sec per file to 30sec per file close to 6 times faster. Is there a due date when the dtms data connector will be available for the script?
Thanks for the great Idea! |
| Chris D |
Posted - 11/11/2020 : 09:15:57 AM
We can investigate some here, but I would suggest you benchmark other applications that can read TDMS files to see if they are noticeably better. Perhaps one is the TDMS Excel Add In but I'm sure there are others. (NOTE added later: Excel Add In not a good example of other apps- very slow).
We do have a newer method for importing TMDS called the "TDMS Data Connector". While it isn't scriptable, you CAN use it for benchmarking. It is accessible from the Data->Connect to File menu. If you don't see it, then choose the Add New... menu entry and add the connector.
Thanks,
Chris Drozdowski
Originlab Technical Support
|
|
|
