The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
Username:
Password:
Save Password
Forgot your Password? | Admin Options

 All Forums
 Origin Forum
 Origin Forum
 Import data from a structured text file
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic

AKazak

Russia
1205 Posts

Posted - 10/04/2023 :  02:57:59 AM  Show Profile  Edit Topic  Reply with Quote  View user's IP address  Delete Topic
OriginPro 2023b (64-bit) SR1 10.0.5.157
Windows 10 Pro for Workstations 22H2 19045.3086

Greetings!

I have many txt structured files (see an example attached) with the data to imported to Origin for further analysis.
What is the best was of parsing this type of files?

Thank you.

https://my.originlab.com/ftp/forum_and_kbase/Images/Sample.txt

---
Andrey

aplotnikov

Germany
165 Posts

Posted - 10/04/2023 :  08:36:48 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
https://www.json.org/json-en.html
https://www.originlab.com/fileExchange/details.aspx?fid=505
Go to Top of Page

YimingChen

1613 Posts

Posted - 10/04/2023 :  09:18:49 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
The data file doesn't adhere to the standard JSON format, for example, there are missing commas at the end of some long strings. Would it be possible to fix that?

James
Go to Top of Page

aplotnikov

Germany
165 Posts

Posted - 10/04/2023 :  10:56:17 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
It looks like corrupted standard JSON format. But the topic starter can use and modify any available source code for JSON parser to consider possible data errors.
The solution consisting in the improvement of the JSON generator seems less time consuming. However, I am not sure, whether it is possible in this particular case.
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 10/04/2023 :  11:37:51 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
This is not JSON format, but output of a proprietary software.
Thus I cannot change the generator...

---
Andrey
Go to Top of Page

aplotnikov

Germany
165 Posts

Posted - 10/04/2023 :  7:02:47 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply

https://my.originlab.com/ftp/forum_and_kbase/Images/Sample_json.txt
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 10/23/2023 :  08:35:03 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
The developer of the software producing the target JSON-like files responded that they use Python stringmap library to generate.

Do you mean that I could use Data Connector Browser to parse these files?

---
Andrey
Go to Top of Page

YimingChen

1613 Posts

Posted - 10/23/2023 :  3:17:25 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
I couldn't find any Python library named stringmap. If it exists, we could explore creating a data connector using Python.

James

quote:
Originally posted by AKazak

The developer of the software producing the target JSON-like files responded that they use Python stringmap library to generate.

Do you mean that I could use Data Connector Browser to parse these files?

---
Andrey

Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 10/26/2023 :  01:31:29 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by YimingChen

I couldn't find any Python library named stringmap. If it exists, we could explore creating a data connector using Python.

James



Dear James,

Please see the library attached.
Can you consider making a relevant data connector for this, please? https://my.originlab.com/ftp/forum_and_kbase/Images/stringmap.py.txt

---
Andrey
Go to Top of Page

YimingChen

1613 Posts

Posted - 10/26/2023 :  11:40:25 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Please try the attached string map connector.opx. It may ask to install the PyYAML module and need to restart Origin. Thanks.



https://my.originlab.com/ftp/forum_and_kbase/Images/String%20Map%20Connector.zip

James
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 10/29/2023 :  06:52:56 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by YimingChen

Please try the attached string map connector.opx. It may ask to install the PyYAML module and need to restart Origin. Thanks.



https://my.originlab.com/ftp/forum_and_kbase/Images/String%20Map%20Connector.zip

James



Dear James,

I tried, but getting:



Can you add "*.gdr" to the import filter list, please?

---
Andrey
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 11/02/2023 :  10:01:47 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Can you share an update to test, please?

---
Andrey
Go to Top of Page

snowli

USA
1388 Posts

Posted - 11/02/2023 :  11:49:48 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi Andrey,

Yiming is on vacation. I will ask some other colleague to take a look.

Thanks, Snow
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 11/02/2023 :  12:23:26 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by snowli

Hi Andrey,

Yiming is on vacation. I will ask some other colleague to take a look.

Thanks, Snow



Dear Snow,

No worries.
I will be waiting until Yiming returns back from his vacation.

---
Andrey
Go to Top of Page

Sam Fang

293 Posts

Posted - 11/03/2023 :  03:08:49 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi Andrey,

I updated the app. It can now support file names with "S*.txt" or "*.gdr". You can test it.
https://my.originlab.com/ftp/forum_and_kbase/Images/StrMapDC.zip

quote:
Originally posted by AKazak

Can you share an update to test, please?

---
Andrey



Thanks.

Sam
OriginLab Technical Services
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 11/05/2023 :  01:45:40 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by Sam Fang

Hi Andrey,

I updated the app. It can now support file names with "S*.txt" or "*.gdr". You can test it.
https://my.originlab.com/ftp/forum_and_kbase/Images/StrMapDC.zip

quote:
Originally posted by AKazak

Can you share an update to test, please?

---
Andrey



Thanks.

Sam
OriginLab Technical Services



Dear Sam,

I tested the version 2, but it seems still unable to read the whole file.

Can share a link to secured file exchange, so I could share more examples, please?

---
Andrey
Go to Top of Page

Sam Fang

293 Posts

Posted - 11/06/2023 :  02:50:34 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi Andrey,

You can click Send File to Tech support button on the top right of the forum page. And it will show you how to upload large files to our ftp.

quote:
Originally posted by AKazak
Dear Sam,

I tested the version 2, but it seems still unable to read the whole file.

Can share a link to secured file exchange, so I could share more examples, please?

---
Andrey



Thanks.

Sam
OriginLab Technical Services
Go to Top of Page

minimax

350 Posts

Posted - 12/19/2023 :  02:21:02 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi Andrey,

When you have time, you may share us more sample files?
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 03/06/2024 :  05:15:56 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by minimax

Hi Andrey,

When you have time, you may share us more sample files?



Greetings!

The real name of the format is YAML format (“JSON with comments“).

---
Andrey
Go to Top of Page

AKazak

Russia
1205 Posts

Posted - 09/24/2024 :  03:44:25 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
quote:
Originally posted by minimax

Hi Andrey,

When you have time, you may share us more sample files?



Done!
Please see "\incoming\GDR 2024-09-24_.zip".

---
Andrey

Edited by - AKazak on 09/24/2024 03:45:43 AM
Go to Top of Page

Sam Fang

293 Posts

Posted - 10/09/2024 :  09:05:01 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hi Andrey,

Thanks for your data files.

I tried your four .gdr files. And all can work well.

Did you fail to import them?

Thanks.

Sam
OriginLab Technical Services
Go to Top of Page
  Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000