| Author |
 Topic  |
|
|
HDMVDM
Japan
4 Posts |
 Posted - 05/28/2002 : 10:48:53 AM
|
I would like to ask a help about importing a large fixed length ASCII file. I have several files (100MB to 400MB), each with about 100 rows and one million records. In File>Import>ASCII Option, I can only specify 7 or so row length, say, 10,2,5,6,7,7,8. Is there any way to specify about 100 rows?
I am using Japanese version of Origin 6.1117.
I really appreciate any help with importing or any shareware that can transform fixed length ASCII file to comma or tab spaced CSV file.
|
|
|
Mike Buess
USA
3037 Posts |
Posted - 05/28/2002 : 12:53:22 PM
|
If fixed length means each number has the same number of digits, select "Fixed, col widths:" in ASCII Options and enter that number to the right. Then you won't need delimiters.
I don't recall off hand what the maximum number of columns is in an Origin 6.1 worksheet, but it's much fewer than 1E6. So you won't be able to import all records to one worksheet in any case.
I'd try to import the whole file to a single column and sort it out later using LabTalk. If each record has the same number of rows (e.g., 100) then it would be straightforward to write a script to extract 500 or so records at a time to separate worksheets. The number of worksheets in your project may become unwieldy, but I can suggest a script to try if you're interested.
Mike Buess
Origin WebRing Member |
 |
|
|
cpyang
USA
1406 Posts |
Posted - 05/28/2002 : 6:29:36 PM
|
From your description I think you meant 100 columns and you need to specify column width. You can specify as many as you can and the last one is used for the all remaining columns.
If you have Origin 7, this can be read easily by writing a simple Origin C program. We have included a sample called "EarthProbeData.c" to show how to read fixed width data into Origin.
CP
|
|
|
|
HDMVDM
Japan
4 Posts |
Posted - 05/28/2002 : 8:01:00 PM
|
Dear Mr.Mike Buess and CP:
Thank you very much for your help. I was wrong and I meant column width. I apologize for my English. Each column has a length of 10C,2N,5C,6C,7C,7C,8C, for instance, where C is the character and N is the numerical number.
CP, could you elaborate on how I can specify as many as I can? I can specify at most 17 single digit column width. If there any simple and easy code that I can use in Origin 6.1? Or is it better to uprade to Origin 7? I have never done any programming in C and I do not know how difficult to modify the exisiting code (the only programming I did was in Matlab).
I was thinking of using Origin just as an alternative to Excel in order to sort this large data set. Do you think it is perhaps better to use database software such as Oracle9i?
I really appreciate further help.
|
|
|
|
Hideo Fujii
USA
1582 Posts |
Posted - 06/06/2002 : 12:10:05 PM
|
Hi HDMVDM,
We are planning to implement to accept a longer input string in the Fixed Column Width field in the ASCII Import Option. Do you think that the field length 400 bytes (as 100*(3byte[column pos]+1byte[comma] in specification) is long enough in your field of application? (Our Tracking# QA1987)
--Hideo
|
|
|
|
HDMVDM
Japan
4 Posts |
Posted - 06/16/2002 : 9:30:52 PM
|
Dear Hideo:
I checked the documentation of several database I have. The longest is 1400 byte. This is a combination of character data, numerical data and 2-byte character data.
I hope this helps!
|
|
|
|
Hideo Fujii
USA
1582 Posts |
Posted - 06/19/2002 : 1:42:33 PM
|
quote:
I checked the documentation of several database I have. The longest is 1400 byte. This is a combination of character data, numerical data and 2-byte character data.
Hi,
could you please tell us how many data items in this 1400-buye-long record?
Thanks!
--Hideo
|
|
|
|
HDMVDM
Japan
4 Posts |
Posted - 06/20/2002 : 10:38:33 AM
|
Dear Hideo:
I sent an e-mail to one of the Japanese distributors who got in touch with me about this point. I sent hime an e-mail and asked him to send you two documentation files for the database I use.
|
|
|
|
Hideo Fujii
USA
1582 Posts |
Posted - 06/20/2002 : 4:45:50 PM
|
quote:
I sent hime an e-mail and asked him to send you two documentation files for the database I use.
Great! Thank you very much!
--Hideo
|
|
|
| |
Topic |
|
about importing large fixed length ASCII file