The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
Username:
Password:
Save Password
Forgot your Password? | Admin Options

 All Forums
 Origin Forum for Programming
 Forum for Origin C
 Import Multiple Ascii Files
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic

amdanby

USA
3 Posts

Posted - 05/17/2005 :  1:19:08 PM  Show Profile  Edit Topic  Reply with Quote  View user's IP address  Delete Topic
Origin Version (Select Help-->About Origin): 7.0 SR4
Operating System: WinXP SP2

I generate several hundred ascii files per experiment containing two columns (X;Y) from which I wish to import the one column of X (X is the same for all files) and all the remaining Y values into one worksheet. I can do this using the import wizard, but then have to remove by hand every other column to rid myself of all the reduntant X columns masquerading as Y columns. After several days of fiddling with OC I have code that will perform certain elements of the task but I cannot get it to work altogether. (I have no programming experience and am also an idiot!)

My first problem is here. This code will add new columns to my worksheet, but only imports the final ascii file.



void ImportFolder()

{

string strPath = "C:\\test\\";

StringArray saFiles;

FindFiles(saFiles,strPath,"asc");

for(int i=0;i<saFiles.GetSize();i++)

ReadASCII(strPath + saFiles[i]);

}
void ReadASCII(string strFile)

{

ASCIMP ascimp;

if(AscImpReadFileStruct(strFile,&ascimp)!=0)

{

out_str("cannot read " + strFile);

return;

}

Worksheet wks = Project.ActiveLayer();

int colnum = wks.AddCol();

if( !wks )
{
printf("Active layer is not a worksheet!\n");
return;
}

wks.ImportASCII(strFile, ascimp);

}



Am I missing someting to instruct which column to place the next ascii file?

Any advice appreciated, but please don't be afraid to tell me to go away if anyone feels I am just out of my depth and should not be polluting this forum with such ineptitude

easwar

USA
1964 Posts

Posted - 05/17/2005 :  1:59:27 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Hello,

First of all please no need to be apologetic about your post - you have a good/valid question.

Please try the code pasted here and post if you need further help.

Easwar
OriginLab


void ImportFolder()
{
string strPath = "C:\\test\\";
StringArray saFiles;
FindFiles(saFiles,strPath,"asc");

for(int i=0;i<saFiles.GetSize();i++)
ReadASCII(strPath + saFiles[i], i);

}

void ReadASCII(string strFile, int iFileNum)
{
ASCIMP ascimp;
if(AscImpReadFileStruct(strFile,&ascimp)!=0)
{
out_str("cannot read " + strFile);
return;

}
// Set import mode to append cols so that data from
// each file is added to new columns
// Constants are defined in OC_types.h, where the ASCIMP struct is also defined
// iMode is the import mode which is being set to "append columns" here
ascimp.iMode = ASCIMP_MODE_APPEND_COLS;

// If file being imported is 2nd file or later, also set partial import so that
// only the 2nd (Y) column is imported
if( iFileNum > 0 )
{
ascimp.iPartial = ASCIMP_PARTIAL_YES;
ascimp.iPartialC1 = 1;
ascimp.iPartialC2 = 1;
}

Worksheet wks = Project.ActiveLayer();
// no need to add columns manually - import will take care of it int colnum = wks.AddCol();
if( !wks )
{
printf("Active layer is not a worksheet!\n");
return;
}
wks.ImportASCII(strFile, ascimp);
}



Go to Top of Page

rlewis

Canada
253 Posts

Posted - 05/17/2005 :  2:01:33 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
the following OriginC function should solve your problem ...


void DeleteAlternateColumns(string strWksName, int StartNum)
{
Worksheet Wks;
if(Wks.Attach(strWksName)
{
uint NumCols=Wks.GetNumCols();
if(StartNum<0 || StartNum>NumCols) return;
uint col=StartNum;
do
{
Wks.DeleteCol(col);
col++;
} while (col<=Wks.GetNumCols());
}
}


After compiling the above function to your workspace .. you can then use the import wizzard as you have done and then call the function to delete the superfluous columns of x-values using something like
"DeleteAlternateCols(WorksheetName, 2);"

Hope this helps
Go to Top of Page

eparent

118 Posts

Posted - 05/17/2005 :  3:23:24 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
amdanby,

Easwar's code is the correct solution. I have cleaned it up to make it more efficient. The following function is all you need.


void importfolder()
{
string strPath = "C:\\"; // change to your data path

StringArray saFiles;
FindFiles(saFiles, strPath, "asc");
if( saFiles.GetSize() < 1 )
return;

ASCIMP ascimp;
AscImpReadFileStruct(strPath + saFiles[0], &ascimp);

Worksheet wks = Project.ActiveLayer();

wks.ImportASCII(strPath + saFiles[0], ascimp);

ascimp.iMode = ASCIMP_MODE_APPEND_COLS;
ascimp.iPartial = ASCIMP_PARTIAL_YES;
ascimp.iPartialC1 = 1;
ascimp.iPartialC2 = 1;

for(int i = 1; i < saFiles.GetSize(); i++ )
wks.ImportASCII(strPath + saFiles[i], ascimp);
}


Go to Top of Page

amdanby

USA
3 Posts

Posted - 05/18/2005 :  1:36:11 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Thanks for the help. It was exactly what I needed and I now understand a little more Origin C!

I have fiddled with the code and have adjusted the number of header lines, column labels and linked it to a button in my worksheet. However, one problem that I cannot overcome (and am not even sure if it is a problem) is this. My data files are generated with the names "FileName #1, FileName #2,........,FileName #10,FileName #11, etc. Consequently when they are imported into columns they are imported in the sequence #1,#10,#11,#12,....#19,#2,#20,#21 etc. Is there any way of importing them as a normal numerical sequence?

Thanks again.

AMD
Go to Top of Page

eparent

118 Posts

Posted - 05/18/2005 :  4:35:52 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
amdanby,

The following code will do what you want. I have added two more functions and then added a single line to the original function.



void test_impfolder()
{
string strPath = "C:\\"; // change to your data path

StringArray saFiles;
FindFiles(saFiles, strPath, "asc");
if( saFiles.GetSize() < 1 )
return;

sort_filenames(saFiles); //--------- sort file names

ASCIMP ascimp;
AscImpReadFileStruct(strPath + saFiles[0], &ascimp);

Worksheet wks = Project.ActiveLayer();

wks.ImportASCII(strPath + saFiles[0], ascimp);

ascimp.iMode = ASCIMP_MODE_APPEND_COLS;
ascimp.iPartial = ASCIMP_PARTIAL_YES;
ascimp.iPartialC1 = 1;
ascimp.iPartialC2 = 1;

for(int i = 1; i < saFiles.GetSize(); i++ )
wks.ImportASCII(strPath + saFiles[i], ascimp);
}

static int get_filename_num(string& strFileName)
{
int i = strFileName.ReverseFind('#');
if( i < 0 )
return -1; // no number
return atoi(strFileName.Mid(i + 1));
}

static void sort_filenames(StringArray& saFiles)
{
string strTmp;

for( int i = 0; i < saFiles.GetSize() - 1; i++ )
{
for( int j = i + 1; j < saFiles.GetSize(); j++ )
{
if( get_filename_num(saFiles[i]) > get_filename_num(saFiles[j]) )
{
strTmp = saFiles[i];
saFiles[i] = saFiles[j];
saFiles[j] = strTmp;
}
}
}
}



Go to Top of Page

amdanby

USA
3 Posts

Posted - 05/19/2005 :  12:05:00 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Eparent,

Thanks for the help. I have a problem compiling this in Version 7 SR4. The line "sort_filenames(saFiles);" gives a "function or variable sort_file_names not found" error. Is this function only avaiable in 7.5?
Go to Top of Page

Mike Buess

USA
3037 Posts

Posted - 05/19/2005 :  4:06:22 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
The function sort_filenames() is not built in to Origin. It's one of the functions posted by eparent. I suspect that a transcription error occurred when you copied those functions to your Origin C file.

Mike Buess
Origin WebRing Member

Edited by - Mike Buess on 05/19/2005 4:08:37 PM
Go to Top of Page

a.abc.b35

175 Posts

Posted - 01/09/2011 :  12:43:29 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
I am importing multiple ASCII files from a folder to worksheet using origin C. All the ASCII data are in X vs Y column format (2 columns in each file,X column is same for all).I am importing the data is a single worksheet with single X column and multiple Y columns.
Below is the code I am using:
.......................................................
void importfolder()
{
string strPath = BrowseGetPath(); // input folder path
printf("Folder Path: ");// outputs the path of the input folder
out_str(strPath); // outputs the path of the input folder

StringArray saFiles; // string vector which will store all the file names in the folder with given extension
FindFiles(saFiles, strPath, "chi");// Finds files with ext 'chi' in strpath and stores file names in saFiles
if( saFiles.GetSize() < 1 ) // checks if there is no file present in the folder
return;

ASCIMP ascimp; // structure for ASCII type data import. see online if detail is needed.
AscImpReadFileStruct(strPath + saFiles[0], &ascimp);// scans and analyzes the given file (1st argument)
//to gather info for ASCII import into worksheet.stores in ascimp.


Worksheet wks; // define a worksheet

wks.Create(); // create a wks for the new data
wks.ImportASCII(strPath + saFiles[0], ascimp); // Imports ASCII data from the file (1st argument)
// and stores in ascimp.

ascimp.iMode = ASCIMP_MODE_APPEND_COLS; //
ascimp.iPartial = ASCIMP_PARTIAL_YES;
ascimp.iPartialC1 = 1; // imports from 2nd column
ascimp.iPartialC2 = 1; // imports upto 2nd column

for(int i = 1; i < saFiles.GetSize(); i++ )
wks.ImportASCII(strPath + saFiles, ascimp);

}

.............................................................
I got this code from this forum !:)
....
Now the problem is the following (like the thread initiator had which ad been answered with a sorting function which is not there by default): ASCII files are numbered as 1,2,....9,10,11,12,...19,20,...29,30....101...etc.And I want them to be imported numerically. But what the code does is that it imports as 1,10,100,101,11,...19,2,21,....etc. So I figured that the files needs to sorted after they are stored in the string array 'saFiles'. For this I wanted to use Array.sort() as in C-programming. But it does not compile and gives the following error :
Error, Member function vector::sort not defined or does not have matching prototype
....
Than I searched a bit more in the origin wiki and used saFiles.Sort().
But it does not solve it,though the program compiles now !
...
Can anyone help me solve this problem ?

AB
Go to Top of Page

a.abc.b35

175 Posts

Posted - 01/11/2011 :  3:24:02 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
i think the best idea is to create the files, at first place, such that the names dont get jumbled up. Like if we save a file series like : myfile_0001.dat etc than there will never be a problem again.
But I am also looking forward to a better and useful solution.:)

AB
Go to Top of Page

rlewis

Canada
253 Posts

Posted - 01/12/2011 :  02:23:08 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
If your files are numbered in order MyFile_1, MyFile_2, MyFile_3 ... MyFile_108 .... then your problem is quite easy to solve ...

Try the following ...

void DoStuff()
{
	vector<string> vctString;
	vctString.RemoveAll();
	// Create a Numerically  indexed string array of MyFile_250, Myfile_249, MyFile_248 ..... MyFile_1
	// sorted in reverse order ...
	for(int i=250;i>=1;i--)
	{
		string strTemp="MyFile_"+("%d",i);
		vctString.Add(strTemp);
	}
	if(SortByIndices(vctString)==true)
	{
		// The Array FilesNames should now be sorted in ascending order and one can 
		// process the Contents in order with 
		for(i=0;i<vctString.GetSize();i++)
		{
			out_str(vctString[i]);
		}
	}
}


bool SortByIndices(vector<string> &vctMyNames)
{
	WorksheetPage wP;
	// Create a temporary WorkBook
	if(wP.Create("origin.otw",CREATE_TEMP)==true)
	{
		Worksheet Wks(wP.Layers(0));
		if(Wks.IsValid()==true)
		{
			Wks.SetSize(vctMyNames.GetSize(),2);
			Column wCol1(Wks,0);
			Column wCol2(Wks,1);
			if(wCol1.IsValid()==false || wCol2.IsValid()==false)
			{
				return (false);
			}
			// Set Column 1 as numeric and Column 2 as Text ...
			if(wCol1.SetFormat(OKCOLTYPE_NUMERIC)==false || wCol2.SetFormat(OKCOLTYPE_TEXT)==false)
			{
				return (false);
			}
			// Place the string Array into Column 2;
			if(wCol2.PutStringArray(vctMyNames)==false)
			{
				return (false);
			}
			vector<int> vctIndices;
			vctIndices.RemoveAll();
			// Parse the String Array to get the numerical Indices of the file names
			// and store these indices in a array vctIndices
			for(int i=0;i<vctMyNames.GetSize();i++)
			{
				string strTemp1=vctMyNames[i];
				string strIndex=strTemp1.GetToken(1,'_');
				vctIndices.Add(atoi(strIndex));
			}
			Dataset dS;
			if(dS.Attach(Wks,0)==false)
			{
				return (false);
			}
			// Place the contents of the array vctIndices int column 1 of the Worksheet Wks
			dS.SetSize(vctIndices.GetSize());
			dS=vctIndices;
			// Sort the Worksheet as per the contnets of Column 1 in ascending order;
			if(Wks.Sort()==false)
			{
				return (false);
			}
			// Copy the sorted array back to vctMyNames
			if(wCol2.GetStringArray(vctMyNames)==true)
			{
				return (true);
			}
		}
	}
	return(false);;
}
Go to Top of Page
  Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000