The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
 All Forums
 Origin Forum
 Origin Forum
 Pre-processing of gigantic time series data

Note: You must be registered in order to post a reply.
To register, click here. Registration is FREE!

Screensize:
UserName:
Password:
Anti-Spam Code:
Format Mode:
Format: BoldItalicizedUnderlineStrikethrough Align LeftCenteredAlign Right Horizontal Rule Insert HyperlinkUpload FileInsert Image Insert CodeInsert QuoteInsert List
   
Message:

* HTML is OFF
* Forum Code is ON
Smilies
Smile [:)] Big Smile [:D] Cool [8D] Blush [:I]
Tongue [:P] Evil [):] Wink [;)] Clown [:o)]
Black Eye [B)] Eight Ball [8] Frown [:(] Shy [8)]
Shocked [:0] Angry [:(!] Dead [xx(] Sleepy [|)]
Kisses [:X] Approve [^] Disapprove [V] Question [?]

 
Check here to subscribe to this topic.
   

T O P I C    R E V I E W
eno Posted - 06/23/2003 : 04:21:38 AM
Hi,

I'm a beginner of Origin 7.
I would like to pre-process gigantic time series data (170,000 rows),
such as moving average, turbulence intensity (standard deviation / mean value).
How do I have to descrive a LabTalk or Origin C script for these functions?

Thanks,
eno
4   L A T E S T    R E P L I E S    (Newest First)
easwar Posted - 06/25/2003 : 12:09:45 PM
Hi eno,

And here is an Origin C solution for the same thing:
Cut and paste the code posted below to a new Origin C file.
Once you compile this file, you can then go to script window and enter a command such as
tscompute data1_b 10
which would then create a new worksheet, and compute min, max, mean, and std dev on groups of 10, for the data in data1_b

Easwar
OriginLab.


void tscompute(string strDatasetName, int iGroupSize)
{
	// Declare dataset and verify
	Dataset dsData(strDatasetName);
	if(!dsData)
	{
		printf("Dataset is not valid!\n");
		return;
	}
	
	// Check group size
	int iSize = dsData.GetSize();
	if ( iGroupSize < 0 || iGroupSize > (iSize/2) )
	{
		printf("Incorrect group size!\n");
		return;
	}
	
	// Create new wks for results
	Worksheet wks;
	wks.Create("origin.otw");
	// Delete all existing columns
	while (wks.Columns(0))
		wks.DeleteCol(0);
	// Add new colums and assign them label names and set them wide enough
	int iColIndex;
	iColIndex = wks.AddCol();
	wks.Columns(iColIndex).SetLabel("Group No.");
	wks.Columns(iColIndex).SetType(OKDATAOBJ_DESIGNATION_X);
	wks.Columns(iColIndex).SetWidth(11);
	iColIndex = wks.AddCol();
	wks.Columns(iColIndex).SetLabel("Group Min");
	wks.Columns(iColIndex).SetWidth(11);
	iColIndex = wks.AddCol();
	wks.Columns(iColIndex).SetLabel("Group Max");
	wks.Columns(iColIndex).SetWidth(11);
	iColIndex = wks.AddCol();
	wks.Columns(iColIndex).SetLabel("Group Mean");
	wks.Columns(iColIndex).SetWidth(11);
	iColIndex = wks.AddCol();
	wks.Columns(iColIndex).SetLabel("Group Std Dev");
	wks.Columns(iColIndex).SetWidth(11);
	// Turn on label display for columns
	wks.ShowLabels();

	// Loop over dataset, set the range for each group, and compute stats
	BasicStats stat;
	int iSteps = iSize / iGroupSize;
	for(int ii = 0; ii < iSteps; ii++)
	{
		int iBegin = ii * iGroupSize;
		int iEnd = iBegin + iGroupSize - 1;
		dsData.SetLowerBound(iBegin);				// set begin marker for dataset
		dsData.SetUpperBound(iEnd);					// set end marker for dataset

		Data_sum(&dsData, &stat);					// compute stats
		wks.SetCell(ii, 0, ii);						// plug values into wks
		wks.SetCell(ii, 1, stat.min);
		wks.SetCell(ii, 2, stat.max);
		wks.SetCell(ii, 3, stat.mean);
		wks.SetCell(ii, 4, stat.sd);
	}
	
	dsData.SetLowerBound(0);						// reset range to full
	dsData.SetUpperBound(iSize - 1);
}




Edited by - easwar on 06/25/2003 12:10:44 PM
Mike Buess Posted - 06/25/2003 : 11:13:19 AM
sum() can operate on a subset of a column...

set col(B) -b 5;
set col(B) -e 8;
sum(col(B)); // operate only on rows 5 through 8

Create 2 columns in the same worksheet and name them "mean" and "sd". To sum inc rows at a time...

get col(B) -e npt; // how many rows?
loop (i,1,npt/inc) {
j=inc*(i-1); k=j+inc; j++;
set col(B) -b j;
set col(B) -e k;
sum(col(B));
col(mean)[i]=sum.mean;
col(sd)[i]=sum.sd;
};
set col(B) -b 1; // reset display range to first row

Mike Buess
Origin WebRing Member
eno Posted - 06/25/2003 : 01:04:25 AM
Hi Mike,
Thank you for your advice.

Can you give me more information about Standard deviation/mean ?

If the length of column B is 24 hours(time series data), and I calculate Standard deviation/mean every ten-minute, I can get 24hours * 60minutes / 10minutes = 144 Standard deviation/mean values. That's what I want. How should I customize these scripts that you suggested.

Thanks,
eno
Mike Buess Posted - 06/24/2003 : 10:46:50 AM
Hi eno,

If your data are in column B of the active worksheet you can do the following.

1> Standard deviation/mean (use sum() function)...
sum(col(B));
// results show up in the following variables
sum.mean= mean value
sum.sd= standard deviation
sum.total= total value
sum.min= smallest value
sum.max= largest value
sum.n= number of values

2> Running average (use Average command). This command changes the dataset it operates on, so you might want to make a copy of it first...
win -c Ave; // create a column named Ave
copy col(B) col(Ave); // copy rows from col B to col Ave
Average -n 3 col(Ave); // replace col Ave with a running average of 7 values (i-3 to i+3)

Hope that helps.


Mike Buess
Origin WebRing Member

The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000