The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
 All Forums
 Origin Forum
 Origin Forum
 normalize data during import

Note: You must be registered in order to post a reply.
To register, click here. Registration is FREE!

Screensize:
UserName:
Password:
Anti-Spam Code:
Format Mode:
Format: BoldItalicizedUnderlineStrikethrough Align LeftCenteredAlign Right Horizontal Rule Insert HyperlinkUpload FileInsert Image Insert CodeInsert QuoteInsert List
   
Message:

* HTML is OFF
* Forum Code is ON
Smilies
Smile [:)] Big Smile [:D] Cool [8D] Blush [:I]
Tongue [:P] Evil [):] Wink [;)] Clown [:o)]
Black Eye [B)] Eight Ball [8] Frown [:(] Shy [8)]
Shocked [:0] Angry [:(!] Dead [xx(] Sleepy [|)]
Kisses [:X] Approve [^] Disapprove [V] Question [?]

 
Check here to subscribe to this topic.
   

T O P I C    R E V I E W
martin.ward Posted - 10/21/2022 : 2:30:44 PM
Using OriginPro 2021 (64bit) 9.8.0.200
Operating system: Windows 10 Pro, 64-bit

Hi all, I have a lot of data that requires importing and normalizing which is a pain to do manually. I havent ever used scripts to perform functions, but I know that a script can be applied during import. Does anyone know of a script that would allow me to do the normalization automatically?

I would be importing either 2 column (XY) or 3 column (XYE) data.

Also, on this topic, if there was such a thing as a good set of tutorials or guides for this sort of thing in Origin I would be glad to find out.

Thanks
6   L A T E S T    R E P L I E S    (Newest First)
easwar Posted - 10/23/2022 : 3:09:36 PM
Hi Martin,

The code I gave before was using wks.ncols in the loop statement, so if you start adding new columns inside the loop, that count keeps increasing which is why you saw the odd behavior you mentioned.

Change the code to the following:


// Get column count
int ncols = wks.ncols;

// Loop over all Y cols: col 2, 5, 8.....
for(int ic=2; ic<=ncols; ic+=3)
{
   // Use XF to normalize col and place result in a new column 
   rnormalize irng:=$(ic) method:=1 orng:=<new>;
}


Easwar
martin.ward Posted - 10/23/2022 : 07:00:27 AM
Hi Easwar, thank you very muchfor that - it does what I need it to do.

I did try to generalize the script to allow the normalized data to be written to new columns:

for(int ic=2; ic<=wks.ncols; ic+=3)
{
rnormalize irng:=$(ic) method:=1 orng:=<new>;
}

This does work, but I think it gets confused with column numbers (maybe?), as it writes copies of columns at the end. For example I have imported 13 files and it has normalized as expected and appended columns to the end of the sheet, but it has then also normalized the
normalized data in previous columns.

This is no big issue as I can manually delete the extra columns, but I am just a bit confused why this is happening?

quote:
Originally posted by easwar

Hello Martin,

Yes, you need a for loop to process all the Y columns.

I am assuming you already know how to set the controls in the multi ascii dialog to bring all files as adjacent columns in same sheet. Then in that dialog, expand the "Scripts" node and paste the following lines of code in the "Script after All Files Imported":


// Loop over all Y cols: col 2, 5, 8.....
for(int ic=2; ic<=wks.ncols; ic+=3)
{
   // Use XF to normalize col and place result in the input itself
   rnormalize irng:=$(ic) method:=1 orng:=$(ic);
}


Then you can click on the black triangle button way at the top right of the dialog and save the settings as a theme. The script will be saved with the theme, so later when you have similar files, you can load the theme from that button and click OK to do the same thing again.

In the above code, I used the rnormalize x-function, as you need to normalize each Y column separately?
https://www.originlab.com/doc/en/X-Function/ref/rnormalize

If your data is XY instead of XYE, then you need to change the for loop to count the Y column positions correctly etc.

Hope this helps.

Easwar
OriginLab

easwar Posted - 10/22/2022 : 7:04:16 PM
Hello Martin,

Yes, you need a for loop to process all the Y columns.

I am assuming you already know how to set the controls in the multi ascii dialog to bring all files as adjacent columns in same sheet. Then in that dialog, expand the "Scripts" node and paste the following lines of code in the "Script after All Files Imported":


// Loop over all Y cols: col 2, 5, 8.....
for(int ic=2; ic<=wks.ncols; ic+=3)
{
   // Use XF to normalize col and place result in the input itself
   rnormalize irng:=$(ic) method:=1 orng:=$(ic);
}


Then you can click on the black triangle button way at the top right of the dialog and save the settings as a theme. The script will be saved with the theme, so later when you have similar files, you can load the theme from that button and click OK to do the same thing again.

In the above code, I used the rnormalize x-function, as you need to normalize each Y column separately?
https://www.originlab.com/doc/en/X-Function/ref/rnormalize

If your data is XY instead of XYE, then you need to change the for loop to count the Y column positions correctly etc.

Hope this helps.

Easwar
OriginLab
martin.ward Posted - 10/22/2022 : 11:27:39 AM
just another quick note on this;

Ive tried using functions like the suggested "normalize iy:=Col(2);", this works if you import each file in to its own sheet/book, but it doesnt work if you import all in to the same sheet i.e. import in to a new column (which is what I wish to do - again I should have mentioned this in my initial post). If you try this you are given a 'normalized Y column for each imported file, but the values are all identical for each normalized column.

Do I need to use a For loop or something else to iterate through each independent 'Y' column?
martin.ward Posted - 10/22/2022 : 08:07:28 AM
Hi Snow,

Thanks for your reply. Sorry I should have given more detail. Yes, I would be using import from multiple file: multiple ASCII and I would typically be importing 30 XYE files each time and I would like to normalize each Y column at the time of import


Thanks
snowli Posted - 10/21/2022 : 4:55:32 PM
Here are the FAQs on this
https://www.originlab.com/doc/Quick-Help/Normalize-Data
https://www.originlab.com/doc/Quick-Help/run-script-after-import

Could you let me know what import tool are you using?
E.g. suppose you are using Data: Import from File: Multiple ASCII... (corresponds to second case in above FAQ)

After selecting files and click OK,
impASC dialog will open.
There is a Scripts node.

U can enter e.g.
normalize iy:=Col(2) suppose you want to normalize the 2nd column.


U can refer this page to see how to set normalize
https://www.originlab.com/doc/en/X-Function/ref/normalize


Some videos that may help
https://www.originlab.com/videos/details.aspx?id=330
https://www.originlab.com/videos/details.aspx?id=52

Thanks, Snow

The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000