The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
Username:
Password:
Save Password
Forgot your Password? | Admin Options

 All Forums
 Origin Forum for Programming
 LabTalk Forum
 Extracting data according to variable conditions
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic

Leif12

Germany
9 Posts

Posted - 01/14/2022 :  10:51:16 AM  Show Profile  Edit Topic  Reply with Quote  View user's IP address  Delete Topic
Hello everybody,
I'm completely new to the Origin Lab user forum, but I'm stuck with a problem and I'm pretty sure one of you can help me.

I hava a large dataset1 with (50000 datapoints x y) were some data need to be extracted according to differnt x values. The x values for the exctraction are in another dataset2 (400 values), but not all of them needs to be used and I need the possibility to change them afterwards to see the results. I might know a logical way to achieve this, but I have no clue how to achieve this.

1st. Define constants from the dataset2 following a loop with different conditions (e.g. define every 5th datapoint as constant with name constI(ii), define every 10th datapoint as constII(ii) ...)

2nd. Extract datapoints from dataset2 y if x value is between const1(ii)+c and const2(ii)-d

Does anyone knows an elegant way to achieve this?

Cheers
Leif

Origin Ver. and Service Release (Select Help-->About Origin): OriginPro 2020 (64-bit) SR1
Operating System: Windows 10

Edited by - Leif12 on 01/14/2022 10:52:54 AM

YimingChen

992 Posts

Posted - 01/14/2022 :  2:50:29 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Can you attach a sample project file here to be more specific on your question?
1. In your 1st step, do you generate two columns constI and constII?
2. 2nd step is more confusing, for each constI and constII values (ii is the index?), you want to extract data from dataset1 within a range?

Thank you
James
Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/16/2022 :  02:20:43 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Thank you for your fast answer. Attached is an example origin file.
I hope you questions are answered by viewing it.

dataset1 are the raw data
dataset 2 are the treshold values

I would like to use the time treshold values of dataset2
(start and end times) to extract data x and y in a new
column if the time (col(A)) is in between the limits E and F

The colored cells marked for example three conditions
I'd llike to investigate:
1st: index +3 starting with 2nd value until 8th value
2st: index +10 starting with 15th value until 35th value
3rd: index +10 starting with 16th value until 36th value

And since I have a lot of data (50000) I'd like to do this automatically
and I may need to change the conditions afterwards.

Best
Leif

https://my.originlab.com/ftp/forum_and_kbase/Images/Origin%20example.opju
Go to Top of Page

YimingChen

992 Posts

Posted - 01/17/2022 :  10:19:03 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
You will need to write script to achieve this. Starting from Origin 2021, we allow using Python function to set column values, see the figure below of the sample code of your project. I also attached the project file that you can check. Thanks.

James
import originpro as op

def filter(val, start, end, step):
    '''F:Fiii'''
    wks = op.find_sheet()
    time = wks.to_list(0)
    time_start = wks.to_list(4)
    time_end = wks.to_list(5)
    result = []
    idx = 0
    idx_thres = start - 1
    while idx < len(time) and idx_thres <= end - 1:
        if time[idx] >= time_start[idx_thres] and time[idx] <= time_end[idx_thres]:
            result.append(val[idx])
        elif time[idx] > time_start[idx_thres]:
            idx_thres += step
            continue
        idx += 1
    return result




https://my.originlab.com/ftp/forum_and_kbase/Images/Origin%20example_YM.opju

James
Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/19/2022 :  12:48:45 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Thanks James for your nice explanation and your example file. This was basically the functino I was looking for.

Unfortunately, I haven't figured out, how to run the script since there are always 2 errors, when I run a script for a column e.g. H in our origin file (translated into English by myself):
1st Error: Math can not be applied to text columns:op.find_sheet()
time = wks.to_list(0)
t
2nd Error: Error while creating the operation for Book1_H due to error in script "set column values"

I only have OrignPro 2018b on this Laptop since I'm on beamtime. Does this might be the issue with the script (setting column values with python)?

Thanks and best greetings
Leif
Go to Top of Page

YimingChen

992 Posts

Posted - 01/19/2022 :  1:31:38 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Right, you need to upgrade to Origin 2021 to have the new Python feature. Origin 2018b is old that we don't support any more.

James

Edited by - YimingChen on 01/19/2022 1:32:01 PM
Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/20/2022 :  10:22:51 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Ok, I upgraded to OriginPro 2022, but I still have some issues.
E.g. if I set the function in col(H) to

Col(H) =
py.filter(B; 2; 11; 3)

Python-function:
import originpro as op

def filter(val. start. end. step):
'''F:Fiii'''
wks = op.find_sheet()
time = wks.to_list(0)
time_start = wks.to_list(4)
time_end = wks.to_list(5)
result = []
idx = 0
idx_thres = start - 1
while idx < len(time) and idx_thres <= end - 1:
if time[idx] >= time_start[idx_thres] and time[idx] <= time_end[idx_thres]:
result.append(val[idx])
elif time[idx] > time_start[idx_thres]:
idx_thres += step
continue
idx += 1
return result


I get the error
ERR0015104: Python file cannot be loaded
and
error while creating the operation for book1_I due to error in script"set column values"

And the Labtalk-windows says:

File "C:\Users\life\AppData\Local\OriginLab\99\TMP\pyTmp\_SCV.py", line 3
def filter(val. start. end. step):
^
SyntaxError: invalid syntax
File "C:\Users\life\AppData\Local\OriginLab\99\TMP\pyTmp\_SCV.py", line 4
def filter(val. start. end. step):
^
SyntaxError: invalid syntax

Best
Leif
Go to Top of Page

YimingChen

992 Posts

Posted - 01/20/2022 :  12:13:20 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
The separator of the Python function arguments should be comma. Can you attach a screenshot showing the dialog like below? Thanks.


James
Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/21/2022 :  05:11:08 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Ok, I found the solution. The issue was that I needed to have a semicolon ";" in the function dialog instead of a comma ","
i.e. Col(H)= py.filter(C; 2; 11; 3)
instead of Col(H)= py.filter(C, 2, 20, 3).

Addtionally, I cannot just change the conditions by simply changing them in the row f(x) in the "origin view" since this changes the commas "," in the python script to dots "."
i.e. "def filter(val, start, end, step):" was changed to "def filter(val. start. end. step):"

I think these issues result from the use of Origin in combination with Python (or maybe the use of Origin in German language?)

But I can work with it.

However, I still have something I need to figure out:
The results, which fit to the conditions are listed one below the other. But I need them to be in the same row as they were in original dataset2. I tried to insert NaN values, but up to now it is not working. I'll continue to figure it out, but if you have an easy solution I would be happy.

Best
Leif
Go to Top of Page

YimingChen

992 Posts

Posted - 01/21/2022 :  09:16:36 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Can you try the code below?

import originpro as op
import numpy as np

def filter(val, start, end, step):
    '''F:Fiii'''
    wks = op.find_sheet()
    time = wks.to_list(0)
    time_start = wks.to_list(4)
    time_end = wks.to_list(5)
    result = []
    idx = 0
    idx_thres = start - 1
    while idx < len(time) and idx_thres <= end - 1:
        if time[idx] >= time_start[idx_thres] and time[idx] <= time_end[idx_thres]:
            result.append(val[idx])
        elif time[idx] > time_start[idx_thres]:
            idx_thres += step
            continue
        else:
            result.append(np.nan)
        idx += 1
    return result
Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/21/2022 :  11:18:59 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
I tried the code after installing the numpy package (Connectivity: Python Packages: install: numpy (v 1.22.1).

But I still get this error code:

Traceback (most recent call last):
File "C:\Users\life\AppData\Local\OriginLab\99\TMP\pyTmp\_SCV.py", line 1, in <module>
import originpro as op
File "C:\ProgramData\OriginLab\99\PyPackage\Py3\originpro\__init__.py", line 9, in <module>
from .config import *
File "C:\ProgramData\OriginLab\99\PyPackage\Py3\originpro\config.py", line 52, in <module>
np.float64: po.DF_DOUBLE,
AttributeError: module 'numpy' has no attribute 'float64'
Traceback (most recent call last):
File "C:\Users\life\AppData\Local\OriginLab\99\TMP\pyTmp\_SCV.py", line 2, in <module>
import originpro as op
File "C:\ProgramData\OriginLab\99\PyPackage\Py3\originpro\__init__.py", line 9, in <module>
from .config import *
File "C:\ProgramData\OriginLab\99\PyPackage\Py3\originpro\config.py", line 52, in <module>
np.float64: po.DF_DOUBLE,
AttributeError: module 'numpy' has no attribute 'float64'

And since I run your code and installed the package numpy I get the same error even if I only run the old script.

Do you know the reason? Thanks for your help.
Cheers
Leif

Edited by - Leif12 on 01/21/2022 11:32:08 AM
Go to Top of Page

YimingChen

992 Posts

Posted - 01/21/2022 :  1:33:21 PM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Have you tried restart Origin? If still doesn't work. You can uninstall the Numpy package and reinstall again.

Go to Top of Page

Leif12

Germany
9 Posts

Posted - 01/24/2022 :  05:24:10 AM  Show Profile  Edit Reply  Reply with Quote  View user's IP address  Delete Reply
Thanks James for your fast and awesome help.
After restarting Origin it worked.
Best
Leif
Go to Top of Page
  Previous Topic Topic Next Topic Lock Topic Edit Topic Delete Topic New Topic Reply to Topic
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000