The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
 All Forums
 Origin Forum
 Origin Forum
 2023b: Optimization Solver failed

Note: You must be registered in order to post a reply.
To register, click here. Registration is FREE!

Screensize:
UserName:
Password:
Anti-Spam Code:
Format Mode:
Format: BoldItalicizedUnderlineStrikethrough Align LeftCenteredAlign Right Horizontal Rule Insert HyperlinkUpload FileInsert Image Insert CodeInsert QuoteInsert List
   
Message:

* HTML is OFF
* Forum Code is ON
Smilies
Smile [:)] Big Smile [:D] Cool [8D] Blush [:I]
Tongue [:P] Evil [):] Wink [;)] Clown [:o)]
Black Eye [B)] Eight Ball [8] Frown [:(] Shy [8)]
Shocked [:0] Angry [:(!] Dead [xx(] Sleepy [|)]
Kisses [:X] Approve [^] Disapprove [V] Question [?]

 
Check here to subscribe to this topic.
   

T O P I C    R E V I E W
AKazak Posted - 11/14/2023 : 07:21:18 AM
OriginPro 2023b (64-bit) SR1 10.0.5.157
Windows 10 Pro for Workstations 10.0.19045.3570

Greetings!

https://my.originlab.com/ftp/forum_and_kbase/Images/Optimization%20Solver%202023-11-14.opju

I am trying to optimize the slope and the offset parameters to perfectly align two peaks (Y1 and Y2) using Optimization Solver app, but getting the following error:



What am I doing wrong?

Thank you.


---
Andrey
7   L A T E S T    R E P L I E S    (Newest First)
AKazak Posted - 11/17/2023 : 12:34:20 AM
Dear Sam,

Thank you for sharing the snippet.
I will try adapting the code to my case.

---
Andrey
Sam Fang Posted - 11/16/2023 : 11:05:56 PM
Hi Andrey,

I converted your function to Python script. It creates a similar result.

from scipy.optimize import minimize
import numpy as np

import originpro as op

def fRSS(p):
    wks = op.find_sheet()
    x1 = np.array( wks.to_list(0) )
    y1 = np.array( wks.to_list(1) )
    x2 = np.array( wks.to_list(2) )
    y2 = np.array( wks.to_list(3) )
    x1b = p[0]*x1 + p[1]
    lb = max( min(x1b), min(x2) )
    ub = min( max(x1b), max(x2) )
    n = 1000
    xn = np.linspace(lb, ub, n)
    y1n = np.interp(xn, x1b, y1)
    y2n = np.interp(xn, x2, y2)
    return np.sum( (y1n -y2n)**2 )/n
    
def opt_min():
    wks = op.find_sheet()
    x0 = np.array( wks.to_list('F') )
    res = minimize(fRSS, x0, method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
    print(f'{res.fun:e}')
    return res.x


In column G, set:
col(G)=py.opt_min()


Initial values in column F, set slope=1, offset=22000, and the result is:
slope=1.10095, offset=20184.41058, RSS=2.16494E9


You can also try other methods in Python.

quote:
Originally posted by AKazak
Can you suggest the proper ways to perform the optimization without using app, that is as a formula or via Set Values dialog, please?
Perhaps using a Python library?...



Sam
OriginLab Technical Services
AKazak Posted - 11/16/2023 : 06:47:42 AM
quote:
Originally posted by Sam Fang

Hi Andrey,

Your columns K and L transformed columns C and D to fit columns A and B. However, your objective function in cell J1 used columns A, B to fit columns C, D.

So you should use slope=1, offset=22,000 instead of -22,000.

I tested results using the app:
------------------------
slope=1, offset=22,000 RSS=2.21181E9
slope=2.18497, offset=0.06992 RSS=3.95274E9


From the RSS result, slope=1, offset=22,000 is better.

Sam
OriginLab Technical Services



Got it.
I will take more care about the initial values.

Can you suggest the proper ways to perform the optimization without using app, that is as a formula or via Set Values dialog, please?
Perhaps using a Python library?...

quote:
Originally posted by Sam Fang
Dynamic Time Warping app can also align two signals, you can try it.
https://www.originlab.com/fileExchange/details.aspx?fid=439

Sam
OriginLab Technical Services



I have tried Dynamic Time Warping app with the current dataset and got
quote:
Dynamic Time Warping: Success!!

in the Log, but columns H, I and J (actual results) are still missing.

---
Andrey
Sam Fang Posted - 11/15/2023 : 10:28:57 PM
Hi Andrey,

Your columns K and L transformed columns C and D to fit columns A and B. However, your objective function in cell J1 used columns A, B to fit columns C, D.

So you should use slope=1, offset=22,000 instead of -22,000.

I tested results using the app:
------------------------
slope=1, offset=22,000 RSS=2.21181E9
slope=2.18497, offset=0.06992 RSS=3.95274E9


From the RSS result, slope=1, offset=22,000 is better.

I used initial values of slope=1, offset=22,000, and the third method, it gives a solution:
slope=1.00314, offset=21999.87106, and RSS=2.20177E9


So for your function, it seems that it is sensitive to initial values. If initial values are not too bad, it can always converge and find a local solution.

For Correlation Shift app, it was created by a customer.

Dynamic Time Warping app can also align two signals, you can try it.
https://www.originlab.com/fileExchange/details.aspx?fid=439

quote:
Originally posted by AKazak
The issue is still there.
The answer you've got:
slope=2.18497
offset=0.06992

Is far from the correct one since the real Offset is about -22,000 and the Slope is about 1.
Just plot AB versus KL and you will see.

Do you have any idea on improving convergence and accuracy of optimization?




Sam
OriginLab Technical Services
AKazak Posted - 11/15/2023 : 06:43:50 AM
It is worth stating, that https://www.originlab.com/fileExchange/details.aspx?fid=466 does fast and accurate Offset search, but does not target slope at all.

What functions does Correlation Shift function uses to align the signals?

---
Andrey
AKazak Posted - 11/15/2023 : 06:39:25 AM
quote:
Originally posted by Sam Fang

Hi Andrey,

It may be caused by your poor initial values.

I used initial values: slope=2, offset=0, and chose the third method. It can give a solution:
slope=2.18497
offset=0.06992

BTW in column F, you should set its nrows=2 (right click cell F2, and select Set as End). And you'd better set Recalculate to None. It can avoid refresh and the app can force it to recalculate.

quote:
Originally posted by AKazak

I am trying to optimize the slope and the offset parameters to perfectly align two peaks (Y1 and Y2) using Optimization Solver app, but getting the following error:




Sam
OriginLab Technical Services



Dear Sam,

The issue is still there.
The answer you've got:
slope=2.18497
offset=0.06992

Is far from the correct one since the real Offset is about -22,000 and the Slope is about 1.
Just plot AB versus KL and you will see.

Do you have any idea on improving convergence and accuracy of optimization?

---
Andrey
Sam Fang Posted - 11/15/2023 : 04:16:51 AM
Hi Andrey,

It may be caused by your poor initial values.

I used initial values: slope=2, offset=0, and chose the third method. It can give a solution:
slope=2.18497
offset=0.06992

BTW in column F, you should set its nrows=2 (right click cell F2, and select Set as End). And you'd better set Recalculate to None. It can avoid refresh and the app can force it to recalculate.

quote:
Originally posted by AKazak

I am trying to optimize the slope and the offset parameters to perfectly align two peaks (Y1 and Y2) using Optimization Solver app, but getting the following error:




Sam
OriginLab Technical Services

The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000