T O P I C R E V I E W |
AKazak |
Posted - 07/04/2021 : 02:38:58 AM OriginPro 2021b (64-bit) SR2 9.8.5.212 Windows 10 21H1 x64
Greetings!
I have a data matrix with missing-value holed inside and want to interpolate them like I would do in XYZ-domain using Interpolate Z from XY command. However I found that 2D Interpolate/Extrapolate command from the matrix domain does not fill the missing values with interpolated values.
Please suggest the proper way of filling the missing values. Here is the data I work with: https://my.originlab.com/ftp/forum_and_kbase/Images/Zone%2001%20-%202D%20Depth%20Area.opju
Thank you.
--- Andrey |
10 L A T E S T R E P L I E S (Newest First) |
AKazak |
Posted - 07/07/2021 : 10:46:30 PM quote: Originally posted by easwar
Hi Andrey,
Our current 2D interpolation tool uses a function from the NAG library shipped with Origin, which lacks support for missing values.
We can look into improving the tool, but for now you could use Python code to do the interpolation.
Here is an example where I started with a matrix with a couple of blocks of missing values (it is one of our graph samples data from the F11 Learning Center dialog). The second matrix is created by the code, with the results of the interpolation. Then I just made the graphs so you can see the results:

Here is the Python code that I found by searching on the web and then added the parts for the Origin matrices etc. I found the code from this page: https://stackoverflow.com/questions/37662180/interpolate-missing-values-2d-python
import originpro as op
import numpy as np
from scipy import interpolate
msheet_in = op.find_sheet('m')
array = msheet_in.to_np2d(0)
x = np.arange(0, array.shape[1])
y = np.arange(0, array.shape[0])
#mask invalid values
array = np.ma.masked_invalid(array)
xx, yy = np.meshgrid(x, y)
#get only the valid values
x1 = xx[~array.mask]
y1 = yy[~array.mask]
newarr = array[~array.mask]
GD1 = interpolate.griddata((x1, y1), newarr.ravel(),
(xx, yy),
method='cubic')
mbook_out = op.new_book('m')
msheet_out = op.find_sheet('m')
msheet_out.from_np(GD1)
You can try with your data and then refer to this page for further information on options for the scipy function: https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.griddata.html
Easwar OriginLab
Dear Easwar,
Thank you for sharing this. It seems to be the best way for me to follow.
--- Andrey |
easwar |
Posted - 07/07/2021 : 4:37:38 PM Hi Andrey,
Our current 2D interpolation tool uses a function from the NAG library shipped with Origin, which lacks support for missing values.
We can look into improving the tool, but for now you could use Python code to do the interpolation.
Here is an example where I started with a matrix with a couple of blocks of missing values (it is one of our graph samples data from the F11 Learning Center dialog). The second matrix is created by the code, with the results of the interpolation. Then I just made the graphs so you can see the results:

Here is the Python code that I found by searching on the web and then added the parts for the Origin matrices etc. I found the code from this page: https://stackoverflow.com/questions/37662180/interpolate-missing-values-2d-python
import originpro as op
import numpy as np
from scipy import interpolate
msheet_in = op.find_sheet('m')
array = msheet_in.to_np2d(0)
x = np.arange(0, array.shape[1])
y = np.arange(0, array.shape[0])
#mask invalid values
array = np.ma.masked_invalid(array)
xx, yy = np.meshgrid(x, y)
#get only the valid values
x1 = xx[~array.mask]
y1 = yy[~array.mask]
newarr = array[~array.mask]
GD1 = interpolate.griddata((x1, y1), newarr.ravel(),
(xx, yy),
method='cubic')
mbook_out = op.new_book('m')
msheet_out = op.find_sheet('m')
msheet_out.from_np(GD1)
You can try with your data and then refer to this page for further information on options for the scipy function: https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.griddata.html
Easwar OriginLab |
AKazak |
Posted - 07/07/2021 : 09:24:45 AM quote: Originally posted by AmandaLu
Hi,
I checked with developer. Spline in 2D Interpolate/Extrapolate cannot support missing values. So it simply replace missing with 0. As a workaround, you can use Spline in Interpolate Z from XY. That is,
1. Convert the matrix into XYZ (Matrix: Convert to Worksheet) with missing value.
2. Convert the matrix into XYZ (Matrix: Convert to Worksheet) without missing value. You can select “Trim Missing Values” in the dialog. Now you get 2 sets, X1Y1Z1 has full XY values and Z with missing; X2Y2Z2 has no missing.
3. Select Analysis: Mathematics: Interpolate Z from XY. XY Values to Interpolate = X1Y1 Input = X2Y2Z2 Method = Spline Smoothing Factor = a very large number such as 10000 Now you get Z3
4. Convert X1Y1Z3 back into matrix (Worksheet: Convert to Matrix: XYZ Gridding).
Thanks, Amanda OriginLab Technical Service
Thank you for suggesting the workaround. I will test it and hopefully, it will solve my issue.
Can you ask the developer to implement filling the missing values with inteprolation results, please?
--- Andrey |
AmandaLu |
Posted - 07/07/2021 : 01:57:46 AM Hi,
I checked with developer. Spline in 2D Interpolate/Extrapolate cannot support missing values. So it simply replace missing with 0. As a workaround, you can use Spline in Interpolate Z from XY. That is,
1. Convert the matrix into XYZ (Matrix: Convert to Worksheet) with missing value.
2. Convert the matrix into XYZ (Matrix: Convert to Worksheet) without missing value. You can select “Trim Missing Values” in the dialog. Now you get 2 sets, X1Y1Z1 has full XY values and Z with missing; X2Y2Z2 has no missing.
3. Select Analysis: Mathematics: Interpolate Z from XY. XY Values to Interpolate = X1Y1 Input = X2Y2Z2 Method = Spline Smoothing Factor = a very large number such as 10000 Now you get Z3
4. Convert X1Y1Z3 back into matrix (Worksheet: Convert to Matrix: XYZ Gridding).
Thanks, Amanda OriginLab Technical Service
|
AKazak |
Posted - 07/05/2021 : 10:49:20 PM quote: Originally posted by AmandaLu
Hi,
If you want to trim the missing values, I suggest that you can convert the matrix into XYZ (Matrix: Convert to Worksheet). You can select “Trim Missing Values” in the dialog. And then convert the worksheet back into matrix (Worksheet: Convert to Matrix: XYZ Gridding). Select griding method like “Radom”, you can get a matrix without missing values.
Thanks, Amanda OriginLab Technical Service
OK, I see. The suggested way is not trivial and laborious.
I would be happy to use 2D Interpolate/Extrapolate command for the matrix. Do you have an idea: why does it fill missing values with zeroes?
--- Andrey |
AmandaLu |
Posted - 07/05/2021 : 10:19:02 PM Hi,
If you want to trim the missing values, I suggest that you can convert the matrix into XYZ (Matrix: Convert to Worksheet). You can select “Trim Missing Values” in the dialog. And then convert the worksheet back into matrix (Worksheet: Convert to Matrix: XYZ Gridding). Select griding method like “Radom”, you can get a matrix without missing values.
Thanks, Amanda OriginLab Technical Service
|
AKazak |
Posted - 07/05/2021 : 05:35:12 AM quote: Originally posted by AmandaLu
Hi,
I tried 2D Interpolate/Extrapolate with the default settings, that is, Method = Spline, and the holes are filled with interpolated values. Could you please tell me which row&column you still see missing after interpolation?
Thanks, Amanda OriginLab Technical Service
Dear Amanda,
Yes, 2D Interpolate/Extrapolate with the default settings fills the holes (and even extrapolates outside), but the filled holes have zeros, instead of interpolated values.
Can you share your results, please?
--- Andrey |
AmandaLu |
Posted - 07/05/2021 : 04:57:52 AM Hi,
I tried 2D Interpolate/Extrapolate with the default settings, that is, Method = Spline, and the holes are filled with interpolated values. Could you please tell me which row&column you still see missing after interpolation?
Thanks, Amanda OriginLab Technical Service
|
AKazak |
Posted - 07/05/2021 : 04:35:59 AM quote: Originally posted by AmandaLu
Hi,
Do you want to interpolate the whole matrix or just the missing values surrounding by Z values? Matrix cannot support extrapolation well so the missing values at the left edge cannot be interpolated.
Thanks, Amanda OriginLab Technical Service
I am OK with the fact that matrix cannot support extrapolation well so the missing values at the left edge cannot be interpolated. I want to interpolate just the "holes" of missing Z values based on surrounding Z values. How do I achieve this?
--- Andrey |
AmandaLu |
Posted - 07/05/2021 : 04:10:06 AM Hi,
Do you want to interpolate the whole matrix or just the missing values surrounding by Z values? Matrix cannot support extrapolation well so the missing values at the left edge cannot be interpolated.
Thanks, Amanda OriginLab Technical Service
|
|
|