| Author |
 Topic  |
|
|
jonni
United Kingdom
58 Posts |
 Posted - 06/02/2003 : 3:19:58 PM
|
Hi,
I would like to create a matrix out of the following data set: a value Z is measured with resect to a variable X; a parameter in the experiment Y is changed between each XZ data set. It is convenient to convert my sets into one worksheet with the variable X in column A (leftmost column) in rows i=2...N, Z values in subsequent columns in rows i=2...N, and my Y parameter for each Z column in row i=1. (Cell data1_a[1]=0 or blank in this arrangement.)
Normally I would delete the X column and Y row and make a direct conversion of the Z data to a matrix, defining its dimensions accordingly. However, the Y set in this case is NOT regularly spaced and hence this approach does not work. The "Random XYZ" also does not work in this instance where only one Z column is required.
The question is therefore what to do in this case. Surely I should be able to create a matrix with missing elements, where the "missing" Y values should be -- interpolation of the missing values may not be physically correct in this instance, so it would be preferable not to do this (if it is the easier option!).
Any help would be appreciated.
Thanks,
jonni |
|
|
easwar
USA
1965 Posts |
Posted - 06/02/2003 : 4:21:42 PM
|
Hi Jonni,
First you need to convert your data format to a simple worksheet with three columns containing the X, Y and Z data. This can be easily done using Origin C or script - post here if you need help with that.
Once your data is in this three-column format, you can then use Edit->Convert to Matrix-> Sparse Conversion to create a matrix that has only the measure values in cells and missing values in other cells. You will need to correctly specify the begin and end values as well as the step value for your x and y.
Another solution is to write an OC function or script that will directly ready the worksheet in your existing format, and then stuff the values into appropriate cells of a matrix, where you first set the matrix up with the correct size/dimensions etc.
Easwar
OriginLab.
|
 |
|
|
edgar.kaiser
Switzerland
29 Posts |
Posted - 06/03/2003 : 03:24:29 AM
|
Hi Jonni,
as easwar already pointed out, you will need to arrange your data into three columns, x y z, before you can convert it into a matrix.
Instead of interpolating the missing matrix elements you might consider a least squares fit using an appropriate model function. I did successful gridding in a project that appears similar to yours using the 2D-B-spline approximation that's available in the NAG library in Origin C. In my project a series of stress(z)-strain(x) curves was acquired on a material testing machine and temperature(y) was varied stepwise.
See: http://www.originlab.com/forum/topic.asp?TOPIC_ID=2197
When you deal with real world data, i.e. measured data, interpolation is very frequently not the best method, because there is a strong tendency towards artifacts, particularly if your data is noisy.
Regards,
Edgar |
|
|
|
jonni
United Kingdom
58 Posts |
Posted - 06/03/2003 : 05:19:56 AM
|
Hi Easwar and Edgar,
Thanks for your interest.
Easwar, please can you help me with the basic script I will need to use -- I am more comfortable using script rather than OriginC unless it will be better/faster to use C for this task.
In your reply you say that I will need to specify a step in Y; I presume this means that I will need to take my irregular Y values and 'map' them on to a regular sequence. If I am to ensure that the exact Y values remain after mapping my follow-up question would then be: is there any way to do this with script, i.e. take a linear array of values (Y) and create a new regular set to include a specific number of elements, both 'phantom' values and the original?
A subsidiary question, to Easwar and Edgar: if this 'phantom' data is necessary to create in order to generate the matrix is there a way to highlight/distinguish it from the real data when it comes to be plotted (it does not appear that you can mask matrix columns/rows as you can in a worksheet)?
Thanks again.
jonni |
|
|
|
edgar.kaiser
Switzerland
29 Posts |
Posted - 06/03/2003 : 06:52:07 AM
|
Jonni,
you don't need to specify a step in one of the independent variables. You just need to arrange your datapoints in x, y and z columns. The gridding process will then map your original xyz data into the grid of the matrix.
It is easy to distinguish original data and mapped data if you do the gridding by an interpolation procedure. The original data that fits exactly into the matrix grid will remain unchanged. You could scan your matrix and generate a second matrix that only contains the original or the mapped data.
If you perform a least squares fit procedure all original data will be replaced by the function value of the fit model at the respective matrix element.
To my knowledge Origin has currently no built in random gridding procedure using a least squares fit, there are only interpolation procedures.
You will need to use the NAG functions to perform a least squares fit.
Anyway Origin C is way faster than Labtalk. This shows up if you do extensive matrix calculations.
Regards,
Edgar
|
|
|
|
easwar
USA
1965 Posts |
Posted - 06/03/2003 : 10:54:39 AM
|
Hi Jonni,
In order to convert your XYYY.. data to XYZ data, download this add-on tool:
http://www.originlab.com/FileExchange/details.aspx?C=5&cid=6&fid=6
This tool, among other things, will do what you want for data conversion. The tool is script based, and so you could just take the script under the appropriate button if you just want that.
Now, once you have your XYYY.. data converted to XYZ, you need to decide whether you should use (a) regular conversion (b) random onversion, or (c) sparse conversion, which are all options under the Edit->Convert to Matrix menu.
(a) regular conversion - this assumes that your X,Y data are perfectly ordered, and none are missing - in other words, there is a row in your XYZ data corresponding to each cell of the destination matrix. This method then sets the matrix dimension and coordinates based on your data.
(b) random conversion - the X,Y data are irregularly spaced, and/or some values are missing as well. In this case, some interpolation of the data is done - the tool offers 5 methods, 3 of which are based on NAG - your resulting matrix will then have interpolated values and not the exact values you started with
(c) sparse conversion - this assumes that your X,Y are regularly spaced, but just some entries are missing. In this case, Origin guesses the start, end, and step value for X and Y based on your data, and then creates a matrix based on these guesses, and fills in the exact Z values in the appropriate cells. What I meant by "you need to specify the begin, end and step size" is that, if many values are missing, our guess for begin, end and step may be wrong and so you need to put in your "known" values in the dialog that comes up.
If you have an algorithm to convert the XYZ data into a matrix using a conversion method that is currently not supported, you can easily code that in OC (I would suggest OC for speed).
Hope this helps.
Easwar
OriginLab.
|
|
|
|
easwar
USA
1965 Posts |
Posted - 06/03/2003 : 10:59:29 AM
|
[quote]
Hi Jonni,
In order to convert your XYYY.. data to XYZ data, download this add-on tool:
http://www.originlab.com/FileExchange/details.aspx?C=5&cid=6&fid=6
This tool, among other things, will do what you want for data conversion. The tool is script based, and so you could just take the script under the appropriate button if you just want that.
Now, once you have your XYYY.. data converted to XYZ, you need to decide whether you should use (a) regular conversion (b) random conversion, or (c) sparse conversion, which are all options under the Edit->Convert to Matrix menu.
(a) regular conversion - this assumes that your X,Y data are perfectly ordered, and none are missing - in other words, there is a row in your XYZ data corresponding to each cell of the destination matrix. This method then sets the matrix dimension and coordinates based on your data.
(b) random conversion - the X,Y data are irregularly spaced, and/or some values are missing as well. In this case, interpolation of the data is done - the tool offers 5 methods, 3 of which are based on NAG - your resulting matrix will then have interpolated values and not the exact values you started with. For details on the methods, please refer to documentation.
(c) sparse conversion - this assumes that your X,Y are regularly spaced, BUT some entries are missing. In this case, Origin guesses the start, end, and step value for X and Y based on your data, and then creates a matrix based on these guesses, and fills in the exact Z values in the appropriate cells. What I meant by "you need to specify the begin, end and step size" is that, if many values are missing, our guess for begin, end and step may be wrong and so you need to put in your "known" values in the dialog that comes up. Also, note that this method will reject X,Y entries that don't exactly fall at a step between the begin and end - so if your X,Y has some "noise", then points may be rejected - you could remove the noise before using this method.
If none of the above satisify what you want to do, you could code the conversion yourself if you have your own algorithm. I would suggest OC, rather than LabTalk, for speed.
Hope this helps.
Easwar
OriginLab.
|
|
|
|
jonni
United Kingdom
58 Posts |
Posted - 06/06/2003 : 05:31:42 AM
|
Hi Edgar and Easwar,
Thank you both for your help. I have succesfully used the add-on tool to convert my data to XYZ format, and for my purposes the sparse conversion to the matrix does what I want.
Thanks again,
jonni |
|
|
| |
Topic |
|
|
|
Creating matrix with irregular data