Since Origin can use Python, and in the upcoming Origin 2021, Python is much easier to use. We will show using Embedded Python in 2021 to process the data with t-SNE.
Here we take the sample data from MNIST, we load 20,000 28x28 images into a matrix book and the corresponding categories into a worksheet column, like this
If you are interested, we can show the code to make this OPJU, but we will focus on the tSNE. The following code assume the above OPJU with a matrix book name "MData" and a Worksheet named "Target"
import numpy as np
import pandas as pd
import originpro as op
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
mat = op.find_sheet('m', 'MData')
wtarget = op.find_sheet('w', 'Target')
#generate the X Y data similar to how you get it from fetch_openml('mnist_784') using the saved data in the OPJU
xdata= mat.to_np3d()
nsize, nc, nr = xdata.shape
xdata = xdata.reshape(nsize, nc*nc)
ydata = np.array(wtarget.to_list2(c2=0)).flatten()
# the following will perform PCA with 50 components first, and then perform tSNE using these PCA results
pca_model_n = PCA(n_components=50) # create PCA model, with 50 components
pca_result_n = pca_model_n.fit_transform(xdata) # fit the model, and transform the data
tsne_model_on_pca = TSNE(n_components=2, perplexity=40, n_iter=300, verbose=0) # create tSNE model, with 2 components, output info
tsne_result_on_pca = tsne_model_on_pca.fit_transform(pca_result_n) # fit the model, and transform the 50-components PCA result data
wks = op.new_sheet()
wks.name = 'Results'
wks.from_df(pd.DataFrame(tsne_result_on_pca, columns=['tsne-on-pca-subset-1', 'tsne-on-pca-subset-2']))
col = tsne_result_on_pca.shape[1]
wks.from_df(pd.DataFrame(ydata, columns=['y']), c1=col, head='L') # put the true y values
Running the code above took about 2 minutes for the 20k dataset. Once the result is in the worksheet, you can just make a colormap scatter plot as shown below:
James
tSNE plot
Posted - 09/10/2020 : 6:44:53 PM
