| Author |
 Topic  |
|
|
Adeptcloth
3 Posts |
 Posted - 07/21/2015 : 5:15:18 PM
|
I have a simple script that creates a time evolution 3d surface plot video:
it takes a 2 dimensional (30x20) array, creates a surface plot with it, makes the necessary changes to the graph, then appends it to a video file created before the loop, and finally deletes the graph and any loose data sets that may remain.
I have to make this iterate for over 1760 graphs, and it becomes extremely slow after about 1100 and starts hogging about 1GB of Memory by then.
I looked well into this and can't find what I'm neglecting in the cleanup.
It seems that there is something being created in the backend of the program every iteration (it nets about a 3MB increase in mem consumption after each graph, even though I delete the graph and the associated virtual matrix).
[main]
//CREATES VID OBJECT
int err = vw.Create(C:\Users\cirl16\Desktop\Denis\15062932\RICBT15062932.avi, codec, 10, 1024, 768);
if(0 == err){
// FOR EVERY GRAPH
loop (i,0,1760){
// activate the window called RICBT
window -a RICBT;
// select the next batch of data
worksheet -s 5 1+i*30 24 30+i*30;
// plot the data
plotvm rowpos:=none colpos:=none xtitle:=DISK ytitle:=DEPTH ztitle:="CURRENT (A)" vmname:=VM1 type:=103 ogl:=<new template:=GLCMAP>;
// pull evolution time to be used as a label
%W = RICBT!Cell(1+30*i,4)$;
// consistent formatting across each frame
themeApply2g theme:="RICBT";
// places label
label -p 90 90 -s %W s;
// Appends the graph to the video as the next frame
vw.WriteGraph(Graph1,1);
// closes the graph and it's relateed dataset (virtual matrix I assume)
window -cd Graph1;
// makes sure to delete the virtual matrix
delete VM1;
// more cleanup, can't tell if they actually do anything
del -a;
del -as;
}
}
//COMPLETES FORMATTING OF VIDEO
vw.Release();
I've noticed that Origin tends to keep increasing in memory consumption no matter what I did within it, I would graph things by hand and remove them and there would still be some residual memory being used.
What can I do to prevent the sluggish behavior caused by the growing amount of memory consumption? Perhaps I might have missed something? Also everyone is welcome to suggest optimizations to the script :D (I've only been working with this for about a week now, so this may seem rather crude).
Thanks |
|
|
jasonzhao
China
262 Posts |
Posted - 07/21/2015 : 11:08:13 PM
|
Hello,
In order to optimise the script, we do not need to delete the graph in every iteration, we can plot the new VM in the same graphlayer, and delete the old VM. which means that we need to construct a 3D graph before the code running, and replace the VM in 3D graph during code running.
Please refer to code below:
loop (i,0,5){
window -a RICBT;
worksheet -s 5 1+i*30 24 30+i*30;
plotvm rowpos:=none colpos:=none xtitle:=DISK ytitle:=DEPTH ztitle:="CURRENT (A)" vmname:=VM2 type:=103 ogl:=[Graph1]1!;
sec -p 1;
delete VM2;
}
Best regards!
Jason
OriginLab Technical Service |
Edited by - jasonzhao on 07/21/2015 11:31:09 PM |
 |
|
|
Adeptcloth
3 Posts |
Posted - 07/22/2015 : 10:52:50 AM
|
quote:
Originally posted by jasonzhao
Hello,
In order to optimise the script, we do not need to delete the graph in every iteration, we can plot the new VM in the same graphlayer, and delete the old VM. which means that we need to construct a 3D graph before the code running, and replace the VM in 3D graph during code running.
Please refer to code below:
loop (i,0,5){
window -a RICBT;
worksheet -s 5 1+i*30 24 30+i*30;
plotvm rowpos:=none colpos:=none xtitle:=DISK ytitle:=DEPTH ztitle:="CURRENT (A)" vmname:=VM2 type:=103 ogl:=[Graph1]1!;
sec -p 1;
delete VM2;
}
Best regards!
Jason
OriginLab Technical Service
Thank you for your input! I implemented it and have observed a HUGE improvement in the efficiency of the code. I still however can see an increase of memory consumption per iteration.
I've made some adjustments to keep my formatting consistent, and tried it on a different dataset (same size).
int err = vw.Create(C:\Users\cirl16\Desktop\Denis\15062932\TEMP15062932.avi, codec, 10, 1024, 768);
if(0 == err){
window -a TEMP;
worksheet -s 5 1 24 30;
plotvm rowpos:=none colpos:=none xtitle:=DISK ytitle:=DEPTH ztitle:="Temperature (K)" vmname:=VM1 type:=103 ogl:=<new template:=GLCMAP>;
themeApply2g theme:="temperature1";
%W = TEMP!Cell(1,4)$;
label -p 90 90 -s -n time %W s;
vw.WriteGraph(Graph1,1);
loop (i,1,1760){
label -r -s %W s;
window -a TEMP;
worksheet -s 5 1+i*30 24 30+i*30;
plotvm rowpos:=none colpos:=none xtitle:=DISK ytitle:=DEPTH ztitle:="Temperature (K)" vmname:=VM2 type:=103 ogl:=[Graph1]1!;
sec -pw Graph1;
win -a Graph1;
themeApply2g theme:="temperature1";
%W = TEMP!Cell(1+30*i,4)$;
label -p 90 90 -s -n time %W s;
vw.WriteGraph(Graph1,1);
delete VM2;
}
}
vw.Release();
It makes sense why the prior code was causing such a rapid increase in memory usage, but I currently can't see why the issue has not been fully remedied.
Any idea as to why the memory is still growing over time?
Thanks again~ |
|
|
|
jasonzhao
China
262 Posts |
Posted - 07/22/2015 : 10:56:20 PM
|
Hello,
To make further improve, we do not need to 'win -a Graph1' at each iteration, because we have already designate vw.WriteGraph(Graph1,1); remove 'themeApply2g theme:="temperature1";'in the loop is also recommended.
I run your code in a worksheet Row=300, iteration=80, however, the total increasing memory is less than 20M, maybe there are some problem still unknown in your OPJ file?
Best regards!
Jason
OriginLab Technical Service |
Edited by - jasonzhao on 07/22/2015 10:58:04 PM |
|
|
|
Adeptcloth
3 Posts |
Posted - 07/23/2015 : 10:39:28 AM
|
quote:
Originally posted by jasonzhao
Hello,
To make further improve, we do not need to 'win -a Graph1' at each iteration, because we have already designate vw.WriteGraph(Graph1,1); remove 'themeApply2g theme:="temperature1";'in the loop is also recommended.
I run your code in a worksheet Row=300, iteration=80, however, the total increasing memory is less than 20M, maybe there are some problem still unknown in your OPJ file?
Best regards!
Jason
OriginLab Technical Service
Those two were necessary because the time evolution label would not change frame to frame (it would actually create a label in the workbook window, since that was last activated), also I require that the color and axis scales remain fixed through the duration of the graphs, which, without those two lines, gives a video that jumps around because of the oscillating nature of the data and displays "0 s" for the entire thing.
Is there anyway I can change the label and apply the theme without activating the window?
I ran 80 iterations (after removing the two lines you suggested just to get a more consistent measurement) and measured that the memory increases from 233MB to 317MB, I doubt that it is the opj specifically because it is consistent across various dat files that I have imported. What were the dimensions of your data every iteration? See if you can reproduce this via a dummy set that has 80 iterations across 30x20 sets (so 2430 rows by 20 columns). I think it is rooted in the actual plotvm X-Function and scales with the actual size of the virtual matrix.
The file sizes I'm working with are bound to get larger, so I'm still persistent on fully diagnosing the cause of this memory deal.
Thank you for all your help so far Jason! |
|
|
|
jasonzhao
China
262 Posts |
Posted - 07/23/2015 : 9:34:43 PM
|
Hello,
In order to fully diagnosing the cause of this memory deal, and try some work around to the problems we mentioned above.
I suggest you sending a sample file to tech@originlab.com, it is not necessary to send a full and large OPJ file, just a small part of data will be OK.
Best regards!
Jason
OriginLab Technical Service |
|
|
| |
Topic |
|
|
|
Memory Leak Whilst Creating a Video?