As far as I have tried in my Origin 8.6, the setting of the option "Significance Level" in the ANOVAOneWay dialog box works as shown in the screenshot below. (Don't be confused with the significance level under "Mean Comparison" branch.)
Oh, thank you for the picture. It's very interesting: I don't have those two boxes for the significane but only one. And with this one box, Origin just does not do the test with another significane than 0.05.
Unfortunately my OriginPro 8G is in German. I hope, you can understand it anyway.
As I have just quickly checked, Origin 8.1 or earlier doesn't have that control, but at least 8.5.1 has it.
This is the reason why it is very important to include your Origin version number in the submission when anyone posts a question to the forum. This makes the thread more effective and less time wasting for readers and participants.