8015660: Test8009761.java "Failed: init recursive calls: 24. After deopt 25"
Windows reserves and only partially commits thread stack. For detecting more thread stack space for execution, Windows installs one-shot page as guard page just before the current commited edge. It will trigger STACK_OVERFLOW_EXCEPTION when lands on last 4 pages of thread stack space. StackYellowPages default value is 2 on Windows (plus 1 page of StackRedPages, 3 pages guarded by hotspot) so the exception happens one page before Yellow pages. Same route executed second time will have one more page brought in, this leads same execution with different stack depth(interpreter mode). We need match Windows settings so the stack overflow exception will not happen before Yellow pages.
Reviewed-by: dholmes
diff --git a/hotspot/src/cpu/x86/vm/globals_x86.hpp b/hotspot/src/cpu/x86/vm/globals_x86.hpp
index 978a1d6..07ab0cf 100644
--- a/hotspot/src/cpu/x86/vm/globals_x86.hpp
+++ b/hotspot/src/cpu/x86/vm/globals_x86.hpp
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -55,7 +55,7 @@
define_pd_global(intx, InlineFrequencyCount, 100);
define_pd_global(intx, InlineSmallCode, 1000);
-define_pd_global(intx, StackYellowPages, 2);
+define_pd_global(intx, StackYellowPages, NOT_WINDOWS(2) WINDOWS_ONLY(3));
define_pd_global(intx, StackRedPages, 1);
#ifdef AMD64
// Very large C++ stack frames using solaris-amd64 optimized builds
