Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assertion failure at openj9/runtime/compiler/control/CompilationThread.cpp:11600: false when PROD_WITH_ASSUMES is enabled #15472

Closed
manashaVetrivelu opened this issue Jun 30, 2022 · 8 comments
Closed

Assertion failure at openj9/runtime/compiler/control/CompilationThread.cpp:11600: false when PROD_WITH_ASSUMES is enabled #15472

manashaVetrivelu opened this issue Jun 30, 2022 · 8 comments
Labels
comp:jit

Comments

@manashaVetrivelu
Copy link
Contributor

In the special.system test suite,
The assert at

/home/jenkins/workspace/Build_JDK11_x86-64_linux_Personal/openj9/runtime/compiler/control/CompilationThread.cpp:11600: false
failed when it was compiled with PROD_WITH_ASSUMES enabled. It was reproducible in all of the tests mentioned below.
It failed during the following tests:

ClassLoadingTest_special_5m_20

The platform used to build and test was x86-64_linux.
The link to the Jenkins Build.
The link to the Grinder Build.

The stack trace for each of the tests are:

ClassLoadingTest_special_5m_20 with the stack trace:

[2022-06-28T19:59:19.730Z] CLT stderr Assertion failed at /home/jenkins/workspace/Build_JDK11_x86-64_linux_Personal/openj9/runtime/compiler/control/CompilationThread.cpp:11600: false
[2022-06-28T19:59:19.730Z] CLT stderr VMState: 0x0005ff0b
[2022-06-28T19:59:19.730Z] CLT stderr 	Unexpected value for method->extra = 0x1747 (method=0x5d240)
[2022-06-28T19:59:19.730Z] CLT stderr 
[2022-06-28T19:59:19.730Z] CLT stderr compiling java/lang/Thread.join(JI)V at level: warm
[2022-06-28T19:59:19.730Z] CLT stderr #0: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x96f845) [0x7efcb8c1d845]
[2022-06-28T19:59:19.730Z] CLT stderr #1: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x97c090) [0x7efcb8c2a090]
[2022-06-28T19:59:19.730Z] CLT stderr #2: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x60169e) [0x7efcb88af69e]
[2022-06-28T19:59:19.730Z] CLT stderr #3: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x601a03) [0x7efcb88afa03]
[2022-06-28T19:59:19.730Z] CLT stderr #4: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x14e1a8) [0x7efcb83fc1a8]
[2022-06-28T19:59:19.730Z] CLT stderr #5: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x1519a3) [0x7efcb83ff9a3]
[2022-06-28T19:59:19.730Z] CLT stderr #6: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x152fa6) [0x7efcb8400fa6]
[2022-06-28T19:59:19.730Z] CLT stderr #7: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x1535c3) [0x7efcb84015c3]
[2022-06-28T19:59:19.730Z] CLT stderr #8: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x15200c) [0x7efcb840000c]
[2022-06-28T19:59:19.730Z] CLT stderr #9: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x152598) [0x7efcb8400598]
[2022-06-28T19:59:19.730Z] CLT stderr #10: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x152632) [0x7efcb8400632]
[2022-06-28T19:59:19.730Z] CLT stderr #11: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9prt29.so(+0x2b3f3) [0x7efcba93a3f3]
[2022-06-28T19:59:19.730Z] CLT stderr #12: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x152a62) [0x7efcb8400a62]
[2022-06-28T19:59:19.730Z] CLT stderr #13: /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/lib/default/libj9thr29.so(+0xe4f6) [0x7efcba7024f6]
[2022-06-28T19:59:19.730Z] CLT stderr #14: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7efcbbdb1609]
[2022-06-28T19:59:19.730Z] CLT stderr #15: /lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7efcbbabf163]
[2022-06-28T19:59:19.730Z] CLT stderr 
[2022-06-28T19:59:19.730Z] CLT stderr Unhandled exception
[2022-06-28T19:59:19.730Z] CLT stderr Type=Unhandled trap vmState=0x0005ff0b
[2022-06-28T19:59:19.730Z] CLT stderr J9Generic_Signal_Number=00000108 Signal_Number=00000005 Error_Value=00000000 Signal_Code=fffffffa
[2022-06-28T19:59:19.730Z] CLT stderr Handler1=00007EFCBABDEB30 Handler2=00007EFCBA939690
[2022-06-28T19:59:19.730Z] CLT stderr RDI=0000000000000002 RSI=00007EFC991B53D0 RAX=0000000000000000 RBX=00007EFCB8D5A780
[2022-06-28T19:59:19.730Z] CLT stderr RCX=00007EFCBBDBD24B RDX=0000000000000000 R8=0000000000000000 R9=00007EFC991B53D0
[2022-06-28T19:59:19.730Z] CLT stderr R10=0000000000000008 R11=0000000000000246 R12=00007EFCB8D58D54 R13=00007EFCB8D5E870
[2022-06-28T19:59:19.730Z] CLT stderr R14=0000000000000000 R15=000000000001AC00
[2022-06-28T19:59:19.730Z] CLT stderr RIP=00007EFCBBDBD24B GS=0000 FS=0000 RSP=00007EFC991B53D0
[2022-06-28T19:59:19.730Z] CLT stderr EFlags=0000000000000246 CS=0033 RBP=0000000000002D50 ERR=0000000000000007
[2022-06-28T19:59:19.730Z] CLT stderr TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=00007F9F822C2780
[2022-06-28T19:59:19.730Z] CLT stderr xmm0 ffffffffffffffff (f: 4294967296.000000, d: -nan)
[2022-06-28T19:59:19.730Z] CLT stderr xmm1 683d4c52555f5353 (f: 1432310656.000000, d: 1.336710e+194)
[2022-06-28T19:59:19.730Z] CLT stderr xmm2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm3 00007efc991b4ea0 (f: 2568703744.000000, d: 6.898311e-310)
[2022-06-28T19:59:19.730Z] CLT stderr xmm4 4000000000000000 (f: 0.000000, d: 2.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm8 0a5d787a23255b20 (f: 589650688.000000, d: 9.583694e-259)
[2022-06-28T19:59:19.730Z] CLT stderr xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2022-06-28T19:59:19.730Z] CLT stderr Module=/lib/x86_64-linux-gnu/libpthread.so.0
[2022-06-28T19:59:19.730Z] CLT stderr Module_base_address=00007EFCBBDA9000 Symbol=raise
[2022-06-28T19:59:19.730Z] CLT stderr Symbol_address=00007EFCBBDBD180
@mpirvu mpirvu added the comp:jit label Jul 2, 2022
@mpirvu
Copy link
Contributor

mpirvu commented Jul 2, 2022 

CLT stderr ----------- Stack Backtrace -----------
CLT stderr raise+0xcb (0x00007EFCBBDBD24B [libpthread.so.0+0x1424b])
CLT stderr _ZN2TR4trapEv+0x47 (0x00007EFCB88AF7D7 [libj9jit29.so+0x6017d7])
CLT stderr _ZN2TR9assertionEPKciS1_S1_z+0xc8 (0x00007EFCB88AFA08 [libj9jit29.so+0x601a08])
CLT stderr _ZN2TR15CompilationInfo24replenishInvocationCountEP8J9MethodPNS_11CompilationE+0x3a8 (0x00007EFCB83FC1A8 [libj9jit29.so+0x14e1a8])
CLT stderr _ZN2TR28CompilationInfoPerThreadBase20postCompilationTasksEP10J9VMThreadP21TR_MethodToBeCompiledP8J9MethodPKvP19J9JITExceptionTablebbP20TR_RelocationRuntime+0xa33 (0x00007EFCB83FF9A3 [libj9jit29.so+0x1519a3])
CLT stderr _ZN2TR28CompilationInfoPerThreadBase7compileEP10J9VMThreadP21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x386 (0x00007EFCB8400FA6 [libj9jit29.so+0x152fa6])
CLT stderr _ZN2TR24CompilationInfoPerThread12processEntryER21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x1e3 (0x00007EFCB84015C3 [libj9jit29.so+0x1535c3])
CLT stderr _ZN2TR24CompilationInfoPerThread14processEntriesEv+0x44c (0x00007EFCB840000C [libj9jit29.so+0x15200c])
CLT stderr _ZN2TR24CompilationInfoPerThread3runEv+0x98 (0x00007EFCB8400598 [libj9jit29.so+0x152598])
CLT stderr _Z30protectedCompilationThreadProcP13J9PortLibraryPN2TR24CompilationInfoPerThreadE+0x82 (0x00007EFCB8400632 [libj9jit29.so+0x152632])
CLT stderr omrsig_protect+0x1e3 (0x00007EFCBA93A3F3 [libj9prt29.so+0x2b3f3])
CLT stderr _Z21compilationThreadProcPv+0x1d2 (0x00007EFCB8400A62 [libj9jit29.so+0x152a62])
CLT stderr thread_wrapper+0x186 (0x00007EFCBA7024F6 [libj9thr29.so+0xe4f6])
CLT stderr start_thread+0xd9 (0x00007EFCBBDB1609 [libpthread.so.0+0x8609])
CLT stderr clone+0x43 (0x00007EFCBBABF163 [libc.so.6+0x11f163])

@mpirvu
Copy link
Contributor

mpirvu commented Jul 2, 2022

replenishInvocationCount() is used when an AOT compilation finishes, but we don't want to execute the compiled code right away. We want to replenish the invocation count and let the method run interpreted to collect more Iprofiler info.
The expectation is that the invocation count is 0 in this case. From the diagnostic data I see the invocation count being 1295 in one case and 5959 in another case. The latter is a strange value.

manashaVetrivelu added a commit to manashaVetrivelu/openj9 that referenced this issue Jul 8, 2022
@manashaVetrivelu
Asserts fired due to PROD_WITH_ASSUMES commented out
98e8900 
Asserts from Issues: eclipse-openj9#15440, eclipse-openj9#15447, eclipse-openj9#15472, eclipse-openj9#15482

Signed-off-by: Manasha Vetrivelu <[email protected]>
@manashaVetrivelu manashaVetrivelu mentioned this issue Jul 8, 2022
Open
@mpirvu
Copy link
Contributor

mpirvu commented Jul 19, 2022 •
edited
Loading

I reproduced this issue while running JCL_Test_SE80_0. It always happens in java/lang/Compiler.<init>()V
The value of extra is a positive small number (e.g. 2001) and we expect it to be 1 or -5. For other methods it is 1, but for this method in particular it is not.

@mpirvu
Copy link
Contributor

mpirvu commented Jul 19, 2022

Debugging the issue I realized that the compilation is triggered through compileMethod() API:

_ZL20internalCompileClassP10J9VMThreadP7J9Class+0x926 (0x00007F18629C7786 [libj9jit29.so+0x179786])
compileClass+0x51 (0x00007F18629C84A1 [libj9jit29.so+0x17a4a1])
Java_java_lang_Compiler_compileClassImpl+0x52 (0x00007F18619702E2 [libjclse29.so+0x112e2])

so, it is quite normal the invocation count to be larger than 1.

@mpirvu
Copy link
Contributor

mpirvu commented Jul 19, 2022 •
edited
Loading

The compilation is triggered from this code:

static IDATA
internalCompileClass(J9VMThread * vmThread, J9Class * clazz)
   {
   J9JavaVM * javaVM = vmThread->javaVM;
   J9JITConfig * jitConfg = javaVM->jitConfig;
   TR::CompilationInfo * compInfo = getCompilationInfo(jitConfig);
   TR_J9VMBase *fe = TR_J9VMBase::get(jitConfg, NULL);

   // To prevent class unloading we need VM access
   bool threadHadNoVMAccess = (!(vmThread->publicFlags & J9_PUBLIC_FLAGS_VM_ACCESS));
   if (threadHadNoVMAccess)
      acquireVMAccess(vmThread);

   J9Method * newInstanceThunk = getNewInstancePrototype(vmThread);

   J9ROMMethod * romMethod = (J9ROMMethod *) J9ROMCLASS_ROMMETHODS(clazz->romClass);
   J9Method * ramMethods = (J9Method *) (clazz->ramMethods);
   for (uint32_t m = 0; m < clazz->romClass->romMethodCount; m++)
      {
      J9Method * method = &ramMethods[m];
      if (!(romMethod->modifiers & (J9AccNative | J9AccAbstract))
         && method != newInstanceThunk
         && !TR::CompilationInfo::isCompiled(method)
         && !fe->isThunkArchetype(method))
         {
         bool queued = false;
         TR_MethodEvent event;
         event._eventType = TR_MethodEvent::InterpreterCounterTripped;
         event._j9method = method;
         event._oldStartPC = 0;
         event._vmThread = vmThread;
         event._classNeedingThunk = 0;
         bool newPlanCreated;
         TR_OptimizationPlan *plan = TR::CompilationController::getCompilationStrategy()->processEvent(&event, &newPlanCreated);
         if (plan)
            {
            plan->setIsExplicitCompilation(true);
            // If the controller decides to compile this method, trigger the compilation and wait here

               { // scope for details
               TR::IlGeneratorMethodDetails details(method);
               compInfo->compileMethod(vmThread, details, 0, TR_no, NULL, &queued, plan);
               }

so one possibility is to replace TR_MethodEvent::InterpreterCounterTripped; with something else like TR_MethodEvent::compileMethodAPI, then have processEvent set a flag in the OptimizationPlan and then test that flag when the assert is thrown.

@0xdaryl
Copy link
Contributor

0xdaryl commented May 9, 2023

@dylanjtuttle : please investigate

This was referenced May 19, 2023
Closed
Closed
@knn-k
Copy link
Contributor

knn-k commented Jun 20, 2023

I encountered this assertion failure on AArch64 Linux with PROD_WITH_ASSUMES enabled.

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk11_j9_sanity.functional_aarch64_linux_Personal_testList_1/205/consoleText:

[2023-06-19T03:14:17.565Z] Assertion failed at /home/jenkins/workspace/Build_JDK11_aarch64_linux_Personal/openj9/runtime/compiler/control/CompilationThread.cpp:11967: false
[2023-06-19T03:14:17.565Z] VMState: 0x0005ff0b
[2023-06-19T03:14:17.565Z] 	Unexpected value for method->extra = 0x1771 (method=0x4e29c8)
[2023-06-19T03:14:17.565Z] 
[2023-06-19T03:14:17.565Z] compiling java/lang/Compiler.<init>()V at level: warm

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk17_j9_sanity.functional_aarch64_linux_Personal_testList_1/226/consoleText:

[2023-06-19T03:10:19.769Z] Assertion failed at /home/jenkins/workspace/Build_JDK17_aarch64_linux_Personal/openj9/runtime/compiler/control/CompilationThread.cpp:11967: false
[2023-06-19T03:10:19.769Z] VMState: 0x0005ff0b
[2023-06-19T03:10:19.769Z] 	Unexpected value for method->extra = 0x1771 (method=0x5a5ec8)
[2023-06-19T03:10:19.769Z] 
[2023-06-19T03:10:19.769Z] compiling java/lang/Compiler.<init>()V at level: warm

@dylanjtuttle dylanjtuttle removed their assignment Jul 21, 2023
dylanjtuttle added a commit to dylanjtuttle/openj9 that referenced this issue Aug 14, 2023
@dylanjtuttle
Comment out assertion eclipse-openj9#15472
a6ac326
dylanjtuttle added a commit to dylanjtuttle/openj9 that referenced this issue Aug 14, 2023
@dylanjtuttle
Comment out assertion eclipse-openj9#15472
c648f3e
luke-li-2003 added a commit to luke-li-2003/omr that referenced this issue Dec 31, 2024
@luke-li-2003
Consider the compileMethod() API in replenishInvocationCount() Assertion
cd06d49 
Add a flag in the optimization plan for compilations triggered by
the compileMethod API, which can have the invocation count be any
positive integer; replenishInvocationCount will not raise the assertion
failure for unexpected count value.

Based on eclipse-openj9/openj9#15472

Signed-off-by: Luke Li <[email protected]>
luke-li-2003 added a commit to luke-li-2003/openj9 that referenced this issue Dec 31, 2024
@luke-li-2003
Consider the compileMethod() API in replenishInvocationCount() Assertion
7b249a7 
Add a flag in the optimization plan for compilations triggered by
the compileMethod API, which can have the invocation count be any
positive integer; replenishInvocationCount will not raise the assertion
failure for unexpected count value.

Based on eclipse-openj9#15472

Signed-off-by: Luke Li <[email protected]>
This was referenced Dec 31, 2024
Open
Merged
luke-li-2003 added a commit to luke-li-2003/openj9 that referenced this issue Dec 31, 2024
@luke-li-2003
Consider the compileMethod() API in replenishInvocationCount() Assertion
f27de16 
Add a flag in the optimization plan for compilations triggered by
the compileMethod API, which can have the invocation count be any
positive integer; replenishInvocationCount will not raise the assertion
failure for unexpected count value.

Based on eclipse-openj9#15472

Signed-off-by: Luke Li <[email protected]>
luke-li-2003 added a commit to luke-li-2003/openj9 that referenced this issue Jan 2, 2025
@luke-li-2003
Consider the compileMethod() API in replenishInvocationCount() Assertion
77067cc 
Add a flag in the optimization plan for compilations triggered by
the compileMethod API, which can have the invocation count be any
positive integer; replenishInvocationCount will not raise the assertion
failure for unexpected count value.

Based on eclipse-openj9#15472

Signed-off-by: Luke Li <[email protected]>
luke-li-2003 added a commit to luke-li-2003/openj9 that referenced this issue Jan 2, 2025
@luke-li-2003
Check for Explicit Compilation in the replenishInvocationCount() Asse…
01824c4 
…rtion

replenishInvocationCount will only raise the assertion
failure for unexpected count value if the compilation was not set
as explicit, since the count value can be any positive integer
in that case.

Based on eclipse-openj9#15472

Signed-off-by: Luke Li <[email protected]>
@mpirvu
Copy link
Contributor

mpirvu commented Jan 3, 2025

Fixed by #20876

@mpirvu mpirvu closed this as completed Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
comp:jit
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@0xdaryl @mpirvu @knn-k @dylanjtuttle @manashaVetrivelu