Bug 115460 - gcc test results deviations
Summary: gcc test results deviations
Status: WAITING
Alias: None
Product: gcc
Classification: Unclassified
Component: testsuite (show other bugs)
Version: 13.2.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-12 13:12 UTC by sadineniharish8446
Modified: 2024-06-18 23:39 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2024-06-12 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description sadineniharish8446 2024-06-12 13:12:56 UTC
I'am using gcc-13.2.0, I have given two consecutive builds with make check-gcc and compared test results, i have observed following deviations: 

Result from first test build:
 WARNING: program timed out                                            
 FAIL: gcc.target/i386/avx512bw-pr94509-2.c (test for excess errors)   
 UNRESOLVED: gcc.target/i386/avx512bw-pr94509-2.c compilation failed to

Result from second test build: 
 PASS: gcc.target/i386/avx512bw-pr94509-2.c (test for excess errors)
 PASS: gcc.target/i386/avx512bw-pr94509-2.c execution test

In yocto we are using gcc-13.2.0 and after running gcc testsuite and comparing results we have observed a lot of test result deviations :
for example, test passed in first iteration is moving to unresolved in the next iteration.

 ptestresult.gcc.gcc.dg/20061026.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/20081223-1.c  (test for errors, line 5): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/Warray-bounds-6.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/asm-4.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/attr-access-read-write-2.c  (test for warnings, line 18): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/bitfld-10.c  (test for errors, line 8): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/c2x-complit-4.c: UNRESOLVED -> UNSUPPORTED
        ptestresult.gcc.gcc.dg/c2x-float-5.c: UNRESOLVED -> UNSUPPORTED
        ptestresult.gcc.gcc.dg/c2x-stdarg-5.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/c99-condexpr-1.c (test for excess errors): PASS -> UNRESOLVED
        ptestresult.gcc.gcc.dg/const-float80-ped.c  (test for warnings, line 5): PASS -> UNRESOLVED
 

 1)What can be the reason for this?
 2)How to resolve this kind of issues?
 
 thanks,
 Harish sadineni
Comment 1 Andrew Pinski 2024-06-12 13:17:21 UTC
Without the full .log of the testsuite runs, it is hard to debug what is going wrong. And I personally have not seen this. The only time I have (recently) seen a timeout compiling has been when the compiler is miscompiled.
Comment 2 Andrew Pinski 2024-06-12 17:18:43 UTC
Note you want to look into the gcc.log file to understand why things are turning into UNRESOLVED. the gcc.sum file is only a summary of what is going on.

Also depending on the board you are using or how you are running the testsuite, it could be that the ssh connection has been broken due to the machine crashing. So you really should look into the gcc.log file to understand the exact failure here.