This is the mail archive of the
mailing list for the GCC project.
Re: Testing compiler reliability using Csmith
- From: Jeff Law <law at redhat dot com>
- To: Andi Kleen <ak at linux dot intel dot com>, Radu Ometita <radu dot ometita at iohk dot io>
- Cc: gcc at gcc dot gnu dot org, Philip Wadler <philip dot wadler at iohk dot io>, gsocsameeran at gmail dot com
- Date: Fri, 7 Dec 2018 08:36:58 -0700
- Subject: Re: Testing compiler reliability using Csmith
- References: <9E61E004-96ED-48F7-8DCC-8739096F2310@iohk.io> <email@example.com>
On 12/6/18 5:02 PM, Andi Kleen wrote:
> Radu Ometita <firstname.lastname@example.org> writes:
>> Hello everyone!
>> We are working on writing a paper about testing the reliability of C compilers by using Csmith (a random C99 program generator).
>> A previous testing effort, using Csmith, found 79 GCC bugs, and 25 of
>> those have been marked by developers as P1
>> <https://www.flux.utah.edu/download?uid=114>): . However, after this
>> paper was published we are unaware of any further testing using
>> Csmith, and we would like to ask you, if you are aware of any such
>> efforts or further results.
> Sameeran has been doing some additional testing with modified csmith.
> There's currently no systematic effort to run csmith regularly
> to find regressions.
That's my understanding as well.
I also got the impression that they'd already hit a significant
inflection point with csmith as the bugs it was finding were ultimately
duplicates of issues it'd already found. csmith was going to need some
significant development to find new ways to stress compilers.
I have a lot of respect for John's work. It's too bad I don't see him
more often. He's only 15 minutes up the road working with my old group
at the U.