This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: GCC Buildbot


On Wed, Sep 20, 2017 at 11:07 AM, Jeff Law <law@redhat.com> wrote:
> On 09/20/2017 09:01 AM, Paulo Matos wrote:
>> Hi all,
>>
>> I am internally running buildbot for a few projects, including one for a
>> simple gcc setup for a private port. After some discussions with David
>> Edelsohn at the last couple of Cauldrons, who told me this might be
>> interesting for the community in general, I have contacted Sergio DJ
>> with a few questions on his buildbot configuration for GDB. I then
>> stripped out his configuration and transformed it into one from GCC,
>> with a few private additions and ported it to the most recent buildbot
>> version nine (which is numerically 0.9.x).
>>
>> To make a long story short: https://gcc-buildbot.linki.tools
>> With brief documentation in: https://linkitools.github.io/gcc-buildbot
>> and configuration in: https://github.com/LinkiTools/gcc-buildbot
>>
>> Now, this is still pretty raw but it:
>> * Configures a fedora x86_64 for C, C++ and ObjectiveC (./configure
>> --disable-multilib)
>> * Does an incremental build
>> * Runs all tests
>> * Grabs the test results and stores them as properties
>> * Creates a tarball of the sum and log files from the testsuite
>> directory and uploads them
>>
>> This mail's intention is to gauge the interest of having a buildbot for
>> GCC. Buildbot is a generic Python framework to build a test framework so
>> the possibilities are pretty much endless as all workflows are
>> programmed in Python and with buildbot nine the interface is also
>> modifiable, if required.
>>
>> If this is something of interest, then we will need to understand what
>> is required, among those:
>>
>> - which machines we can use as workers: we certainly need more worker
>> (previously known as slave) machines to test GCC in different
>> archs/configurations;
>> - what kind of build configurations do we need and what they should do:
>> for example, do we want to build gcc standalone against system (the one
>> installed in the worker) binutils, glibc, etc or do we want a builder to
>> bootstrap everything?
>> - initially I was doing fresh builds and uploading a tarball (450Mgs)
>> for download. This took way too long. I have moved to incremental builds
>> with no tarball generation but if required we could do this for forced
>> builds and/or nightly. Ideas?
>> - We are currently running the whole testsuite for each incremental
>> build (~40mins). If we want a faster turnaround time, we could run just
>> an important subset of tests. Suggestions?
>> - would we like to run anything on the compiler besides the gcc
>> testsuite? I know Honza does, or used to do, lots of firefox builds to
>> test LTO. Shall we build those, for example? I noticed there's a testing
>> subpage which contains a few other libraries, should we build these?
>> (https://gcc.gnu.org/testing/)
>> - Currently we have a force build which allows people to force a build
>> on the worker. This requires no authentication and can certainly be
>> abused. We can add some sort of authentication, like for example, only
>> allow users with a gcc.gnu.org email? For now, it's not a problem.
>> -  We are building gcc for C, C++, ObjC (Which is the default). Shall we
>> add more languages to the mix?
>> - the gdb buildbot has a feature I have disabled (the TRY scheduler)
>> which allows people to submit patches to the buildbot, buildbot patches
>> the current svn version, builds and tests that. Would we want something
>> like this?
>> - buildbot can notify people if the build fails or if there's a test
>> regression. Notification can be sent to IRC and email for example. What
>> would people prefer to have as the settings for notifications?
>> - an example of a successful build is:
>> https://gcc-buildbot.linki.tools/#/builders/1/builds/38
>> This build shows several Changes because between the start and finish of
>> a build there were several new commits. Properties show among other
>> things test results. Responsible users show the people who were involved
>> in the changes for the build.
>>
>> I am sure there are lots of other questions and issues. Please let me
>> know if you find this interesting and what you would like to see
>> implemented.
> I'd strongly recommend using one of the existing infrastructures.  I
> know several folks (myself included) are using Jenkins/Hudson.  There's
> little to be gained building a completely new infrastructure to manage a
> buildbot.

Python Buildbot is not a completely new infrastructure.  It is widely
deployed and used for many projects.  LLVM utilizes a Buildbot
cluster.

I strongly support that we explore how to utilize this offer.  Please
don't bikeshed this.

Thanks, David


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]