statically linked gcc executables

Ted Byers r.ted.byers@rogers.com
Tue Jan 29 16:30:00 GMT 2008


--- John Carter <john.carter@tait.co.nz> wrote:
> On Thu, 24 Jan 2008, Ted Byers wrote:
> 
> > You're half right.  If your program uses library
> X,
> > and  that library has a subtle bug in the function
> > you're using, then the result you get using a
> > different library will be different.  The fix is
> not
> > to ensure that you use the same library all the
> time,
> > but to ensure your test suite is sufficiently well
> > developed that you can detect such a bug, and use
> a
> > different function (even if you have to write it
> > yourself) that routinely gives you provably
> correct
> > answers.
> 
> Alas, Reality bites, we all suck, nobody on planet
> with a non-trivial
> product has perfect test coverage of code and state,
> and we all have
> bugs.
> 
True.  No-one is perfect.  i never said that anyone
had achieved perfection.  At best, perfection is a
state one must strive for, but which can never be
achieved.  But that doesn't stop us from beginning
with unit tests, and proceeding to integration tests
and usability tests, &c., and adopting a protocol that
requires the test suite to be expanded every time a
new bug is found, and prohibiting new code from being
added to an application's code base unless all
existing tests pass.  Such a practice generally
results in the number of bugs per line of code
diminishing through time, although the total number of
bugs may not.  You never stop trying when the kind of
application you're helping develop could have
catastrophic consequences, for the company for which
you're developing it, or for people using it, or
affected by facilities where it is used, should your
application fail in a bad way.

> And even if you have really really good coverage,
> you seldom have the
> time to rerun _every_ test after every change.
> 
True.  But standard practice here is to run the full
test suite, with no failures, before code is committed
to the code-base.  That may be overkill for an
application supporting only drawing cartoons, but in
other industries, where real and significant harm can
be done if an application is wrong, it is a price no
one questions.

> So given how much reality sucks, one of eminently
> practical things you
> can do is reduce the variance between what you have
> tested and what
> you ship.
> 
Right.  So what is the problem with not upgrading all
your development machines to a new release of the tool
chain you're using until you have proven the new
version of the toolchain won't break your code?  Or
that the new version has found a bug in your code the
previous version didn't (when it produces results
inconsistent with a previous version of your
application), and that you have fixed the bug and
extended your testsuite accordingly?

> Test what like you fly, fly what you test.
> 
Right.  All tests, for the kinds of applications I
develop, in the test suite must pass before the
application can be released for general use (generally
by consultants with doctorates in some aspect of
environmental science).

> And that applies to shipping products to customers,
> it applies to
> internal products like shipping cross compilers to
> colleagues.
> 
Right, we upgrade ASAP when a new release is available
for our development tools, but this process includes
stress testing them, especially to prove that they
don't break existing code.  If a test in our existing
suite fails upon using a new tool, we have no option
but to investigate to see if the problem lies with
something that was missed in our previous testing (in
which case, the bug revealed is fixed and additional
tests developed to improve our QA), or with something
in the new tool (for which we must find a solution). 
All this must be done before a project can be migrated
to the new tool.  But we do it in anticipation of
relatively continual improvement in our tools as new
releases become available.

> As I said, Reality truly sucks.
> 
Yup.  There is a reason it is more expensive to
develop applications in some disciplines than it is in
others.

> Hint: C/C++ based reality sucks even more since,
> unless you test
> heavily under Valgrind, most code has subtle
> uninitialized data bugs
> that often don't fire under even the heaviest
> testing. One of the
> reasons I like dynamic languages like Ruby.
> 
This is debatable, and this probably isn't the forum
to debate it.  Each programming language has its own
problems, and some problems transcend the language
used.  What really matters is the experience and
discipline of the team doing the work, including
especially the senior programmers, architects, team
leads, &c.: people who know well the potential worst
case consequences of a bug in the application they're
developing, and design and implement accordingly, with
due attention paid to QA and testability.

No one will be too upset if a tool used for animation
in the entertainment industry occasionally fails
(apart, perhaps, from the people who paid for it, or
for a good animation), but if an application could
result in the loss of life or an adverse effect on
someone's health, should it fail (e.g. an application
used in aircraft, such as the autopilot or the
navigation software, or in medicine, or in risk
assessment in environmental protection), one goes the
extra mile to try to ensure such failures don't
happen.

Good QA is more about the people doing the work, and
the protocols they use, than it is about the tools
they have at their disposal.  This is part of why I
tried to explain to the OP that instead of going
through major hoops on your developer's machines, you
have a smaller team working to assess the new tool, or
suite of tools, and deploy it to the core developers
only after you have proven that the new tools produce
correct code when used on your existing application
and test suite.   Once you have THAT proof, you can
proceed confidently with a routine deployment of the
new tool on all the developer's machines.  If there is
insufficient manpower or time to do it right, then
don't upgrade until you do; and recognize that if
there is always insufficient manpower or time to do it
right, then those paying to have it done can't really
afford to pay to get it done right (which is a scary
notion to me, with the kinds of applications I
routinely develop).  This is a protocol that will
likely be more efficient than one in which a major
effort is put into altering all your developer's
machines to use the same versions of the same tool
chain. 

I try to keep my tools current, expecting continual
improvement in their quality, but I would never go
through the kinds of hoops the OP described as that
struck me as counterproductive: time not spent either
developing new code for the application or
trouble-shooting the combination of the new tool chain
with the existing codebase and testsuite.  I can work
around deficiencies in the tools I use, since I know
my tools, and while I upgrade my tools as soon as
practicable, I don't do it until I know the new tool
won't break my existing code, and if it does, I
investigate to see where the problem really lies: and
if it is in my code, I fix it and if it is in the new
version of the tool, I develop a solution for the
problem.  Only once I know the code I get from the new
tools is correct do I proceed with an upgrade.

Cheers,

Ted



More information about the Gcc-help mailing list