[Bug c/106002] New: RFE: complain about incorrect checks of return values (CWE-253)

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106002

            Bug ID: 106002
           Summary: RFE: complain about incorrect checks of return values
                    (CWE-253)
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: diagnostic
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: dmalcolm at gcc dot gnu.org
  Target Milestone: ---

Some APIs return NULL to signify an error; others return a negative int.  It's
possible to mess up by confusing these.

We currently don't warn at -Wall on these:

#include <stdlib.h>
#include <stdio.h>

extern void do_something_with (void *);

int test_1 (void)
{
  void *tmp = malloc(1024);
  if (tmp < 0 )
    return -1;

  do_something_with (tmp);

  free (tmp);
  return 0;
}

int test_2 (void)
{
  char buf[256];
  if (fgets (buf, sizeof (buf), stdin) < 0)
    return -1;

  do_something_with (buf);

  return 0;
}

where both error checks are testing for < 0, when they should be checking for
NULL; looks to me like we ought to warn for this.

I'm not sure if it's possible to reliably warn for the opposite error without
lots of false positives (checking for pointer NULL, rather than int 0?)

Filing against "c" since I think this could probably happen in the frontends
rather than the analyzer.

See CWE-253:
  https://cwe.mitre.org/data/definitions/253.html