locale: ctype<char> classifications fails for chars with high bit set

[Per Liboriussen <liborius at stofanet dot dk>]

The current version of ctype<char> doesn't work for chars with the 8th bit
set, in implementations where char is signed.

The problem is in the files "config/os/.../bits/ctype_inline.h". The
various classification functions are given arguments of type char (of
pointer to char), which are used directly to index the classification
table. They must be cast to unsigned char before being used, otherwise
chars with the high bit set will be promoted to negative integers, and thus
reference random memory.

Unless I'm very much mistaken, there is a similar problem in the
ctype_noninline.h files. The static_cast<> is in place there, but
unfortunately it is to int, not unsigned char, which doesn't help much.
However, this problem seems to be hidden on systems using glibc, since it
uses the __ctype_b table from there directly, and this table has been set
up to allow for an index in the range [-128,255]. ("for broken old
programs", the ctype.h header says ;-)

I have verified the problem on a GNU/Linux system (Debian 2.2) with GCC
3.0.4, but it looks like the problem exists in all the ctype_xxx headers.

This problem is already in the GNATS database as number 4457, where it is
marked as closed. The problem is still there in the current CVS head, though.
I suspect the reason is that the original submitter used Solaris, whereas
the reviewer reports it as fixed on linux. It is probably the glibc thing
mentioned above that has hidden it. (The example included in the bug report
didn't supply its own table to the ctype<char> constructor.)

A simple program that shows the problem is this:

  #include <iostream>
  #include <iomanip>
  #include <locale>

  int main()
  {
    std::ctype_base::mask maskdata[256];
    for (int i=0; i<256; ++i)
      maskdata[i] = std::ctype_base::alpha;
    std::ctype<char>* f = new std::ctype<char>(maskdata);
    std::locale global;
    std::locale loc(global, f);
    for (int i=0; i<256; ++i) {
      if (i % 16 == 0)
        std::cout << std::setw(3) << i << ": ";
      char ch = i;
      std::cout << std::isalpha(ch, loc) << (i % 16 == 15 ? "\n" : " ");
    }
  }
  

It should print all 1's, but the values from 128 to 255 are random 0's and 1's.


Per Liboriussen