[PATCH] codecvt<wchar_t, char, mbstate_t>

[Pétur Runólfsson <peturr02 at ru dot is>]

Hi,

This patch is basically a new implementation of
codecvt<wchar_t, char, mbstate_t> that should handle all stateless
encodings and also fixes several minor problems in the current code,
including DR 75.

Things that need to be done to support stateful encodings:
* do_unshift needs to be implemented.
* do_encoding needs to check if encoding is stateful.
* do_max_length is wrong for encodings that may contain redundant
  shift sequences.

There are some places where extra help from the C library would be
useful:
* do_encoding needs to know if encoding is stateful, I don't think
  there is any way to get this info from the C library at present.
* do_in, do_out and do_length convert one character at a time,
  which is probably not the fastest way to do it.

I don't think this is appropriate for 3.3:
* The resolution of DR 75 breaks the ABI.
* The member _M_c_locale_codecvt is needed for this to be useful.
* There are also many bugs in basic_filebuf that need to be fixed
  for this to be useful.

The patch causes the tests
22_locale/codecvt/{encoding,max_length}/wchar_t/wrapped_locale.cc
to fail. These tests are wrong, the tests 1.cc check for the expected
return values for the "C" locale, which happen to be different from
the ones for the locale "ja_JP.eucjp". I don't know what the correct
return values are so I haven't tried to fix this.

Tested on linux on x86 with glibc 2.3.

Petur

2003-02-04  Petur Runolfsson  <peturr02@ru.is>

	DR 75
	DR 305
	PR libstdc++/9028 (partial)
	PR libstdc++/9224
	PR libstdc++/9246
	PR libstdc++/9247

	* src/codecvt.cc
	(codecvt<wchar_t, char, mbstate_t>::do_encoding,
	codecvt<wchar_t, char, mbstate_t>::do_length,
	codecvt<wchar_t, char, mbstate_t>::do_max_length):
	Move...
	* config/locale/generic/codecvt_members.cc:  ...here.
	* config/locale/gnu/codecvt_members.cc:  ...and here.

	* config/locale/generic/codecvt_members.cc,
	* config/locale/gnu/codecvt_members.cc
	(codecvt<wchar_t, char, mbstate_t>::do_encoding
	codecvt<wchar_t, char, mbstate_t>::do_in,
	codecvt<wchar_t, char, mbstate_t>::do_length,
	codecvt<wchar_t, char, mbstate_t>::do_max_length,
	codecvt<wchar_t, char, mbstate_t>::do_out):
	New implementation that handles stateless encodings,
	including UTF-8.

	* config/locale/generic/codecvt_members.cc,
	* config/locale/gnu/codecvt_members.cc,
	* config/locale/ieee_1003.1-2001/codecvt_specializations.h,
	* include/bits/codecvt.h,
	* src/codecvt.cc
	(codecvt::length, codecvt::do_length):
	Change type of first argument of length and do_length from
	'const state_type&' to 'state_type&' according to DR 75.

	* testsuite/22_locale/codecvt/always_noconv/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/always_noconv/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/always_noconv/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/always_noconv/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/encoding/wchar_t/1.cc:
	Cleanup and check for correct return value from encoding
	for "C" locale.
	* testsuite/22_locale/codecvt/encoding/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/encoding/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/encoding/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/in/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/5.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/6.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/7.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/8.cc:  New test.
	* testsuite/22_locale/codecvt/in/wchar_t/9.cc:  New test.
	* testsuite/22_locale/codecvt/length/char/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/length/char/2.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/length/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/5.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/6.cc:  New test.
	* testsuite/22_locale/codecvt/length/wchar_t/7.cc:  New test.
	* testsuite/22_locale/codecvt/max_length/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/max_length/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/max_length/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/max_length/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/out/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/4.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/5.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/6.cc:  New test.
	* testsuite/22_locale/codecvt/out/wchar_t/7.cc:  New test.
	* testsuite/22_locale/codecvt/unicode/1.cc:  New test.
	* testsuite/22_locale/codecvt/unshift/wchar_t/1.cc:  Cleanup.
	* testsuite/22_locale/codecvt/unshift/wchar_t/2.cc:  New test.
	* testsuite/22_locale/codecvt/unshift/wchar_t/3.cc:  New test.
	* testsuite/22_locale/codecvt/unshfit/wchar_t/4.cc:  New test.

Attachment: codecvt5.diff.gz
Description: codecvt5.diff.gz