std::codecvt::out() returns a no-op in marginal situations

[Sam Varshavchik]

In certain marginal situations when std::codecvt::out() (gcc 4.1.2) is 
invoked with:

1) from < from_end

and

2) to < to_end

then out() returns with

1) from_next=from
2) to_next=to
3) and the return code of partial

In other words, out() does not consume any input in [from, from_end), and 
does not produce any output in [to, to_end). One may interpret this in two 
ways: either out() wants more input, or it wants a bigger output buffer, but 
there's nothing to tell me which one it is. Example:

	std::locale test_locale("en_US.utf-8");

	const std::codecvt<wchar_t, char, std::mbstate_t> &codecvt=
		std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t> >
		(test_locale);

	wchar_t warray[1];

	warray[0]=0x00e8;

	const wchar_t *from_next;
	char *to_next;

	std::mbstate_t init_state=std::mbstate_t();

	std::codecvt_base::result res=
		codecvt.out(init_state, warray, warray+1, from_next,
			    outarray, outarray+sizeof(outarray), to_next);

	std::cout << "from_next=" << from_next-warray << std::endl;
	std::cout << "to_ext=" << to_next-outarray << std::endl;

	switch (res) {
	case 	std::codecvt_base::ok:
		std::cout << "res=ok" << std::endl;
		break;
	case 	std::codecvt_base::partial:
		std::cout << "res=partial" << std::endl;
		break;
	case 	std::codecvt_base::error:
		std::cout << "res=error" << std::endl;
		break;
	case 	std::codecvt_base::noconv:
		std::cout << "res=noconv" << std::endl;
		break;
	}

The output of this is:

from_next=0
to_ext=0
res=partial

Unicode U+0x00e8 is 0xc3+0xa8 in utf-8. Changing outarray to a 2-byte buffer 
lets out() produce this sequence, so, in this case, out() wants a bigger 
output buffer. But how exactly am I supposed to now this?

Note, that even though the result is "partial", out() does not advance the 
from_next pointer, and sets it to "from".

If out() wants a bigger output buffer, I would expect it to put 0xc3 into 
the buffer, update the state, return with to_next==to_end, then later have 
unshift() produce the 0xa8, in the ordinary course of business.

So, if out() returns with:

1) from_next < from_end, and

2) to_next < to_end

then can I assume that out() wants a larger output buffer? And, is the 
similar true for inp() as well? I don't seem to find anything in ISO/IEC 
14882 that suggests this.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/libstdc++/attachments/20080121/3289406d/attachment.sig>