Bug 78448 - Container max_size() functions don't consider the range of their difference_type
Summary: Container max_size() functions don't consider the range of their difference_type
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libstdc++ (show other bugs)
Version: unknown
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
: 87027 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-11-21 13:41 UTC by Jonathan Wakely
Modified: 2018-08-22 22:24 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2018-08-20 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Wakely 2016-11-21 13:41:26 UTC
#include <vector>
#include <limits>
#include <cassert>

int main() {
 using V = std::vector<char>;
 V v;
 assert(v.max_size() <= std::numeric_limits<V::difference_type>::max());
}

The requirement is that max_size() returns:

   distance(begin(), end()) for the largest possible container

which must fit in Container::difference_type
Comment 1 Jonathan Wakely 2018-08-20 09:59:33 UTC
*** Bug 87027 has been marked as a duplicate of this bug. ***
Comment 2 Jonathan Wakely 2018-08-20 10:00:27 UTC
Testcase from PR 87027:

The program below outputs 2:
 
/usr/bin/g++ -m32 -std=c++14  ex.cpp && ./a.out
Vector maximal size: 4294967295 and actual size: 2415919104
Found value 2

It should output 3 or throw std::length_error.

See also
https://stackoverflow.com/questions/51907247/lower-bound-algorithm-stl-usage-preconditions/51910011#51910011  

// compile with -m32 switch
#include<algorithm>
#include<iostream>
#include<stdexcept>
#include<vector>
using namespace std;
int main() {
 try {
  vector<uint8_t> v((1ULL << 28) * 9, 2); // 2.25 G entries
  v.back() = 3;                           // the last of which is greater
  cout<< "Vector maximal size: "<<v.max_size()<< " and actual size: " << v.size() <<endl;
  uint8_t val=3;
  auto x= lower_bound(v.begin(), v.end(), val );
  if (x!=v.end() && !( val< *x ) ) {
   cout << "Found value " << int(*x) << endl;
  } else {
   cout << "Not Found " << endl;
  }
 } catch (exception const & ex){
  cerr<< ex.what()<<endl;
 }
}

//----------
$ /usr/bin/g++ --version
g++ (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Comment 3 Jonathan Wakely 2018-08-20 10:02:37 UTC
UBsan shows where the overflows happen:

/home/jwakely/gcc/9/include/c++/9.0.0/bits/stl_vector.h:359:59: runtime error: pointer index expression with base 0x6760f010 overflowed to 0xf760f010
/home/jwakely/gcc/9/include/c++/9.0.0/bits/stl_algobase.h:777:20: runtime error: pointer index expression with base 0x6760f010 overflowed to 0xf760f010
/home/jwakely/gcc/9/include/c++/9.0.0/bits/stl_algobase.h:778:24: runtime error: pointer index expression with base 0x6760f010 overflowed to 0xf760f010
Vector maximal size: 4294967295 and actual size: 2415919104
Found value 2

We just need to fix our max_size() as per comment 0.
Comment 4 Jonathan Wakely 2018-08-22 22:23:12 UTC
Author: redi
Date: Wed Aug 22 22:22:40 2018
New Revision: 263789

URL: https://gcc.gnu.org/viewcvs?rev=263789&root=gcc&view=rev
Log:
PR libstdc++/78448 limit vector::max_size and deque::max_size

The container requirements imply that max_size() can't exceed the
maximum value of the container's difference_type. Enforce this for
std::vector and std::deque, and add checks to ensure the container
doesn't grow larger than that.

	PR libstdc++/78448
	* include/bits/deque.tcc (deque::_M_range_initialize): Use
	_S_check_init_len to check size.
	(deque::_M_push_back_aux, deque::_M_push_front_aux): Throw length
	error if size would exceed max_size().
	* include/bits/stl_deque.h (_Deque_base::size_type): Remove typedef.
	(_Deque_base(_Deque_base&&, const allocator_type&, size_t)): Use
	size_t instead of size_type.
	(deq(size_type, const allocator_type&)
	(deq(size_type, const value_type&, const allocator_type&)
	(deque::_M_initialize_dispatch): Use _S_check_init_len to check size.
	(deque::max_size): Call _S_max_size.
	(deque::_S_check_init_len, deque::_S_max_size): New functions.
	* include/bits/stl_vector.h (vector(size_type, const allocator_type&))
	(vector(size_type, const value_type&, const allocator_type&))
	(vector::_M_initialize_dispatch, vector::_M_range_initialize): Use
	_S_check_init_len to check size.
	(vector::max_size): Call _S_max_size.
	(vector::_M_check_len): Prevent max from being expanded as a
	function-like macro.
	(vector::_S_check_init_len, vector::_S_max_size): New functions.
	* include/bits/vector.tcc (vector::_M_assign_aux): Use
	_S_check_init_len to check size.
	* testsuite/23_containers/deque/capacity/max_size.cc: New test.
	* testsuite/23_containers/vector/capacity/max_size.cc: New test.

Added:
    trunk/libstdc++-v3/testsuite/23_containers/deque/capacity/max_size.cc
    trunk/libstdc++-v3/testsuite/23_containers/vector/capacity/max_size.cc
Modified:
    trunk/libstdc++-v3/ChangeLog
    trunk/libstdc++-v3/include/bits/deque.tcc
    trunk/libstdc++-v3/include/bits/stl_deque.h
    trunk/libstdc++-v3/include/bits/stl_vector.h
    trunk/libstdc++-v3/include/bits/vector.tcc
Comment 5 Jonathan Wakely 2018-08-22 22:24:33 UTC
Fixed for std::vector and std::deque.

For the other containers the overhead of the nodes means that the allocator's max_size will never be larger than ptrdiff_t anyway.