Incorrect transition times in std::chrono::time_zone::get_info()

[Jonathan Wakely]

On Fri, 26 Jul 2024 at 11:04, Edward Welbourne <edward.welbourne@qt.io> wrote:
>
> Hi libstdc++ folks,
>
> I'm the Qt Project's maintainer of dates, times and localization.  We're
> currently adding a backend to QTimeZone that uses std::chrono::tzdb and
> friends.  In the course of testing this I've hit some issues with the
> implementation in libstdc++.  I'm currently using the version GCC 14
> uses by default on Debian/testing (but dpkg -S doesn't admit to which
> actual package supplies /lib/x86_64-linux-gnu/libstdc++.so.6 so I don't
> know which that version is).
>
> One issue is that the supported range of times is rather limited; Qt
> uses a 64-bit signed count of milliseconds from the UTC start of 1970,
> which does admittedly have an insanely wide range, but (I am happy to
> report) glibc's time_t functions seem to cope with that whole range just
> fine.

For a 64-bit time_t they'll have an even larger range, since they'll
work with a 64-bit signed number of seconds, not milliseconds.

For 32-bit time_t, you have bigger problems anyway.


> However, the std::chrono::time_zone::get_info() range is
> considerably narrower (I haven't yet pinned down its limits).  That's
> bearable, since the out-of-range times are of course only ever seen in
> artificial test-cases; and get_info() raises a std::runtime_error for
> out-of-range values, so we can catch that and cope by other kludges
> (that we need on other platforms anyway).

What exactly is the problem you see? Do you have a testcase for this?

time_zone::get_info converts its argument to sys_seconds and calls
_M_get_sys_info. That should have exactly the same range as time_t
(64-bit signed number of seconds, with 1970-01-01 as the epoch).


> The main issue, however, is that the times of many zone-transitions are
> wrong.  Specifically, there are many transitions where the IANA DB, as
> digested into the contents of /usr/{share,lib}/zoneinfo/ as part of the
> standard Linux tzdata package, gives different answers for the time of a
> transition to those coming out of get_info() as the begin and end
> values.  Here's my test program:
>
> #include <chrono>
>
> #include <iostream>
> #include <algorithm>
>
> template <typename R, typename T>
> static auto *binary_search_name(R &&r, const T &t)
> {
>     using std::begin;
>     using std::end;
>     auto before =  [](const auto &e, const T &t) { return e.name() < t; };
>     auto it = std::lower_bound(begin(r), end(r), t, before);
>     return it != end(r) && it->name() == t ? &(*it) : nullptr;
> }
>
> static const std::chrono::time_zone *idToZone(std::string_view id) noexcept
> {
>     auto &tz = std::chrono::get_tzdb();
>     if (auto zone = binary_search_name(tz.zones, id))
>         return zone;
>     if (auto link = binary_search_name(tz.links, id))
>         return binary_search_name(tz.zones, link->name());
>     return nullptr;
> }
>
> int main(int argc, const char *args[])
> {
>     using namespace std::chrono;
>     year_month_day ymd(year(2014), month(12), day(21));
>     const sys_days date = ymd;
>     for (int i = 1; i < argc; ++i) {
>         const time_zone *zone = idToZone(args[i]);
>         if (zone) {
>             const sys_info info = zone->get_info(date);
>             const zoned_time local(zone, info.begin);
>             std::cout << '\n' << args[i] << " time at its transition was " << local;
>         }
>     }
>     std::cout << '\n' << __GLIBCXX__ << '\n';
>     return 0;
> }
>
> If I've misunderstood what some of the API should be doing, please let
> me know.  Assuming it does what I intended, this reports the last
> transition before 2014-12-21; pass the resulting program some zone names
> on the command-line, and it reports on each.  I have a script that runs
> a program for each zone known to the tzdata package; so have seen its
> results for all.  Many are correct, but many are not.  Observed results
> include:
>
> Europe/Kaliningrad  : 2014-10-26 04:00:00 EET (offset +2)
> Europe/Moscow       : 2014-10-26 05:00:00 MSK (offset +3)
> Asia/Bishkek        : 2005-08-12 06:00:00 +06
> Asia/Bangkok        : 1920-04-01 07:00:00 +07
> Asia/Almaty         : 2004-10-31 08:00:00 +06
> Asia/Pyongyang      : 1945-08-24 09:00:00 KST (offset +9)
> Pacific/Bougainville: 1945-08-21 10:00:00 +10
> Pacific/Kosrae      : 1999-01-01 11:00:00 +11
> Pacific/Kwajalein   : 1993-08-20 12:00:00 +12
> Pacific/Kanton      : 1994-12-31 13:00:00 +13
> Pacific/Honolulu    : 1947-06-07 14:00:00 HST (was 02:00, offset -10)
> Pacific/Gambier     : 1912-09-30 15:00:00 -09
> America/Metlakatla  : 1983-10-29 16:00:00 PST (was 1983-10-30 02:00, offset -8)
> America/Phoenix     : 1968-03-20 17:00:00 MST (was 1983-03-21, offset -7)
> America/Regina      : 1960-03-30 18:00:00 CST (was 1960-04024 02:00, offset -6)
> America/Jamaica     : 1983-12-31 19:00:00 EST (was 1983-10-30 02:00, offset -5)
> America/Boa_Vista   : 2000-10-14 20:00:00 -04 (was 2000-10-15)
> America/Araguaina   : 2013-08-31 21:00:00 -03 (was 2013 Sep)
> America/Noronha     : 2002-09-30 22:00:00 -02 (was 2002-10-01)
> Asia/Karachi        : 2009-10-31 23:00:00 PKT (was 2009-11-01 00:00, offset +5)
>
> Transitions in the middle of the day are not a thing that zones really
> do.  Some of those near the start and end of this list are almost
> plausible, but the ones I've checked are different in tzdata.
>
> Many of the above look suspiciously like UTC's start of day, or in some
> cases its 02:00 (the default in the IANA DB's runic format), converted
> to the relevant zone, so I have to conclude that libstdc++ is
> misunderstanding Olson's arcane grammar - either that or I'm misusing
> the APIs above, in some way.

There's a known bug (well, known to me) where libstdc++ should
inerpret the transition time relative to the current wall clock time,
not UTC. There are a couple of relevant comments in the code:

    // XXX The ri.until() time point should be
    // "interpreted using the rules in effect just before the transition"
    info.end = ri.until();

      // XXX UNTIL field should be interpreted
      // "using the rules in effect just before the transition"
      // so might need to store as year_month_day and hh_mm_ss and only
      // convert to a sys_time once we know the offset in effect.
      inf.m_until = sys_days(year(y)/m.m/day(d)) + seconds(t.time);

So if a transition should happen at (for example) 2am local time,
libstdc++ might be applying it at 2am UTC in some cases.

> So I first want to check whether I'm using the API right and, if so, to

I'll take a look at your testcase and check if it matches my understanding.

> know where to report the bug.

All libstdc++ bugs should be reported in GCC's bugzilla.

> Available evidence of internet searches
> points to the general GCC bug-reporting, but I wanted to first check in
> case you have a separate bug-tracker for libstdc++,
>
>         Eddy.
>
>