How to tell (in advance) if a date-time is ambiguous?

[Binarus]

Dear experts,

a few days ago I have got great help from this list, so I hope I may ask
another (probably stupid) question (I am now having the opposite problem
than back then):

Using DateTime, is it possible to tell in advance if a certain date-time
which is given in a certain locale will be ambiguous due to switching
from DST to standard time?

Example (taken from DateTime's documentation):

my $dt = DateTime->new(
year => 2003,
month => 10,
day => 26,
hour => 1,
minute => 30,
second => 0,
time_zone => 'America/Chicago',
);

Now $dt is ambiguous: The clock has been turned back to 01:00:00 h at
02:00:00 that night, so the time 01:30:00 h has occurred twice.

Even after thinking many hours about it, I haven't found a reasonable
general method to determine if an arbitrary date-time is ambiguous in
the sense above.

Possibly, I could subtract different time spans from the date-time in
question and check if the result is the expected one and use that to
find out if it is ambiguous, but this would cost much CPU time.

So I would like to ask if somebody knows a general, reasonable method
for solving that problem, given the following conditions:

1) We don't know the time span the clock is turned back when switching
from DST to standard time. It might be one hour in most time zones /
countries, but after all, some weird person could decide that it is 18
minutes and 13 seconds or 5 hours, 53 minutes and 42 seconds.

2) We don't know whether the point in time when the switch occurs is
exactly at an hour's end / begin. Again, some weird person could decide
that the switch happens at 08:48:27 am.

I know that there are not many persons on the world that are *that*
weird, but on the other hand, I don't want to implement an algorithm
which uses assumptions which are not safe.

As a last resort, there is at least one other module (AFAIK) which I
could use to extract the daylight saving switching times and time spans
from the time zone database, and I could use that information to solve
my problem. But this would probably mean to reinvent the wheel, so I'd
like to avoid it.

Thank you very much in advance,

Binarus

[Zefram]

Post by Binarus
Using DateTime, is it possible to tell in advance if a certain date-time
which is given in a certain locale will be ambiguous due to switching
from DST to standard time?

That is tricky. I don't think our APIs provide any way to do it.
Thinking about the facilities available a bit lower down, the way I'd
probably approach it is to look at the list of all the offsets ever used
in that timezone (not available through any API, but extractable from
the tzfile). I'd compute for each the UT time that would be represented
by the specified local time with that offset, and then use the regular
API to convert that UT time to the local time in the specified timezone
(or equivalently just to look up the zone's offset for that UT time).
Look at which local times come out matching the specified local
time (which offsets match the candidate offset that we were trying).
If there's more than one match, that's an ambiguous local time. If there
are no matches, it's a non-existent local time.

Something close to this can actually be done in the C time API. You
don't get to ask what all the zone's offsets are, but in the local->UT
conversion you can specify whether DST is in effect. Giving both states
of that flag gives you two UT times, which you can then convert back
to local to check whether they come out with the same DST flag state.
This will work for regular DST changes, but not for offset changes that
are unrelated to DST.

-zefram

[Binarus]

Post by Zefram

Post by Binarus
Using DateTime, is it possible to tell in advance if a certain date-time
which is given in a certain locale will be ambiguous due to switching
from DST to standard time?

That is tricky. I don't think our APIs provide any way to do it.
Thinking about the facilities available a bit lower down, the way I'd
probably approach it is to look at the list of all the offsets ever used
in that timezone (not available through any API, but extractable from
the tzfile). I'd compute for each the UT time that would be represented
by the specified local time with that offset, and then use the regular
API to convert that UT time to the local time in the specified timezone
(or equivalently just to look up the zone's offset for that UT time).
Look at which local times come out matching the specified local
time (which offsets match the candidate offset that we were trying).
If there's more than one match, that's an ambiguous local time. If there
are no matches, it's a non-existent local time.

At first, thank you very much for your answer!

I see. I always thought that the tzfile only contained the offsets for
standard time (and not DST). Obviously, I have been wrong (I never had a
look into a tzfile yet because I hope that I could solve my problem
without reading it out directly).

Post by Zefram
Something close to this can actually be done in the C time API. You
don't get to ask what all the zone's offsets are, but in the local->UT
conversion you can specify whether DST is in effect. Giving both states
of that flag gives you two UT times, which you can then convert back
to local to check whether they come out with the same DST flag state.
This will work for regular DST changes, but not for offset changes that
are unrelated to DST.

This is very interesting. Actually, I am doing more things in C than in
Perl (but on other architectures), but since I thought that DateTime was
mirroring all capabilities libc has in this respect, I did not look into
the C time API yet. I have no dislike against writing a part of the
application in C (but then will have to learn how to call C from Perl).

Again, thanks for all advice,

Binarus

[Zefram]

As the documentation tells us, DateTime always chooses the later time
when calculating with ambiguous times,

This logic is actually in DateTime::TimeZone, where DateTime invokes
it via the ->offset_for_local_datetime method. The internal logic
is able to walk the sequence of observances, and when it finds a
matching observance it looks ahead to the next one to check specially
for ambiguity. It returns the later span if there's an ambiguity due
specifically to a change from DST to non-DST; it doesn't have guaranteed
behaviour in any other ambiguity scenario. DT:TZ doesn't document this
feature, and doesn't offer any other form of ambiguity detection in its
API (though of course it would be possible to add some).

and if you subtract an hour from
the later (ambiguous) time, you'll get the same time, but the earlier
one (provided the clock is turned back by an hour when switching back).

You can't rely on offset changes, even specifically ones for DST, being of
an hour. Australia/Lord_Howe (Lord Howe Island) does a regular half-hour
DST jump.

-zefram

[Binarus]

Post by Zefram

As the documentation tells us, DateTime always chooses the later time
when calculating with ambiguous times,

This logic is actually in DateTime::TimeZone, where DateTime invokes
it via the ->offset_for_local_datetime method. The internal logic
is able to walk the sequence of observances, and when it finds a
matching observance it looks ahead to the next one to check specially
for ambiguity. It returns the later span if there's an ambiguity due
specifically to a change from DST to non-DST; it doesn't have guaranteed
behaviour in any other ambiguity scenario. DT:TZ doesn't document this
feature, and doesn't offer any other form of ambiguity detection in its
API (though of course it would be possible to add some).

Now I know why I didn't find according documentation. At least I was not
completely wrong in believing that its all in there (although I
suspected the wrong module).

So is there any change to add an according API function to DT:TZ?

Post by Zefram

and if you subtract an hour from
the later (ambiguous) time, you'll get the same time, but the earlier
one (provided the clock is turned back by an hour when switching back).

You can't rely on offset changes, even specifically ones for DST, being of
an hour. Australia/Lord_Howe (Lord Howe Island) does a regular half-hour
DST jump.

That was the reason why I have mentioned the weird preconditions in my
first post.

Another thing is that I eventually have to re-think the behavior of my
application. In some of your posts, you indirectly have mentioned that
there might be other reasons for ambiguous times than switching from DST
to standard time, and of course, you are right.

While my application will not be concerned by leap seconds (the
application actually could do its thing when the respective moment is
passed the first time, and ignore the second occurrence), it will be
concerned when the time is turned back by at least 15 minutes, whether
or not this happens due to DST switching or permanently because some
dictator has gone nuts. In that case, the application had to do its
thing upon the first occurrence of the respective time as well as upon
the second occurrence.

That means that the API which I would like DT::xx to provide would have
to tell me if a certain arbitrary date-time will occur twice *whether or
not* this will happen due to DST switching.

I see that this is difficult to implement. I have to think about the
situation and will try to find a solution at the application level.

Thank you very much,

Binarus

[Zefram]

Post by Binarus
So is there any change to add an according API function to DT:TZ?

Yes, but we wouldn't want to rush it. There's more than one
implementation of the API, and we want to be sure to design it correctly
the first time.

Perhaps it could be a ->offsets_for_local_datetime method (note
plural in the name), which returns a sorted list of all the timezone
offsets that are applicable to a specified local time. Normally the
list would have length 1, the element being the same value returned by
->offset_for_local_datetime. Ambiguous local times (regardless of the
source of ambiguity) yield more than one offset. Non-existent local times
(skipped due to clocks going forward) yield an empty list without error.

-zefram

[Binarus]

Post by Zefram

Post by Binarus
So is there any change to add an according API function to DT:TZ?

Yes, but we wouldn't want to rush it. There's more than one
implementation of the API, and we want to be sure to design it correctly
the first time.
Perhaps it could be a ->offsets_for_local_datetime method (note
plural in the name), which returns a sorted list of all the timezone
offsets that are applicable to a specified local time. Normally the
list would have length 1, the element being the same value returned by
->offset_for_local_datetime. Ambiguous local times (regardless of the
source of ambiguity) yield more than one offset. Non-existent local times
(skipped due to clocks going forward) yield an empty list without error.

That would be absolutely great. Could you please let us know when you
have started working on it? I didn't see a datetime-announce mailing
list yet :-)

(In the meantime, I'll try to work around the problems somehow).

Regards and thank you very much again,

Binarus

[Binarus]

If you're trying to avoid these, the best advice I could give would be to
avoid the 12am-4am window, which AFAIK is when most (all?) transitions have
occurred historically.

Most, but there are both historical and current exceptions.
America/Godthab (west Greenland) changes 22:00->23:00 and 23:00->22:00
(base offset -03:00, with transition at 01:00 UT per EU rules).
America/Santiago (Chile) changes 00:00->23:00. Pacific/Easter (Easter
Island) changes 22:00->23:00 and 22:00->21:00 (changing at the same
time as Chile, but with base offset 2 hours to the west). Historically,
Africa/Casablanca (Morocco) changed 12:00->13:00 in a DST change in 1967.
And Pacific/Kwajalein (part of the Marshall Islands) jumped across the
international date line in the unfashionable direction, repeating 23
hours of 1969-09-30.
-zefram

Wow, that's quite a cool bit of information. Thank you, very
interesting! Did you memorize the tzfile of 1969 :-)

Regards,

Binarus

[Zefram]

Post by Binarus
Did you memorize the tzfile of 1969 :-)

I looked through the Olson source files. I could also have automated
a search through the compiled zone data.

-zefram