RE: Working with time zones and XQuery

Bill sent a draft of this enquiry to me off-list a few days ago, and I gave
a very similar response to Jim's. Bill phrased the questions in terms of the
current spec and I believe we've answered that. But it leaves the underlying
question unanswered: what are users supposed to do about the irritating
habit of governments who like to deceive us into getting up earlier in the
morning by pretending that the time is later than it really is? 

(One approach is: fight back. Leave your wristwatch on UTC all the year
round. Use UTC when you post meeting announcements. Get out of bed when you
choose, not when the government chooses. There's no law against it, yet. I'm
not being entirely facetious: if you use this approach within your
applications, and only use local civil time when you really have to, you can
save a lot of hassle.) 

Seriously though, it does seem a little strange that we provide so much
machinery for date and time handling, and yet fail to address this real
issue.

I suspect there is room for a third party function library here. Something
that works with a database of civil time zone information provided as an XML
document, and perhaps some complex data types that combine a zoneless
dateTime with the identifier of a civil time zone, allowing people to ask
questions like the one I had today "if the agenda says the telcon is at
2007-03-16T11:00:00 EST, what time do I need to pick up the phone in the
UK?"

But it's very hard to get these things right, and a lot of the burden will
always fall at the application level. Like Jim, I can't immediately think of
anything that we could or should do at the language level. I don't think
more built-in data types is the answer - if the issue can't be tackled using
user-defined types, then there's something badly wrong with the
extensibility of the type system.

Michael Kay
http://www.saxonica.com/



> -----Original Message-----
> From: public-qt-comments-request@w3.org 
> [mailto:public-qt-comments-request@w3.org] On Behalf Of Jim Melton
> Sent: 16 March 2007 21:45
> To: Bill Patton
> Cc: public-qt-comments@w3.org
> Subject: Re: Working with time zones and XQuery
> 
> 
> Mr. Patton,
> 
> Thanks very much for your enquiry and comments.  The 
> responses that I have embedded below amongst your text 
> represent only my own opinion and do not necessarily reflect 
> the Working Groups' opinions.
> 
> At 3/16/2007 12:28 PM, Bill Patton wrote:
> 
> >Dear Sirs,
> >
> >I am currently working on implementing all the date/time 
> data types in 
> >XQuery and associated F&O functions.  This has all been straight 
> >forward except when I get to time zones.  I have read the 
> W3C technical 
> >note "Working with Time Zones", which points out the distinction 
> >between a "time zone offset" and a real "time zone" which has a 
> >historical schedule for daylight savings time, DST.  The note points 
> >out that comparing zoned with unzoned dateTimes is problematic.
> >
> >The XQuery specs and the XML Schema specs remain strangely 
> silent about 
> >DST and, according to the the technical note, misuses the term time 
> >zone for time zone offset.
> 
> Thanks for your observation about the distinction between "time zone" 
> and "time zone offset".  Although XQuery 1.0 does not use the 
> term "time zone offset", we believed that the intent was 
> implicit.  Your comment might be considered by the Working 
> Groups as a potential change to the language in the document.
> 
> 
> >I believe that there are two consistent interpretations of the spec: 
> >one where the spec describes date/time values which use 
> standard time 
> >and no DST, and one that allows for DST by computing time 
> zone offsets 
> >for the date and time in question.
> 
> As Michael Rys has responded, the specifications do not 
> acknowledge DST at all, for reasons that I hope will become 
> clear in my further remarks below.
> 
> >The goal is to have a query where the data does not change
> >  give the same answer when run in the summer or winter.  The 
> >implementation should also be consistent for some time zone 
> preferably 
> >the default time zone for the computer running the query.
> >
> >XQuery uses the "implicit time zone" from the dynamic 
> context to fill 
> >in missing information during operations involving unzoned 
> date/times. 
> >If we use functions time-zone-by-utc(utc) and 
> time-zone-by-wall(wall), 
> >which return the time zone offset for a specific dateTime, 
> and we use 
> >these in the places where the spec calls for the 
> implicit-timezone from 
> >the dynamic context, we get an implementation that is consistent and 
> >allows for DST.  In this model
> >implicit-timezone() might be defined as 
> time-zone-by-utc(current-dateTime()).
> >
> >Here are some questions I am not clear about that lead me to the 
> >conclusion that we should really be using a time zone offset 
> function 
> >rather than a timezone offset constant. Please note the 
> questions may 
> >reflect a poor understanding on my part.
> >
> >1)  Should the XQuery implicit time zone stored in the 
> dynamic context
> >     reflect daylight savings time, DST?   Should the 
> implicit time zone
> >     for a computer on the east coast of the US change from 
> -5:00 hours
> >     in the winter to -4:00 hours in the summer, or should 
> this variation
> >     be handled in some other way?
> 
> You are probably aware that the U.S. Congress passed a law 
> last year that changed the dates on which Daylight Savings 
> Time begins (by three weeks) and ends (by one week).  There's 
> been a fair amount of frantic effort to issue patches for 
> various pieces of software to ensure some consistency within 
> computer systems and the real world.
> 
> How would you suggest the specification for a language such 
> as XQuery deal with that situation?  We know of no meaningful 
> mechanism to encode the always-changing, always-political, 
> most-definitely-human rules governing DST into a 
> specification with a long life.
> 
> We believed that the subject of DST has to be handled by 
> mechanisms outside the control of the specification itself, 
> which lead us to the notion of the implicit time zone 
> [offset] taken from the dynamic context.
> 
> However, no contributor to that dynamic context (e.g., the operating
> system) can predict with 100% confidence the dates on which 
> DST will terminate even this year, much less for the next 
> decade.  The only possible advice is that humans must be 
> aware of the potential for changes in the rules for DST and 
> schedule their appointments accordingly (when doing so on an 
> electronic medium such as a computer).
> 
> 
> >2)  Is a dateTime without an explicit time zone supposed to represent
> >     a local dateTime in the time zone of the computer 
> running XQuery?
> 
> I'd say that it's more accurate to say that the answer to 
> that question depends on what information is inserted into 
> the dynamic context.  Under normal conditions, I would expect 
> that information to correspond to "local" conditions -- 
> HOWEVER, those local conditions might apply to a server or a 
> client (in environments where those concepts are distinguished).
> 
> 
> >3)  Should the date, time, and zone offset fields returned by the
> >     current-dateTime() function reflect wall clock time 
> relative to UTC
> >     time?   That is, for a computer on the east coast of 
> the US would we
> >     expect to see times like 2002-07-01T10:00:00-04Z in the 
> summer and
> >     2002-01-01T10:00:00-05Z in the winter?
> 
> Again, that depends on how the dynamic context is populated.  
> Casually speaking (as in the response just above), I think 
> this would normally be true.
> 
> 
> >4)  Should adjust-dateTime-to-timezone(current-dateTime(), ()) always
> >     return the wall clock time in the computer room?
> 
> In Functions & Operators, section 10.7.1, 
> "fn:adjust-dateTime-to-timezone" 
> (http://www.w3.org/TR/xpath-functions/#func-adjust-dateTime-to
> -timezone),
> I found: If $timezone is not specified, then $timezone is the 
> value of the implicit timezone in the dynamic context.
> 
> That makes it clear to me that it returns the wall clock time 
> implied by the dynamic context.  That might (often/usually) 
> be the time in the computer room, but that depends on how the 
> dynamic context is populated.
> 
> 
> >5)  Are the following identities always true for any time 
> zone and any
> >     time of the year?
> >
> >     timezone-from-dateTime(current-dateTime()) eq 
> implicit-timezone()
> 
> The rules of current-dateTime() say: Returns the current 
> dateTime (with timezone) from the dynamic context.
> The rules of timezone-from-dateTime() say: Returns the 
> timezone component of $arg if any.
> The rules of implicit-timezone() say: Returns the value of 
> the implicit timezone property from the dynamic context.
> 
> I don't believe that there is any ambiguity about the 
> expression's semantics.  To be clear, the answer is "Yes".
> 
> >     adjust-dateTime-to-timezone(current-dateTime()) eq 
> > current-dateTime()
> 
> The rules of adjust-dateTime-to-timezone() say: If $timezone 
> is not specified, then $timezone is the value of the implicit 
> timezone in the dynamic context.
> The rules of current-dateTime() say: Returns the current 
> dateTime (with timezone) from the dynamic context.
> 
> I don't believe that there is any ambiguity about the 
> expression's semantics.  To be clear, the answer is "Yes".
> 
> I am not going to address items 6) through 9) because I 
> believe that the answers to those questions are as 
> unambiguous as the answers to items 4) and 5).  That is, the 
> specification is clear and unambiguous about the meaning of 
> each expression.
> 
> 
> >6)  We should always be able to convert from wall time to UTC and
> >     back again.  Are the following examples, which adjust 
> wall time to
> >     zoned time, correct for a computer using EST time?
> >
> >     adjust-dateTime-to-timezone(xs:dateTime("2002-07-01T10:00:00"))
> >     2002-07-01T10:00:00-04:00 : xs:dateTime
> >
> >     adjust-dateTime-to-timezone(xs:dateTime("2002-01-01T10:00:00"))
> >     2002-01-01T10:00:00-05:00 : xs:dateTime
> >
> >7)  Are the following examples, which adjust each of the 
> results above to
> >     UTC time, correct?
> >
> >     adj xs:dayTimeDuration("PT0S"))
> >     2002-07-01T14:00:00Z : xs:dateTime
> >
> >     
> adjust-dateTime-to-timezone(xs:dateTime("2002-01-01T10:00:00-05:00"),
> >         xs:dayTimeDuration("PT0S"))
> >     2002-01-01T15:00:00Z : xs:dateTime
> >
> >8)  Are the following examples, which adjust UTC time to 
> local time, correct
> >     for a computer running on EST?
> >
> >     adjust-dateTime-to-timezone(xs:dateTime("2002-07-01T14:00:00Z"))
> >     2002-07-01T10:00:00-04:00 : xs:dateTime
> >
> >     adjust-dateTime-to-timezone(xs:dateTime("2002-01-01T15:00:00Z"))
> >     2002-01-01T10:00:00-05:00 : xs:dateTime
> >
> >9)  Are the following examples, which convert the results 
> above to unzoned
> >     time, correct?
> >
> >     
> adjust-dateTime-to-timezone(xs:dateTime("2002-07-01T10:00:00-0
> 4:00"), ())
> >     2002-07-01T10:00:00 : xs:dateTime
> >
> >     
> adjust-dateTime-to-timezone(xs:dateTime("2002-01-01T10:00:00-0
> 5:00"), ())
> >     2002-01-01T10:00:00 : xs:dateTime
> >
> >10) When comparing dateTimes without a time zone offset, 
> should we, in
> >     effect,  historically correct them to UTC, taking into 
> account the zone
> >     offset for that particular date and time?
> 
> I can find nothing in the XQuery 1.0 spec or the Functions 
> and Operators spec that would even hint that this should be 
> the case.  The words "daylight", "summer", and "DST" to not 
> appear anywhere in either spec, so I don't understand how you 
> might have gotten the impression that such an approach was intended.
> 
> Any such model would depend on the XQuery implementation's 
> ability to know what the historical time offset changes 
> occurred at every possible location where the server might be 
> located or that might be the origin of some piece of data.  
> That is possible for the past, dubious for the present, and 
> patently impossible for the future.  And, certainly, if the 
> implementation cannot do this, the spec should not be 
> expected to define it.
> 
> >If the implicit time zone
> >     changes from summer to winter, some XQuery computations 
> will produce
> >     different results when run in the summer rather than in 
> the winter,
> >     even when the data itself has not changed.  Applying 
> the historical
> >     time zone offset corrections eliminates this problem, 
> since it ties
> >     zone offsets to the data rather than the implicit time zone.
> 
> The only thing that needs to happen in order for this nice 
> resolution to take place is for each piece of data to include 
> the timezone with which it is associated.  There is certainly 
> no need to depend on "implicit" timezones to make this happen.
> 
> >An
> >     implementation might achieve this type of comparison by 
> simply using
> >     the long utc millisecond times corresponding to 
> dateTime values for
> >     comparison operations.
> >
> >     Using a correction such as this changes the result of XQTS
> >     K-DatesSubtract-1 which is,
> >
> >     xs:date("1999-07-19") - xs:date("1969-11-30") eq
> >         xs:dayTimeDuration("P10823D")
> >
> >     from true to false for a computer running on EST time.  This
> >     is because xs:date("1999-07-19") - xs:date("1969-11-30") is
> >     P10822DT23H.  One hour is lost in April
> 
> Sorry, but the clock is NOT SET AHEAD IN APRIL, at least not 
> within the United States starting with 2007.  I have not 
> actually done the research, so am not able to cite exact 
> dates and years, but I am fairly certain that the dates on 
> which DST started and ended has changed several times since 
> 1969.  I know for a fact that certain locales in the USA 
> (e.g., parts of Indiana, all of New Mexico, and at times 
> other locations) do not always follow the same DST rules as 
> the rest of the USA.  Therefore, your assumptions are 
> demonstrably false, and thus the solution to the problem 
> isn't a solution at all.
> 
> >when the clock is set ahead
> >     to DST.  None of the other XQTS tests seem to be affected.
> >
> >11) What does the XQuery spec mean when it says in section 10.2
> 
> Actually, the XQuery spec doesn't have a section 10.2, so I 
> assume that you meant to cite Functions & Operators.
> 
> >"Before
> >     comparing or subtracting xs:dateTime values, this local 
> value must
> >     be translated or normalized to UTC"?  Does it mean we 
> should make
> >     historical time zone correction?
> 
> What it means is that all of those values must be 
> (effectively) normalized to some timezone; UTC was chosen as 
> the only timezone whose meaning never changes (by 
> definition).  It has nothing at all to do with historical 
> corrections, since it is impossible to know what historical 
> corrections to apply unless the dateTime value also has 
> locale information attached to it.
> 
> >It is my understanding that making
> >     historical corrections only makes sense when all the 
> data originates
> >     in the same local time zone as the computer running the query.
> 
> And, even then, it is not 100% reliable.
> 
> 
> >12) During the spring when the local time suddenly jumps 
> ahead there is
> >     a one-hour hole in the time continuum between 2 AM and 
> 3 AM EST where
> >     2:30 AM is forbidden.  Even worse, in the fall when the 
> time jumps
> >     from 2:00 AM to 1:00 AM there is a one-hour period 
> where times occur
> >     twice in a day, so there are two 1:30 AM's.  I will 
> refer to this
> >     period as the "Twilight Zone". In order to convert 1:30 
> AM to UTC
> >     during the twilight zone you need to know which 1:30 
> you are talking
> >     about - the 1:30 EDT or the 1:30 EST which occurs one 
> hour later.
> >     The local time, of course, doesn't have an extra bit to 
> express which
> >     1:30 AM it represents, so which one should we pick?
> 
> You've identified an issue that gives many, many people 
> heartburn.  I do not know of any solution that has achieved 
> wide acceptance.  Neither XQuery nor F&O acknowledge the 
> problem.  If you believe you know of a solution that is 
> widely agreed, please suggest that the Working Groups implement it!
> 
> 
> >13) If you naively use the Java GregorianCalendar class to 
> compute the
> >     number of days elapsed since 1 AD, you will be off by 2 
> days from the
> >     XML Schema algorithm for adding a duration to a 
> dateTime.  Setting
> >     the Gregorian/Julian cutover to the smallest negative integer
> >     produces the correct answer.  Does this mean that the 
> XQuery dateTime
> >     is based purely on a Gregorian calendar, which is 
> sometimes called
> >     the "Proleptic Gregorian Calendar"?  Using XQuery to do dateTime
> >     arithmetic spanning 1582 will give results different 
> from the dates
> >     recorded in history books, which are based on the 
> Julian calendar
> >     for dates before 1582.
> 
> Ummmm....in which country/countries?  Russia did not convert 
> over until it was absorbed into the Soviet Union in 1918.  
> Too many people cite 1582 as though it were a universal 
> cutover date.  It most certainly was not in most countries 
> and locales on the planet.  Even in what is now the USA, it 
> didn't happen until 1752.
> 
> 
> >14) What implementation-specific query context options and 
> settings are
> >     needed to make dateTimes really useful?
> >
> >What I find perplexing about all this is that I can calculate times 
> >from 9999 BC to 9999 AD with 1 millisecond accuracy knowing only the 
> >4yr, 100yr, 400yr rule for leap year
> 
> That's incorrect for two reasons.  First, you overlooked the 
> 4,000 year rule, so your calculation would be off by about 
> 4.5 days.  More importantly, you can't make that 
> determination meaningfully without knowing the exact GPS 
> coordinates and the precise political history that governed 
> those coordinates.
> 
> I am sorry if my response here (and perhaps above, too) seem 
> antagonistic.  I don't mean them to be, because I truly 
> appreciate the fact that you took the time and care to submit 
> your comment.  That's something we really encourage.
> 
> I really intended to make the point that it's not that 
> simple.  (In fact, virtually any proposal in this area is 
> inadequate for a variety of reasons.)  I highly recommend 
> "Calendrical Calculations" by Dershowitz and Reingold, ISBN 
> 0-521-56474-3.  I think that, unfortunately, the book is out 
> of print; most of us who have one wouldn't give it up in 
> exchange for food or medicine ;^)  But there are undoubtedly 
> used copies available somewhere on the web.
> 
> >and get the same answers as the Java GregorianCalendar, but I don't 
> >know how to get simple wall clock times out of XQuery.
> 
> Yes, I believe that you do.  fn:time() does exactly that.  Of 
> course, as I said above, the value returned depends on how 
> the dynamic context in which the query is run was populated.  
> In most "reasonable" environments, the dynamic context will 
> reflect the political reality at your GPS coordinate, and 
> therefore the result of the query will match what your wrist 
> claims, plus or minus the accuracy of the various 
> time-keeping instruments.
> 
> Hope this helps,
>     Jim
> 
> >Wall clock
> >time is ubiquitous; most of us carry it strapped to our wrists.
> >
> >Please let me know if you have any answers.
> >
> >Bill Patton
> >http://www.cogneticsystems.com
> 
> ==============================================================
> ==========
> Jim Melton --- Editor of ISO/IEC 9075-* (SQL)     Phone: 
> +1.801.942.0144
>    Co-Chair, W3C XML Query WG; F&O (etc.) editor    Fax : 
> +1.801.942.3345
> Oracle Corporation        Oracle Email: jim dot melton at 
> oracle dot com
> 1930 Viscounti Drive      Standards email: jim dot melton at 
> acm dot org
> Sandy, UT 84093-1063 USA          Personal email: jim at 
> melton dot name
> ==============================================================
> ==========
> =  Facts are facts.   But any opinions expressed are the 
> opinions      =
> =  only of myself and may or may not reflect the opinions of 
> anybody   =
> =  else with whom I may or may not have discussed the issues 
> at hand.  = 
> ==============================================================
> ========== 
> 
>

Received on Friday, 16 March 2007 23:32:56 UTC