For the Intervals API, we’re wrestling with issues surrounding data input validation. This recently became interesting when the matter of date validation came up. Ordinarily, Intervals allows many, many different date formats, dependent on the locale that the customer is using (for example, Intervals may expect the date format ‘mm/dd/yyyy’ for US customers, ‘dd.mm.yy’ for a customer in Austria).

For our API developers, we wanted to use a common, universal format, one that would be easily compatible with our application and database layers. For that we selected ISO 8601, which is great in terms of widespread use, but not so great in terms of how complicated its specifications are.

Generally, ISO 8601 looks something like ’2009-05-20′ for dates and ’2009-05-20 12:30:30′ for date/time combinations. These two examples encompass 98% of the user input we’re likely to encounter. But we wanted to make sure that if we told developers they could use ISO 8601 dates, our system would support it. Unfortunately, there’s not a lot of code out there for the validation of ISO 8601 dates (especially regular expressions), and most of the stuff that is out there doesn’t encompass the entirety of the ISO 8601 spec.

Starting off, here are some dates that the validator should match (all these are valid ISO 8601 dates to the best of my knowledge):

2009-12T12:34
2009
2009-05-19
2009-05-19
20090519
2009123
2009-05
2009-123
2009-222
2009-001
2009-W01-1
2009-W51-1
2009-W511
2009-W33
2009W511
2009-05-19
2009-05-19 00:00
2009-05-19 14
2009-05-19 14:31
2009-05-19 14:39:22
2009-05-19T14:39Z
2009-W21-2
2009-W21-2T01:22
2009-139
2009-05-19 14:39:22-06:00
2009-05-19 14:39:22+0600
2009-05-19 14:39:22-01
20090621T0545Z
2007-04-06T00:00
2007-04-05T24:00

Added Feb 19 2010:
2010-02-18T16:23:48.5
2010-02-18T16:23:48,444
2010-02-18T16:23:48,3-06:00
2010-02-18T16:23.4
2010-02-18T16:23,25
2010-02-18T16:23.33+0600
2010-02-18T16.23334444
2010-02-18T16,2283
2009-05-19 143922.500
2009-05-19 1439,55

And here are some of the strings that the validator should not match (ie. reject):

200905
2009367
2009-
2007-04-05T24:50
2009-000
2009-M511
2009M511
2009-05-19T14a39r
2009-05-19T14:3924
2009-0519
2009-05-1914:39
2009-05-19 14:
2009-05-19r14:39
2009-05-19 14a39a22
200912-01
2009-05-19 14:39:22+06a00

Added Feb 19 2010:
2009-05-19 146922.500
2010-02-18T16.5:23.35:48
2010-02-18T16:23.35:48
2010-02-18T16:23.35:48.45
2009-05-19 14.5.44
2010-02-18T16:23.33.600
2010-02-18T16,25:23:48,444

The code we came up with was the following:

Updated Feb 19 2010:
^([\+-]?\d{4}(?!\d{2}\b))((-?)((0[1-9]|1[0-2])(\3([12]\d|0[1-9]|3[01]))?|W([0-4]\d|5[0-2])(-?[1-7])?|(00[1-9]|0[1-9]\d|[12]\d{2}|3([0-5]\d|6[1-6])))([T\s]((([01]\d|2[0-3])((:?)[0-5]\d)?|24\:?00)([\.,]\d+(?!:))?)?(\17[0-5]\d([\.,]\d+)?)?([zZ]|([\+-])([01]\d|2[0-3]):?([0-5]\d)?)?)?)?$

I guess I should add the caveat that this code doesn’t support the time interval or duration part of the ISO 8601 spec, so I didn’t include it. And it only supports dates or date/times, since right now we don’t have to deal with time input (for the Intervals API, all time is input in decimal format, rather than ISO 8601). But it should support everything else. Please let me know if this works for you or doesn’t, of if you can fine tune it.

In our PHP code, we are using the ISO-8601 week number of year, as specified on the PHP date function page, but we weren’t using ISO-8601 for the year. The ISO-8601 week number specifies the last monday of a year as the first week of the new year, if that new year begins before thursday. Intervals thought it was the first year of 2007!

In PHP, the fix was as easy as converting all instances of date(‘Y’) to date(‘o’), according to php.net:

ISO-8601 year number. This has the same value as Y, except that if the ISO week number (W) belongs to the previous or next year, that year is used instead. (added in PHP 5.1.0)

That fixed everything on the PHP side of things. But next we had to dig into the SQL queries and get them to use the ISO Year.

Snag.

PostgreSQL 8.2.5 doesn’t support ISO Year in the Extract function. EXTRACT(ISOYEAR, timestamp) is being included in PostgreSQL 8.3, as specified here in the RC1 documentation. But PostgreSQL 8.3 hasn’t been released yet, and we needed to fix things immediately.

Our final PostgreSQL fix was to instead use the TO_CHAR(timestamp, ‘IYYY’) function. It’s not ideal to be using a string formatting function for data comparisons, because it slows down some of the queries. But we had to trade some performance to get things working properly again in the new year. As soon as the PostgreSQL developers release a stable version of 8.3, we’ll change our queries back to using EXTRACT(ISOYEAR, timestamp).