Saturday, December 26, 2015

Preparing for GeoPackage 1.0.2

The GeoPackage Standards Working Group (SWG) has voted to recommend the working version of the GeoPackage specification to the OGC Technical Committee for adoption as version 1.0.2. Because of some substantive changes that affect conformance, we decided to make version 1.0.2 a technical amendment.

I have been asked by OGC to provide release notes for this version. This is in progress and a draft version is available here [1]. For those of you who are not inclined to wade through the boilerplate text and other stuff, here is a summary of the substantive changes. (The hyperlinks at the beginning of each line point to the relevant GitHub Issue).

  • 102. Requirement 45 is new and therefore there is no risk to existing implementations. Conforming to the new requirement would improve interoperability between systems.
  • 130. Two columns in the Schema Extension were renamed to conform to the rule that all column names are lower case. There are no known uses of this table in the field so the risk of this change is low.
  • 132. In the past, information regarding a single extension was scattered in at least four parts of the document. It was likely that a reader would not be able to find all of the relevant information about a particular extension. In addition, if a new extension was subsequently added to the document, it would change the numbering of the other annexes. In response, we decided to move all information regarding extensions into a single annex (Annex F), with one extension per sub-annex.
    While technically an administrative edit because no requirements were actually changed, a number of internal references were updated. References to numbered elements may break. (References to linked elements are unchanged.) Because of the potential impact to users of the standard (in documents, code samples, etc.), this is listed as substantive.
  • 137. OGC has adopted 12-063r5, a new standard for describing coordinate reference systems as well-known text. The SWG determined that it would be too disruptive to update the references in the document to the new standard. In order to encourage adoption of the new standard, a new extension was formed. Implementations are new free to adopt the new standard at their own pace by conforming to the extension. Since this is a new extension, there is no risk to existing implementations.
  • 147. Reports from the community indicated that the schema and metadata sections (formerly Sections 2.3 and 2.4 respectively) were not being used. Mandating those tables was creating work for implemententers with no tangible benefit. In response, the SWG determined that the best course of action would be to demote them to extensions. This reduces the length of the core of the standard, improving readability. Because of the lack of use, the risk of this change is low. 
The next steps are to create a PDF version of the document and to initiate a vote in the OGC Technical Committee. If I understand correctly, technical amendments can be approved via an electronic vote after an appropriate review period.

[1] Note: this file was updated on January 13. Feedback is welcome!

Tuesday, December 22, 2015

The One That Got Away

As a SWG chairman, I take pride in building consensus. We have tackled some tricky challenges this past year and I am pleased to say that we have gotten there with only one roll call vote.  What I found most of the time is that when there was objection to unanimous consent, there was often further work to be done. In a few situations we were able to come up with creative solutions that ended up satisfying everyone.

What was that one roll call vote? I was actually on the losing end of it. I wanted the elevation extension to encode the data in a purely binary encoding. I felt that introducing another format to the mix was adding an unnecessary layer of redirection that some developer was going to have to deal with down the line. The fact that libraries existed for many formats was not important to me because a) these libraries tended to be overkill for what we needed and b) reading a simple binary format is not particularly hard.

I lost. In the end I had to resign myself to the fact that people wanted a self-describing format (even if that made no sense to me). However, the story didn't end there. When all was said and done, we ended up building two extensions. The first laid out the format and specified PNG files for storing integer data. The second allowed floating point data to be stored in TIFF files. It is a reasonable compromise and I think it will work.

Wednesday, December 16, 2015

What I Want for Christmas (or 2016)

In my last blog post I talked about what the GeoPackage SWG accomplished in 2015. Now I want to look ahead to 2016. 

This is my wish list:
  • From OGC:
    • A way to publish multiple versions of the GeoPackage specification on geopackage.org (right now it just shows the working version)
    • Speedy adoption of GeoPackage 1.0.2
    • A repeatable process for producing HTML (and possibly other document types) from AsciiDoc so that we can easily publish other draft specifications in an easily readable form
    • Commitment to encourage other SWGs to use the GitHub-based specification process that we pioneered here
    • For users to automatically get added as observers in the project portal when they accept an observer agreement (you have no idea how big a nuisance this is!)
  • From developers and/or software vendors:
    • More content for geopackage.org, particularly FAQs and guidance on how to use the trickier parts of the standard (e.g., some of the extensions)
    • Accurate, timely information on what products support what versions and/or portions of the standard
  • From data providers:
    • A commitment to publish their geospatial data as GeoPackage and/or GeoJSON instead of or in addition to proprietary formats or arcane government formats
    • Feedback on what works well (or doesn't work well)
  • From Image Matters (my employer):
    • Support to continue in my role as SWG chair regardless of fickle funding situations
    • A proper retirement sendoff for Paul Daisey, without whom GeoPackage never would have gotten this far
See you next year!

Tuesday, December 1, 2015

2015 Recap

As I type this, OGC is preparing for its Technical Committee meeting in Sydney, Australia. Unfortunately I cannot be there in person. Nevertheless, I want to take this opportunity to recap all of the work the GeoPackage Standards Working Group (SWG) has done in 2015. We have:
It has been a busy year and I doubt we will get this much done next year. We do have a few things on our radar for the first half of 2016:
  • Releasing Version 1.0.2
  • Completing the GPKG-EE IE and hopefully releasing the Elevation Extension as an adopted extension
  • Releasing at least a Discussion Paper regarding a potential extension for symbology and styling
However, the main measure of success next year will be utilization. We believe the standard is in a good state and that it solves the problems it was intended to solve. We ask data providers to consider using GeoPackage as an alternative (if not a complete replacement) to other formats. We ask data consumers to consider using GeoPackages when they are available and to let us know when using them proves beneficial. The SWG is here to help but the rest is up to you.