Introducing Thomas Neirynck

We have had a small shake-up in the GeoPackage Standards Working Group (SWG). Roy Rathbun (NGA) has recently accepted another position within NGA and will not be as available to support OGC initiatives. Roy has therefore resigned as SWG vice-chair effective the selection of a replacement.

Thomas Neirynck (Luciad) was recently selected as the new SWG vice-chair. As an active developer of software that uses GeoPackage and similar technologies, he brings a perspective that neither Roy nor I could provide. He also represents the European geospatial community, a community that is vital to GeoPackage's long-term success. In the short term, I would like for him to oversee any changes to the emerging elevation extension that arise as part of the Interoperability Experiment. This will allow me to focus on getting GeoPackage 1.0.2 through the adoption process.

Thanks to Thomas for accepting the position and to Roy for his service to the SWG.

Preparing for GeoPackage 1.0.2

The GeoPackage Standards Working Group (SWG) has voted to recommend the working version of the GeoPackage specification to the OGC Technical Committee for adoption as version 1.0.2. Because of some substantive changes that affect conformance, we decided to make version 1.0.2 a technical amendment.

I have been asked by OGC to provide release notes for this version. This is in progress and a draft version is available here [1]. For those of you who are not inclined to wade through the boilerplate text and other stuff, here is a summary of the substantive changes. (The hyperlinks at the beginning of each line point to the relevant GitHub Issue).

• 102. Requirement 45 is new and therefore there is no risk to existing implementations. Conforming to the new requirement would improve interoperability between systems.
• 130. Two columns in the Schema Extension were renamed to conform to the rule that all column names are lower case. There are no known uses of this table in the field so the risk of this change is low.
• 132. In the past, information regarding a single extension was scattered in at least four parts of the document. It was likely that a reader would not be able to find all of the relevant information about a particular extension. In addition, if a new extension was subsequently added to the document, it would change the numbering of the other annexes. In response, we decided to move all information regarding extensions into a single annex (Annex F), with one extension per sub-annex.
  While technically an administrative edit because no requirements were actually changed, a number of internal references were updated. References to numbered elements may break. (References to linked elements are unchanged.) Because of the potential impact to users of the standard (in documents, code samples, etc.), this is listed as substantive.
• 137. OGC has adopted 12-063r5, a new standard for describing coordinate reference systems as well-known text. The SWG determined that it would be too disruptive to update the references in the document to the new standard. In order to encourage adoption of the new standard, a new extension was formed. Implementations are new free to adopt the new standard at their own pace by conforming to the extension. Since this is a new extension, there is no risk to existing implementations.
• 147. Reports from the community indicated that the schema and metadata sections (formerly Sections 2.3 and 2.4 respectively) were not being used. Mandating those tables was creating work for implemententers with no tangible benefit. In response, the SWG determined that the best course of action would be to demote them to extensions. This reduces the length of the core of the standard, improving readability. Because of the lack of use, the risk of this change is low. 

The next steps are to create a PDF version of the document and to initiate a vote in the OGC Technical Committee. If I understand correctly, technical amendments can be approved via an electronic vote after an appropriate review period.

[1] Note: this file was updated on January 13. Feedback is welcome!

The One That Got Away

As a SWG chairman, I take pride in building consensus. We have tackled some tricky challenges this past year and I am pleased to say that we have gotten there with only one roll call vote.  What I found most of the time is that when there was objection to unanimous consent, there was often further work to be done. In a few situations we were able to come up with creative solutions that ended up satisfying everyone.

What was that one roll call vote? I was actually on the losing end of it. I wanted the elevation extension to encode the data in a purely binary encoding. I felt that introducing another format to the mix was adding an unnecessary layer of redirection that some developer was going to have to deal with down the line. The fact that libraries existed for many formats was not important to me because a) these libraries tended to be overkill for what we needed and b) reading a simple binary format is not particularly hard.

I lost. In the end I had to resign myself to the fact that people wanted a self-describing format (even if that made no sense to me). However, the story didn't end there. When all was said and done, we ended up building two extensions. The first laid out the format and specified PNG files for storing integer data. The second allowed floating point data to be stored in TIFF files. It is a reasonable compromise and I think it will work.

What I Want for Christmas (or 2016)

In my last blog post I talked about what the GeoPackage SWG accomplished in 2015. Now I want to look ahead to 2016. 

This is my wish list:

• From OGC:
• A way to publish multiple versions of the GeoPackage specification on geopackage.org (right now it just shows the working version)
• Speedy adoption of GeoPackage 1.0.2
• A repeatable process for producing HTML (and possibly other document types) from AsciiDoc so that we can easily publish other draft specifications in an easily readable form
• Commitment to encourage other SWGs to use the GitHub-based specification process that we pioneered here
• For users to automatically get added as observers in the project portal when they accept an observer agreement (you have no idea how big a nuisance this is!)
• From developers and/or software vendors:
• More content for geopackage.org, particularly FAQs and guidance on how to use the trickier parts of the standard (e.g., some of the extensions)
• Accurate, timely information on what products support what versions and/or portions of the standard
• From data providers:
• A commitment to publish their geospatial data as GeoPackage and/or GeoJSON instead of or in addition to proprietary formats or arcane government formats
• Feedback on what works well (or doesn't work well)
• From Image Matters (my employer):
• Support to continue in my role as SWG chair regardless of fickle funding situations
• A proper retirement sendoff for Paul Daisey, without whom GeoPackage never would have gotten this far

See you next year!

2015 Recap

As I type this, OGC is preparing for its Technical Committee meeting in Sydney, Australia. Unfortunately I cannot be there in person. Nevertheless, I want to take this opportunity to recap all of the work the GeoPackage Standards Working Group (SWG) has done in 2015. We have:

• Released Version 1.0.1 of the standard (OGC Document 12-128r11). This was a corrigendum release that corrected a bunch of errors that were identified, particularly in the features section of the document.
• Revamped geopackage.org to provide more precise information, including better FAQs, sample data, implementations, and guidance
• Completed the migration of almost all SWG activities to GitHub (everything but internal SWG communication and meeting information).
• Completed the Tiles Plugfest (as documented in this Engineering Report) which gave us a good idea of what interoperability problems we had with the tiles portion of the standard.
• Made a number of specification changes towards getting Version 1.0.2 ready for release. The most important changes are:
• Clarifying the tiles portion of the standard
• Organizing all extensions under a single annex
• Demoting the Schema and Metadata sections to an extension
• Creating a new extension to support OGC's new standard for describing Coordinate Reference Systems
• Produced a draft specification for an Elevation Extension and initiated an Interoperability Experiment (GPKG-EE IE) to evaluate it
• Conducted a survey to gauge interest in potential extensions.
• Started the discussion regarding a potential extension for symbology and styling.

It has been a busy year and I doubt we will get this much done next year. We do have a few things on our radar for the first half of 2016:

• Releasing Version 1.0.2
• Completing the GPKG-EE IE and hopefully releasing the Elevation Extension as an adopted extension
• Releasing at least a Discussion Paper regarding a potential extension for symbology and styling

However, the main measure of success next year will be utilization. We believe the standard is in a good state and that it solves the problems it was intended to solve. We ask data providers to consider using GeoPackage as an alternative (if not a complete replacement) to other formats. We ask data consumers to consider using GeoPackages when they are available and to let us know when using them proves beneficial. The SWG is here to help but the rest is up to you.

Clean-up in Section 2

We made a mistake and it is time to clean it up. Two sub-sections in the GeoPackage Specification, Schema and Metadata, have failed to provide any measurable benefit to users. I believe they were added to the specification with the best of intentions. Alas, as far as I can tell GeoPackage providers have been leaving the tables blank and even when they have been populated, they were not being read by clients. In the end all we are doing is making more work for people by making the presence of these tables mandatory.

In response, the Standards Working Group (SWG) voted to relegate the two sections into extensions. We will not remove these sections entirely but they will no longer appear in Section 2. As described separately, we have already agreed to compile all OGC-approved extensions into a single Annex. These changes will shorten the core of the standard and make the whole document more organized. We hope that this will make it more palatable to implementers. I plan to do the reorganization over the course of the next two weeks.

Meanwhile, the SWG's work is not done on this topic. I would like to reach the point that implementers look for opportunities to support extensions like these that do exist. Emerging profiles such as the National System for GeoINT Profile will use them and that is a good start.
To get beyond that, we will need to make a stronger case for the community benefits of providing this information.

Powers-of-Two Scale Sets

When we built the GeoPackage standard, there was a big push to limit tile sets to powers-of-two zoom levels. This is how most tile-based web maps (Google Maps, OpenStreetMap, etc.) work and there is a certain wisdom to it from a simplicity standpoint. However, I pushed back on this limitation for three reasons:

1. Existing cartographic products are based on a fixed scale which is some round number. When a cartographer designs a map to look right at 1:100,000 or 1:25,000 or whatever, it does not make sense to rescale it to 1:136,495 or 1:34,124[1] or some other arbitrary rational number.
2. Image products require a lossy transformation process to convert the raw (or base) image to a scale that matches the tile set. This transformation reduces image quality. When someone goes through the effort of collecting imagery at 0.5m (or better), it is a shame for that content to be degraded unnecessarily before reaching the end user.
3. When people look at imagery, they are not interested in looking at the second-highest zoom level. It just doesn't happen - they either zoom out enough to get their bearings or they zoom in as far as they can. If your tile set is based on powers-of-two, your second-highest zoom level adds about 25% of waste.

It doesn't have to be this way. "Because it is easier for the developer" is not a good justification for reducing quality or wasting space. Besides, It is not that hard for an application to look up a tile set and provide an appropriate set of zoom levels. 

Alas, we compromised by relegating support for non-powers-of-two zoom levels to an extension. I will be interested to see what happens as organizations commit to distributing raster products (especially imagery products) via GeoPackage and the ramifications of the powers-of-two approach become more apparent.

---

[1] These values are for the equator; the values at ±30° latitude are about 3.5% lower.