Tuesday, September 29, 2015

Core and Extensions Model

The GeoPackage specification is organized using a core and extensions model. The core represents the “least common denominator” that all compliant systems must conform to. It is fairly compact; there are two required metadata tables (contents and coordinate reference systems), three optional metadata tables (one for extensions and two for user-defined metadata), and two core content types (features and tiles, at least one of which must be present).

When existing GeoPackage capabilities are insufficient, new ones can be added through extensions. This allows systems to satisfy additional requirements in an interoperable way without adversely affecting the other systems which do not. If a system reading the GeoPackage recognizes an extension then it can use the associated data as intended. If not, the associated data is ignored. When in doubt, system developers should stick to the core because arbitrary systems are only expected to conform to it.

Extensions may be developed inside or outside the SWG and the standard describes how this is done. When considering whether the SWG should approve a new extension, I look for three things:

  • Clear use case
  • Sound technical approach that does not adversely affect the core
  • A commitment to implementation by multiple implementers 
These things keep us focused on producing extensions that practical, relevant, and interoperable.


[1] Extensions that are approved by OGC get a name with the “gpkg_” prefix.

Wednesday, September 23, 2015

Specification Stability vs. Maintenance

Changing a specification is disruptive because software developers have to make changes and systems need to be retested to ensure that everything still works as intended. Some organizations do not have the budget or schedule flexibility to do this sort of thing properly and in a timely manner. With this in mind, substantive changes to the core are to be avoided whenever possible. If it is up to me, there will be no 2.0 release of GeoPackage and I hope to avoid a 1.1 release (and certainly a 1.3 release) as well.

Like it or not, standards do need to be maintained.
The GeoPackage Standards Working Group (SWG) uses the corrigenda[1] process to make corrections to the standard. We continue to review and inspect it to ensure that it is as clear as possible. We do as much as possible in public, using GitHub Issue Tracker to manage issues and relying on public feedback. For each issue, we review the passages in question, come to consensus on their intent, and rewrite them as needed.

A few months ago, we released a corrigendum that focused on the features portion of the standard. We are in the process of producing a second corrigendum that will focus on tiles[2]. We believe this second corrigendum will further improve the readability of the standard and reduce the risk of misinterpretation. While the formal PDF document is not ready yet, the changes that we have already agreed upon are reflected in the on-line specification.

[1] In printed works, a corrigendum is an insert listing errors and corrections. In the web-based world, that is a bit of an anachronism but at a minimum we need a way to fix errors, make clarifications, and show what has been updated.
[2] The GeoPackage Plugfest was an important part of this process because it identified parts of the standard that were not interpreted consistently.

Wednesday, September 16, 2015

GeoPackage.org and GitHub

Start at geopackage.org. That is my message every time I present about GeoPackage. GeoPackage is the first OGC project with its own home page. (The old structure was really only good for finding downloadable versions of the specification documents.) The new site provides access to most project-related resources including:
  • A web-based version of the standard
  • Implementations
  • Sample data
  • Relevant hyperlinks for partipants
  • Frequently Asked Questions
Meanwhile, we have migrated most of our work to GitHub:
  • The AsciiDoc of the specification itself [1]
  • Issue Tracker
  • GitHub Pages (for maintaining geopackage.org itself)[2]
We have had good results using the common GitHub workflows. The issue tracker has allowed much more public input than was ever possible before. It is much easier to manage changes to  geopackage.org through pull requests. The processes for managing the specification itself are still being improved but at least there is a short turnaround between a SWG decision and the corresponding update to the online specification. 

There are just a few things that remain on the old OGC site:
  • Email lists (one public, one for SWG members only)
  • Portal (for hosting internal files like presentations and for files that are too big to host on GH Pages)
  • Wiki (now used primarily for meeting agenda, minutes, and a detailed change log)

[1] A back end process produces HTML from the specification AsciiDoc so that changes to the AsciiDoc are immediately reflected in the on-line version of specification.
[2] Via the gh-pages branch (particularly index.html) in the https://github.com/opengeospatial/geopackage repository

Wednesday, September 9, 2015

GeoPackage Vision

I am regularly asked about the vision for GeoPackage. For example, is it a simple container or does it have any higher level semantics? While the GeoPackage Standards Working Group (SWG) has not produced a formal vision statement, I believe something like the following would be appropriate:
The GeoPackage SWG develops and maintains the GeoPackage Encoding Standard as the standard for interoperable exchange of geospatial data and information using SQLite.
GeoPackage is simply a schema for data stored in an SQLite database. Note that I made no explicit mention of mobile here. GeoPackage is not just a standard for mobile! While mobile is clearly a major thrust of GeoPackage, it is not the only one. The great thing about SQLite is that it can be used in nearly every modern computing environment from a mobile device to the desktop up to the cloud. 

Because GeoPackage is so flexible, we envision it being used in a wide variety of ways. We foresee GeoPackage being a partial replacement[1] for the Shapefile format[2]. Organizations within the US government are exploring ways to distribute geospatial data assets in the format. As organizations continue to adopt GeoPackage, they will continue to find new and innovative uses for it. Some of them will be mobile-based and some will not.


[1] I don’t want to overstate this point. I use the term “partial” because GeoJSON is a perfectly suitable replacement for Shapefile for many scenarios. Both formats overcome many of Shapefile’s limitations. GeoPackage is more complex than GeoJSON but it also provides a number of additional capabilities that some scenarios demand.
[2] See the Shapefile Challenge at http://geometalab.tumblr.com/post/119804340842/the-shapefile-challenge

Wednesday, September 2, 2015

Introduction

GeoPackage is an Open Geospatial Consortium (OGC) standard for storing geospatial data and information in an SQLite database. The GeoPackage specification describes a set of conventions for storing the following content types:

  • vector features
  • tile matrix sets (pyramids) of imagery and raster maps at various scales
  • schema
  • metadata
  • extensions


I was named the chairman of the GeoPackage Standards Working Group (SWG) about six months ago. The more I get acclimated to this role, the more I realize that this initiative will only be successful if it is embraced by the geospatial community at large (not just OGC, its members, or their customers). For that to happen, I need to be more active in reaching out. This blog is part of that effort.

In this blog, I will share my thoughts about where I think GeoPackage needs to go and what the SWG is doing to make that happen. While this content is my own, I do intend for it to reflect the consensus positions the SWG has reached. My style is to keep things brief, direct, and to the point so posts will be roughly the length of this one. I plan to post something about once a week for the next couple of months and then taper off to about once a month.