Clean-up in Section 2

We made a mistake and it is time to clean it up. Two sub-sections in the GeoPackage Specification, Schema and Metadata, have failed to provide any measurable benefit to users. I believe they were added to the specification with the best of intentions. Alas, as far as I can tell GeoPackage providers have been leaving the tables blank and even when they have been populated, they were not being read by clients. In the end all we are doing is making more work for people by making the presence of these tables mandatory.

In response, the Standards Working Group (SWG) voted to relegate the two sections into extensions. We will not remove these sections entirely but they will no longer appear in Section 2. As described separately, we have already agreed to compile all OGC-approved extensions into a single Annex. These changes will shorten the core of the standard and make the whole document more organized. We hope that this will make it more palatable to implementers. I plan to do the reorganization over the course of the next two weeks.

Meanwhile, the SWG's work is not done on this topic. I would like to reach the point that implementers look for opportunities to support extensions like these that do exist. Emerging profiles such as the National System for GeoINT Profile will use them and that is a good start.
To get beyond that, we will need to make a stronger case for the community benefits of providing this information.

Powers-of-Two Scale Sets

When we built the GeoPackage standard, there was a big push to limit tile sets to powers-of-two zoom levels. This is how most tile-based web maps (Google Maps, OpenStreetMap, etc.) work and there is a certain wisdom to it from a simplicity standpoint. However, I pushed back on this limitation for three reasons:

1. Existing cartographic products are based on a fixed scale which is some round number. When a cartographer designs a map to look right at 1:100,000 or 1:25,000 or whatever, it does not make sense to rescale it to 1:136,495 or 1:34,124[1] or some other arbitrary rational number.
2. Image products require a lossy transformation process to convert the raw (or base) image to a scale that matches the tile set. This transformation reduces image quality. When someone goes through the effort of collecting imagery at 0.5m (or better), it is a shame for that content to be degraded unnecessarily before reaching the end user.
3. When people look at imagery, they are not interested in looking at the second-highest zoom level. It just doesn't happen - they either zoom out enough to get their bearings or they zoom in as far as they can. If your tile set is based on powers-of-two, your second-highest zoom level adds about 25% of waste.

It doesn't have to be this way. "Because it is easier for the developer" is not a good justification for reducing quality or wasting space. Besides, It is not that hard for an application to look up a tile set and provide an appropriate set of zoom levels. 

Alas, we compromised by relegating support for non-powers-of-two zoom levels to an extension. I will be interested to see what happens as organizations commit to distributing raster products (especially imagery products) via GeoPackage and the ramifications of the powers-of-two approach become more apparent.

---

[1] These values are for the equator; the values at ±30° latitude are about 3.5% lower.

Extensions Survey

Over the summer, we conducted an online survey to get a feel for what potential extensions to GeoPackage people were most interested in. The candidate extensions came from a wide range of discussions with people inside and outside of OGC. The survey was anonymous and open to the public. The timestamps suggest that we got participation from across the globe.

The survey had three parts[1]. 

1. Whether the participant was more on the builder side (developer/integrator) or user side (administrator/user) of things
2. The attitude (strongly support, mildly support, neutral, or oppose) for each extension
3. Top three favorites.

I produced the chart below in mid-August[2] and the results speak for themselves. 

We plan to address symbology and styling next[3]. Our next step is to gather input on the desired use cases for this extension. This will be captured in a discussion paper that will be presented to OGC at an upcoming Technical Committee meeting. If you have any thoughts on this topic, let us know! If you comment here, over email, or on Twitter, I will be sure to see it.

---

[1] If I had a chance to do it over again, I would allow participants to provide free-form comments.
[2] The poll is still out there and responses continue to trickle in but it doesn't look like they will change the overall results significantly.
[3] Meanwhile, we are going to evaluate whether our recently published guidance on feature data is sufficient to cover the multi-resolution feature data case.

One Geometry per Table

Note: This post was mostly written by another GeoPackage SWG member, Thomas Neirynck of Luciad. Any editing mistakes are mine.

The GeoPackage encoding standard mandates in Requirements 22 and 30 that vector feature user tables shall only have a single geometry. This is advantageous for two reasons:

• It keeps the semantics of the data model concise: a feature has one shape. This is in line with other vector data formats with which GeoPackage is often compared, such as Shapefile or GeoJSON. By sharing this restriction, converting data from one format to another is more straightforward. This is particularly beneficial in enterprise workflows. For example, mobile components use a GeoPackage to view the business data, while the web-clients use a GeoJSON version of the same.
• Allowing multiple geometries per feature table would compromise GeoPackage's position in the GIS application ecosystem. Most ready-made GIS data viewers handle data formats in a uniform manner. Opening a file creates a layer on map and the layer shows the features contained in that file. There are usually some UI-capabilities to view properties of a feature or change the styling of a feature. If feature tables were to contain multiple geometries, the visualization of a GeoPackage in such a COTS-viewer would be much more complicated. What geometry should be shown on the map? All of them? Only the first? Editing tools would create a similar set of problems.

This restriction should not be seen as a limitation. Since a GeoPackage is a relational database, users can create rich data models to describe the required scenarios. Instead of adding more geometry columns to a single table, the general approach is to extract these geometries into a separate table, creating a one-to-many relationship between the geometries and the feature.

Because of the feedback we received on this topic, we recognized that we needed to produce some guidance to aid developers in producing compliant GeoPackages that handle non-trivial scenarios. This guidance has been published on geopackage.org. As always, we encourage public input on the specification itself and anything else that we can do to make things easier for the community.

Elevation Extension

Currently the GeoPackage Standards Working Group (SWG) is developing an extension to support the storing of elevation data in support of visualization and analysis operations[1]. We see this as a good thing because without it, there is no interoperable way to exchange elevation data in SQLite[2]. In the interest of simplicity, we are basing the design on the existing tile model. We use PNG files to store the elevation values as unsigned integers and there is an additional extension that supports the use of TIFF files to store floating point values.

So how does this extension stack up in my list of things that make a good extension?

• Clear use case? Check.
• Sound technical approach? We believe so.
• A commitment to implementation by multiple implementers? In progress...

Our next step is to conduct an interoperability experiment[3] to investigate the degree to which implementers are able to produce GeoPackages that conform to the extension and their software is able to use compliant GeoPackages effectively. We are in the planning stages now and are looking for participants. We plan to do the bulk of the work in January and February and present the results at the OGC Technical Committee Meeting in March 2016. 

If this experiment is successful, it proves both the technical approach and the commitment to implementation. At that point, the SWG will be ready to adopt the new extension.

---

[1] The ongoing work is available at: https://github.com/opengeospatial/geopackage-elevation
[2] As with any other extension, if a client does not conform with it, it will not see the elevation data but it will be able to see the features and tiles as it always has.
[3] See the Activity Plan

Coordinate Reference Systems

It is no secret. Coordinate reference systems (CRSs) are a constant issue in the geospatial community. Unfortunately they are a fact of life for many of us[1]. The GeoPackage standard requires that the CRSs in use be defined[2]. The standard encoding for CRS definitions that was available at the time was included in OGC 01-009 but that one is pretty dated and it has been causing problems in the field. In response, OGC recently adopted 12-063r5 to bring things up to date. The problem is getting people to start using it.

It is a classic chicken and egg problem. Consumers do not want to build support for data that isn’t being produced yet. Providers don’t want to produce data that consumers can’t read. Naturally, we’ve been asked to break the cycle. The traditional approach is to issue a new standard version that cites the new document but as mentioned previously, this is something we’re trying to avoid.
 
Our solution is to create another extension and to let a specific domain try it out. In this case it is the US National System for GeoINT (NSG) which represents the US defense and intelligence community. NSG has been working on a profile for GeoPackage which constrains and extends the standard. Among other things, the profile will contain an extension that updates the reference for CRS definitions from 01-009 to 12-063r5. Systems designed to work in this domain will be mandated to support the extension but others can adopt it at their own pace.

---

[1] For example, NGA does not support the use of Web Mercator and has developed other CRSs for use in tiles.
[2] There are three defaults, one for EPSG::4326, one for undefined geographic systems, and one for undefined cartesian systems.

Core and Extensions Model

The GeoPackage specification is organized using a core and extensions model. The core represents the “least common denominator” that all compliant systems must conform to. It is fairly compact; there are two required metadata tables (contents and coordinate reference systems), three optional metadata tables (one for extensions and two for user-defined metadata), and two core content types (features and tiles, at least one of which must be present).

When existing GeoPackage capabilities are insufficient, new ones can be added through extensions. This allows systems to satisfy additional requirements in an interoperable way without adversely affecting the other systems which do not. If a system reading the GeoPackage recognizes an extension then it can use the associated data as intended. If not, the associated data is ignored. When in doubt, system developers should stick to the core because arbitrary systems are only expected to conform to it.

Extensions may be developed inside or outside the SWG and the standard describes how this is done. When considering whether the SWG should approve a new extension, I look for three things:

• Clear use case
• Sound technical approach that does not adversely affect the core
• A commitment to implementation by multiple implementers 

These things keep us focused on producing extensions that practical, relevant, and interoperable.

---

[1] Extensions that are approved by OGC get a name with the “gpkg_” prefix.