2018 — Blog — open.gis.lab

Sorry I’m been on hiatus for a few months. But I’m back with a short post on adding and viewing geojson data in QGIS and ArcGIS Desktop. I was inspired to write this post after seeing a recent map in the New York Times that showed all the buildings in the U.S. Check out their interactive map of every building in the United States if you haven’t seen it already. It showcases the use of Microsoft (Bing) building footprints database for all 50 States. The NYT maps showed some interesting patterns of the urban landscapes in America which I’ve taken snapshots of and are shown here.

The Bing building footprints are open source and can be downloaded for use in geojson format - which is what this post is about. Once you download the geojson file, how do you view it? Anyway, I was interested in seeing what the building footprints look like for Hawaii as well as wanting to do a quick comparison of the Microsoft/Bing buildings outlines to the one that the City and County of Honolulu has and the OSM buildings extract. QGIS can natively open geojson file but it doesn’t always consistently work – at least for me, and I’m not sure why it works sometimes or doesn’t. You can try doing one of the following options below to add geojson to QGIS for viewing.

Adding a local geojson data in QGIS

If you have a geojson file on your local hard drive or network and want to view/use it in QGIS, you can just drag and drop it from the Browser Panel into the Layers Panel or just double click on the file will add it to the Layers Panel. If trying to add the original *.geojson file crashes the you can try changing the files extension to either *.json or even as *.geojson.txt. I’ve tried all 3 different file extensions and they all seem to work.

Adding a geojson data via URL

SEE ADDENDUM POST (02/01/2020): GOT GEOJSON ERROR WHEN ADDING VIA URL

You can also add geojson fiel via a web service URL or a local networked drive. Just click the Open Data Source Manager button >> Vector Data >> Protocol HTTP(S), cloud, etc. >> Select GeoJSON type >> URI path >> Add.

Adding and viewing geojson data in ArcGIS Desktop and ArcGIS Pro

ArcGIS desktop and Pro can also read geojson files but it’ll cost you a pretty penny ($$$) because you need the data interoperability extension. In this case I would stick with QGIS, but I listed the steps for ArcGIS below for your information.

Note: Viewing geojson file in ArcGIS Desktop and ArcGIS Pro requires a Data Interoperability Extension, and it needs to be activated or turned on first.

I have Data Interoperability extension so was able to just add the geojson file in ArcMap. I guess ArcGIS desktop with the Data Interoperability extension turned on it somehow automatically imported the geojson file in ArcMap for me. I tried to see if I can also view the geojson file in ArcGIS Pro but the file format was not recognized. With ArcGIS Pro you have to convert geojson file first to json or something else that ArcGIS Pro can use. ArcGIS Online will read geojson though. I’m using the most current version of ArcGIS Desktop 10.6.1 and ArcGIS Pro 2.2.3.

To add and view geojson file in ArcMap, just use the Add Data button as you would to add a shapefile or geodatabase feature class - like the example below.

Below is the building geojson file as viewed in ArcMap.

As I said earlier, ArcGIS Pro doesn’t recognize geojson. You’ll need to convert the file to a shapefile or feature class first below you can view it. You can follow the steps for converting JSON to Features tools in AG Pro.

Once you convert the geojson file then you can use it in Pro. This is how Bing’s building footprints look like in Pro.

Comparison of OSM, Microsoft/Bing, and City & County of Honolulu’s Building Footprints

A quick comparison of OSM, Bing, and City & County of Honolulu building footprints suggests that the local source (CCH) is the most accurate of the three followed by OSM, and then Bing. Also note that the Bing building footprints (at least this version of the data update) does not have height information. OSM building extracts can either have height info or not depending on the file format and source - for example shapefiles from Geofabrik doesn’t have this info but *osm.pbf has more info. CCH’s building footprints has height info, although it is an older dataset, captured between 2004-2009.

OSM building footprints (yellow polygons) overlaid on Bing Aerial

Microsoft/Bing building footprints (red polygons) overlaid on Bing Aerial

City and County of Honolulu building footprints (orange polygons) overlaid on Bing Aerial

I would say, always stick with your local source for building footprints but OSM and Bing building footprints will suffice for areas that don’t have data. Use whatever that will fit your purpose.

That’s all I have for now. Thanks for reading! :)

Part 2: QGIS Server and Lizmap

This post is a continuation of the previous Updated Installing Apache, Lizmap Web Client, and QGIS on Windows OS blog post. You need to have a GIS server set up first that uses the Lizmap Web Client. Then you use the Lizmap plugin in QGIS Desktop to help you publish the QGIS project as a web app to your GIS server.

For this tutorial, I assume you already have QGIS Desktop installed. I have both QGIS Desktop LTR 2.18.21 and 3.2 installed on my computer but for this tutorial I am using the LTR 2.18.21 version – as that is the version of my QGIS Server and that my Lizmap Web Client is using. The Lizmap plugin will also work on QGIS desktop 3.2 but is listed as “experimental” so I take it that it hasn’t been fully tested yet(?). For this tutorial, I assume you already know the basics of using QGIS Desktop.

The basic workflow is as follows:

Set up working project directory
Create and configure QGIS project for the web mapping
Install and configure Lizmap plugin
Publish QGIS project as web map to Lizmap Web Client
View web map online via the Lizmap Web Client

Part 1: Set up working project directory - optional

This part is optional but highly recommended for organizational purposes. Create a working project directory. For my set up I create a batch file (*.bat) to create my folders. You can of course also create your folder and subfolders manually. Because I have everything (QGIS Server, Lizmap Web Client, and QGIS desktop) all on my laptop for testing, I’m just going to put my webapp folder inside my webserver directory - most likely in reality your GIS Server would be on a separate server. Your folder structure can be whatever fits your needs.

1. I use a batch file to make my folders. You can create a batch file in a text editor (such as NotePad or NotePad++) to create these folders. Below is an example of my folders.

mkdir C:\webserver\webapp\common\
mkdir C:\webserver\webapp\document\
mkdir C:\webserver\webapp\document\svg\
mkdir C:\webserver\webapp\prod\
mkdir C:\webserver\webapp\prod\maprep\
mkdir C:\webserver\webapp\prod\maprep\data
mkdir C:\webserver\webapp\prod\maprep\data\vector
mkdir C:\webserver\webapp\prod\maprep\data\raster
mkdir C:\webserver\webapp\prod\maprep\media\
mkdir C:\webserver\webapp\prod\maprep\media\js\
mkdir C:\webserver\webapp\prod\maprep\media\image\
mkdir C:\webserver\webapp\preprod
mkdir C:\webserver\webapp\preprod\maprep\
mkdir C:\webserver\webapp\preprod\maprep\data
mkdir C:\webserver\webapp\preprod\maprep\data\vector
mkdir C:\webserver\webapp\preprod\maprep\data\raster
mkdir C:\webserver\webapp\preprod\maprep\media\
mkdir C:\webserver\webapp\preprod\maprep\media\js\
mkdir C:\webserver\webapp\preprod\maprep\media\image\

2. Save your *.bat file (e.g. createDir.bat) and then run it in the Command Prompt window.

In the Command Prompt window: cd into your directory that contains the *.bat file
Run the *.bat file

Part 2: Create and configure QGIS project

1. Create a new QGIS project - add your data layers, style and symbolize them. Save your project to your \webapp\preprod\maprep folder. I am working in my preprod folder or environment and I put everything for my project into their respective folders (created in above step)

Here is a look at my sample project.

2. Configure QGIS project properties. Go to File menu >> Project Properties

See Lizmap configuration for more info: https://docs.3liz.com/en/publish/project_for_web.html

General Tab: Use option Save Paths = relative ; fill or select other parameter option as you choose.

Note: For Project Scales you can enter manually or write them out into an *.xml file so it can be used for other projects. So example, here is my map scales in feet and as EPSG3857 (e.g. similar to Google Maps)

CRS tab: check the option to for enable on the fly CRS transformation and use WGS84 Pseudo Mercator as the project CRS

Identify Layers tab: Select the layers that you want identifiable features

OWS Server tab: Select Options
- Enable Service capabilities by checking the box
- Title - enter a title for your project (this will show up in Lizmap client)
- WMS Capabilities - check Advertised extent and Use Current Canvas Extent (this will use current map view); Add in CRS restriction if any
- WFS Capabilities - check layers you want published
- WCS Capabilities - check layers you want published (note: these are from external web services)
- Test Configuration - click Launch button and hopefully everything will be good
- Click OK and save your project

Part 3: Install and configure Lizmap plugin

This part just shows you some of the basic configurations. See Lizmap configuration documentation for more info: https://docs.3liz.com/en/publish/lizmap_configuration.html

1. In QIGS, go to Plugins menu >> Manage and Install Plugins. In the Plugins window, search for Lizmap then install the plugin

2. Configure Lizmap. Click on the Lizmap button (looks like a green globe) or go to Web menu >> Lizmap >> Lizmap

In the Lizmap window: There are many options and parameters. You can choose whatever you like.

See Lizmap configuration documentation for more info: https://docs.3liz.com/en/publish/lizmap_configuration.html

But here's the options I'm using for this example:

Layers tab options: Input metadata and options for each layer in your project

Baselayers tab options: input basemap layer options.

Locate by layer tab options: Here I am adding the Tax Map Parcel (TMK) layer, so I can search by TMK number.

Attribute table tab: Again here I am adding in the Tax Map Parcel layer so it's attribute table will be shown.

That's just some of the basic configuration set up I'm using for this example. After you are done with your set up, click Apply then Close. Also don't forget to save your QGIS project as well. After you apply and save the Lizmap configuration a configuration file (*.qgs.cfg) will be written with the same name of your project. Save your QGIS Project, and if you're happy with everything, then you can copy the project from your preproduction environmental to production for Lizmap web client to consume. For me that means copying everything from ../preprod/maprep to ../prod/maprep.

Thumbnail image for your web app

If you want a custom thumbnail image to be displayed for your web app in the Lizmap web client view PROJECTS page, you can do so by taking an image and naming it with *.qgs.png file extension. Note in my example, I have a *.qgs.png image with same name as my QGIS project, which is really a snapshot of my QGIS map canvas. This image (*.qgs.png) will be used as the thumbnail image of my project web app. The default image for a web project in Lizmap is a black and white world map. You must include the .qgs file extension to your filename and saved in the same location as your *.qgs.cfg and *.qgs files - like this example.

Part 4: Publish QGIS project as web app to Lizmap web client

This step is basically just pushing your QGIS projects to Lizmap web client. See the Lizmap Administration guide for more info: https://docs.3liz.com/en/admin/index.html

1. Copy everything from ../preprod/maprep to ../prod/maprep

2. Log into Lizmap web client's Administration dashboard. Go to http://localhost/admin.php and login

3. In the Lizmap Configuration tab: Create a New Repository. You can also remove the default demo and intranet projects if you'd like.

See link for more info: https://docs.3liz.com/en/admin/lizmap-configuration.html#repositories

ID: Id of your repository (e.g. oahumap). it seems like you can only use lowercase with no spaces or special characters
Label: Name of your repository (e.g. Demo Project)
Local Folder Path: path to your QGIS project (e.g. /webserver/webapp/prod/maprep/). Do not include Drive letter or use back slashes in your path.
Check the box to allow use of repository themes
Click Save - you will see then automatically be able to view the user group rights and check what each types of user group can do
Save your modifications

3. Optional - Theme settings: Go into the Theme settings and Modify options if you want for your web app.

If you wan to do more advanced theme creations, see advance lizmap configuration link: https://docs.3liz.com/en/publish/advanced_lizmap_config.html#creating-simple-themes

4. Go to the PROJECTS page (button in top right) and you should see your projects. Note you can have more than 1 projects in your repository - I have 2 projects in mine.

Here's what my web map example looks like.

That's basically it for a basic configuration and publishing of QGIS project to Lizmap web client. You'll notice in my web app that I have added a google-satellite basemap and an overview map - these are advance configurations which are explained below.

Step 5: Some Advance Configurations and Caveats

For more info on advance configuration of Lizmap plugin and web client see: https://docs.3liz.com/en/publish/advanced_lizmap_config.html

Adding an overview map

To add an overview map to your web app, you just add a group layer in QGIS with the named "Overview" (without quotes) - this will indicate to Lizmap web client to use it as the overview map. Then add your data layers to the group - in my example, I'm only using a stamen-toner web service in my overview map.

In QGIS, create a group layer and name it Overview
Add your data layers to your Overview Group

Adding basemaps for viewing and printing

Note: If you need help adding basemaps, see my posting on Adding Basemaps in QGIS

There are basically 2 ways you can add basemaps to your web app: 1) via Baselayers tab and 2) as a group or layer(s) in QGIS project

The first way is quick and easy way - you just check the basemap options (e.g. Google, OpenStreetmap, Bing, IGN (France) in Baselayers tab in the Lizmap plugin configuration window. However, adding basemaps via the Baselayers tab will not print as these basemaps are external to your QGIS project - see https://docs.3liz.com/en/publish/advanced_lizmap_config.html#allow-printing-of-external-baselayers.

To work around this, you need to add the basemap(s) as a group or layer in QGIS whose name is part of the following list - these corresponds with the external basemaps in the Baselayers tab of Lizmap plugin.

NOTE: Remember you still have to be in compliance with the licensing terms of all the basemaps you use.

osm-mapnik for OpenStreetMap
osm-mapquest for MapQuest OSM
osm-cyclemap for OSM CycleMap
google-satellite for Google Satellite
google-hybrid for Google Hybrid
google-terrain for Google Terrain
google-street for Google Streets
bing-road for Bing Road
bing-aerial for Bing Aerial
bing-hybrid for Bing Hybrid
ign-scan for IGN Scan
ign-plan for IGN Plan
ign-photo for IGN Photos

Below is the options I used in my example - also see snapshot.

In QGIS, I used the QuickMapServices plugin to add in these basemaps: Google Satellite, Open Topo Map, and ESRI Topo.
Rename Google Satellite to "google-satellite" (without quotes and to match the naming convention from list above). I left OpenTopoMap and ESRITopo as is since they're not in the list above)
Create a Group and call it "hidden". Then create sub-group named "osm-mapnik" which contains the OSM Standard basemap, and sub-group named "osm-stamen-toner" which contains the Stamen Toner Lite basemap
In Lizmap plugin configuration, Layers tab:
- For google-satellite, OpenTopoMap, and ESRITopo, check base layer? option (these are not part of the default Lizmap baselayers and I want to use them as Basemaps). Also check option for use Single Tile?
- For hidden groupings: No need to check option to use as base layers since these are hidden anyway and will only be used for printing purpose. Also these are already part of the default lizmap baselayers options and you don't want duplicate basemaps in your web app.
Check box to Hide checkboxes for Groups
Save your Lizmap configuration and QGIS Project

Printing basemaps in your web app in Lizmap web client viewer

Printing basempas in your web app via the Lizmap web client viewer is a bit tricky. First you need to have a print composition in your QGIS project and it needs to have at least 1 map in it and you also need to enable the printing function in Map options tab of the Lizmap plugin configuration window.

Caveat

Printing basemaps in Lizmap may not always work. One big caveat is, you cannot have text labels in your print composition - otherwise you will get an "Internal Server Error" and your map will not print. It seems to me that the printing of basemaps can sometimes be finicky. I've gotten errors even after I've removed text labels in my print compositions - something about the basemap layer not being found or define (e.g. Layer "osm-mapnik" not found). What I did for this error was to clear my cache and log off and log back into my Lizmap Administration page - this worked for me.

I'm not sure if printing basemap issues are from Lizmap's end or QGIS Server. I've seen on some forums, suggestions for installing a fake X server but that's for Linux and/or Mac OS - I don't think that's an option for Windows OS. This makes me think it's a QGIS Server issue, but I don't know. By the way, printing of basemaps in the QGIS project works fine - it's just the printing in the web app that can be a problem.

Here's an example of my print composition. It is a simple A4 landscape layout. Create your print compositions and save your project, then in the web app you can select the layout to use as template for printing.