Intelligent Routing LogoIntelligent Routing · Blog

Responsive Tableau Geocoding for South African Municipal Wards

At Intelligent Routing, we use Tableau for data exploration and reporting. Since our customers need to visualize data with a geographical component in Africa, we often have to build geocoded visualizations that are not available in the default Tableau installation.

At last week’s Johannesburg Tableau User Group, a question was asked on how to use custom Tableau Geocoding with shape data. Specifically: How can we help Tableau to stay responsive when visualizing municipal areas without reverting to circles located at shape centroids. If you are geocoding US data in Tableau, you are in luck with embedded shapes that can be linked to geocodes with a high level of detail. For the South African context, you need a bit of data processing and simplification to get a smooth user experience with the default polygon-shaded maps.

Tableau Geocoding with Shapes

In this post, we build a Tableau visualization of voter population in South Africa’s wards, linked to the geographical shape of each ward. We make sure that the visualization stays responsive without hiding too much detail when zooming down to 1:50000 scale.

Here is a static copy of the final visualization:

Voter Population Per Municipal Ward in South Africa

Map of South Africa’s voting population (2011) for each municipal ward.

To get an idea of the amount of underlying data, here is the same visualization, but colored by discrete ward identifier:

Wards Colored By Ward ID

When visualized in this manner, the amount of detail in the dataset becomes clear.

Now that we know where we are heading, let’s get going.

Summary of Steps

  1. Find and download your South African shape-file,
  2. Explore the file in QGIS to get an idea of data,
  3. Simplify the shape file to desired level (topojson toolchain)
  4. Evaluate simplification results in QGIS
  5. Export CSV version (MMQGIS plugin)
  6. Add an order column to the CSV
  7. Create and join your Tableau Visualization
  8. Rinse, repeat, enjoy

Finding Open Data Shape Files for South Africa

It is good to see that the South African government has many initiatives that drive open data. Data that was gathered with public money is becoming more accessible. This is great news for startups and small companies that can’t afford traditional GIS related fees. Open data is especially well established for voting areas and ward demarcations. There is a wealth of information available at the Municipal Demarcation Board of South Africa. In the downloads section, one can find a shape file that details all wards in South Africa, including figures of the registered voting population within each ward. Other levels of detail are also available on the site. For this post we will focus on ward level shape file available at http://www.demarcation.org.za/index.php/downloads/boundary-data/boundary-data-main-files/wards/11457-wards/file. Download the Wards.zip file and extract it into its own directory. There will be 7 files of which one is Wards2011.shp which can be opened in QGIS.

Another good starting point to find South African GIS base data is at the CSIR geoportal.

QGIS

QGIS allows you to open and explore the shape file. Once QGIS is installed (version 2.8 in this post) click on the add vector layer icon QGIS Add Vector Layer in the top left corner. This adds the layer in the layer panel. Double-click the layer to change the marker style and select transparent fill. This will allow us to overlay the simplified layers in later steps and choose the appropriate simplification. To zoom to an exact scale, choose it from the drop-down on the status bar. You can also see the latitude and longitude in the status-bar. If the lat-long does not match your expectation, check that the projection of the project is set to EPSG4328.

QGisTransparentLayer1to50000PretoriaEast

Ward transparent layer zoomed to 1:50000 in the east of Pretoria.

We will use the layer above when checking our simplifications to confirm that we don’t loose too much detail.

Easy does it, almost: Using Builtin Simplify Geometries and Why We can do Better

QGIS 2.8 has some powerful toolboxes that come with the standard installation. Menu items are available that allow you to run powerful external tools with the defaults selected. One of these menu items can be found under Vector–>Geometry–>Simplify Geometries…SimplifyGeometriesMenu

When the simplification is run with a tolerance of 0.001 degrees we save 95% on the number of nodes, which would be very handy in speeding up the Tableau visualization.

Simplifying geometries at 0.001 tolerance result in 95% savings on nodes.

But if we inspect the layer in QGIS, we can clearly see the problem with the default simplification that is not topology aware. This is discussed in this GIS Stackexchange question. Each ward is simplified on an independent basis which results in gaps (silvers) between the shapes. To see the effect more clearly, set the simplified shape layer colors based onward number.

Simplified Geometries at 0.001 Tolerance Colored Original Wards Overlaid

At 1:50000 one can see that the wards still follow the original ward lines relatively well. But if we hide the original transparent layer (as will be the case in the final Tableau viz), those gaps become quite prominent. If we set the tolerance down to 0.01 degrees, the effect becomes very pronounced.

Simplified Geometries at 0.01 Tolerance Colored

We can do better (with some help).

A tool for Topologically Aware Simplification: topojson

Mike Bostok, the author of D3.js (and many splendid visualizations based on it) has written a tool that solves exactly this problem. The topojson utility simplifies maps while keeping the topological integrity as far as possible. For this post, install both the node topojson package and, since QGIS 2.8 only understands geojson, the topo-geo-json-converter package.

A script is worth a thousand words (if it leads to a visualization). So here is the list of commands run from the Wards2011 directory to get to an 8% simplified version of the original.

$ mkdir topojson
$ mkdir geojson
$ topojson –no-quantization -p –simplify-proportion 0.08 -o topojson/sa_wards0.08.json — Wards2011.shp \
&& topojson-geojson topojson/sa_wards0.08.json \
&& mv Wards2011.json geojson/sa_wards0.08.geojson

bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 7.80e-10
simplification: retained 149815 / 1872666 points (8%)
prune: retained 14700 / 14702 arcs (100%)

After running these, the directory structure is as below.

 [~/git/gis/Wards2011]$ tree
.
├── Wards2011.dbf
├── Wards2011.prj
├── Wards2011.sbn
├── Wards2011.sbx
├── Wards2011.shp
├── Wards2011.shp.xml
├── Wards2011.shx
├── geojson
│   └── sa_wards0.08.geojson
└── topojson
└── sa_wards0.08.json

2 directories, 10 files

How to determine the percentage that works best? Amongst some more precise methods based on scale and maximum zoom level, the topojson reference mentions: When in doubt, try it and see!

In this spirit, we can combine the commands into a short script for the command-line to generate some simplification percentages that we can then evaluate in QGIS:

Short script to try out different levels of simplification.

[~/git/gis/Wards2011]$ for i in 01 02 03 04 05 06 07 08 09 10 15 20 25; do
  echo "$i%:" \
  && time topojson –no-quantization -p –simplify-proportion 0.$i -o topojson/sa_wards0.$i.json — Wards2011.shp \
  && time topojson-geojson topojson/sa_wards0.$i.json \
  && mv Wards2011.json geojson/sa_wards0.$i.geojson
done

01%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 0.000145
simplification: retained 29405 / 1872666 points (2%)
prune: retained 14691 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.12s user 0.73s system 98% cpu 12.059 total
topojson-geojson topojson/sa_wards0.$i.json 0.28s user 0.04s system 97% cpu 0.328 total
02%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 1.91e-7
simplification: retained 37456 / 1872666 points (2%)
prune: retained 14691 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.46s user 0.75s system 100% cpu 12.209 total
topojson-geojson topojson/sa_wards0.$i.json 0.30s user 0.04s system 97% cpu 0.350 total
03%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 2.73e-8
simplification: retained 56181 / 1872666 points (3%)
prune: retained 14694 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.47s user 0.74s system 99% cpu 12.282 total
topojson-geojson topojson/sa_wards0.$i.json 0.39s user 0.05s system 97% cpu 0.452 total
04%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 8.88e-9
simplification: retained 74908 / 1872666 points (4%)
prune: retained 14694 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.20s user 0.67s system 99% cpu 11.933 total
topojson-geojson topojson/sa_wards0.$i.json 0.44s user 0.05s system 98% cpu 0.502 total
05%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 3.91e-9
simplification: retained 93635 / 1872666 points (5%)
prune: retained 14697 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.19s user 0.60s system 100% cpu 11.745 total
topojson-geojson topojson/sa_wards0.$i.json 0.51s user 0.06s system 98% cpu 0.577 total
06%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 2.04e-9
simplification: retained 112361 / 1872666 points (6%)
prune: retained 14699 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.36s user 0.66s system 100% cpu 11.937 total
topojson-geojson topojson/sa_wards0.$i.json 0.59s user 0.07s system 98% cpu 0.669 total
07%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 1.21e-9
simplification: retained 131088 / 1872666 points (7%)
prune: retained 14700 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.45s user 0.62s system 100% cpu 11.969 total
topojson-geojson topojson/sa_wards0.$i.json 0.64s user 0.07s system 98% cpu 0.718 total
08%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 7.80e-10
simplification: retained 149815 / 1872666 points (8%)
prune: retained 14700 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.38s user 0.63s system 100% cpu 11.984 total
topojson-geojson topojson/sa_wards0.$i.json 0.70s user 0.07s system 98% cpu 0.785 total
09%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 5.35e-10
simplification: retained 168541 / 1872666 points (9%)
prune: retained 14700 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.24s user 0.61s system 100% cpu 11.796 total
topojson-geojson topojson/sa_wards0.$i.json 0.79s user 0.09s system 98% cpu 0.890 total
10%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 3.81e-10
simplification: retained 187268 / 1872666 points (10%)
prune: retained 14702 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.21s user 0.61s system 100% cpu 11.756 total
topojson-geojson topojson/sa_wards0.$i.json 0.81s user 0.10s system 98% cpu 0.928 total
15%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 1.10e-10
simplification: retained 280901 / 1872666 points (15%)
prune: retained 14702 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.57s user 0.66s system 100% cpu 12.191 total
topojson-geojson topojson/sa_wards0.$i.json 1.14s user 0.12s system 97% cpu 1.285 total
20%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 4.55e-11
simplification: retained 374534 / 1872666 points (20%)
prune: retained 14702 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.78s user 0.61s system 100% cpu 12.276 total
topojson-geojson topojson/sa_wards0.$i.json 1.46s user 0.14s system 96% cpu 1.662 total
25%:
bounds: 16.45189000000005 -34.83417000299994 32.944984945000044 -22.125030057999936 (spherical)
topology: 14702 arcs, 1872666 points
simplification: effective minimum area 2.31e-11
simplification: retained 468168 / 1872666 points (25%)
prune: retained 14702 / 14702 arcs (100%)
topojson –no-quantization -p –simplify-proportion 0.$i -o — Wards2011.shp 11.72s user 0.64s system 100% cpu 12.301 total
topojson-geojson topojson/sa_wards0.$i.json 1.74s user 0.17s system 97% cpu 1.960 total

There is no need to decide now, but from topojson‘s output seen in the script above, we see a small jump between 09% and 10%. At 10% all arcs are retained, thus hinting in this direction.

Evaluate the Simplified Layers in QGIS

After running the script the directory geojson now contains a list of files that can be opened and evaluated in QGIS.

Directory structure after running the script.

 [~/git/gis/Wards2011]$ tree
.
├── Wards2011.dbf
├── Wards2011.prj
├── Wards2011.sbn
├── Wards2011.sbx
├── Wards2011.shp
├── Wards2011.shp.xml
├── Wards2011.shx
├── geojson
│   ├── sa_wards0.01.geojson
│   ├── sa_wards0.02.geojson
│   ├── sa_wards0.03.geojson
│   ├── sa_wards0.04.geojson
│   ├── sa_wards0.05.geojson
│   ├── sa_wards0.06.geojson
│   ├── sa_wards0.07.geojson
│   ├── sa_wards0.08.geojson
│   ├── sa_wards0.09.geojson
│   ├── sa_wards0.10.geojson
│   ├── sa_wards0.15.geojson
│   ├── sa_wards0.20.geojson
│   └── sa_wards0.25.geojson
└── topojson
    ├── sa_wards0.01.json
    ├── sa_wards0.02.json
    ├── sa_wards0.03.json
    ├── sa_wards0.04.json
    ├── sa_wards0.05.json
    ├── sa_wards0.06.json
    ├── sa_wards0.07.json
    ├── sa_wards0.08.json
    ├── sa_wards0.09.json
    ├── sa_wards0.10.json
    ├── sa_wards0.15.json
    ├── sa_wards0.20.json
    └── sa_wards0.25.json

 

In QGIS we can add the geojson files as vector layers and choose the one that fits our needs. To quantify the differences visually it is good to keep a copy of our original transparent layer handy. Here is a quick summary how to do it.

Show the reference layer line with map unit width

  • Layer Style Properties - Wards2011 Layer Style

    Doubleclick the Wards2011 layer to get to the Layer Properties and choose Style to set the single symbol simple fill style.

  • Layer Syle - Simple fill - Colors Fill Transparent and Border Soft

    Keep the transparent fill, but choose a soft color (baby blue will do for now).

  • Layer Style - Border width set to 0.001 Map unit - which equates to about 112 meters in SA

    In order to evaluate each simplification we’ll set the line width of the original layer to map units of 0.001 degrees which equates to about 112m in Gauteng, South Africa. Since the line width is in map units, it will scale with the map and one can more clearly visualize when parts of the simplified map fall outside this boundary. If you are unsure about the width in your part of the world you can measure the resulting width/height with the measurement tool to check it.

  • Layer Style - Layer blending mode set to Multiply

    With a blend mode of Multiply, the layers below are not hidden, but rather their color is multiplied by the color in this layer (that’s why it is a good idea to choose a soft color, since it won’t overwhelm the underlying layers when multiplied).

We are now ready to compare layers (with fill set to transparent) below the full detail layer. Choosing a relatively detailed part of our original area and zooming in to 1:10000 reveals that for a 55m margin the 10% simplification hardly ever cuts a and 8% is not much worse.

Original compared to 10 and 25 percent simplification

Original compared to 8 percent simplificationThe first image compares the original layer to 10% and 25% simplification. In the second image we see that 8% still cuts minimal corners in this snapshot when compared to a +-100 m wide line of the original.

From here it seems that we could go with the 8% simplifications and allow our users of the final Tableau visualization to zoom down to 1:10000 scale without distorting their view of the wards too much. Another handy way to check the effect of simplification at different scales is to set the original layer to dark and overlay it on categorized fill colors.

Check with categorized fill colors on the simplified layer

  • Layer Style Properties - Wards2011 Layer Style

    Doubleclick the Wards2011 layer to get to the Layer Properties and choose Style.

  • Categorized coloring

    Instead of choosing Single Symbol, choose Categorized.

  • [Categorized coloring by Column WARDNO

](/wp-content/uploads/2015/06/Categorized-coloring-by-Column-WARDNO.png) In the column field choose WARDNO and click on the Categorize button.

This, together with the original layer gives an opportunity to check the simplification at different scales. Here is the 5% simplification.

Original compared to 5 percent with fill colors

If we set the original layer back to a dark overlay and zoom to 1:50000 in our original area, we can see that 8% is nearly perfect in coloring the areas given the boundary. 5% takes some shortcuts but is not too bad either. To quantify “not too bad” we can zoom out to 1:250000.

Original 1 to 50000 compared to 8 percent with filled colors

8% simplified at 1:50000

Original 1 to 50000 compared to 5 percent with filled colors

5% simplified at 1:50000

At 1:250 k scale the 5% simplifications are nearly lost in the pixel averages and for 1:500 k and 1:1 M scales, they become imperceptible.

Original compared at 250000 scale to 5 percent with filled colors zoomed to pixel level

Original compared at 2500000 scale to 5 percent with filled colors zoomed to pixel level.

 

Original compared at 5000000 scale to 5 percent with filled colors zoomed to pixel level

Original compared at 5000000 scale to 5 percent with filled colors zoomed to pixel level.

Original compared at 10000000 scale to 5 percent with filled colors zoomed to pixel level

Original compared at 10000000 scale to 5 percent with filled colors zoomed to pixel level

 

A CSV file via the MMQGIS Plugin

Since 8% gives a good approximation down to 1:50000 scale, that seems like a good choice. Since we want to see the Tableau Visualization responsiveness it is worthwhile to export a few versions and try them out. For the conversion to CSV we use the MMQGIS Plugin.

MMQGIS Export Geometry to CSV Menu Item  MMQGIS Export Geometry to CSV

After exporting 8%, 5% and 1%(to check speed gains in Tableau) the directory structure looks like this:

Directory structure after exporting the csv.

[~/git/gis/Wards2011]$ tree
.
├── Wards2011.dbf
├── Wards2011.prj
├── Wards2011.sbn
├── Wards2011.sbx
├── Wards2011.shp
├── Wards2011.shp.xml
├── Wards2011.shx
├── csv
│   ├── sa\_wards0.01\_attributes.csv
│   ├── sa\_wards0.01\_nodes.csv
│   ├── sa\_wards0.05\_attributes.csv
│   ├── sa\_wards0.05\_nodes.csv
│   ├── sa\_wards0.08\_attributes.csv
│   └── sa\_wards0.08\_nodes.csv
├── geojson
│   ├── sa_wards0.01.geojson
│   ├── sa_wards0.02.geojson
│   ├── sa_wards0.03.geojson
│   ├── sa_wards0.04.geojson
│   ├── sa_wards0.05.geojson
│   ├── sa_wards0.06.geojson
│   ├── sa_wards0.07.geojson
│   ├── sa_wards0.09.geojson
│   ├── sa_wards0.10.geojson
│   ├── sa_wards0.15.geojson
│   ├── sa_wards0.20.geojson
│   └── sa_wards0.25.geojson
└── topojson
    ├── sa_wards0.01.json
    ├── sa_wards0.02.json
    ├── sa_wards0.03.json
    ├── sa_wards0.04.json
    ├── sa_wards0.06.json
    ├── sa_wards0.07.json
    ├── sa_wards0.08.json
    ├── sa_wards0.10.json
    ├── sa_wards0.15.json
    ├── sa_wards0.20.json
    └── sa_wards0.25.json

3 directories, 39 files

This step can be automated via a script, and automation would probably be necessary if you have a lot of files to convert. For now we will leave it at some extra clicks. The files are almost ready for Tableau.

One last step is needed, though, before we can do the Tableau Visualization.

Adding Render Order to the csv Files

Since Tableau needs render order for the polygons, we will need to post-process the csv node files. This can be done in Excel®, but I prefer to do it with awk on the command line.

Scripts to add render order to the csv, this can be done in a spreadsheet.

[~/git/gis/Wards2011/csv]$ awk 'NR<=1 {printf("RenderOrder,%s\n",$0)} NR>1 {printf("%s,%s\n",NR-1, $0)}' sa_wards0.01_nodes.csv > sa_wards0.01_nodes_with_renderorder.csv
[~/git/gis/Wards2011/csv]$ awk 'NR<=1 {printf("RenderOrder,%s\n",$0)} NR>1 {printf("%s,%s\n",NR-1, $0)}' sa_wards0.05_nodes.csv > sa_wards0.05_nodes_with_renderorder.csv
[~/git/gis/Wards2011/csv]$ awk 'NR<=1 {printf("RenderOrder,%s\n",$0)} NR>1 {printf("%s,%s\n",NR-1, $0)}' sa_wards0.08_nodes.csv > sa_wards0.08_nodes_with_renderorder.csv

Directory structure after adding render order to the csv.

 [~/git/gis/Wards2011]$ tree
.
├── Wards2011.dbf
├── Wards2011.prj
├── Wards2011.sbn
├── Wards2011.sbx
├── Wards2011.shp
├── Wards2011.shp.xml
├── Wards2011.shx
├── csv
│   ├── sa\_wards0.01\_attributes.csv
│   ├── sa\_wards0.01\_nodes.csv
│   ├── sa\_wards0.01\_nodes\_with\_renderorder.csv
│   ├── sa\_wards0.05\_attributes.csv
│   ├── sa\_wards0.05\_nodes.csv
│   ├── sa\_wards0.05\_nodes\_with\_renderorder.csv
│   ├── sa\_wards0.08\_attributes.csv
│   ├── sa\_wards0.08\_nodes.csv
│   └── sa\_wards0.08\_nodes\_with\_renderorder.csv
├── geojson
│   ├── sa_wards0.01.geojson
│   ├── sa_wards0.02.geojson
│   ├── sa_wards0.03.geojson
│   ├── sa_wards0.04.geojson
│   ├── sa_wards0.05.geojson
│   ├── sa_wards0.06.geojson
│   ├── sa_wards0.07.geojson
│   ├── sa_wards0.08.geojson
│   ├── sa_wards0.09.geojson
│   ├── sa_wards0.10.geojson
│   ├── sa_wards0.15.geojson
│   ├── sa_wards0.20.geojson
│   └── sa_wards0.25.geojson
└── topojson
    ├── sa_wards0.01.json
    ├── sa_wards0.02.json
    ├── sa_wards0.03.json
    ├── sa_wards0.04.json
    ├── sa_wards0.05.json
    ├── sa_wards0.06.json
    ├── sa_wards0.07.json
    ├── sa_wards0.08.json
    ├── sa_wards0.09.json
    ├── sa_wards0.10.json
    ├── sa_wards0.15.json
    ├── sa_wards0.20.json
    └── sa_wards0.25.json

3 directories, 42 files

Now on to Tableau to check if we are indeed successful in our endeavors.

The Tableau Visualization

After all of the pre-processing we now have files in our Tableau Geocoding Toolbox that will allow us to do shape based visualizations for the whole of South Africa, without breaking the golden standard of responsive interaction.

Add the Node and Attribute Data sources

For a particular simplification level (8% in this case) add the node and attribute data sources.

Datasource added

Ensure that Shapeid is set to text since some of the wards have text id’s.

To change the Geographic role of X and Y simply click on the # and select Longitude for X and Latitude for Y.

Set the geographic role for X as Longitude

The Columns should now look like this.

Shapeid as string and X as Long Y as Lat

Return to the workbook and add the second datasource for the attribute table. Ensure that shapeid is marked as text.

Add Attributes table and ensure ShapeId is marked as text

Ideally we would like to do the data join at this level, but for some reason Tableau does not like the Shapeid as join field with text. We will have to investigate further in another post.

Create the First Responsive Ward Visualization with Render Order

To create the visualization, double-click on X and Y and drop ShapeId onto the color pane. In case you have been wondering why we went to the trouble to add render order, the view at this step should give a good indication of why Tableau needs the Render Order for the Path.

Polygon with out Render Order on Path

South Africa’s Wards without render order on the path show why the column was added to the csv as part of the processing.

Now drop Render Order on to the Path pane to get the visualization of all municipal wards of South Africa.

Responsive Visualization of SA Wards in Tableau

Responsive Visualization of SA Wards in Tableau

 

Clicking on one of the wards confirms that the detail is on the map. Thus the shapes are there, ready to be linked to an external attribute.

 

 

Creating the Ward Population visualization with a join

In the last step we had a responsive view of all the wards. This is not very useful for visualizing external data. In order to visualize, for example, the population of each ward we need to join the attributes to the ward shapes via the shapeid.

  • To do this, duplicate the sheet that was created in the previous step (right click on tab->Duplicate) and choose the attributes datasource.

  • Secondary Ward Attribute Datasource is Marked Orange

    Note that the dimensions change color to orange, which indicates that this is secondary data and needs a linking field.

  • If there is a broken chain next to ShapeId, click on it.  ShapeId will be used as link field - click on it

    This should give you the linked version of the data.

    ShapId is linked - the chain is orange
  • Important step: First move the ShapeId to the Detail pane. This gives a rather bland view of South Africa.

    ShapeId in the detail pane before dropping other visualizations

    Keep ShapeId in the detail pane before dropping other linked visualization attributes.

    To show that the ShapeId is in the detail, one ward was selected in the screenshot below.

    Confirmed ShapeId in the detail pane before dropping other visualizations by selecting one of the wards.

    Clicking on one of the wards confirms that the detail is on the map. Thus the shapes are there, ready to be linked to an external attribute.

    If this step is left out and WardPopulation is dropped onto color, Tableau will most likely hang on the display side. My best guess is that the visualization is undefined for ward population across a combination of all points in SA, which could result in a huge join query. That is just a guess and the bug is possibly fixed in Tableau version 9. If you get to a point where all of your visualizations (including bar graphs etc.) have a little map in the visualization pane, you know you have triggered this issue. Save, restart Tableau and continue with the ShapeId in the detail pane first.

    • Final Step (Yay!): Drop the Ward Population onto the Color pane to get the promised visualization:

SA Ward Population

Next Steps

While we are at it we can check the population density across South Africa by dividing the Population into the Shape Area to get the following visualization.

SA Ward Population Density

From here you can link, create dashboards and generally go ahead and have good Tableau fun with your data. As an example here is a snapshot comparing 8% and 5% simplification on a dashboard.

Some wards side by side in Tableau

 

Or the 9 Provinces based on the province attribute field and the ward shapes.

SA Provinces based on Ward Shape data

In a future post we could look at dashboards linking tables to the relevant wards.

Alternatives to Standard Tableau Polygon Shaded Maps

Now that we have come to the end of implementing custom South African geocodings as a standard Tableau Polygon Shaded Map you may well ask about alternatives.

One of the obvious first choices would have been to import all shapes into a geocoded Tableau hierarchy. But alas, Tableau only supports importing a single point (Lat, Long) in the standard workflow, which means your geocoded data would work well to represent spherical cows.

At last weeks meeting Andrew from Slipstream BI mentioned that Geocoded shape data is on its way to South Africa. If you don’t want the diversions via external tools and/or need direct Tableau support, it would be a good idea to contact Andrew and discuss your specific geocoding requirements.

In the meantime: Richard Leeke has developed an alternative for windows users. He wrote an application that embeds shape data into custom Tableau geocoding hierarchies. This approach looks very promising for exploring Tableau Geocoding options, but I did not get round to test a similar approach for the Mac.

If you are happy with point visualizations and looking for background layers that are pre-populated with South African open data, take a look at www.1map.co.za, where a lot of effort has gone into presenting ward and SG data on layers that are available at a monthly subscription.

We’ll keep on the lookout for improvements on the basic Tableau Geocoding capabilities. If you know of tips and tricks that can improve the experience, leave a message for the Intelligent Routing team in the comments.

Excel® is a registered trademark of Microsoft Corporation in the United States and other countries.


Written by the Intelligent Routing team who work hard to make vehicle route optimization software available to every business that runs a fleet.
You should follow them on Twitter