Workflow improvement with FME in Skedsmo municipality
Anders Hveem Malum (Geodata AS) and
Turid Brox Nilsen (Skedsmo municipality)
16.05.2014
Skedsmo Kommune, Teknisk sektor
1
Skedsmo municipality
• Situated between Oslo and
Gardermoen airport
• Lillestrøm is the administrative centre of
a growing municipality
• 50 000 inhabitants (2013)
• Area 77 km2
• 25 000 jobs
• Lillestrøm Centre of Expertise – one of
Norway’s largest research- and
technology environments
• Skedsmo is host of Norway Trade Fairs
28.01.2013
Skedsmo Kommune, Teknisk sektor
2
GIS in Skedsmo municipality
• Geodata department with 23 employees
– Surveying
– Updating the cadastre
– Property taxation
– Manage other public fees
– One team of six people adminster databases
and makes maps available
• ArcGIS
• FME Server and Desktop
• GeoSOSI Professional GDMMapper
16.05.2014
Skedsmo Kommune, Teknisk sektor,
3
Geodata AS
• About Geodata
–
–
–
–
Established 1988
Distributor of Esri and Safe Software in Norway
115 employees
Offices in Oslo, Stavanger, Hønefoss and Steinkjer
• What do we offer?
–
–
–
–
–
–
–
16.05.2014
ArcGIS software from Esri
FME software from Safe
Customized solutions
Consulting and assessment services
Map products and GIS - data
Online hosting and GIS Services in the cloud
Courses and support
Skedsmo Kommune, Teknisk sektor,
Geodata AS
4
FME use case: Zoning plans from A to Z
• The workflow:
–
–
–
–
–
–
Initiator (member of the public / private firm / government agency) constructs and sends us the
zoning plan as a SOSI-file
Skedsmo converts the SOSI-file to a work-geodatabase (GDB) using FME and GeoSosi
Professional (SOSI Plugin from Geodata)
Quality control in ArcMap
Is there already data where the new plan is situated?
• Old data is extracted into a backup geodatabase (GDB)
Data from the GDB is loaded into the full database (SDE) using an ETL Tool in ArcMap
Users access FME projects as jobs on FME Server or as ETL Tools in ArcMap.
• Software used:
–
–
–
16.05.2014
FME Workbench 2014, extended to ArcGIS
GeoSOSI Professional FME plugin from Geodata
ArcMAP 10.2.1 with Python 2.7
Skedsmo Kommune, Teknisk sektor
Geodata AS
5
FME use case: Zoning plans from A to Z
• Most of the tools for zoning plans are FME ETL Tools, combined with ArcGIS ModelBuilder
and some Python scripting:
Toolbox
Python scripts
FME ETL tools
Model
16.05.2014
Skedsmo Kommune, Teknisk sektor
Geodata AS
6
FME use case: Zoning plans from A to Z
• Example 1:
–
–
16.05.2014
Loading proposed plan from SOSI-file to Geodatabase, and create a layer – file for the plan.
Tool input:
Skedsmo Kommune, Teknisk sektor,
Geodata AS
7
FME use case: Zoning plans from A to Z
• Example 1 – what happens when the user pushes the OK – button?
–
–
–
16.05.2014
The FME ETL – tool is executed:
• A new file geodatabase is created using a xml – schema template for proposed plans
• FME reads input SOSI file
• Data from the SOSI file is imported, and mapped to the correct feature classes and subtypes
in the newly created file geodatabase.
Python script 1 starts
• This script loops through the file geodatabase and deletes any feature classes in the
template schema that are empty after the SOSI - import.
Python script 2 starts
• This script uses standard presentation from GeoAreal (geodata product) to symbolize the
newly created file geodatabase.
• All feature classes from the new file geodatabase is added to the map document with correct
representation and symbols
• Zoning plan topology is also validated and symbolized.
Skedsmo Kommune, Teknisk sektor,
Geodata AS
8
FME use case: Zoning plans from A to Z
• The log from FME and from the python scripts is written to the GP – tool results:
FME log
Python log
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
9
FME use case: Zoning plans from A to Z
• And it is also available from the Geoprocessing results dialog in ArcMAP afterwards:
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
10
FME use case: Zoning plans from A to Z
• Sample output from the tool:
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
11
FME use case: Zoning plans from A to Z
• Example 2:
–
Extracting «expired» data from the SDE database:
To be deleted
from SDE
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
12
FME use case: Zoning plans from A to Z
• Example 2:
–
16.05.2014
Extracting «expired» data from the SDE database:
Skedsmo Kommune, Teknisk sektor,
Geodata AS
13
FME use case: Zoning plans from A to Z
• Example 2:
–
Extracting «expired» data from the SDE database:
Input SDE
database
Outline of
new plan
File GDB with
clipped data
Delete/update
on SDE
database
Clipper
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
14
Water and sewerage installation data
• Skedsmo municipality maintains their water and sewerage installations data in an Oracle
database using the program Gemini from the Norwegian company Powel.
• FME Workbench is used to convert those data into an ArcGIS Geodatabase.
• The FME Project also detects logical errors in the original data, and writes error logs and
feature classes containing the objects with errors.
–
–
Code errors (deviations from the Norwegian SOSI - standard)
Network errors
• A scheduled Python script runs this conversion every week, and sends an e-mail containing
extracts of the logs to system administrator.
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
15
Water and sewerage installation data - example
Topology/network
errors
Code errors
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
16
Water and sewerage installation data - log from scheduled routine
FME Errors
FME Warnings
FME Statistics
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
17
Use case: «Sak i kart»
• A map layer that shows cases
we are working on from our
electronic filing system
• Ie. Building applications, new
addresses, new/changed parcels
• FME extracts data from our
electronic filing system…
…using the cadastral/property unit number (gnr./bnr.) to georeference the points
• A python script in FME spreads the points from one parcel apart so they don’t
overlap
• Sceduled job on FME Server updates the points nightly.
• Data is presented in a map inside our firewall, there are links directly from the
points to the file in the electronic filing system
• Very useful to get an overview of everything that is happening on a parcel
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
18
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
19
Use case: work list for changes to the cadastre
• Norwegian: «Arbeidsliste for matrikkelførere»
• Changes in our cadastre as a result of building permits is input into software from the
Norwegian Mapping Authority.
• Challenge: How do we find out which building permits/changes to a building needs to be
input into the cadastre?
• FME goes through our electronic filing system, looks at case statuses and writes this to an
Excel spreadsheet
• The spreadsheet is updated nightly using a scheduled job on FME Server
• The person updating the cadastre edits the spreadsheet to show when a case has been
updated in the castre
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
20
Use case: data compilation and adaptation in a property tax project (1/3)
• We need to appraise (taksere) 15 000 parcels in Skedsmo
• A valuation board (takstnemnd) approves all parcel values
• We make an estimate of a parcel’s value based on data in the cadastre (building
size etc.)
• One important factor in the estimate is the «neighborhood factor»
–
–
–
in an area with normal parcel prices this will be 1
in an area with relatively expensive parcels higher (for instance 1,3)
in an area with relatively cheap parcels lower (for instance 0,8)
• The «neighborhood factor» is calculated based on actual sales figures
• The «neighborhood factor» is different in parcels with different building types (for
instance detached houses vs. apartment blocks)
• Challenge: assign a «neighborhood factor» to records in the property tax database
• Building blocks: property tax database, cadastre data, polygons containing the
neighborhood factor
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
21
Use case: property tax project (2/3)
• Buildings in the cadastre have a
building type
Georeferenced
tax record
• Tax records needs to have the same
information
• Solution: use FME to assign building
type to the parcel, then use the
geometry of the parcel to assign a
building type value to the tax record
• Then use the building type of the tax
record to assign it a «neighborhood
factor»
Parcel
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Point from the
cadaster with
Building type
Ie. “detached
house”
Geodata AS
22
Use case: property tax project (3/3)
Obligatory illustration showing
a confusing FME project
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
23
What we gain by using FME
• Programming made visual
• Ensures consistency and lessens the possibillity of errors when loading/
extracting data
• Saves employees in Skedsmo lots of time by automating tasks
• FME in combination with the ArcGIS platform is a very powerful and efficient way
of improving workflows for every day GIS tasks.
• Get in touch: Anders Hveem Malum ([email protected]) / Turid
Brox Nilsen [email protected]
16.05.2014
Skedsmo Kommune, Teknisk sektor,
Geodata AS
24