Harnessing the power of open data to improve the city and its services.
Staggering fact, but it’s a figure that will only increase as our world becomes more and more connected to the internet, from “smart home” electronics that allow you to control your heating or lighting via your smartphone through to “smart city” devices like the intelligent street lights being trialled in Glasgow.
By 2020, it is predicted that there will be 50 billion devices connected to the internet - from smart meters to self-driving cars. The number of people using the internet will be outnumbered by the number of devices using it.
We are on the eve of the next era in the information age. Harnessing this new, rich source of data can be valuable but it can also be overwhelming, particularly across an entire city.
There are already vast swathes of data about our cities but unfortunately much of it is difficult to find. It is typically locked away within different organisations, each using it for a specific purpose but not necessarily understanding the value it could provide to the city as a whole.
By encouraging organisations across Glasgow to make non-sensitive and non-personal data discoverable via the internet we can empower everybody (the public, voluntary, academic, private sectors and communities,) to harness it, use it and combine it in new ways. We can all contribute to making Glasgow a better place to live, work and play.
The opportunities are almost endless, a fact that is both exciting and challenging. The Future City Glasgow programme is opening up data about the city, looking at innovative ways to harness it, and make it discoverable for everyone's use, we’ve been doing it on an unprecedented “Big Data” scale.
Around the world, organisations are opening up data by making it discoverable and available via the internet - to be more transparent, improve partnership working and stimulate innovation. This movement is backed by the G8 who signed an Open Data Charter in 2013 and promoted in the UK by the Open Data Institute (ODI) and Open Knowledge Foundation (OKF). Glasgow is at the forefront of this movement in the UK.
People and organisations who make data available (the “data providers”)
People and organisations who can make use of this newly accessible data to inform their decisions (the “data users”)
People or organisations who can bridge the gap between what the data providers can make available and what the data user needs (the “data developer”)
The “data developer” creates new value to users by enriching data, such as creating a visual of the data, creating a smart phone app, combining the data with other data, or undertaking complex analysis of the data.
This new value chain provides new economic opportunities for businesses within cities, where universities and companies (small and large) can innovate using a new commodity and all businesses can all use it to better understand their market.
By making data discoverable and available, open data can provide benefit to everyone in our city by helping people make smarter decisions, whether it’s an immediate decision like “how should I travel to work today”, or longer term decisions such as those involved in the introduction of new policies.
Most open data initiatives have traditionally been driven by a desire for transparency (opening up data about salaries or expenses claims for example), some are driven from recognition that allowing others to use data can stimulate innovation, such as data being used by a software developer to develop a smart phone app.
In Glasgow we recognise that the benefits are more fundamental and broad-ranging than that. We are evaluating how open data can:-
As part of our commitment to open up data across the city, we have created an open data catalogue - bringing together a growing collection of data streams from over 60 different organisations. Organised by relevant and meaningful category groupings, the catalogue provides a one-stop-shop for data about the city.
This is only the beginning. Additional features will be added in coming months which will make it easier to discover and understand data. These include a quality and standards rating system.View Data Catalogue
Glasgow is demonstrating how cities can harness data that is already available but not yet universally accessible, and data that will be available in the future from sensors and smart devices such as Intelligent Street Lights.
It has set its sight on a world where data from across the city is published as "open by default" - from real time traffic system data to large (more static) datasets about the built environment.
As the shop window, the Data Catalogue provides users with the means to discover and gain access to raw ingredients - data. And like a shopper in a supermarket, users do not need to worry about how the content got there.
Behind the scenes many Open Data Catalogues typically rely on people taking copies of data as files and manually uploading the files to the catalogue. It’s equivalent to a shop-keeper collecting his produce from all of his suppliers one-by-one. It’s fine if there isn’t much to carry and there aren’t many places to go to, but if there is a lot of content, a lot of destinations and a lot of change, it isn’t sustainable.
Like in retail, if we want open data at a city-scale, where organisations are publishing their operational data (which is currently managed within their own mission-critical IT systems) as open by default, where more and more real-time data is available from smart devices, we needed to build a distribution centre for data (a “data hub”).
No off-the-shelf solutions were available. Through the Future Cities Demonstrator, the city now has a data hub that allows organisations to automate the publication of their data, allows it to be stored and makes it available on a large scale so that it is easy to access via the catalogue. It helps make the publication of open data sustainable for all in the city.
An example of this scale and the value it can provide is traffic data. The data hub platform is currently collecting data from 800 vehicle sensors embedded within traffic junctions in the city. These 800 sensors generate a total of 3Mbytes per minute. Historically our data catalogue just provided access to the latest file. Using the data hub platform we now store the history so that people can analyse traffic flow trends across the whole city over time more easily, allowing them to correlate it against other datasets such as weather events, disruption to public transport, major events in the city.
But that’s not all. The data hub is designed to provide value to data users and developers, not just publishers. It provides advanced tools to help make data more accessible (for example through application programming interfaces (APIs), more visual, and provides advanced “Big Data” analytics technology which allows people to undertake complex analysis of the various datasets it makes available).
From a technology perspective, the Platform uses some of the most advanced technologies and major industry trends available; bringing together Cloud Computing, Big Data Analytics (Hadoop, Machine Learning) and the Internet of Things (IoT).The Platform is built to be modular so it can be expanded by developers using APIs, and where appropriate we have used open source to facilitate collaborative development. For example, the data catalogue itself uses an off-the-shelf open source product that should allow Glasgow to share development with other cities (and vice versa).
We can make use of open-source software for these user interfaces to create opportunities for innovation and for sharing with other cities.