Urban big data
For as long as data has been generated about cities various kinds of data-informed urbanism have been occurring. The data employed is typically sampled, generated on a one-off or occasional basis, limited in scope, and often produced by state agencies such as national statistics offices, writes Prof. Rob Kitchin
Such data includes censuses, household, transport, environment and mapping surveys, and commissioned interviews and focus groups, complemented with various forms of public administration records. In general, this data is analysed at the aggregate level and provides snapshots of cities at particular moments.
Increasingly, these datasets are being supplemented with new forms of urban big data. Big data has fundamentally different properties from traditional datasets, being generated and processed in real-time, exhaustive in scope, and having a fine resolution.
Rather than data being derived from a travel survey with a handful of city dwellers during a specific time period, transport big data consists of a continual survey of every traveller: for example, collecting all the tap-ins and tap-outs of Oyster cards on the London Underground, or using automatic number plate recognition-enabled cameras to track all vehicles.
This transformation from slow and sampled data to fast and exhaustive data has been enabled by the roll-out of a raft of new networked, digital technologies embedded into the fabric of urban environments that underpin the drive to create so-called smart cities.
"Urban big data enables a highly granular, longitudinal system, whole system understanding of a city system or service"
Such technologies include digital cameras, sensors, transponders, meters, actuators, GPS, and transduction loops that monitor various phenomena and continually send data to an array of control and management systems, such as city operating systems, centralised control rooms, intelligent transport systems, logistics management systems, smart energy grids, and building management systems that can process and respond in real-time to the data flow. In addition, a multitude of smartphone apps and sharing economy platforms generate a range of location, movement and activity data.
The result is a vast deluge of real-time, fine-grained data that is routinely generated about cities and their citizens by a range of public and private organisations, including:
- utility companies (use of electricity, gas, water)
- transport providers (location/movement, travel flow);
- mobile phone operators (location/movement, app use, behaviour)
- travel and accommodation websites (reviews, location/movement, consumption)
- social media sites (opinions, photos, personal info, location/movement)
- crowdsourcing and citizen science (maps, such as OpenStreetMap; local knowledge, such as Wikipedia; weather, such as Wunderground)
- government bodies and public administration (services, performance, surveys)
- financial institutions and retail chains (consumption, location)
- private surveillance and security firms (location, behaviour)
- emergency services (security, crime, policing, response)
- home appliances and entertainment systems (behaviour, consumption).
Although some of this data is generated by local authorities and state agencies, much of it is considered a private asset. The latter is generally closed in nature, though it might be shared with third-party vendors (such as city authorities, often for a fee). In some cases it is open in nature, often on a limited basis (through data infrastructures or APIs).
The Dublin Dashboard, funded by Science Foundation Ireland, is an interactive website and portal that provides access to a wide range of datasets about the city and a suite of visualisation and analysis tools.
It is designed to enable users to gain detailed, up-to-date intelligence about the city that aids everyday decision-making and fosters evidence-informed analysis. The underlying data is drawn together from the four Dublin local authorities, Dublinked, the Central Statistics Office, Eurostat, and government departments.
The site consists of several modules, each of which contains a number of apps. Users can:
• Examine how Dublin is performing on a number of metrics and compared to other cities and regions;
• View what is happening with transport and the environment in real time;
• Interact with maps of the Census, crime, live register, companies, housing, and planning;
• Find city services near to them;
• Report issues in their area; and
• Download data to conduct their own analysis or build apps.
Planning and governing the city
The consequence of the emerging data deluge is that data-informed urbanism is increasingly being complemented and replaced by data-driven urbanism; and this is changing how we plan and govern cities, both within particular domains (for example, transport, environment, lighting, waste management, etc) and across them.
Urban big data enables a highly granular, longitudinal, whole system understanding of a city system or service. For example, it is possible to determine patterns of travel or pollution across times of the day, days of the week, and seasons, and to do this for all nodes on the network (for example, junctions, bus stops, sensor locations).
Moreover, the data can be used to create and improve models and simulations to guide future urban development – for example, to simulate what might happen to travel patterns or land values by closing a road or siting a new hospital on the network.
At present, making sense of these new types of data is proving somewhat of a challenge given their volume, velocity and exhaustivity. However, nascent academic fields such as urban informatics and urban science are advancing quite quickly to provide new tools and forms of analysis. One response has been the creation of urban dashboards that enable city planners, and also citizens, to visualise and interact with a range of data, both traditional and real-time (see box: Dublinked).
In addition to urban dashboards, urban big data can also flow into systems designed to enable on-the-fly management of city services. While many of these systems are domain specific, such as intelligent transport systems, there is a move to try to centralise and link them within urban operating systems (such as Microsoft’s CityNext, Urbotica’s City Operating System and Plan-IT’s Urban Operating System) and urban operation centres to provide a more synoptic view of a city.
The archetypal example of an urban operations centre is the The Centro De Operacoes Prefeitura Do Rio in Rio de Janeiro (see below). This was built in the lead-up to three major sporting events – The Confederations Cup; The World Cup; and the Olympics – to aid the city administration in managing and controlling a large, diverse, complex city, by breaking down the walls between data silos in the city’s administration.
Rio - a synoptic view
Operated on a 24-hour basis, seven days a week, and staffed by 400 professional workers employed over three shifts, the centre draws together data streams from 30 agencies, including traffic and public transport, municipal and utility services, emergency services, weather feeds, social media, and information sent in by the public via phone, internet and radio.
This is complemented by a virtual operations platform accessible by mobile devices that enable city officials to login from the field to access real-time information. For example, police at an accident scene can use the platform to see how many ambulances have been dispatched and when, and to upload additional information.
The team of centre analysts uses various data analytics software to process, visualise, analyse, and monitor the vast deluge of live service data, along with data aggregated over time and the huge volumes of public administration data that is released on a more periodic basis.
The data is principally used for real-time decision-making and problem solving. It is also mashed together to investigate particular aspects of city life and change over time, and to build predictive models with respect to everyday city development and management and disaster situations such as flooding. In cases of emergencies, the centre becomes a crisis management centre.
The verge of a new era
We are entering a new era in which a deluge of contextual and actionable real-time data about cities is being generated, analysed and deployed. This is enabling a transition to a form of data-driven urbanism in which the planning and governance of cities will increasingly occur in a much more responsive and evidence-informed manner.
Urban big data and the smart city technologies underpinning it, its advocates argue, will break down silos between city agencies and departments and enable joined-up thinking and better coordination. This in turn will lead to improved governance and service delivery, more resilient critical infrastructure, increased transparency and accountability, a stronger economy, better quality of life, improved sustainability, and increased safety and security.
This is not to say that such data-driven urbanism is not without issues, however. The generation, processing, analysing, sharing and storing of large amounts of actionable data also raises a number of concerns and challenges.
Key among these are privacy, data protection, and data security. Much urban big data consist of personally identifiable information and household level data about citizens – their characteristics, their location and movements, and their activities – and links these data together to produce new derived data, using them to create profiles of people and places and to make decisions about them.
As such, there are concerns about what harms might arise from the sharing, analysis and misuse of such data. Moreover, the wide-scale generation and collation of fine-scale data raise concerns with respect to ‘Big Brother’ scenarios and the creation of a highly surveillant and authoritarian state. The challenge, then, is to try to gain the benefits of producing and using urban big data while systematically minimising any pernicious effects and harms. This is a pressing concern given the rapid rate at which new smart city technologies are being deployed.
Professor Rob Kitchin works at the National University of Ireland Maynooth, where he is a European Research Council Advanced Investigator. He is the principal investigator for the Programmable City project and the Dublin Dashboard. The research for this paper was provided by a European Research Council Advanced Investigator Award, ‘The Programmable City’ (ERC-2012-AdG-323636). Follow Rob on Twitter at @robkitchin
Life, connected: Samer Bageaeen looks at how the Internet of Things is driving a consumer revolution - and what that means for planners
An unusual proposal involving housing and various ecological enhancements on a site that straddles the border between two planning...
Outline permission for a development west of Newcastle-under-Lyme has been granted, despite the fact that residents would be largely reliant...
Plans for a 500-home development near Princes Risborough have been rejected despite a housing shortfall in the area, after an inspector...