Untangling urban data signatures: unsupervised machine learning methods for the detection of urban archetypes at the pedestrian scale

Abstract

Urban morphological measures applied at a high-resolution of analysis may yield a wealth of data describing varied characteristics of the urban environment in a substantial degree of detail; however, these forms of high-dimensional datasets are not immediately relatable to broader constructs rooted in conventional conceptions of urbanism. Data science and machine learning methods provide an opportunity to explore such forms of data by applying unsupervised machine learning methods. The dimensionality of the data can thereby be reduced while recovering latent themes and identifying characteristic patterns which may resonate with urbanist discourse more generally.

Dimensionality reduction and clustering methods, including Principal Component Analysis (PCA), Variational Autoencoders, and an Autoencoder based Gaussian Mixture Model, are discussed and demonstrated for purposes of ‘untangling’ urban datasets, revealing themes bridging quantitative and qualitative descriptions of urbanism. The methods are applied to a morphological dataset for Greater London. The spatial aggregations and morphological measures are computed at pedestrian walking tolerances at a 20m network resolution using the cityseer-api Python package, which utilises a local windowing-methodology with distances computed directly over the network and with aggregations performed dynamically and with respect to the direction of approach, thus preserving the relationships between the variables and retaining contextual precision.

Whereas the demonstrated methods hold tremendous potential, their power is difficult to convey or fully exploit using conventional lower-dimensional visualisation methods, thus underscoring the need for subsequent research into how such methods may be coupled to interactive visualisation methods to further elucidate the richness of the data and its potential implications.

Introduction: Detection of urban archetypes with unsupervised machine learning methods

Vibrant pedestrian districts manifest an affinity for complexity and its requisite diversity, as do complex systems more generally (Page 2011). However, urban master plans have historically demonstrated a proclivity towards reductionism. Cities were increasingly rearranged around motor vehicles and reconceived in the abstract on drawings boards; the more granular, dense, mixed-use, and visually ‘messy’ artefacts of evolved cities were swept-aside for grandiose compositions that were ultimately too large, too homogenous, and too resistant towards change for pedestrian-based forms of urbanism to thrive (Harvey 1989; Lyon 1999). Though initially associated with high-modernism, aspects of these patterns are still prevalent in various forms of contemporary urbanism manifesting across the spectrum from suburbia to romanticised smart city master-plans, explicitly or implicitly emphasising idealised efficiencies at the expense of complexity (Greenfield 2013; Townsend 2013; Sterling 2014). This paradigm can stifle the oft unpredictable and chaotic forms of interaction aiding processes of discovery and diffusion within complex adaptive systems (Kauffman & Johnsen 1991; Kauffman 1995).

The complex systems interpretation of cities, replete with dynamics from emergence to non-linearities to phase-changes (Batty & Longley 1994; Batty 2005; Batty 2013; Allen 2012), resists simple averages and crude models (Jacobs 1961). A dilemma faces architects, urban designers, and planners tasked with the challenge of how to plan for inherently unpredictable processes at the urban scale (Portugali 2012; Marshall 2009). Whereas it is not possible to anticipate every last action of city citizens — and how these chains of interaction might bifurcate or coalesce through space — it is possible to gauge, more generally, how that certain forms of urbanism may be more conducive to large numbers of permutations of complex interactions (Alexander 1967). Complex systems derived methods, including network centralities and land-use diversity measures, are proxies for urban complexity: they foreshadow networks of potential interactions available to city citizens. Whereas our ability to model the full complexity of urban systems will always be constrained — not least by sensitivity to initial conditions — it remains possible to explore the spatial manifestations of complex processes present in historical cities and to compare these to new forms of development. The question then arises: if we apply pedestrian-scale centrality and mixed-use measures using sufficiently precise and high-resolution analysis, then can emerging forms of data analysis and unsupervised machine learning methods be used to ‘untangle’ and ‘sift out’ signature patterns from the mass of ensuing data? These terms are used in the literal sense because large and high-dimensional datasets require teasing apart to reveal latent themes, which may ultimately help bridge the gap from quantitative forms of urban analytics to qualitatively framed conceptions of cities (Portugali 2012).

At first glance, the use of data science or machine learning models to understand or even predict aspects of healthy cities may appear ironic and misguided given planning’s problematic past. Examples such as Robert Moses’ ruthless dismemberment of New York City (Flint 2011) and failed housing schemes such Pruitt-Igoe are stark reminders that reliance on shallow or overly abstract interpretations of urbanism can lead to misguided decisions and problematic forms of urban policy. It is precisely these issues that provoked Jane Jacobs’ The Death and Life of Great American Cities (Jacobs 1961) which would forever change perspectives on urbanism. Jacobs laments that planners had misconstrued the nature of cities and had mistakenly assumed that decisions made in the abstract were somehow sufficient to deal with the emergent complexity of healthy cities. She articulates her thoughts with reference to Warren Weaver’s seminal paper Science and Complexity (Weaver 1948) which casts the nature of scientific problems into three classes: ‘problems of simplicity’, described and modelled using simple sets of variables and equations that behave predictably; ‘problems of disorganised complexity’, where the behaviour of large quantities of elements such as gas molecules in a container are modelled collectively through statistical methods even though the behaviour of each constituent part is chaotic; and, heralding the complexity sciences, ‘problems of organised complexity’, which do not adequately yield to either of the approaches above and present some difficulty in solving because they exhibit non-linear, emergent, and adaptive behaviours. She places the nature of cities squarely in the last category — problems of organised complexity — entailing “a sizable number of factors which are interrelated into an organic whole” and present “situations in which a half-dozen or even several dozen quantities are all varying simultaneously and in subtly interconnected ways” (Jacobs 1961, p.2). Attempts at planned settlements tend to eschew complexity in favour of, per Garden Cities, dumbed-down linear ratios between abstract quantities such as housing and jobs or, as symbolised by Le Corbusier’s Radiant City, conceptions of urbanism rooted in larger-scale abstractions wherein people are reduced to statistical aggregations, again treated as simpler linear combinations of variables. By recasting cities through a reductionist lens, planners had come to make decisions that were detached from the complexities of functioning neighbourhoods and treated inhabitants as simplistic aggregations tantamount to “grains of sand, or electrons or billiard balls” (Jacobs 1961, p.437). Jacobs proceeds to offer a prescription: our understanding of cities should instead develop out of “the microscopic or detailed view… rather than on the less detailed, naked-eye view suitable for viewing problems of simplicity or the remote telescopic view suitable for viewing problems of disorganised complexity” (Jacobs 1961, p.439). She outlines three principles to this end: first, think about city ‘processes’: elements within cities have different effects depending on their combinations with other elements and the varied interactions between them; second, reason from an inductive rather than deductive approach, meaning from particularities to generalisations instead of from generalities to particulars; third, look for ‘unaverage’ clues such as peculiarities, outliers, or nascent trends that help elucidate the workings of cities rather than fixating on statistical methods rooted in large-scale ‘averages’ which may offer little explanation for how constituent elements may be operating within a complex system, particularly if applied at a more localised scale.

A knee-jerk reaction may be to reject machine learning out-of-hand for its links to mathematics and statistics more generally; however, on closer scrutiny, the synthesis of locally precise urban morphological metrics combined with machine learning methods affords the use of highly detailed datasets capable of capturing and preserving contextual particularities; facilitates the use of high-dimensional datasets with significant assortments of variables and potentially complex and varied non-linear relationships between them; and, in the form of unsupervised methods combined with deep neural networks, allows for structures to be unearthed directly from within the data without the imposition of reductionist theories or formulas. Compared to traditional statistical methods applied to larger-scale spatial aggregations, machine learning applied to high-resolution and contextually-anchored spatial data resembles an approach akin to proceeding from the particular to the general. Despite the large volumes of information, the data-space is (in effect) explored ‘line-by-line’ with model losses computed and updated over comparatively small batches of data. Patterns are ‘sniffed out’ using exploratory and bottom-up-like procedures with the more prevalent of these congealing over successive iterations to reveal thematic patterns that have arisen directly from the data. Emphatically, this reasoning only holds if working with pedestrian-scale metrics gathered using sufficiently high-resolution analysis, with the measures processed directly from each location. Use of intervening levels of spatial aggregation, interpolation from larger to smaller units of scale, or overly large units of analysis would otherwise result in the attrition of information and, critically, discards or otherwise masks local-scale inter-relationships between the variables.

Traditional forms of urban morphological analysis have been challenging to apply at scale because of reliance on manually collated observations and wearisome calculations. Geographic Information Systems (GIS) have permitted larger scales of quantitative analysis. However, the lack of comprehensive and granular data sources combined with computational constraints meant that these methods have oft been applied against larger units of spatial aggregation and relied on simplified distance metrics (Logan et al. 2017; Araldi & Fusco 2016). More recently, however, the increased availability of detailed datasets has facilitated a finer scale of analysis while retaining the ability to process larger areal extents (Araldi & Fusco 2019), thus prompting the adoption of multi-variable and multi-scalar workflows. The ensuing large and high-dimensional datasets can be combined with unsupervised exploratory methods and have engendered interest in how urban morphological analysis can be applied not only to the exposition of existing cities but also in the capacity of a rigorous design-aid for newly planned forms of development (Serra et al. 2016; Gil et al. 2009; Berghauser Pont et al. 2017).

Please see the linked preprint paper for additional information.