Embedded Files
SONIFICATION SHOWCASE
A Sonification of the Warming Stripes
A Sonification of the Warming Stripes
By C.M. Harrison, J.W. Trayford, and R.D. Shepherd
Overview
Overview
We present a sonification of the temperature anomalies (deviations from an average, reference value) on the Earth's near-surface over the period 1930-2024. The sonification is combined with the famous "Warming Stripes" visualisation to create an animated audio-visual. Similarly to the visualisation, the sonification is designed to create a sense of urgency and an appreciation of the gravity of the situation of ever-increasing temperatures. A synthetic mix of sounds is used, which is modified so that with increasing temperature the sound appears to increase in intensity (by modifying a filter cut-off frequency and the amplitude) and increase in harmonic tension (by modifying relative pitches). To represent the passage of time, there's an additional 'tick' for each year of the data, which also could be interpreted as an ominous countdown to disaster. It is intended as a data journalism piece for general public audiences, with a secondary goal of making an accessible representation of the "Warming Stripes" for those with sight loss. This article describes the sonification methods, and with it, we release the Python code used to produce the sonification.
The data files and associated code for this sonification will be released on the Audio Universe project page of data.ncl.ac.uk.
Version Note: This is Version 2024 of the sonification. It is updated, using a newer dataset, since the original version was released on YouTube in May 2024. The sonification approach is the same as the original.
Summary
Summary
Data Domain/Topic: Climate Science
Primary Goal: Data Journalism
Audience/User: General Public
Secondary Goal: Accessibility
Analytical vs. Narrative: Mostly Narrative
Sonification Type: Parameter Mapping
Sound Type(s): Synthesised Mix
Multi-Modalities: Sonification + Animation
Sonification
Sonification
Figure 1: This is an audio-visual representation of the Earth's temperature anomalies from 1930 to 2024. There is one tick per year of the data (i.e., one tick per data point). The tick also has a stereo pan to indicate the passage of time from full left (earliest year) to full right (latest year). The temperature anomaly data is used to control the properties of the synthesised sound. With increasing temperature anomaly, the pitch is raised of one of the composite sounds (to create a harmonic tension), the range of frequencies is increased (by increasing a filter cut-off frequency parameter) and the amplitude (`volume') of the sound is increased. The visualisation represents the same data, with below average anomalies in blue, and above average in red (inspired by the "Warming Stripes" designed by Ed Hawkins). In the audio-visual, a black stripe highlights the current year being played in the sonification. Data is from the UK Met Office Hadley Centre observations, using the annual global temperature anomaly (ensemble averages).
Context & design rationale
Context & design rationale
There is consensus among the scientific community that the Earth's temperature is rising and human activity is significantly contributing to these global rises. The situation is serious with respect to the world's climate and ecosystems, and interventions are critical to prevent catastrophic levels of temperature rises in the immediate future (WMO 2024). Communicating this crisis to the public and policy makers is critically important to raise awareness and engage people with necessary climate actions. Towards this, in 2016, Ed Hawkins (University of Reading) created a novel and engaging approach to represent global temperature with a visualisation. The so-called "Warming Stripes" consists of a set of coloured stripes, one per year, which are colour-coded by temperature anomalies on Earth (either globally, or at specific locations). A temperature anomaly is defined as the temperature offset from some reference period (typically at least 30 years). Anomalies above this value are represented by increasingly dark red colours, and below this reference point as increasingly dark blue colours. This visualisation went viral, particularly initiated by the Social Media campaign #ShowYourStripes.
Sonification provides an alternative, or supplementary, method to traditional visualisations to engage the public with the relevant data. Furthermore, sonification could invoke more of an emotional response than a passive experience of looking at a visualisation of data. It is in this context we decided to create the "Sonification of Warming Stripes". Below, we describe the rational of our sonification approach, using the terminology and design frameworks as set out in the Data Sonification Canvas of Lenzi et al. (2024) and the Data Sonification Archive of Lenzi et al. (2020).
Our goal was to create a sonification invoking a sense of urgency and highlighting the seriousness of the global temperature rises. The primary audience (or 'user') is intended to be the general public, listening to the sonification in the context of online browsing (i.e., through websites of social media channels). In this sense it could be classed as 'data journalism'. Consequently, we aimed more for a 'Narrative' than an 'Analytical' approach for our sonification design, where we aimed to communicate a message, rather than require the listener to be able to identify specific data values. This is similar to the "Warming Stripes" visualisation, in that it is the trend of the data which is the main message being communicated, as opposed creating an approach to accurately identify specific temperature anomaly values for any particular year.
We intended to communicate temperature rises of the Earth, which can not easily be represented in an indexical way (i.e., a sound directly produced by the phenomena), nor an iconic way (i.e., similar to the phenomenon). Therefore, we aimed to produce a sound that is symbolic of the phenomena, but portraying the intended association. This led to our choice of base sounds (outlined below), which where chosen to portray a gentle sense of calm for lower temperatures and a sense of power and 'fire-like' sounds for the highest temperatures. Therefore, the listening experience is intended to be semantic in that the listener needs to de-code the association between the sound they hear and what is represents. Nonetheless, we made choices to make this de-coding very simple and intuitive; i.e., for increasing temperatures we designed the sonification such that sound would become increasingly intense and harmonically in tension.
Source data & processing
Source data & processing
The data for the sonification was obtained from the UK Met Office, Hadley Centre observations datasets. Specifically, we made use of the ensemble means data, averaged over the Northern and Southern hemispheres, and calculated annually (i.e., one datapoint per year). We used Version 5.0.2.0 of the data (HadCRUT.5.0.2.0). The data traces the near-Earth temperature, by blending surface air temperature and sea-surface temperature, with a median across regional time series data computed across 200 ensemble members. This is described in Morice et al. (2021).
The data are presented as a time series, as temperature "anomalies" (in degrees Centigrade), relative to the average temperature over the period 1961-1990. For the purposes of our sonification, we truncated the data to the dates 1930-2024 (i.e., 94 years of data). We did not process the data in any other way.
Sonification method
Sonification method
Our sonification approach was to perform parameter mapping on synthesised audio clip samples. We used the open-source Python package STRAUSS (introduced in Trayford & Harrison 2023) to read in these samples (see below), and then manipulated these sounds based on a set of parameters whose values were determined by the anomaly values. The Python notebook used to produce the sonification, STRAUSS_Stripes.ipynb, is released with this article (on data.ncl.ac.uk).
Base sounds
Base sounds
As the base sounds for our sonification we created two synthesised mixed sound samples, with lengths longer than the intended sonification. These samples have very little time variation with an almost constant sound over time. One base sound was chosen to represent a sense of calm, which we refer to the 'gentle sound' and another was chosen to represent a sense of intensity/power/danger, which we refer to as the 'powerful sound'. The sounds are represented as spectrograms in Figure 2, and are released with the file names: gentleSound_a4.wav and powerfulSound_bb1.wav, respectively.
The gentle sound was constructed with a synthesiser, using notes D, F, B flat and C. The sound is consequently dominated by a set of four frequencies around 200\,Hz (see panel (a) in Figure 2). The perceived tonic (or main) pitch is of a 'B flat' note. The synthesised powerful sound was also produced with a dominant perceived pitch of B flat, but three octaves lower than the gentle sound, with a frequency at ~30 Hz (see panel (b) in Figure 2). In addition to the pitches, the powerful sound was mixed with noise and a field-recording of a wood fire. We chose a familiar fire sound to represent the sun's burning plasma.
Figure 2: A spectrogram of the two base sound samples used to create the sonification (before any parameter mapping applied). In each case an 8 second representative section is shown. The logarithmic colour scaling has white representing 4 orders of magnitude higher amplitude than dark blue. Panel (a) is the 'gentle sound', dominated by mid-pitch tones, and panel (b) is the 'powerful sound', which is strong mix of a low-pitch tone and noise.
Mapping & data relationships
Mapping & data relationships
The sonification was chosen to last 47 seconds, corresponding to 2 years of the data per second. We designed our parameter mapping approach such that the 'gentle sound' would dominate the overall sonification when the temperature anomalies are low (mostly at early times), and the 'powerful sound' would dominate the overall sonification when the temperature anomalies are high (mostly at later times). Furthermore, the pitch was also shifted with the anomalies data. Together, this leads to an increasing sense of harmonic tension between the two sounds as the anomaly value becomes increasingly high. To aid the interpretation of how the parameter mapping choices influenced the sound, we show in Figure 3, a spectrogram of the final sonification in addition to showing how the sound parameters were altered throughout the sonification.
We made use of a few, of the many, possibilities within the STRAUSS sonification package to apply parameter mapping to the base sound samples. To produce the perceived effect of increasing intensity and increasing sense of harmonic tension with increasing temperature, we made use of the parameters volume, pitch_shift and cutoff and applied these only to the powerful sound. We set the volume parameter to scale the relative amplitude of the powerful sound between a value of 0 (for the minimum data point) to 1.0 (for the maximum data point). The effect of this is that the powerful sound makes zero contribution to the overall sonification at the minimum temperature (i.e., see solid white curve at year 1933 in Figure 3), and has its maximal contribution at the maximum temperature (see year 2024 in Figure 3)
The pitch_shift, modifies the pitch of the sound continuously. We set this parameter between a value of 0 semi-tones for the minimum temperature anomaly and a maximum value of 6 semi-tones (i.e., 1 tri-tone, as shown in Figure 3) for the maximum temperature anomaly. The effect is an overall increase in frequency of the base sound of ~40% (or 0.15 dex) from the start (dominated by low temperature anomalies) to the end (dominated by the highest temperature anomalies) of the sonification. Figure 3 shows how this pitch shift varies as a function of time, and the corresponding data values (see the dot-dashed line).
The cutoff parameter refers to the cut-off frequency of a low-pass filter. Specifically, a Butterworth filter with a 24dB roll-off is used.
Figure 3 shows how the value of the filter-cut off frequency varies with the data values and corresponding time in the sonification (see dashed white curve).
We did not modify properties of the gentle sound in the sonification. This is because we intended this to be a constant throughout, but becoming more/less overwhelmed by the powerful sound for higher/lower temperatures.
We applies one other effects to help the listener perceive the passage of time throughout the sonification. We added a 'ticking' sound for every year of the data. This was created by using a short burst of noise at every data point (creating white noise and then applying short volume envelope). We panned this tick sound from full left (for the beginning year) to full right (for the final year), for a stereo audio format.
The spectrogram in Figure 3 visualises some of the key characteristics of the final sonification; including:
The vertical stripes, every 0.5 seconds, due to the tick sound every year.
The four horizontal stripes near, and around, ~200 Hz, originate from the (non modified) 'gentle sound' (see panel (a) in Figure 2). These are persistent throughout the sonification.
The increasing amplitude, with increasing temperature, of the noise and lower frequencies bands are caused by the 'powerful sound' (see panel (b) in Figure 2) having its amplitude and filter cut-off frequency modified throughout.
Figure 3: A visual representation of the audio, the values of the parameter mappings applied to the powerful sound, and the underlying data for the "Warming Stripes Sonification". The main panel shows a spectrogram of the full sonification, where the logarithmic colour scaling has white representing 4 orders of magnitude higher amplitude than dark blue. The dashed white curve is the varying low-pass frequency cut-off parameter (values in hertz, shown by left axis). The solid white curve shows the variation of the amplitude, which is calculated relatively (linearly) from the maximum (amplitude = 1.0) to minimum (amplitude = 0.0) corresponding data value. The dot-dashed black curve shows the variation of the pitch-shift. The units are in tri-tones, such that the maximal value of 1.0 corresponds to a 6 semitones positive shift and a value of 0 semitones shift is applied for the lowest data value (see right axis). The top panel shows a visualisation of the temperature anomaly data used to map onto to the sound parameters. The visualisation uses the approach of the famous "Warming Stripes", with one coloured bar per year. The corresponding colour-scale is shown in the bottom-left of the figure.
Multi-modality
Multi-modality
We combined the sonification with a visual representation of the data to create the final audio-visual file. The visualisation is created following the approach of the famous "Warming Stripes", with one coloured bar per year. The corresponding colour-scale is shown in the bottom-left of Figure 3. The visualisation was animated by turning the bar black for the corresponding year that is being played in the sonification. This visualisation was created by producing a set of frames in our Python notebook, and then combining these frames with the sonification audio file using ffmpeg.
Bibliography
Bibliography
Lenzi S., Ciuccarelli P., Liu H., and Hua Y. (2020), Data Sonification Archive. http://www.sonification.design.
Lenzi, S., and Ciuccarelli, P. (2024), Designing tools for designers: The Data Sonification Canvas, in Gray, C., Ciliotta Chehade, E., Hekkert, P., Forlano, L., Ciuccarelli, P., Lloyd, P. (eds.), DRS2024: Boston, 23–28 June, Boston, USA. https://doi.org/10.21606/drs.2024.730
Morice, C. P., Kennedy, J. J., Rayner, N. A., Winn, J. P., Hogan, E., Killick, R. E., et al. (2021). An updated assessment of near-surface temperature change from 1850: the HadCRUT5 data set. Journal of Geophysical Research: Atmospheres, 126, e2019JD03236. https://doi.org/10.1029/2019JD032361
Trayford, J. and Harrison, C.M. (2023), Introducing STRAUSS: A flexible sonification Python package, 28th Proceedings of the International Community of Auditory Displays (ICAD2023), p249-256, https://hdl.handle.net/1853/73935
World Metereological Organization (WMO) 2024, State of the Climate 2024 (Update for COP29), https://library.wmo.int/idurl/4/69075
Acknowledgements
Acknowledgements
CMH acknowledges an United Kingdom Research Innovation grant (code: MR/V022830/1). JWT and RDS acknowledges support from the Science and Technology Facilities Council (grant codes ST/X004651/1 and ST/W006790/1, respectively). Thanks to Maximilian Nöhe for providing code that was used as the basis for making the visualisation in Python, based on the original design of Ed Hawkins.
Page updated
Google Sites
Report abuse