Illinois musicians, chemists use sound to better understand science

Date

02/20/22
Illinois music professor and composer Stephen Andrew Taylor, chemistry professor and biophysicist Martin Gruebele and Illinois alumna, composer and software designer Carla Scaletti are using sound to depict biochemical processes and better understand them

By Jodi Heckel, Arts and Humanities Editor, U. of I. News Bureau


Musicians are helping scientists analyze data, teach protein folding and make new discoveries through sound.

A team of researchers at the University of Illinois Urbana-Champaign is using sonification – the use of sound to convey information – to depict biochemical processes and better understand how they happen.

Music professor and composer Stephen Andrew Taylor; chemistry professor and biophysicist Martin Gruebele; and Illinois music and computer science alumna, composer and software designer Carla Scaletti formed the Biophysics Sonification Group, which has been meeting weekly on Zoom since the beginning of the pandemic. The group has experimented with using sonification in Gruebele’s research into the physical mechanisms of protein folding, and its work recently allowed Gruebele to make a new discovery about the ways a protein can fold.

Taylor’s musical compositions have long been influenced by science, and recent works represent scientific data and biological processes. Gruebele also is a musician who built his own pipe organ that he plays and uses to compose music. The idea of working together on sonification struck a chord with them, and they’ve been collaborating for several years. Through her company, Symbolic Sound Corp., Scaletti develops a digital audio software and hardware sound design system called Kyma that is used by many musicians and researchers, including Taylor.

A graphic from the journal article illustrates the process of a protein (the chain of spheres at the top) folding, unfolding or misfolding as it moves around. The structural parameters of the protein are coded into sound, producing a variable-pitch sound waveform, for example. Timbre, pan, loudness and other parameters can be adjusted based on data input to represent different aspects or qualities of a protein.
A graphic from the journal article illustrates the process of a protein (the chain of spheres at the top) folding, unfolding or misfolding as it moves around. The structural parameters of the protein are coded into sound, producing a variable-pitch sound waveform, for example. Timbre, pan, loudness and other parameters can be adjusted based on data input to represent different aspects or qualities of a protein. Courtesy Biophysics Sonification Group

Scaletti created an animated visualization paired with sound that illustrated a simplified protein-folding process, and Gruebele and Taylor used it to introduce key concepts of the process to students and gauge whether it helped with their understanding. They found that sonification complemented and reinforced the visualizations and that, even for experts, it helped increase intuition for how proteins fold and misfold over time. The Biophysics Sonification Group – which also includes chemistry professor Taras Pogorelov, former chemistry graduate student (now alumna) Meredith Rickard, composer and pipe organist Franz Danksagmüller of the Lübeck Academy of Music in Germany, and Illinois electrical and computer engineering alumnus Kurt Hebel of Symbolic Sound – described using sonification in teaching in the Journal of Chemical Education.

Gruebele and his research team use supercomputers to run simulations of proteins folding into a specific structure, a process that relies on a complex pattern of many interactions. The simulation reveals the multiple pathways the proteins take as they fold, and also shows when they misfold or get stuck in the wrong shape – something thought to be related to a number of diseases such as Alzheimer’s and Parkinson’s.

The researchers use the simulation data to gain insight into the process. Nearly all data analysis is done visually, Gruebele said, but massive amounts of data generated by the computer simulations – representing hundreds of thousands of variables and millions of moments in time – can be very difficult to visualize.

“In digital audio, everything is a stream of numbers, so actually it’s quite natural to take a stream of numbers and listen to it as if it’s a digital recording,” Scaletti said. “You can hear things that you wouldn’t see if you looked at a list of numbers and you also wouldn’t see if you looked at an animation. There’s so much going on that there could be something that’s hidden, but you could bring it out with sound.”

For example, when the protein folds, it is surrounded by water molecules that are critical to the process. Gruebele said he wants to know when a water molecule touches and solvates a protein, but “there are 50,000 water molecules moving around, and only one or two are doing a critical thing. It’s impossible to see.” However, if a splashy sound occurred every time a water molecule touched a specific amino acid, that would be easy to hear.

Taylor and Scaletti use various audio-mapping techniques to link aspects of proteins to sound parameters such as pitch, timbre, loudness and pan position. For example, Taylor’s work uses different pitches and instruments to represent each unique amino acid, as well as their hydrophobic or hydrophilic qualities.

“I’ve been trying to draw on our instinctive responses to sound as much as possible,” Taylor said. “Beethoven said, ‘The deeper the stream, the deeper the tone.’ We expect an elephant to make a low sound because it’s big, and we expect a sparrow to make a high sound because it’s small. Certain kinds of mappings are built into us. As much as possible, we can take advantage of those and that helps to communicate more effectively.”

The highly developed instincts of musicians help in creating the best tool to use sound to convey information, Taylor said.

“It’s a new way of showing how music and sound can help us understand the world. Musicians have an important role to play,” he said. “It’s helped me become a better musician, in thinking about sound in different ways and thinking how sound can link to the world in different ways, even the world of the very small.”

Videos:

This research in the news:

Editor’s notes: To contact Stephen Andrew Taylor, email staylor7@illinois.edu. To contact Martin Gruebele, email mgruebel@illinois.edu. To contact Carla Scaletti, email carla@symbolicsound.com.

The paper “Sonification-enhanced lattice model animations for teaching the protein folding reaction” is available online and from the U. of I. News Bureau.

DOI: 10.1021/acs.jchemed.1c00857

Related People

mgruebel

Directory

scheelinAlexander
Scheeline
bjmccallBenjamin
McCall
r-gennisRobert
Gennis
j-gerltJohn
Gerlt
sgranickSteve
Granick
mgruebelMartin
Gruebele
hergenroPaul
Hergenrother
huangRaven
Huang
mlkraftMary
Kraft
leckbandDeborah
Leckband
yi-luYi
Lu
martinisSusan
Martinis
snairSatish
Nair
eoldfielEric
Oldfield
cmsCharles
Schroeder
zanZaida
Luthey-Schulten
selvinPaul
Selvin
sksScott
Silverman
s-sligarStephen
Sligar
tajkhorsEmad
Tajkhorshid
zhao5Huimin
Zhao
pbraunPaul
Braun
mdburkeMartin
Burke
jeffchanJefferson
Chan
sdenmarkScott
Denmark
dlottDana
Dlott
foutAlison
Fout
agewirthAndrew
Gewirth
ggirolamGregory
Girolami
sohirataSo
Hirata
jainPrashant
Jain
jkatzeneJohn
Katzenellenbogen
nmakriNancy
Makri
douglasmDouglas
Mitchell
jsmooreJeffrey
Moore
murphycjCatherine
Murphy
r-nuzzoRalph
Nuzzo
dimerPhilip
Phillips
rauchfuzThomas
Rauchfuss
joaquinrJoaquín
Rodríguez-López
sarlahDavid
Sarlah
kschweizKenneth
Schweizer
jsweedleJonathan
Sweedler
vddonkWilfred
van der Donk
renskeRenske
van der Veen
vuraweisJosh
Vura-Weis
mcwhite7M.
White
sczimmerSteven
Zimmerman
beakPeter
Beak
wklemperWalter
Klemperer
jdmcdonaJ.
McDonald
pogoreloTaras
Pogorelov
mshen233Mei
Shen
dewoonDavid
Woon
wboulangWilliam
Boulanger
rxbRohit
Bhargava
qchen20Qian
Chen
jianjuncJianjun
Cheng
hy66Hong
Yang
andinomaJosé
Andino Martinez
decosteDonald
DeCoste
thhuangTina
Huang
tjhummelThomas
Hummel
dkellDavid
Kell
doctorkMichael
Koerner
marvilleKelly
Marville
crrayChristian
Ray
tlbrownTheodore
Brown
rmcoatesRobert
Coates
thdjrThom
Dunning
dykstraClifford
Dykstra
j-jonasJiri
Jonas
j-lisyJames
Lisy
shapleyJohn
Shapley
pshapleyPatricia
Shapley
zumdahl2Steven
Zumdahl
ksuslickKenneth
Suslick
jcoxJenny
Cox
sqdSean
Drummond
sheeleySarah
Sheeley
jsmaddenJoseph
Madden
cknight4Connie
Knight
schulzeHeather
Schulze
kbaumgarKeena
Finney
adkssnBeatrice
Adkisson
trabari1Katie
Trabaris
metclfKara
Metcalf
ljohnso2Lori
Johnson
lchenoweLeslie
Chenoweth
wdedoWolali
Dedo
spinnerDavid
Spinner
plblumPatricia
Simpson
stevens2Chad
Stevens
lsagekarLori
Sage-Karlson
bertholdDeborah
Berthold
kecarlsoKathryn
Carlson
sdesmondSerenity
Desmond
axelson2Jordan
Axelson
scbakerStephanie
Baker
pflotschPriscila
Falagan Lotsch
dgrayDanielle
Gray
thennes2Tom
Hennessey
holdaNancy
Holda
aibarrAlejandro
Ibarra
kimshSung Hoon
Kim
kocherg2Nikolai
Kocherginsky
philipk2Philip
Kocheril
legare2Stephanie
Legare
alewandoAgnieszka
Lewandowska
smccombiStuart
McCombie
jdm5Justin
McGlauchlen
egmooreEdwin
Moore
myerscouKathleen
Myerscough
snalla2Siva
Nalla
romanovaElena
Romanova
roubakhiStanislav
Rubakhin
shvedalxAlexander
Shved
asoudaAlexander
Soudakov
xywangXiying
Wang
kwilhelKaren
Wilhelmsen
wilkeyRandy
Wilkey
silongSilong
Zhang
schlembaMary
Schlembach
trimmellAshley
Trimmell
emccarr2Elise
McCarren
cmercierChristen
Mercier
atimpermAaron
Timperman
niesShuming
Nie
hshanHee-Sun
Han
mmgMutha
Gunasekera
kknightsKatriena
Knights
lisawLisa
Williamson
keinckKatie
Einck
kneef1Kate
Neef
txiang4Tiange
Xiang
j-hummelJohn
Hummel
i-paulIain
Paul
munjanjaLloyd
Munjanja
glnGayle
Nelsen
agerardAnna
Gerard
powerskaKimberly
Powers
lolshansLisa
Olshansky
miricaLiviu
Mirica
qingcao2Qing
Cao
lisa3Lisa
Johnson
tinalambTina
Lamb
baronpBaron
Peters
bransle2Sarah
Bransley
dylanmh2Dylan
Hamilton
raegansRaegan
Smith
apm8Angad
Mehta
leverittJohn
Leveritt
xingwXing
Wang
emillrEva
Miller
jmill24Jacqueline
Miller
jlbass2Julia
Bass
ramonarRamona
Rudzinski
tlcraneTracy
Crane
cejohnstCelia
Johnston
adlAmber
LaBau
cnsolomoCandice
Solomon-Strutz
mikaelbMikael
Backlund
jacksonnNick
Jackson