Timbre space representation as a tool
for conceiving of how to compose with
or analyze timbral structures
David Romblom, Gabriel Vigliensoni
Music Technology Area
Schulich School of Music
McGill University
Outline
Timbre space introduction
Technique in modern music
Technique in computer music
Navigating timbre space
Technique in modern music production
Timbre space introduction
Summary of general timbre space generation method
Timbre space represents the geometrical organization of perceptual distances, measured in
dissimmilarity ratings, between tones with equal pitch, loudness and perceived duration.
Selection of materials for study
Collection of (dis)simmilarity judgements
Representation of the (dis)simmilarity judgements
Psychoacoustic interpretation of the structure
Verification of the structure in musical situation
Perceptual e!ects of spectral modifications
on musical timbres
(Grey and Gordon, 1978)
An original timbre space was defined using 16 synthetic tones and MDS methods.
(Gordon and Grey, 1977)
A second study was conducted, where 8 of these tones swapped spectral envelopes.
(Gordon and Grey, 1978)
All of the swapped tones also swapped places in the timbre space.
Big deal: modifications to acoustic descriptors cause predictable and meaningful perceptual changes.
This opens the door for meaningful operations on timbre, we can modify descriptors and get (hopefully) musically
useful results!
Perceptual scaling of synthesized musical timbres:
Common dimensions, specificities, and latent subject
classes (McAdams et al., 1995)
A data set of timbre dissimilarity judgements was analyzed using the CLASCAL MDS method to have five classes of
subjects, called latent classes.
The first two classes weighted all of the dimension approximately equally, meaning that the dimensions were
roughly equal perceptually.
The other classes did not, indicating that some dimensions were perceptually “heavier” for them.
Big deal: timbre space might be an individual perception in some cases.
Musical training didn't matter as much as one might guess. Everybody uses timbre every day to “figure out what is
going on out there.”
Big deal: timbre is important to everybody, as such, musicians should be using it!
Specificities are acoustic qualities specific to a given instrument. Considering these in the analysis allowed a
reduction from: 6 hard to interpret dimensions to 3 psychophysically meaningful dimensions: log-rise time,
spectral centroid, and spectral flux.
Big deal: this means 3 dimensions might be enough to “get around” in timbre space.
Acoustic correlates of timbre space dimensions:
A confirmatory study using synthetic tones.
(Caclin et al., 2005)
Aim
Participants
Stimuli
Statistical
Analysis
Results
To determine if the structure of the
perceptual space is closely parallel to the
acoustical space, particularly using attack
time, spectral centroid, spectral flux, and
attenuation of high harmonics as the physical
dimensions.
What is the auditory mapping between the
physical parameter and the perceptual values
3 experiments, 30 subjects
musicians and non-musicians
16 synthetic tones, perceptually equalized in
duration and loudness, homogeneously
distributed in space
variations in the attack time (log steps), SCG (linear
steps), spectral fux (equally spaced steps)
Outliers data discarded
Dissimilarity ratings analyzed using CLASCAL and
CONSCAL MDS models
Attack time and SCG were confirmed as
salient timbre parameters, but spectral flux
was not used in dissimilarity ratings when
attack time and SCG varied concurrently.
The experiments shed new lights about possible
third dimensions of timbre spaces, but the type of
spectro-temporal information that listeners use
remains unclear.
The items conserve their position along the
perceptual dimension, but their actual
position is derived from the physical values by
a continuous monotone spline function which
is di!erent for each person
In other words, people hear music in di!erent
ways.
The Timbre Toolbox:
Extracting audio descriptors from musical signals
(Peeters et al., 2011)
Aim
To provide a comprehensive set of audio
descriptors to perform perceptual research.
Analysis Corpus of 6037 pitched and non-pitched sounds
Technique
Results
For assesing the redundancy of information of the
descriptors, an experiment was developed to found
their correlation, thus allowing the selection of the
most independent ones.
53 audio descriptors for capturing the temporal,
spectral, spectro-temporal and energetic–global or
time-varying–characteristics of the set of sounds.
Descriptive statistics for characterizing
time-evolving descriptors
Correlational analysis and hierarchical clustering
in the set of descriptors is performed for examining
the information redundancy
The analysis suggests ten classes of relatively
independent audio descriptors
The toolbox is a multidimensional instrument for
the measurement of the acoustical structure of
complex sound signals
Technique in modern music
“If it is possible to make compositional structures from
sounds which di!er according to pitch, structures which we
call melodies, sequences producing an e!ect similar to
thought, then it must also be possible to create such
sequences from the timbres of that other dimension from
what we normally and simply call timbre. Such sequences
would work with an inherent logic, equivalent to the kind of
logic which is e!ective in the melodies based on pitch. All
this seems a fantasy of the future, which it probably is. Yet I
am firmly convinced that it can be realised.”
(Schoenberg, 1911:470-471)
J.S.Bach 'Ricercare a 6'. BWV 1079 (scores taken from Erickson, 1974)
Gestalt timbre:
Individual timbres are part of a whole
Cogan and Escot uses the spectrogram to study timbral orchestration
(Sonic Design, 1976) (Slawson, 1985)
Technique in computer music
“Though more research needs to be done, the notion of
transposing a sequence of timbres by forming another
sequence geometrically parallel to it in timbre-space thus
appears to be a reasonable and musically viable idea.”
(Wessel, 1979)
McAdams and Cunible (1992) show the validity of timbre
analogies, suggesting that timbre could be subject to musical
operations.
Similar to intervals in pitch and in time, the sucession of
intervals in timbre space defines a timbre melody
Vectors through timbre space - Technique in MDS
As sounds are located in the timbre space, we can think of timbre intervals as
vectors in the timbre space, and we can apply cartesian operations on these
vectors.
However, some questions arise:
What is the minimum distance interval (a.k.a. 'the semitone') in terms of timbre?
Does that distance change per dimension?
What is the role of rotation? Is the 2" spiral the timbral octave?
Could we develop a symbolic notation for timbre?
Pitch technique of the Western Art Canon can be extended
Chromatic transposition adds a fixed intervalic o!set to every pitch class.
Diatonic transposition adds the same interval class to every pitch class
Chromatic inversion is the negation of the pitch intervals, as measured in
semitones.
Diatonic inversion is the negation of the intervalic class.
Contour mantains direction, but manipulates the size of the intervals
Sticky Points
Timbral glissando, or discrete steps?
What does diatonic mean in the context of timbre?
It seems that chromatic and modern set theory techniques are most appropriate!
Transposition = O!set
Take a given vector path in timbre
space, and move it to a new starting
point
Inversion = negate a dimension
Take a given vector path in timbre space, and
negate one, many or all dimensions.
Note the possibility of a partial inversion –
this is novel to timbre!
Contour = scaling
Scale any given dimension
Tension and release
Tension based upon psycho-acoustics – amount of information and auditory
roughness.
If tension in timbre is achieved through roughness, is release achieved through
auditory smoothness? Release might be more than simplicity, sine tones are
smooth but unpleasant.
In equal tempered music, certain intervals have certain tendencies, e.g., the
dominant seventh chord 'points'.
Such parallels are not clear in common practice harmony, however.
Perhaps only the very general parallels to tension and release (of modern
harmony) are valid.
Hyper-timbres
It would be interesting to think about extreme, hyper timbres because
they could define, limit our sonic working space.
What are the extreme sounds that we could generate in a timbre space?
Are the impulse and silence located at the boundaries of the timbre space?
Does the timbre space have boundaries?
Would it be possible to go beyond the boundaries of a timbre space?
Discrete vs. Continuous
'Common practice' instruments are clustered in the timbre space. Here,
categorically identification may actually impede the use of timbre as a
compositional tool!
Instead of hearing smooth transitions, temporal/timbral segmentation wrecks the
intended timbre melody.
Modern composers began working with blend to 'work around' this problem.
Synthethic sounds can be manipulated to fill the space continuously – the sky is
the limit.
Music for 18 musicians, S. Reich
Musical Analysis of Timbre Structures
It has been common in the western art tradition for music theorists to analyze the
vocabulary of the accepted masters of composition.
In acoustic music up to the modern masters, an analysis of pitch and duration can
be derived from the score or by skilled ears. An analysis of categorical timbre can
also be derived from either the score or by ear – it only answers the question of
what instrument played what part.
Scientific analysis is more di#cult, especially once all of the parts are combined
into a given recording. The spectrogram tools suggested by Cogan and Escot
(Sonic Design, 1976, Slawson 1985) allow us to view the timbre structure at a very
high level, but would be unlikely to reveal anything similar to a timbre melody.
The problem is similar in computer and electro-acoustic music, but is confounded
by the extension of music beyond the note. Skilled ears might be able to detect
patterned timbre composition in these works, whereas spectrogram tools will give
a general outlay of the piece's overall timbre structure.
Finer grained analysis tools might give more insight into fine grained structures,
however a theory of analysis is not yet obvious.
Navigating timbre space
Étude des 1000 collants
Karlheinz Stockhausen
(1952)
Open
John Oswald
‘Plexure’ (1993)
Music to observe
electronic sheep to
Part I
Sven König (2006)
Designing control systems from the perceptual
representations
“The timbre space representation suggests relatively
straightforward schemes for controlling timbre. The basic idea
is that by specifying coordinates in a particular timbre
space, one could hear the timbre represented by those
coordinates.”
“The most natural way to move about in the timbral space
would be to attach the handles of control directly to the
dimensions of the space.”
(Wessel, 1979)
Frelia
Mandel and Momeni (2005)
CSS Technique
Concatenative sound synthesis techniques allow a user to segment a collection of
sounds into small units, analyze their sonic identity by means of the extraction of
their acoustic and perceptual features using descriptors, and arrange the sound
units into a multi-dimensional descriptor space according to their values.
By means of the manual or automatic selection of sound units from the database
and their concatenation, we can synthesize new sonic textures.
CataRT (Schwarz et al., 2006)
CataRT (Schwarz et al., 2006)
TimbreID (Brent, 2010)
SoundCloud (Vigliensoni, 2011)
SoundCloud extends the interaction possibilities of previous concatenative sound
synthesis systems by mapping the three-dimensional performance space to a threedimensional user-defined descriptor space.
The system allows a musician to perform with a corpus of sounds distributed in a
three-dimensional descriptor space by exploring a performance space with her hands.
SoundCloud (Vigliensoni, 2011)
ArcSyn (Romblom, 2007)
ArcSyn attempted to put timbre on the same footing as pitch and loudness for
performer:
"For dynamic evolution, Arcsyn uses both ! and pp steady-state spectral
information. Spectral morphing is achieved by adding a second, entirely
distinct timbre with its own spectral information. The third dimension of
interpolation is pitch; here we specify new spectral information at each welltempered note."
The spectral database was packaged as an "analog oscillator" with
dimensions of dynamics, timbre, and pitch.
Technique in modern music production
Technique in modern production
Some of the main ideas in timbre research are well represented in modern music
production.
In particular, the timbre space axes are related to popular forms of signal
processing: compressor/expanders, equalizer, non-linear distortion, and envelope
followers.
Axis 1 of many timbre space representations is log-attack time. This is related to
compressor / expanders, which manipulate the onset and sustain of instruments.
Axis 2 of many timbre space representations is spectral centroid. DAWs and
mixing consoles provide equalizers and filters for modeling the spectral shape of
a sound and changing its centroid. This processing technique allows producers to
give instruments and sounds their own frequency space to 'cut' in the mix.
Multiples timbres give rise to an overall timbre - Gestalt timbre!
We have speculated that timbral tension results from auditory roughness. Nonlinear distortion creates new partials, many times no longer harmonic. This gives
rise to beating and auditory roughness. Envelope followers allow producers to
give more life to dull sounds by changing their spectral flux.
Technique in modern production
The backbeat is a timbre interval, and it is omnipresent in popular music.
Second only to vocal treatment, the drum treatment is the next most important
production element in most genres.
Timbre matters: changing drum tones can alter the entire meaning of the song.
Wrap and Summary
Perceptual Issues
our perceptual auditory
system is adaptive
di!erent sound stimuli arise
di!erent timbre spaces
flexible, adaptive
strategies for working
with timbre
all reviewed
experiments used
isolated sounds
no context was
considered
does context a!ect our
timbre perception
our perceptual
dimension weighting
and mapping is unique
all of us listen di!erent
common ground of
timbral perception
Syntactic Issues
music is structure
music is context
define minimum,
constituent parts to
create the whole
"If the timbral di!erence between adjacent notes is large, then one tends to perceive
interleaved descending lines formed by the notes of the same timbral type."
(Wessel, 1979)
Wessel demonstrated that auditory stream formation and rhythmic organization of
klangfarben sequences could be predicted from a timbre space.
timbre space depends
on the context
timbre space is non-linear
unified timbre theory
COMPOSERS:
TIMBRE MATTERS!
THANKS!