Musical Creativity: Strategies and Tools in Composition and Improvisation

Computational Music Science

Series Editors Guerino Mazzola Moreno Andreatta

For further volumes: www.springer.com/series/8349

Guerino Mazzola r Joomi Park r Florian Thalmann

Musical Creativity Strategies and Tools in Composition and Improvisation

Prof. Dr. Guerino Mazzola School of Music University of Minnesota Minneapolis, MN, USA [email protected] Department of Informatics University of Zurich Zurich, Switzerland

Dr. Joomi Park McNally Smith College of Music Saint Paul, MN, USA [email protected] Florian Thalmann School of Music University of Minnesota Minneapolis, MN, USA [email protected]

ISSN 1868-0305 e-ISSN 1868-0313 Computational Music Science ISBN 978-3-642-24516-9 e-ISBN 978-3-642-24517-6 DOI 10.1007/978-3-642-24517-6 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011942218 ACM Classification (1998): H.5.5, J.5 © Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Every act of creation is first of all an act of destruction. (Pablo Picasso)

Yes, but always with a constructive intention. (Guerino Mazzola, Joomi Park, Florian Thalmann)

Preface

Writing this book was a special challenge because we knew that creativity is a core but quite mysterious topic in music. We had many ideas about creativity in theory, and we had also been working creatively as composers, improvisers, and music software programmers. But there was one very special and demanding part of the book that we wanted to write, namely the practical tutorial. We planned to complement theoretical and other high-end perspectives with a really concrete, practical, and teachable contribution. The result of this effort is the hundred-page tutorial, Part II. It significantly precedes the theoretical Part III because we wanted to offer a presentation that works for undergraduate students or for any reader who wants to see how the theory works when you apply it, without having to go through long and annoying theoretical discourses, also called “general nonsense”1 . Writing the tutorial was in no way a replication of known material. In each unit of the tutorial, we had to break through the standards of that theme and develop creative extensions of the status quo. We are by no means claiming that our solutions are unique or even optimal ones, but we hope that they demonstrate the validity of our general method for working creatively in music theory, technology, and performance. All tutorial chapters have been written as an exchange of ideas between Joomi Park (as a composer and pianist) and Guerino Mazzola (as a scientist). Joomi’s practical and musical contributions have strongly enhanced the value of the tutorial chapters as a model for a future course syllabus, as well as the case studies and the theoretical chapters of this book. 1

“General nonsense” is a well-known qualification of utterly abstract and general mathematical theories, in particular category theory. It was a style developed in the 1970s following Alexander Grothendieck’s success in algebraic geometry, which was achieved in this style, but then used by not-so-creative mathematicians. Around 1980, this style was strongly criticized, also relating to teaching abstract set theory in elementary school.

vii

viii

Preface

This book’s approach to creativity is a bit different from others since it follows less a psychological method than a semiotically shaped procedure. Of course, psychology is extremely important in individual and collective creative dynamics. But we wanted to present a method that can be taken as a backbone for any given challenge in musical creativity and that is operational, i.e. one can start working without first having to go through psychotherapeutic warmups. And although this approach is applied to musical creativity in this book, we believe that its generic character enables you to apply it in many creative environments and problem fields. Because our approach is so specific, we felt obliged to include a short review of what has been done in creativity research in the past and present times. Florian Thalmann has been charged with this delicate subject, and we are very grateful for his diligent presentation. We hope that our present contribution may help demystify the mysterious perspective in the popular creativity discourse, moving to a relaxed understanding of the term and following Albert Einstein’s statement, “Creativity is knowing how to hide your sources.” As in the previous book of this Springer series on performance theory, Emily King has been an invaluable help in transforming our text to a valid English prose; thank you so much for your patience with non-native English. We are pleased to acknowledge the strong support for writing such a demanding treatise by Springer’s science editor Ronan Nugent. Minneapolis, August 2011

Guerino Mazzola, Joomi Park, Florian Thalmann

Contents

Part I Introduction 1

What the Book Is About . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2

Oniontology: Realities, Communication, Semiotics, and Embodiment of Music . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Realities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Semiotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Embodiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 The Baboushka Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 6 7 7 8 9

Part II Practice 3

The Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4

The General Method of Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5

Getting Off the Ground . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6

Motivational Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

23 23 24 24 25 25 26 27 ix

x

Contents

7

Rhythmical Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

29 29 30 31 33 34 34 35

8

The Pitch Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

37 37 38 38 39 42 43 45

9

The Harmonic Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 9.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

47 48 48 49 49 51 53 55

10 Melodic Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 10.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

57 58 59 60 61 63 66 67

11 The Contrapuntal Aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 11.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

73 74 76 77 77 77 79 82

Contents

12 Instrumental Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 12.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . .

xi

85 85 86 87 88 88 89 90

13 Creative Aspects of Musical Systems: The Case of Serialism 93 13.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 13.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 13.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 13.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 13.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 100 13.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 101 13.6.1 Another Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 13.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 105 14 Large Form Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 14.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 14.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 14.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 14.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 14.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 112 14.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 113 14.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 115 15 Community Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 15.1 What is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 15.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 15.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 15.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 15.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 120 15.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 121 15.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 121 16 Commercial Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 16.1 What Is Your Open Question? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 16.2 Let Us Describe the Context! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 16.3 Find the Critical Concept! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 16.4 We Inspect the Concept’s Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 16.5 Try to Soften and Open the Walls! . . . . . . . . . . . . . . . . . . . . . . . . . . 125 16.6 How Can We Extend Opened Walls? . . . . . . . . . . . . . . . . . . . . . . . . 126 16.7 Final Step: Testing Our Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 126

xii

Contents

Part III Theory 17 Historical Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 17.1 The Concept of Creativity through (Western) History . . . . . . . . . 132 17.2 Creativity in Early Psychology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 17.3 Creativity Research in Recent Years . . . . . . . . . . . . . . . . . . . . . . . . 137 18 Present Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 18.1 The Creative Process Today . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 18.1.1 The Four P’s of Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 18.1.2 The Creative Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 18.2 Musical Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 19 Our Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 19.1 Approach to Creativity: A Semiotic Presentation . . . . . . . . . . . . . 149 19.1.1 The Open Question’s Context in Creativity . . . . . . . . . . . . 149 19.1.2 Motivation for a Semiotic Extension . . . . . . . . . . . . . . . . . . 150 19.1.3 The Critical Sign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 19.1.4 Identifying a Concept’s Walls . . . . . . . . . . . . . . . . . . . . . . . . 152 19.1.5 Opening a Wall and Displaying Its New Perspectives . . . . 154 19.1.6 Visual Representation of the Wall Paradigm . . . . . . . . . . . 155 19.1.7 Evaluating the Extended Walls . . . . . . . . . . . . . . . . . . . . . . . 157 19.2 Approach to Creativity: A Mathematical Model . . . . . . . . . . . . . . 157 19.3 The List of the Creativity Process . . . . . . . . . . . . . . . . . . . . . . . . . . 159 20 Principles of Creative Pedagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 20.1 Origins of Creative Pedagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 20.2 Applying Our Concept of Creativity to Creative Pedagogy . . . . . 163 20.3 Creative Pedagogy for Musical Creativity . . . . . . . . . . . . . . . . . . . . 164 20.3.1 Conceiving Our Tutorial in Creative Pedagogy for Musical Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 21 Acoustics, Instruments, Music Software, and Creativity . . . . 169 21.1 Acoustic Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 21.1.1 First Sound Anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 21.1.2 Making Sound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 21.1.3 Fourier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 21.1.4 FM, Wavelets, Physical Modeling . . . . . . . . . . . . . . . . . . . . . 180 21.2 Electromagnetic Encoding of Music: Audio HW and SW . . . . . . 185 21.2.1 General Picture of Analog/Digital Sound Encoding . . . . . 185 21.2.2 LP and Tape Technologies, Some History . . . . . . . . . . . . . . 190 21.2.3 The Digital Approach, Sampling . . . . . . . . . . . . . . . . . . . . . 191 21.2.4 Finite Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 21.2.5 Fast Fourier Analysis (FFT) . . . . . . . . . . . . . . . . . . . . . . . . . 196

Contents

xiii

21.2.6 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 21.2.7 MP3, MP4, AIFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 21.2.8 Filters and EQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 21.3 Symbolic Formats: Notes, MIDI, Denotators . . . . . . . . . . . . . . . . . 214 21.3.1 Western Notation and Performance . . . . . . . . . . . . . . . . . . . 214 21.3.2 MIDI: What It Is About, Short History . . . . . . . . . . . . . . . 217 21.3.3 MIDI Networks: MIDI Devices, Ports, and Cables . . . . . . 219 21.3.4 MIDI Messages: Hierarchy and Anatomy . . . . . . . . . . . . . . 220 21.3.5 Time in MIDI, Standard MIDI Files . . . . . . . . . . . . . . . . . . 222 21.3.6 Short Introduction to Denotators . . . . . . . . . . . . . . . . . . . . . 224 21.4 Creativity in Electronic Music: Languages and Theories . . . . . . . 231 22 Creativity in Composition and Improvisation . . . . . . . . . . . . . . . 233 22.1 Defining Composition and Improvisation . . . . . . . . . . . . . . . . . . . . 233 22.2 Creativity in Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 22.2.1 Composition by Objectivation . . . . . . . . . . . . . . . . . . . . . . . 236 22.2.2 Creativity in Composition with Symbolic Objects . . . . . . 237 22.3 Creativity in Improvisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 22.3.1 Improvisational Creativity in the Imaginary Time-Space . 241 22.3.2 Improvisational Creativity with Gestural Embodiment . . 243 22.4 Instant Composition and Slow-Motion Improvisation . . . . . . . . . . 245

Part IV Case Studies 23 The CD Passionate Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 23.1 The General Background of This Production . . . . . . . . . . . . . . . . . 251 23.1.1 The Overall Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 23.1.2 Joomi’s Compositional Approach . . . . . . . . . . . . . . . . . . . . . 252 23.1.3 Guerino’s Improvisational Approach . . . . . . . . . . . . . . . . . . 256 23.2 Softening One’s Boundaries in Creativity . . . . . . . . . . . . . . . . . . . . 258 23.2.1 Embodied Creation and the Crisis of Contemporary Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 23.3 The Problem of Creativity in a Dense Cultural Heritage of Compositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 23.3.1 First Wall: Composition, an Object? . . . . . . . . . . . . . . . . . . 262 23.3.2 Second Wall: Originality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 24 The Escher Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 24.1 A Short Review of the Escher Theorem . . . . . . . . . . . . . . . . . . . . . 267 24.1.1 Gestures and Hypergestures . . . . . . . . . . . . . . . . . . . . . . . . . 267 24.1.2 The Escher Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 24.2 The Escher Theorem and Creativity in Free Jazz . . . . . . . . . . . . . 273 24.3 Applying the Escher Theorem to Open Walls of Critical Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

xiv

Contents

25 Boulez: Structures Recomposed . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 25.1 Boulez’s Idea of a Creative Analysis . . . . . . . . . . . . . . . . . . . . . . . . 279 25.2 Ligeti’s Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 25.3 A First Creative Analysis of Structure Ia from Ligeti’s Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 25.3.1 Address Change Instead of Parameter Transformations . . 284 25.3.2 The System of Address Changes for the Primary Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 25.3.3 The System of Address Changes for the Secondary Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 25.3.4 The First Creative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 288 25.4 Implementing Creative Analysis on RUBATO . . . . . . . . . . . . . . . 288 25.4.1 The System of Boulettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 25.5 A Second More Creative Analysis and Reconstruction . . . . . . . . . 292 25.5.1 The Conceptual Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 294 25.5.2 The BigBang Rubette for Computational Composition . . 297 25.5.3 A Composition Using the BigBang Rubette and the Boulettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 25.5.4 Was This “Creative Analysis” a Creative Success? . . . . . . 302 26 Ludwig van Beethoven’s Sonata opus 109: Six Variations . . . 305 26.1 Uhde’s Perspective Metaphor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 26.2 Why a Sixth Variation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

Part V References, Index References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

Part I

Introduction

1 What the Book Is About

This book is about creativity. After a short introduction (Part I), it presents its practical aspects in a tutorial (Part II), the theoretical background (Part III), and four extensive case studies (Part IV). The practical tutorial is conceived as a skeleton for a university undergraduate course syllabus. It can be taught either in a one-semester course with a straightforward strategy going through the entire tutorial, or in a two-semester course with the suggested practical exercises in each of the tutorial’s chapters. It starts with an informal presentation of the subject of creativity (the formal theory is presented in Part III). We then discuss eleven topics in tutorial units. These units all deal with fundamental themes from music theory, performance, and social as well as commercial issues. These themes are all deployed to challenge the future composer in creative extensions of known material and methods. Our treatment of these topics is by no means thought to replace the traditional standard teaching in theory or performance. The idea is to lead these students to new frontiers that transcend standard points of view. The philosophy of this plan is that music is an ever-evolving field of artistic and scientific expression, and that such extensions can be achieved by following a general process scheme of creative exploration. Our approach should make clear that creativity is not that mysterious uncontrollable phenomenon of human originality which still dominates the overall discourse on this subject. Creativity is neither divine inspiration nor random spontaneity. We hope the reader will agree on this point after having run through the tutorials’ storyboard. The theoretical part is devoted to two main threads: an overview of the history and presence of creativity theories, together with our own theoretical approach, including a presentation of creative pedagogy; and a thorough discussion of the creative potential of music technology in hardware and software as well as a comparison of creativity in composition and improvisation. Music technology is included because we believe that any serious endeavor in new music must take care of the hard facts of embodiment. Music is not only a G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 1, © Springer-Verlag Berlin Heidelberg 2011

3

4

1 What the Book Is About

matter of thoughts and emotions; it relies on the physical environment where sounds are produced and where instrumental interfaces are designed. Part IV, case studies, is a spectrum of four topics where creativity plays a crucial role. Our selection includes (1) a discussion of creativity in a recent CD production, involving authors Joomi Park and Guerino Mazzola; (2) the analysis of a systematic procedure, the Escher Theorem, for realizing creative extensions; (3) a technology-oriented discourse of a software that enables a recomposition of Pierre Boulez’s Structures pour deux pianos; (4) an interpretation of Ludwig van Beethoven’s six variations in the third movement of piano sonata op. 109 as a creative achievement in sonata theory. We hope that our threefold perspective (Parts II, III, IV) on creativity will inspire students, instructors, and researchers to think in new ways about what creativity is about.

2 Oniontology: Realities, Communication, Semiotics, and Embodiment of Music

Before we embark on the detailed investigation of musical creativity, we should ask about the localization of such creation: Where are those works which composers and improvisers are generating? And also: Where are the generative processes and gestures located, and is there an option to view all these objects and activities in a big existential topography? To answer this complex questionnaire, we want to prepend the technically detailed discussion with a complete picture of musical existentiality: where music exists and how it comes into being. Musical ontology is precisely the philosophy and field of knowledge that deals with these questions. This approach will give us the necessary conceptual architecture to unfold a presentation of the subject that comprises all relevant perspectives and that enables us to interconnect them in the framework of a unified understanding of music. We shall now describe the general setup of musical ontology. Although general musical ontology has been described in a concise way in [85], in [82], and in [89], we want to recapitulate it here for the sake of a self-contained text, and also to stress certain aspects to be more adapted to the topic of creativity. Let us start with what we have coined musical oniontology. It is the classical musical ontology that was introduced in [72], comprising the dimensions of realities, communication, and semiotics, but now enriched by a fourth dimension, embodiment. Since this fourth dimension splits into three layers, the classical ontology is given a triple-layered structure, hence the somewhat fancy name of an “oniontology” (see Figure 2.5). This oniontology presents a topographic landscape of musical ontology: It is a geographic display of localities determined by coordinates as specified from the four dimensions of ontology. In other words, musical oniontology is a conceptual space on which phenomena of musical existence are distributed. This spatial display enables us to understand creativity as a dynamic process that retrieves its contextual data from the oniontological landscape. So let us present all the dimensions of musical oniontology.

G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 2, © Springer-Verlag Berlin Heidelberg 2011

5

6

Oniontology: Realities, Communication, Semiotics, Embodiment of Music

2.1 Realities

Fig. 2.1. The three fundamental realities of music.

This dimension describes the three fundamental values of reality involved in music: physical reality, psychological reality, and mental or symbolic reality (see Figure 2.1). So, acoustical phenomena relate to physics, emotional effects to psychology, and symbolic structures, e.g. mathematical descriptions in music theory, to the mental reality. Observe that the mental reality is not conceived as being a part of the psychological one, since mathematical reality (for example) is independent of our human psyche. Differentiation of realities is crucial for avoiding widespread misunderstandings about the nature of musical facts. For example, the retrograde of a melody is a clear mathematical fact in the mental reality, but its reality in the psychological understanding of a listener is mostly inexistent: You probably cannot recognize the retrograde of a melody when hearing it.1 Methodologically, there is no reason nor is it ontologically possible to reduce one reality to others. For example, it is a logically vicious circle to try to reduce mental reality to physical reality, as happens in fashionable neuroscience. In fact, explaining mathematical thoughts by neuroscience would mean describing them by chemical and physical processes. But their description would enforce quantum mechanics of chemistry and other basic theories of physical processes. Such descriptions, however, would be based on the complex mathematics of quantum mechanics and therefore generate a vicious circle: explaining maths by maths. 1

A more complex example of this problem is Fourier’s theorem, roughly stating that every periodic function is a unique sum of sinusoidal components, but see section 21.1.3 for a precise discussion. Its a priori status is a mental one, a theorem of pure mathematics. In musical acoustics, it is often claimed that—according to Fourier’s theorem—a sound “is” composed of “pure” sinusoidal partials. However, there is no physical law to support this claim. Without a specific link to physics, Fourier’s statement is just one of an infinity of mathematically equivalent orthonormal decompositions based on “pure” functions of completely general character. To give the claim a physical status, it would be necessary to refer to a concrete dynamical system, such as the cochlea of the inner ear, which is physically sensitive to the first seven partials in Fourier’s sense.

2.3 Semiotics

7

The problem is rather to describe the transformation rules from the manifestation of a phenomenon in one reality to its correspondences within the others. To be clear, a neurophysiological transformation (“explanation”) of a psychological phenomenon does not, however, conserve the psychological ontology of the phenomenon. The specific phenomenon within the psychological realm corresponds to another phenomenon within the physiological realm. But ontologically, the phenomena do not collapse.

2.2 Communication

Fig. 2.2. The three stages of communication in music.

Following the famous scheme of Jean Molino and Paul Val´ery [72], music deals with communication from the first value, the poietic position of the composer or creator, to the creator’s work, which is the material essence and output of the second value, called neutral level by Molino (see Figure 2.2). Communication, as encoded in the work, targets the third value: the aesthesic position of the listener, the addressee of the composer’s message. Val´ery coined the word “aesthesic” to differentiate it from the aesthetic understanding. Aesthesis means perception and can be acoustical, psychological, or analytical, and it needs not relate to aesthetical evaluation. The aesthesic instance could even be computer software that takes a MIDI file as input and processes an analytical task thereof.

2.3 Semiotics This axis (Figure 2.3) comprises all sign-theoretic aspects of music. It is articulated in the three classical constituents of a sign: expression, content, and signification. Expression, the first value on this axis, relates to the surface of a sign, something that stands for the sign’s meaning or content. The latter, content, is the second value—the “aliquo” in the classical definition “aliquid stat pro aliquo” (“something stands for something else”) of a sign. The third value is the signification part of a sign. It refers to the middle word “stat pro” of the

8

Oniontology: Realities, Communication, Semiotics, Embodiment of Music

Fig. 2.3. The three positions in musical semiotics.

classical definition and explains the way or process engaged for the transfer of the surface value of expression to the “hidden” value of content. For example, when reading the musical expression for a fermata, the reader must invoke a complex machinery to understand the expression, i.e. produce the symbol’s content. In a semiotic system, there are three main structural aspects: The pointer from expression to content is called semantics; the connection between different signs, such as is built in sentences of a language, is called syntagm; and the mechanisms of signification applied by the user of the semiotics is called pragmatics. The classical three-dimensional cube of musical topography is shown in Figure 2.4.

2.4 Embodiment The very making of art is a level that is not articulated in that threedimensional cube of musical ontology. Not one of its twenty-seven (3 × 3 × 3) positions grasps the gestural aspect of making art (and science). The cube, strictly speaking, only deals with the ontology of facts, of “what is the case” in Ludwig Wittgenstein’s sense [127]. It does not include the processual level of ontology. Formally speaking, processes are the diagrams of spaces and transformations that describe the interaction of components of a complex system. We have to differentiate between processes and their products, the output of pro-

2.5 The Baboushka Principle

9

Fig. 2.4. The classical three-dimensional cube of musical ontology.

cessual dynamics. Processes are a kind of factory for facts, but not the facts themselves. The processual level is fundamentally different from its output products. Processes and facts are instances of different ontologies. Going still farther in the initiated direction, processes are also an abstraction from a more basic layer—namely, the gestural layer—where all processes and their facts are initiated. Processes are disembodied gestures, reduced to their referential system of transformations. This means that a new dimension must be added to the cube of musical ontology. This fourth dimension is called dimension of embodiment. Its three values are: facts, processes, and gestures. They deal with, respectively, these activities: “what is the case,” “to refer to,” and “to make.” In this scheme, the transition from gesture to process is dominated by disembodiment and schematization, whereas the transition from process to facts is dominated by evaluation and dissection (from the relating transformations). Together with the previous three-dimensional cube of ontology, this fourth dimension creates a four-dimensional cube, which we call the hypercube of musical oniontology. It takes the form of a three-layered onion of gestural, processual, and factual levels of ontology, as shown in Figure 2.5.

2.5 The Baboushka Principle The above dimensions do not mean that musical ontology is indecomposably inscripted in such coordinates. It mostly happens that the 3 × 3 × 3 × 3 coordinates are themselves encapsulated subsystems of the same nature. This

10

Oniontology: Realities, Communication, Semiotics, Embodiment of Music

Fig. 2.5. The hypercube of musical oniontology defined by the fourth dimension of embodiment. The graphics illustrate facts (the M¨ obius strip as a configuration of tonal degrees), processes (a diagram from Lewin’s transformational theory), and gestures (a model of the pianist’s hand).

reiteration of the hypercube’s structure is called the Baboushka principle. It does not mean that new dimensions are generated, but that each position in the hypercube can recursively be the compact representation of still a finer hypercube of the same type. Let us make this clear on the two examples of semiotics and communication. In the semiotic dimension, it is a classical result from Louis Hjelmslev’s investigations [52] that the expressive surface of a semiotic system may be a semiotic system in its own. This is the case, for example, in so-called double articulation in language. Here, the words—expressions of the language sign system—are also signs with a graphical expression—the written level of alphabetization—that signifies its acoustical content. This level or semiotic ramification within the expressive level of the top system is called connotation. If, on the other end, the content level is itself a semiotic system, the comprising system is called a metasystem. And if the middle layer of signification is a semiotic system, the comprising system is called a motivated semiotics. It can be shown that music is built from a repeated imbrication of connotative subsystems [76, 77].

2.5 The Baboushka Principle

11

In the dimension of musical communication, the overall poietic position may be seen (and this was our example above) as articulated in composer, work (score), and interpreter, whereas this entire communicative unit is the poiesis that generates the neutral level of a performed work (on the acoustical level, say, when the performance is taking place in a concert), which in turn reaches the aesthesic level of the audience. We leave it to the reader to imagine such Baboushka configurations for the dimensions of realities or embodiment. Hint: Think of the physical reality of a symbol, the symbolic representation of a physical sound, the facticity of a gesture, or the processual scheme underlying a gestural utterance, etc.

Part II

Practice

3 The Tutorial

This book has two parts, a first practical one, and a second theoretical one. We have decided to prepend the practical part because we believe it is possible and also our obligation to the students and instructors to access creativity without having to go through heavy theories. Those who want to learn about more theoretical topics will find a lot of theory and case studies in the theoretical part. The practical part is essentially a tutorial in creative pedagogy of music. In chapter 20 we give an in-depth definition and discussion of what is creative pedagogy in general, and what we add to the general approach in this book. For the present practical aspect, let us just say that creative pedagogy is a pedagogical method to teach whatever field of knowledge or art by letting the student experience learning as a creative activity. So we do not teach creativity, but via creativity. However, in our special field of music, we will teach not only via creativity, but also the specific creativity in musical composition. We believe that this double presence of creativity will give the student an excellent education, which should free him/her from that mysticism that usually radiates from the word “creativity,” a mysticism that immediately refers us to those great geniuses of music Bach, Mozart, Beethoven, Schumann, Liszt, Schoenberg, Berg, Webern, Scriabin, Bartok, and so on, and that paralyzes our own power of invention instead of boosting it. We do not claim that creativity is a mechanical competence you can learn like you can learn to solve linear equations, but we claim that this tutorial and the backing theory can eliminate a good number of wrong obstructions of psychological and methodological natures. This tutorial is conceived to be used as a skeleton of a syllabus for a university course in musical creativity at a school of music. It is structured in eleven units—each of which could be covered in a week over three hours— which deal with specific aspects of musical creativity. Our choice is by no means unique, but it is a prototypical setup that could, of course, be shaped differently according to the weights chosen by the instructors. Here are our eleven aspects: G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 3, © Springer-Verlag Berlin Heidelberg 2011

15

16

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

3 The Tutorial

Motivational Aspects, chapter 6 Rhythmical Aspects, chapter 7 The Pitch Aspect, chapter 8 The Harmonic Aspect, chapter 9 Melodic Aspects, chapter 10 The Contrapuntal Aspect, chapter 11 Instrumental Aspects, chapter 12 System Aspects, chapter 13 Large Form Aspects, chapter 14 Community Aspects, chapter 15 Commercial Aspects, chapter 16

These units are not independent of one another. For example, the harmonic aspect unit presupposes the pitch aspect. The overall image of dependence is known as “leitfaden” and is depicted in Figure 3.1 below. The leitfaden enables different flow paths through the total material if preferences vary.

Fig. 3.1. The tutorial’s leitfaden.

4 The General Method of Creativity

While we give a theoretically founded presentation of creativity, its history, present trends, and our own approach in chapters 17, 18, and 19, we want to describe here the essentials of our approach to creativity in practical terms to be used in the tutorial. Our approach to creativity is not a psychological trick nor does it incite a state of ecstasy, possibly boosted by drugs and similar devices. It is a process of discovery and invention that begins with an open question and continues with a run through a sequence of well-defined operational steps. It might not be successful in finding a solution to the question, but we can start over and repeat the processual run until we find a solution. Creativity cannot be guaranteed, but there are good reasons for approaching it based upon a clear strategy. Again, we discuss many examples of successful creativity in the theory part and hope that the reader will accept our approach here as a basis for our practical work. The process of creativity is composed of the following steps: 1. 2. 3. 4. 5. 6.

Exhibiting the open question Identifying the semiotic context Finding the question’s critical sign or concept in the semiotic context Identifying the concept’s walls Opening the walls and displaying its new perspectives Evaluating the extended walls

Let us briefly describe them, together with a simple example: the invention of 3M’s Post-It (see Figure 4.1). Step 1, exhibiting an open question. Creativity should start with the will to find something new that will answer an open question. It has a target. Example: In 1968, Dr. Spencer Silver, a chemist at Minnesota’s 3M company, developed an adhesive that did not really glue. He did not know what to do with it. What would be the usage of such a substance? Step 2 consists of identifying the context of this question. This is a natural requirement: We want to know what is the overall position of the question, since G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 4, © Springer-Verlag Berlin Heidelberg 2011

17

18

4 The General Method of Creativity

for one individual the question might be open but trivial, while for another it might be a big challenge. Also, we stress that the context is a semiotic one. What does this mean? It means that the context should be one of things, thoughts, and signs that mean something. The open question then would be one that creates new contents when answered. It would extend the given meaningful context. So creation in our understanding is about creating new contents, not just forms. Example: The context is the chemical industry. Including a variety of more than 50,000 3M products. Step 3 is the moment of focusing on a specific location in the context, a critical concept or thing where we guess that the open question could be made more precise. Example: The critical concept here was “adFig. 4.1. The Post-It notes. hesive.” Dr. Silver had invented an adhesive that was critical—it did not glue as required, but only “half of it.” Step 4 is very important. It asks us to identify the concept’s walls. Walls are a metaphor for properties, characteristics, and specificities of the concept that circumscribe the concept in a more or less explicit form. It is a delicate task here, since some properties might be so subtle that one is barely capable of recognizing them. Example: The walls of the concept “adhesive” was that one would expect that an adhesive has to glue, meaning 100 percent not just somewhat. It was a wall because it was so evident that an adhesive had to glue, by its very definition! Step 5 asks us to consider these walls and to try to ‘soften’ and to ‘open’ them. This means that we ask to what extent these walls are necessary for the critical concept, and whether we could possibly find ways to open them and to recognize new perspectives on the other side of the given walls. Example: The softening of the “glue” wall meant that one had to put into question this strong requirement of 100 percent gluing. It was Arthur Fry, who attended one of Silver’s seminars, who successfully opened that wall. He sang in a church choir and was frustrated with the paper bookmarks he used to mark the songs in his hymnal because they would not stay put. Fry realized that Silver’s adhesive met exactly his needs. Step 6 is the terminal step. It consists of the evaluation of wall extensions we may have found in step 5, and then the judgment of this evaluation’s result. Is it a successful extension of the original critical concept or didn’t we find an answer to the open question? If we are successful, everything is OK; otherwise, we have to go back to step 4 and find new walls and new extensions, or even to step 3 and look for new critical concepts. Example: In 1980 Post-It Notes were introduced nationwide in the U.S. and soon became a big success.

4 The General Method of Creativity

19

Remark 1 It is, of course, easy to give an example of a creative strategy that very probably would not be successful. Suppose that an open question in number theory is given. The critical concept could be “number.” And the wall one could discover could be “the number’s color” when you write it down on paper, on a blackboard, or on a computer screen. You could have the ‘ingenious’ idea that the number’s color could be extended from the usual black, blue, or red to any fancy color. But it is quite clear that such an extension would not solve the original number-theoretical question. Remark 2 We should not conclude this informal introduction to our creativity process scheme without pointing out the deep impact of representing ideas and methods in a sensual way (touch, smell, taste, hearing, sight). We have already established the sight- and touch-oriented metaphor of a wall described in steps 4, 5, and 6. When unfolding creative processes, we should always embed our activity in an object-oriented environment, where we feel at home and comfortable. Humans cannot think in abstract categories without using sensual metaphors. Even the most abstract mathematical thoughts are always embodied in objects that can be manipulated by human gestures. This is one of the secrets of successful mathematicians. Einstein, for example, admitted that his thoughts were always performed with an intense chewing activity. He was literally chewing ideas! Last but not least, we should be aware that probably the most creative force is love for life. After all, this is the strongest motor of human propagation. For a successful creation, the atmosphere should be a warm and loving one1 ; creation out of hate or indifference is never for life. A creation is a mental baby; it needs a ‘mother’ and her carrying body.

1

Sidney Lanier says, “Music is love in search of a word.”

5 Getting Off the Ground

The following tutorial is not a full-fledged syllabus, but it should give you, as a student or as an instructor, enough information to complete such a syllabus and to run through a course in creative musical composition applying creative pedagogy. For convenience, in the tutorial, we use the term “composition” to represent both, composition and improvisation. If we review the tutorial’s topics, one might have the impression that this course replaces music theory courses, counterpoint, etc., but this is not the aim of the tutorial. It is more a fresh look at the fundamental target of creativity in music, using this special type of pedagogy. We could imagine giving such a course to freshmen or sophomores. But as a course in composition, it would also be an ideal educational contribution. Why? Because composition, and musical creativity in general, is not viewed as comprising theoretical fields, too. Theory is given, and might or might not be used in composition, but to include theory in one’s compositional creativity is not standard, although making music should not halt in front of theory. We have strong evidence that musical laws are not yet in their final state; musical evolution is still in its Planck time (that very short initial time after the Big Bang when the natural laws were not defined yet). This tutorial, therefore, is a trajectory through music as a creation that the composer realizes with full responsibility—not just taking gadgets from the shelf, not doing just anything, but creating in a thoroughly constructive way. It might also seem that the trajectory through this tutorial starts with a huge spectrum of freedom and chaos, and then successively restricts one’s freedom by the variety of structural castings. This is only appearance. Creativity is not the limitless extension and chaos and destruction of limits; it is an evolutionary process that builds up spiritual cathedrals, much as biology has its most sophisticated creations in a very limited environment on a very special planet and for a very short time, presumably. In our tutorial, we shall always suppose that a ‘model student’ is taking the course and therefore also suppose that his/her achievements in the previous G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 5, © Springer-Verlag Berlin Heidelberg 2011

21

22

5 Getting Off the Ground

lessons could be used in later lessons. This creates a natural coherence in the pedagogical development. We also might suppose that our model student has some preliminary knowledge in musical notation or even might know how to play an instrument.

6 Motivational Aspects

This is the very beginning of our tutorial. In this initial unit, we want to approach creatively the very motivation you, the student1 , may discover and set up for the entire study lying ahead: more precisely, for the will to compose music. This is a very general situation, as motivations to compose music may stem from very different areas of human engagement. And the student might also envisage very different musical styles he/she wants to compose: classical, contemporary Western music, jazz, pop, rap, world musics, etc. Or the student just has a certain sound in his/her ears without any stylistic specification. Further, it is very important to also discuss the psychological motivations backing the student’s decision. This is a wide and complex field, but the instructor should also give it a chance in this first tutorial unit. The student we shall present here makes a specific choice for his/her motivation. But we encourage the instructor of a real course to group motivational directions of the students and to develop all of these threads in the creative process.

6.1 What Is Your Open Question? You have chosen to want to compose music. To be a creator of musical works. That’s why you are a student in this course. Beautiful! Let us first listen to different genres of music you might want to compose and then discuss your ideas on the styles you want to compose: classical Western music (Mozart, Beethoven), contemporary Western art music (Schoenberg, Nono, Stockhausen), classical jazz (Ellington, Parker), modern jazz (Davis, Coltrane, Taylor), rock ’n Roll (Preseley), pop (Michael Jackson), rock (Alanis 1

We shall henceforth always use this classical pedagogical form of a fictitious dialog with a student.

G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 6, © Springer-Verlag Berlin Heidelberg 2011

23

24

6 Motivational Aspects

Morissette), hip-hop (Meshell Ndegeocello), rap (Public Enemy), world music (Don Cherry), etc. Let us now also discuss what and why you want to compose. Exercise 1 Tell us (in your own words) the music style in which you would like to compose. It does not have to be precise, but it is important that you actively choose or even create this style. Exercise 2 Tell us your emotional, social, artistic, or economic reasons for wanting to compose. Let us know about your open questions! We now want to follow your open question and show how the creativity process may help solve it. Your question when becoming a composer is about the chance to compose new music. This is what you want to achieve. So let us first envisage your unanswered question: Why should you be motivated to create new, unheard musical works?

6.2 Let Us Describe the Context! There is nothing more important than the context where this initial question must be positioned. After all, you are not alone on Earth, and it is a big burden to want to compose in a culture where so many great (and not so great) works have already been written and performed. It is also important to think about for whom you want to compose, and for which communication, media, style, etc. Let us know your understanding of the context. We will suppose here that for you the context is the entire society, with its history, cultural diversity, music media, and technology.

6.3 Find the Critical Concept! What is the critical concept on which the question of your motivation would focus in the broad context you have just defined? You seem to be concerned with the problem of wanting to compose new works in a context that first of all gives you the impression that everything has already been said, or at least said in a way that nobody could outperform. Who can write more beautifully than Mozart, who can swing better than Count Basie, who can play the piano more energetically than Cecil Taylor, who can write more divine structures than Bach, who can invent more substantially than Beethoven? And so on. Also let us listen to some of the music you have in mind, music that impresses you a lot or music that you would never want to compose.

6.5 Try to Soften and Open the Walls!

25

So, what is the critical concept here? Let us guess that you choose create new music, and let us specify for the experts that you write “new” and “music” with lowercase letters, not uppercase letters, which would be the name of a very special movement in composition that took place in the first half of the twentieth century. So this is what you are concerned with: writing new music in a context where everything seems to be there already, or at least essentially.

6.4 We Inspect the Concept’s Walls! We now want to discuss the critical concept’s walls! What is the concept’s limitation in your chosen context? The strongest wall is that ominous attribute new. You have to make something new in the context of a world of already created works. Is new music still possible? This is how you judge your situation: You are a young student of composition and you stand in front of a big wall asking to create really new music. At first, you feel like this is an impossible attribute; how can new music be created at all? Your critical concept looks like an impossible task. A second wall is connected to this first one: creation. How can you make such new music? Do you have to invent it or can you possibly find it in an eternal repertory as it is described for the great composers, who are inspired by a divine message and who do not essentially make new music by their own action? So let us investigate the meaning of “new” and the role of the associated “creative” action.

6.5 Try to Soften and Open the Walls! Let us first attack the question of novelty. Is new music still possible? Look at all the scores that have been written in Western notation. Think about how many signs for notes, pauses, bar lines, etc., you have in such a score. Without going into combinatorial details, you will find out that almost all potential scores that can be written have not been composed yet, when you are using a given huge number of such signs and limiting the signs to be contained in reasonable value containers. So it is not true that no new music is possible, at least when speaking about the score form. In other words, all potential scores are already given in an abstract mathematical sky—you only have to find the good ones. This opens two problematic threads: First, if this were the point of novelty, Beethoven would not have created new music but just found it in an eternal repertory. Second, the first wall would throw you back to the second wall: Creation is not possible, so it is only about inspiration, finding music in a huge combinatorial repertory. But then, what is your chance to share this inspiration?

26

6 Motivational Aspects

Do you have any chance to be connected to those divine resources of ingenious inspiration? These are the walls in their seemingly impenetrable rigidity. Creating new music appears as the perfect illusion that offers no vital motivation for becoming a composer. In the Middle Ages, it was in fact believed that no creation by humans was possible, and all music came from God. The solution against this apparent impenetrability is to become aware of what it means to create a composition. It is certainly not about creating new combinations of score signs, but about the meaning, which is added to the given understanding of music. It is similar to the writing of a poetic text. The combinatorial question is also clear: Every possible reasonable text is already contained in a mathematical sky of combinations of letters and spaces. But what we add is not this formal level of letter conglomerates; you add your own meaning to given semiotic contexts. Don’t try only to compose unheard sounds. Einstein’s energy formula E = mc2 is a simple combination of letters, but its meaning is revolutionary, and this one cannot be reduced to letter combinations. In music, even the empty score can be an example of new music. John Cage’s famous composition 4 33 , composed in 1952, is an empty score and its only specification is the duration of 4 minutes and 33 seconds. But its meaning is a novelty: It puts into question the score form as such. Is the score form already music? Is it the sounds that happen during the performance of 4 33 that define its music? Is the score music, as much as an empty canvas is painting? To put it into simple words, the softening of these walls consists of questioning the meaning of music that you may create. You do not create scores, but the meaning of scores. So let us follow this thread and see what could be added to extend these opened walls.

6.6 How Can We Extend Opened Walls? But what is the direction of our opening gesture? Is it about the score signs only? We have learned that it is about extending the meaning of a combination of signs. The score is such a combination, but if we think about the general situation, it means that you may think of adding meaning to any formal sign combination in the music you encounter, and, more generally speaking, to any explicitly determined formal system of musical entities. Does this make sense? Exercise 3 Think of all the constraints you have when composing music: It is not only about the score signs, but also about the rules that are applied when setting the score signs. It is about counterpoint, harmony, any kind of music theory, and it is about the use of instruments, which are also rigid machines for sound production according to well-defined rules of usage. Also think about

6.7 Final Step: Testing Our Extension

27

your personal psychological or social constraints when composing. Perhaps you would never compose without getting less than $ 1 million. . . What we learn here is that the extension of our walls could lead you to question any fixed system of formal objects and rules, to think about extension of meanings, of signification, the making of music as a meaningful human activity.

6.7 Final Step: Testing Our Extension So, suppose now that you have discovered what kind of musical composition you are aiming at, given your personal situation. You have just developed an extension of the walls of novelty and creation in music by the dimension of meaning and by the requirement of added contents. This is a good direction: It opens novelty to a potentially infinitely deepening of meaningful music, and it also dispenses you from waiting for divine inspiration. It is now your own responsibility to add new contents, and to explore how musical works may become meaningful to our society and with respect to the works which your predecessors have composed. So let us see how strongly you can deepen musical meaning in the following topics. Try to always question given meanings, given forms, and given standard procedures.

7 Rhythmical Aspects

We now suppose that you found your motivation to delve into the art and business of inventing music. Let us stress that we are not presupposing any given musical system of rules, instruments, or musicians. We want to see how music can be created from scratch, from the chaotic universe of sound and its possible generators. You are totally free to start this enterprise, but you want to create music, and we want therefore to investigate the basic realities of music. The first of these realities is time. Time is a mysterious thing. Saint Augustine once stated that he knew what time was, but when asked about time, could not define it. So let us get started with that fascinating phenomenon of time.

7.1 What Is Your Open Question? Let us first listen to some examples of music and focus on the music’s time structures. Suggested examples: Bach: Well-tempered Piano, Beethoven: Appassionata, Scriabin: Vers la flamme, Ligeti: Po`eme Symphonique for 100 metronomes, Japan’s traditioal Nˆ oh Music, Coltrane: Meditations, Taylor: Live in the Black Forest. You will observe that there are so many different ways to deal with time in music. Exercise 4 Please describe (in your own words, no music theory needed!) the different variants of time in the music that we have just listened to. You become aware that music unfolds in time, but it also creates time, it is an art of time. It strongly matters how time is shaped in music. It is similar to painting, which is an art of space; the canvas is given, but the painter has to make the artistic space when shaping the canvas with brush and paints. You (the model student in our tutorial) therefore come up with the big open question: G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 7, © Springer-Verlag Berlin Heidelberg 2011

29

30

7 Rhythmical Aspects

How can we shape the body of time? And what kind of time is this? Is it given—do we just fill in our notes in a given timeline—or do we make it, and how?

7.2 Let Us Describe the Context! What is the context of your open question? We think about shaping time in music. So the context is anything that relates to time in music. Let us name those time-related concepts that come to mind. Tempo: This relates to the speed with which you produce sound events. In music, there are classical names for different tempi, such as largo, andante, moderato, allegro, presto, etc. But tempo is not the critical concept we are looking for, because it just tells us how fast the shaping of time is performed. Beat: The beat is a basic regular pulse in a music piece, and it is usually perceived by musical events being played louder. Beat is also played at a certain tempo, so it helps shape time at an elementary level, but it is totally regular and cannot claim to be the critical concept. It is only auxiliary like tempo. Groove: This is a feeling mainly created by rhythm sections (bass, drums, guitar, keyboards). It is akin to beat because it also refers to regularly played material. But it is very subjective and might be too complex, too ‘critical,’ to give us the critical concept. Also, there are many music styles where there is no groove, and the shaping of time is very strong, like Nˆ oh music. Duration: This refers to how long a sound lasts. Of course this is important, but it is far from central. Conduct an experiment: Play a melody with its notes having normally set durations, and then play it with all durations being very short (staccato). You will still recognize the melody, although it sounds strange. So there is something more essential than duration when shaping time. Observe that when you look at a Western score, durations are given by the notes’ heads and a number of flags for short notes. The upper half of Figure 7.1 shows notes of different durations. And also note that the written duration in the score is not the real duration, in seconds, say. This one is only defined if we know at which tempo the piece is played. The metronome, invented by Johann Nepomuk M¨ alzel in 1815, is the machine that gives us the tempo with its clicks of variable speed. Pause: In music, we do not only have events, but also eventless moments, moments when you should not play anything and you should do this for a certain duration. Pauses are precisely the symbols invented to prevent a musician from playing sounds. One might at first be astonished to think of pauses as being positive structures in music, but if we think about time being also shaped when nothing sounds, this is not miraculous anymore: Silence is a positive musical quality. The lower half of Figure 7.1 shows pauses of different durations. Time Signature: When musical scores organize time, they use the socalled time signature. This means that we divide time into units of the same

7.3 Find the Critical Concept!

31

Fig. 7.1. Notes and pauses of different durations in Western score writing. The durations are, from left: 1, 1/2, 1/4, 1/8, 1/16, 1/32, 1/64. It is wrong to think of durations as multiples of certain beats, such as the whole note being composed of four beats. Durations and beats are totally independent concepts.

duration, such as 2/4, 3/4, 4/4, 6/8, and so on, and each of these units is delimited by barlines and defines a measure. So in signature 3/4, the measure has duration 3/4, i.e. three quarter-note durations. But it is more than that: The numerator also tells us into how many beats the measure is divided and which note carries the beat. So, there are three beats in 3/4, but six beats in 6/8. Although the duration of one measure is the same (3/4 = 6/8), the beat of the music is different. Gesture: In music, when we view it as related to dance and to body movements of performing musicians, gestures play an important role in the shaping of time. Gestures are, however, very complex and difficult to describe in precise terms. Like groove, they might be ‘too critical’ as a concept to get off the ground with shaping time in music. All these concepts define the context of time-related phenomena in music. Some are more symbolic and relate to scores, some are more physical and relate to the real time in performed music. But you will agree that all of them are somehow special, secondary, or even too critical to get off ground with our basic understanding of time in music.

7.3 Find the Critical Concept! There is still one concept that we have not named but is well known in timecritical discussion of music, and this is rhythm. Rhythm shapes time by definition via a collection of time events, each having a determined duration and being played at a constant or variable tempo. The other sound parameters, such as pitch or loudness, are not our concern here. Let us look at a typical rhythm in Figure 7.2. The figure shows the fanfare in the first measures of Beethoven’s op. 106, the famous Hammerklavier sonata. In the upper half we see the score, in the lower half we see the corresponding rhythm. We have eight notes, a first group of four (the red points) and a second group of four

32

7 Rhythmical Aspects

Fig. 7.2. Above: The fanfare in Beethoven’s Hammerklavier Sonata, op. 106, the beginning of first movement (Allegro). Below: The onset times and durations of the fanfare notes in multiples of 1/8.

(the green points). The collection of eight time events is shown in the lower half of the figure time unit is 1/8. The first note has duration 1/8, the second has duration 3/8, the third one again duration 1/8, the fourth one duration 2/8, the fifth and sixth one have duration 1/8, the seventh and eighth have duration 2/8. This rhythm is not only a collection of time events, but also shares a characteristic internal shape, namely that the first and second pair of events are each movements from short to longer notes, they are some kind of repeated slowing down. The second four-note group relates to the first in that it now is not a repetition of slowing down, but a slowing down of a repetition: Eighth notes are repeated and then their repetition is slowed down and we see a repetition of quarter notes. Beethoven’s tempo prescription (138 1/2 notes per minute) of this fanfare is quite fast, so we are hearing an energetic shape of time as an initial stamp given to this famous sonata. Rhythm has what psychologists call a “gestalt,” a shape; it is a collection of time events with a significant internal structures. Exercise 5 Look at other rhythms in music and try to describe them by onset and duration of their notes. Try also to play rhythms and if possible to notate them, even in your private notation.

7.4 We Inspect the Concept’s Walls!

33

So the open question is strongly connected to the question of how we may shape musical time (onsets and durations of notes and pauses) in rhythmical structures.

7.4 We Inspect the Concept’s Walls! Exercise 6 Write down some rhythms you imagine in your own notation. Then try to normalize them according to time signatures on a score. What is the experience with time signature? Exercise 7 Now, do the converse: Give yourself a time signature and then write down your rhythms within this time signature on a score. Try out some not so common time signatures, such as 5/4 or 7/8. What is its effect on your possibilities? Does it help you or restrict your efforts? We see that first of all time signature is a wall: It helps normalize rhythms and could inspire your rhythmical ideas, but it might also imprison you into an artificial time frame. A second type of wall is the role of pauses. At first whern you’re writing, the pauses are just written to organize the notes’ onsets. If you want to have a second note not come immediately after the first, you may write a pause to indicate how much later the second note must be played. The Beethoven fanfare shows this technique in the upper staff, where the first chord must wait 3/8 after the barline (pauses 1/4 + 1/8 = 3/8 before the chord). Exercise 8 Write down some rhythms with pauses and try to feel that silence is also an important musical event. Why are pauses a wall? You feel after these exercises that what is not played is not just the absence of rhythm, but suggests a whole universe of what could have been played in the pause. Remark 3 Overuse of the pedal frequently happens to amateur keyboard players. It is a method that in gastronomy is known as covering the main ingredients with a thick sauce to give the recipe body and substance. In reality, it destroys the substantial flavor and also the possibly bad quality of the cooking. In music, this might be caused because they rely on the covering pedal effect, as they are afraid of silence and are insensitive to the body of time of a performed sound. The third wall for rhythms relates to who is shaping time. This looks like a subtlety. But when composing music, and then performing it in a creative way, we must make sure that the responsibility is clearly taken by those who make the music. When looking at the first wall, time signature, it may happen

34

7 Rhythmical Aspects

that you feel that time is already there, and you just have to fill the slots. And also with pauses: They look like empty space, dead silence. This is the subtle third wall: The normalization of music notation on scores (on paper or in computerized representations) suggests that you are just a servant of an already given time out there, like in Newtonian physics, where we all had to follow the one and only divine time. But creative composition and performance would be paralyzed by such a passive role in artistic expression. Music is not a set of divine laws; we have to make it from its first fundamental moment on.

7.5 Try to Soften and Open the Walls! In order to soften these three walls, complete this series of exercises: Exercise 9 Take a rhythm written in a given time signature. Now forget the time signature and just play the rhythm. Try to develop it without any reference to the time signature. Do you feel that wall now? Exercise 10 Now try to play enriched rhythms using the pauses in our previous rhythms. Investigate the richness of the possibilities not being played in your previous pauses. Also try to vary the tempo of your rhythm and find out what this changes in the enrichment of the rhythm’s pauses. Exercise 11 To open the third wall, the passivity toward a given ‘divine’ time, play one of your rhythms and sing with it. Then stop and wait, do not follow any given external framework. Decide spontaneously, or just following your feeling about when to play. Do not count; try to make time into something that’s only there because you’re playing and making effort on your instrument. Remark 4 The system of time signatures is what you choose and utilize. But the time signature should not cast your composition. It might come after having set your rhythm.

7.6 How Can We Extend Opened Walls? Let us now try to extend the three walls of the rhythm concept. Before we make our own extension, let us listen to some famous composers who in fact successfully extended these walls: Cecil Taylor: Garden, Guerino Mazzola: Synthesis, third movement, Byungki Hwang: Jasi (Night Watch), The Beatles: All You Need Is Love. The extension we are proposing after your experiments and after listening to some famous extended rhythm creations is that

7.7 Final Step: Testing Our Extension

35

1. we take the full responsibility of positioning time events when we want them, as long as we want, 2. we don’t count any time signature to cast our time events—such a time signature may come after we have created rhythm, not as a precondition, 3. we open our ears for the silence in pauses as an enormous potential of notyet-played rhythms, of a universe of silent tongues1 that will sing when we let them do so. We should also add that it is not implied in our model of rhythm that only one voice is performing the rhythm. It may very well happen that several voices perform different rhythms simultaneously (generating a polyrhythm) with overlapping sounds.

7.7 Final Step: Testing Our Extension With these extended walls of the concept of rhythm, the student now embarks in a series of exercise performances to test the effects of these extended rhythmical powers. Exercise 12 Play your rhythms, if possible also together with other students, everyone on his/her own instrument. Try to step from a time-signature-driven one to a freer one, or to another time signature. Try to “ride the horse,” as they say in free jazz, to take full responsibility of what you are playing. Try to listen to what you have not played yet, in the silence of pauses, but also behind the sounds you are producing. Don’t follow a given scheme or memory of heard music—make your own creation! You are the only one who determines what to do and what not to do!

1

This is a title of one of the most famous solo piano LPs by Cecil Taylor [116].

8 The Pitch Aspect

When shaping the body of time in music, we have to think about what material we want to distribute over time. Time as such has no sensual quality: You don’t see, smell, hear, touch it. We need to think about what is being dealt with in time. This tutorial unit deals with this theme. And it, in fact, will answer it in such a way that we understand why music deals with sound and not with, say, odors or visuals.

8.1 What Is Your Open Question? To begin with, we want to look at all possible ways to shape the body of time. We do not want to shape it in an abstract reality of pure thoughts, but in a sensual way. We are concerned with art and human expression for other humans and therefore are focusing on the five human senses1 : touch, smell, taste, hearing, and sight. Note that we have other senses, such as kinesthetic sense, senses for temperature, pain, balance, and acceleration. But let us focus on the most prominent ones. Exercise 13 Describe the five classical human senses and their qualities. What can you express with these senses? How precise is this expression? How can you perceive them? How can you produce them? How about their communication to other humans? Your discussion evokes the open question of this tutorial unit: Why are we using sound in music, and how do we deal with sound? 1

These senses are classical. They were classified by Aristotle but also recognized in other cultures, such as the “five material faculties” in Buddhism or as the “five horses drawing the chariot of the body” in the Katha Upanishad of Hinduism.

G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 8, © Springer-Verlag Berlin Heidelberg 2011

37

38

8 The Pitch Aspect

It is, of course, not the question of defining music as the art of sound (in time), but the question of why this sense of hearing, and not any other, has such a prominent position in human artistic expression.

8.2 Let Us Describe the Context! We are now looking at the musical sense of hearing in the wider context of all five senses. The context is the multisensorial human reality of perception and utterance. Following your discussion of the qualities of our five senses, we can write down the following rough list of properties.

Sense touch smell taste hearing sight

Precision low low low high high

Complexity low average average high high

Production high low low high low

Perception high high high high high

Speed fast slow slow fast fast

Memory low average low high high

In this sensual context, hearing is the most skilled. More precisely, it is the only one where we can make fast and fine-grained sensorial output (sounds) and perceive it with no extra efforts. We have a voice and are also sensitive to sound for reasons of biological survival! We can communicate by sounds. The sense of sight is a passive one, since we can see but we cannot make what we see, whereas the sense of hearing can both receive and produce sounds. Exercise 14 Try to invent and use different instruments to make sounds. Try the same for visuals, touch, taste, and smell.

8.3 Find the Critical Concept! Within the context of the sense of hearing, we want to find a critical concept. This would be a concept that is central to our sense of hearing and ‘sounding.’ Let us first listen to different sounds from instrumental and environmental recordings. Exercise 15 Describe the different concepts coming to your mind when talking about sound. You have exhibited a number of such concepts: Loudness, the sound’s character (bright, dark, cold, hard, etc.), the development of the sound’s presence (envelope, play a piano sound backward to hear the envelope!), the spatial position and distribution of the sound, its male or female voice, etc.

8.4 We Inspect the Concept’s Walls!

39

There is one concept whose role seems to be very strong in sound perception and production: high and low, male or female sounds. We are extremely sensitive to the level of a sound. This might be for biological reasons: Babies have high voices and attract our attention; sexual selection also relates to male and female voices. This fact adds to the observation that we may perceive different heights of sounds being played at the same time! In music, several voices of a choir may sing and be perceived simultaneously. The quality of sound ‘height’ is called pitch. It is a crucial quality of sound in every musical culture. Let us choose this concept as a critical one in this tutorial unit. Exercise 16 Listen to different music with pitch and without pitch. Try to describe the types of pitch your perceive—if any. For example: piano, human voice, whistle, siren, snare drum, bass drum, saxophone, triangle, explosion, river, sea waves, wind. When looking at the physical phenomenon responsible for sound and pitch, we learn from acoustics (see section 21.1.1 for technical details) that sound stems from more or less regularly altered air pressure around its atmospheric average value, acting on our ears’ eardrums, and that pitch arises when the alteration is periodic (it repeats after a time period of P seconds) and it is proportional to the frequency f = 1/P , measured in units Hz (Hertz, 1/second) of this alteration. For example, the concert pitch a has frequency 440 Hz, i.e. the sound alteration repeats 440 times per second. A piano sound of this frequency 440 Hz is shown in Figure 8.1. A drum sound typically has no regular pressure variation; we see an example in Figure 8.2.

Fig. 8.1. The periodic pressure variation of a piano sound of concert pitch at 440 Hz.

Fig. 8.2. A nonperiodic sound variation of a drum, no pitch is perceived.

8.4 We Inspect the Concept’s Walls! So far, we have learned about pitch as an important quality for music. We have seen that pitches can be perceived and produced simultaneously, yielding what are called chords. Pitch is a very powerful specification of sound qualities. But it is also imprisoned by a number of walls. A first wall relates to the very simple question: How do you make sound with pitch?

40

8 The Pitch Aspect

Exercise 17 Invent ways to make pitch, look for instruments that have pitch and describe how it is produced. When you compare a piano and a violin, for example, you will notice that you cannot produce pitches on the piano, beyond those that are defined by pressing one of the 88 keys that in turn hit the metal strings with felted hammers. But on the violin, you can produce an infinity of pitches (within the range of the violin’s overall pitches) by just pressing down the string on a variable position over the fingerboard (see Figure 21.5 for the violin’s anatomy). So the first type of wall is the instrumental accessibility of pitches. Some instruments may restrict your access by their mechanical construction.

Fig. 8.3. A selection of historical and present music notational systems. Most of them are locally displaying note events. Only the Tibetan Yang-Yig notation (middle) has a more gestural flavor.

A second type of wall relates to the question of how you memorize pitch. How do you communicate it to fellow musicians, scholars, and others who want to study your compositions? Exercise 18 Try to invent a notation for pitch on different instruments (also your own invented instruments, of course). And look for already existing such notations, as in Figure 8.3, which shows some old and new notation systems.

8.4 We Inspect the Concept’s Walls!

41

The wall character of notation is based on the fact that notation always not only represents pitch, but also limits pitch to what can be notated in the given notation standard. Let us look at the standard Western notation for piano, as shown in Figure 8.4.

Fig. 8.4. A classical score in Western notation.

Here we have two staffs of five lines each, a treble clef for the upper and a bass clef for the lower staff (starting at the end of the second measure). You can place note heads for specific pitches on each line or in between, and also on or between some higher or lower ledger lines. The line encircled by the treble clef is the pitch g below the concert note a. The line between the two points of the bass clef is the pitch f one octave and a major second below the g of the treble clef. The pitches that are possible in this system are the white keys on a piano. If you add some alteration signs ( for next lower (black) key on piano,  for next higher (black) key on piano), you can describe all keys (the white and the black ones) that can be played on the piano.

Fig. 8.5. A scale with all pitches on a piano between two octaves.

Exercise 19 Play all keys from middle c on the piano upward one octave (playing each key, black and white, in order as you move to the right). Write this also in Western notation as a ‘scale.’ See Figure 8.5 for a solution.

42

8 The Pitch Aspect

We see here that the notation just selects a system of pitches and does not allow for notation of any other pitch. For example, if we want a pitch between e and f , Western notation does not allow it. For such a reason, Arabic maqam music, for example, cannot be notated with our Western system. Some composers and theorists, such as Ferrucio Busoni and Alois H´aba, have proposed and composed pitches between the piano key pitches. These systems are called microtonal and can yield very interesting music. Let us hear a quarter tone (a pitch in the middle of each successive pitch of the Western system) string quartet composed by H´aba. Exercise 20 Create all scales of any seven different pitches within one octave starting with c (select from the 12 pitches from c to the next upper b). For example, the seven white keys on the piano (the major scale: ascending, c, d, e, f, g, a, b, see also Figure 8.5). Notate your scale’s seven pitches on a piano score system. One of them would be c, d, e, f, g, a, b, the so-called harmonic minor scale. The third type of wall that we want to exhibit is perhaps the most important, but it is also the most hidden one. If we want to organize pitch for music, the whole organism we want to construct cannot just consist of a set of disconnected pitch events in time. In other words, and in a somehow circular spirit, the third wall is pitches themselves as isolated entities. We need to connect pitches to make music. We need something like a networking structure between pitches.

8.5 Try to Soften and Open the Walls! So let us try to soften these three walls of pitch. Exercise 21 How could you soften the instrumental wall? Play an instrument or a sound software (such as Max MSP) on a computer and try to go beyond their pitch limitations. Don’t be shy, be experimental! Exercise 22 How could you soften the notation wall? Could you invent new notations for microtonal pitch or for your own pitch selection? Try! Exercise 23 How could you soften the pitch isolation wall? How could you find a bridge between two pitches? These exercises are very important to let you experience the walls as limits of musical expression and also the possibilities to break them down against established norms!

8.6 How Can We Extend Opened Walls?

43

8.6 How Can We Extend Opened Walls? The opened walls have been extended in different ways; let us look at some of them: The instrumental wall of the piano has been extended by playing inside the piano, by using rubber or metal pieces to change the sound, and by inventing piano sound software allowing for new pitches or slightly changed original pitches. These software are built by sound synthesis methods, which we describe in detail in sections 21.1.3, 21.1.4.1, 21.1.4.2, and 21.1.4.3 of the theoretical part. Other extensions have been realized on the saxophone, by jazz saxophonist Pharoah Sanders, using overblowing techniques. On the violin, there is a whole arsenal of sophisticated techniques to alter the sound (see section 21.1.2). The notational wall has been opened and extended by many contemporary composers and theorists. H´ aba has written a harmony book [48] using his notation for microtonal pitch. Many electronically working composers have quit the traditional notation and work directly with frequencies. Of course, such notation cannot be used for normal instrumental performance, but it is absolutely adequate for computerized music production. It is also important to acknowledge the oral tradition, those composers and musicians who do not rely on notated pitch but play it directly from their instrumental skill and their practical memory. This is true in particular for many traditional Arab and Asian music cultures. The third wall, isolation, is the most important. How can we overcome this one? In your experiments in the above exercises, you have suggested that we try to measure the step when moving from one pitch to another, for example, when moving from c to g. There must be a way to fashion this movement between two pitches. Remark 5 The isolation of the selected seven pitches of a scale is a state of structural non-understanding of the scale’s anatomy. The naive listeners might just feel that there is some character of happiness (major scale) or sadness (minor scale), but they do not know what is responsible for this emotional qualification and therefore nurture the mystification of music. With isolated pitches, any music would be possible because it would neglect the possible relationships between its pitches. Conceiving the movements between a scale’s pitches creates a scale’s anatomy and connects its pitch spots. Exercise 24 Play several sequences of two pitches on the piano. Play c to g, then d to a. Or else d to f  or g to b. What do you notice? Any suggestions? You rightly observe that some movements between two pitches sound the same. The movement c to g sounds like d to a, for example. This means that there is a relationship between c and g that is also the case for the movement d to a. This means that there is a connection between pitches that forgets about the absolute pitches and only expresses their ‘distance.’

44

8 The Pitch Aspect

This quality is called interval between pitches. It is a quality that classical music theorists in ancient Greece, the Pythagoreans, discovered. They worked with three such intervals. First: the octave, which means that two pitches x, y are in octave interval relation if the frequency f (y) of y is the double of f (x), f (y) = 2.f (x). Second: the perfect fifth, which means that x, y are in perfect fifth interval relation if f (y) = 3/2.f (x). Third: the perfect fourth, which means that x, y are in perfect fourth interval relation if f (y) = 4/3.f (x). Exercise 25 Play an octave (two adjacent c notes), perfect fifth (e.g. c to next higher g), and perfect fourth (e.g. g to next higher c) on the piano. Try the same on a violin or on a flute. For the Pythagoreans, it as a miraculous experience to create these intervals on their simple monochord, an instrument having just one string spanned over a resonance box. Exercise 26 You can also try this out! Take a string of the violin and pluck it, then fix it at its middle. Plucking a half of it yields an octave higher pitch. Now fix the string at 2/3 and you get a perfect fifth, finally fix it at 4/3 and you get a perfect fourth. Exercise 27 Make again a scale of any number of pitches in one octave (think of scales with four, nine, eleven pitches!), or even including microtones if you can notate them and ideally also play them on an instrument or computer. When you choose the pitches, be aware of the semitone contents of the intervals that you generate with this scale. Choose carefully what pitches you want while considering these interval qualities (see Figure 8.6). Those are just not the intervals between successive scale notes but they generate different value/flavor depending on your choices. This is called a well-tempered tuning: All twelve successive interval steps are equal and add up to the octave. These small intervals are called semitone or minor second intervals, and the interval of two successive semitones is called a whole tone or major second interval. For this reason, one also represents the octave as a clock with twelve hours, where every hour is a pitch, starting from c at 12, then c at 1, etc. until b at 11. But why a closed circle? It is true that after an octave one is at a higher pitch. But music theorists have taken into account that notes that are one or several octaves apart from each other sound very similar. This has motivated them to even identify pitches that are one or several octaves apart from each other. Therefore we may use a clock, where we return to the same point after 12 interval steps.

8.7 Final Step: Testing Our Extension

45

8.7 Final Step: Testing Our Extension In music, the concept of an interval has been very successful, and you should be aware that there are infinitely many intervals even within a single octave. The interval between two pitches x and y would be defined by the ratio f (y)/f (x) of their frequencies. Exercise 28 Try out any possible interval on a violin or trombone. Exercise 29 Listen to various existing scales. Here are some famous examples: melodic minor, harmonic minor, gypsy major, gypsy minor, whole tone, messiaen 2, messiaen 3, blue notes, major (= ionian), dorian, phrygian, lydian, mixolydian, aeolian, locrian, pentatonic, octatonic. Each scale has its own intervallic property and flavor.

Fig. 8.6. These are the classical intervals, in parenthesis the numbers of semitone steps: Unison U (0), minor second m2 (1), major second M2 (2), minor third m3 (3), major third M3 (4), perfect fourth P4 (5), tritone TT (6), perfect fifth P5 (7), minor sixth m6 (8), major sixth M6 (9), minor seventh m7 (10), major seventh M7 (11), octave P8 (12).

The classical intervals in Western music are shown in Figure 8.6. The Western counterpoint and harmony are based upon these interval categories, so the intervals are of practical relevance. Their relevance comes not from their abstract formal characterization as frequency relations, but from what their expression means to the theorist, composer, performer, and listener. Counterpoint and harmony are essentially defined upon the meaning of such intervals. For example, counterpoint is built upon the distinction of consonant (sounding agreeably) and dissonant intervals (sounding disagreeably). Exercise 30 Write a melody (a sequence of pitches) and determine all the successive intervals. Exercise 31 Play all these intervals as pairs of simultaneous notes. Do you experience different qualities of these intervals? Or do they all sound same to you? Also play intervals on the violin; do you experience different qualities?

46

8 The Pitch Aspect

Summarizing: There is an infinity of intervals. We may try to play them on certain instruments and also try to notate them, but then we would have to extend the classical Western musical notation. The relation of intervals shows different qualities, some sound nice, others rather harsh. But we should think of the entire variety of intervals when creating music. This relational thought of intervals does not mean however that the absolute pitch is irrelevant. There are many musicians and composers who perceive absolute pitch. So intervals are important but cannot replace pitch as an original sound quality.

9 The Harmonic Aspect

Harmony is a magic word in music theory and practice. It is not only a technical term but also refers to more transcendental layers of existence. In ancient Greece, harmony was the ultimate ground of the universe. In Pythagorean philosophy, the ultimate formula of the universe was their tetractys, a triangular arrangement of ten points starting at the base with 4, then 3, then 2, then 1 on top. You have already seen in the previous chapter on intervals that the successive ratios of these triangular numbers—2/1, 3/2, 4/3—are the intervals of an octave, perfect fifth, and perfect fourth. So the tetractys was the universal law behind both music and the universe. This formula was an expression of harmony since these intervals were considered to be particularly agreeable, Fig. 9.1. Johannes Kepler or “consonant,” as music theorist call them. In wrote the five books Haropposition to consonant intervals, they speak of monices mundi about the “dissonant” intervals; for example, the minor sec- world’s harmony and its reond or the tritone are dissonant and sound less lation to the five Platonic agreeable to many persons—but not all. We come bodies as a guiding principle back to these interval qualities in tutorial 11 about of astronomy. counterpoint. In 1619, when astronomy was developed using modern mathematics, Johannes Kepler wrote the five books Harmonices mundi (see Figure 9.1) about the world’s harmony and its relation to the five Platonic bodies as a guiding principle of astronomy. He was concerned more with the musical harmony of planet motion than with any other rationales for natural laws. And we should not forget that a modern foundation of physics, string theory, is also based upon vibrations of microscopic strings, much like the vibrations of violin strings. G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 9, © Springer-Verlag Berlin Heidelberg 2011

47

48

9 The Harmonic Aspect

9.1 What Is Your Open Question? Now that you’ve read about the history of harmony and its deep impact on the understanding of the universe, what do you think is the role of harmony in these philosophical approaches driven by music? Exercise 32 Think about harmony and its role in the grand view of this universe. Also think about music-related harmony. What could be its overall role and structure? Your answer might be this: Harmony gives the world a deeper content. And this is what we are also experiencing with harmony in music. Now, in music such harmony is built starting from the traditional harmony as expressed by those consonant Pythagorean intervals. So our open question would be about how the composing or playing of combined pitches—possibly referring to their intervallic relations—constitutes harmony in the sense of a deeper content in music.

9.2 Let Us Describe the Context! Let us now locate the open question within a reasonable context. Exercise 33 If you are to create meaning with music, what context of such action would you propose? Think about it in the sense of creating a communication by music as if you were speaking in a language. This means to deal with music as a system of meaningful signs. If you recall the introduction to this book, section 2.3, you will remember that systems of meaningful signs are called semiotics. Harmony then is to be viewed in the context of a music semiotics. We want music to be a system of meaningful signs. And then we are also concerned with connecting such signs, like when you build sentences that connect signs functioning as subject, verb, object, and so forth. Recall from section 2.3 that this is called syntagm, while the pointer from expression to meaning or content is called semantics. In other words, harmony relates to the making of meaning in music in the dimensions of semantics and syntactics. But it relates to this process in a special way—namely, when combining different pitches. The classical situation is that such a combination is played simultaneously, which is called a chord. In Figure 7.2 we saw a number of chords in Beethoven’s fanfare. Although this seems straightforward, it is an important topic in music philosophy to ask whether music means anything at all. In fact, most music does not refer to a thing or object in the external world’s reality. When we say “tree,” this refers to a meaning showing an object in the real world. In music, such a reference is absent (except for program music—for example, mimicking

9.4 We Inspect the Concept’s Walls!

49

the sound of sea waves—but this covers only a small part of musical expressivity). Let us keep this conflict in mind and come back to it at the end of this chapter.

9.3 Find the Critical Concept! The open question about the mechanism for constituting substantial meaning in music by combined pitch performance is a semiotic one. It must focus on the musically central concept of harmonic signification. The classical concept for this is a harmonic function. This function attributes to any combination of pitches (played simultaneously or successively, it does not matter, we abstract from the pitches’ onset time or duration) a meaning in the semiotic system of music. Exercise 34 Suggest some kind of harmonic function. For example, a chord could mean an emotion, color, objects, etc. What does a major triad chord mean to you? And any rule works, just think about how it would help you give meaning to music. For example, a diminished seventh chord followed by an augmented triad could mean “distorted reality being resolved to a floating dream.” This exercise might sound strange, since it looks like you are allowed to invent any kind of meaning in harmony, irrespective of any other given meanings of musical objects (melodies, rhythms, instruments, etc.). The situation is, in fact, somehow wild. Of course, one may attribute emotional or biological meanings to musical chords or similar pitch-defined objects. But a reduction of musical meaning to such everyday signification would not satisfy the idea of musical content or meaning. Exercise 35 Listen to Bach’s Well-tempered Piano and think about its content beyond emotions and biology. The work is known for its careful melodic and harmonic construction. Exercise 36 Now let us try to define more precisely what you think is the meaning of a chord. Try to define the emotional meaning of chords by looking at their intervals and using their consonances. But recall from our earlier observation that the intervals’ consonance/dissonance are subjective.

9.4 We Inspect the Concept’s Walls! If we think about the harmonic function of a chord, the first wall thereof is that we automatically think of a fixed meaning, like for signs in language: The sign “cat” has a uniquely determined meaning. It should not depend on the

50

9 The Harmonic Aspect

day time, weather, or other contingencies. Is it reasonable to look for such an unchangable harmonic function in music, too? Exercise 37 Think about reasons for rigid or nonrigid harmonic functions for chords. For example, whether functions could define the harmonic environment of a chord, or whether the harmonic environment defines the function of that chord. The second wall is the domain of values we may attribute to a chord. What do we expect from such valuations? Let us look at the classical harmonic functions in music theory. Here a chord can only take three quite abstract values: tonic, dominant, and subdominant. More precisely, if we are given a tonality, like the one determined by the C-major scale (c, d, e, f, g, a, b), the triad chord {f, a, c} would have harmonic subdominant function (denoted by S), while the triad {g, b, d} has dominant function (denoted by D) and the triad {c, e, g} has tonic function (denoted by T). Remark 6 We should notice that the main theorist of harmonic function theory, Hugo Riemann, in fact defined tonality very elegantly by the harmonic function attributed to chords (all possible chords were envisaged!). For example, the C-major tonality would be defined by the function values (out of three values D, S, T) attributed to chords as described above. In other words, these functions create the tonality. We could then write C-major({c, e, g}) = T, etc., but F-major({c, e, g}) = D, etc. This is elegant because we do not have to restrict our pitches to the particular scale; chords with pitches not in that scale would also be given harmonic function values with respect to that tonality. See Figure Fig. 9.2. A cadence in C- 9.2 for a sequence of harmonic functions. This semajor with the standard se- quence, T, S, D, T, is a typical syntagmatic figure: It is called a cadence in C-major. Any chord sequence T, S, D, T. quence with this harmonic function associated is then called a cadence in C-major. This is a type of ‘grammatical’ sentence similar to a language sentence. It is used to say in a few chords that we are working in this tonality. You will find this very frequently in classical and in popular music. Listen to some examples: Go to the end of these works, please, and listen to the final chords. Exercise 38 Why would you want to have only these three main values of harmonic functions? Why not much more, and different values? Could you also think of taking fuzzy values: not only “yes” or “no,” but something between, 60 percent of a value, say? Listen to a piano solo by jazz pianist and composer Thelonious Monk. It is full of beautiful dissonances. Try to give such fuzzy values to his chords.

9.5 Try to Soften and Open the Walls!

51

A third wall can be recognized when thinking about aesthetics in music, dealing with what is beautiful or ugly. Does such a function have to represent beauty or truth? Or both? Setting ugly chords in beautiful way also works in music! Does beauty follow a logic, or does logic shape beauty? Recall the famous poem Ode on a Grecian Urn by George Keats (1819), concluding with the beautiful verses “Beauty is truth, truth beauty,”—that is all Ye know on earth, and all ye need to know. Exercise 39 Think about beauty as a possible value of a harmonic function. Isn’t this much more important than an abstract logical value or such an abstract value like tonic, dominant, or subdominant? We don’t have to stress that these properties are really walls, i.e. they prevent the harmonic function concept from becoming connected to what useful meaning would give us. We should develop the keen sensitivity about how narrow the function concept is if one keeps these walls upright. The point of this sensitivity is that we do not want to compose music within a priori fixed semantics. We are by no means limited by given external, real world contents. As said above, the situation is somehow wild, but it is not true that anything goes and that meaning does not matter. It matters a lot, but it is not set and fixed forever. Composers and improvisers should actively set such contents in their music. This is probably one of the most exciting moments in musical creativity! We not only may create musical forms, objects, chords, melodies, and rhythms, but we are also asked to organize their meanings. We have to go to the initial moment where there is virtually no given meaning, at least beyond those elementary emotional or biological elements. In this moment, we create the harmonic functions with their relation to tonality.

9.5 Try to Soften and Open the Walls! We have these three walls: • • •

Rigidity: The harmonic functions are unchangeable. Abstractness: The poor value set of domains of harmonic functions. Beauty: The question about aesthetical values of a harmonic functions.

We could summarize the three walls and say (quite exaggerated) that harmonic function tends to be rigid, abstract, and aesthetically useless. Why should we need this type of semantical values at all? We needn’t because they are paralyzing any creative work that searches for meaning and not just formal games. So let us run against these walls and try to break them down! The first wall of functional rigidity already shows fissures. We have seen that harmonic functions in Riemann’s theory are not absolutely rigid. The

52

9 The Harmonic Aspect

formula C-major({c, e, g}) = T is relativizing functional values for the sake of a definition of tonality. This is an exciting new perspective in that it opens the wall toward the use of harmonic functions for the construction of higher musical contents. Tonality is such a higher content, and Riemann’s idea was to use harmonic functions to imagine and construct the concept of tonality. Why is this a decisive step? Because larger musical sentences, not just single chords, are often conceived as expressing tonality—the composition is in C-major, F-minor, etc. This means that we are interested in having the function values of the composition’s chords be connected by their role as expressions of tonality. Tonality and then modulation, i.e. the transition from one tonality to another one, are extremely important in the construction of harmonic meaning in music. Therefore, the first opening of this wall consists of the question of what the value of harmonic functions is for establishing larger musical coherence. Could such a scope be a door to overcome the rigidity of harmonic functions? Remark 7 We should add the observation that despite the prominent position of Riemann’s approach, it has never been completed, i.e. it has never been possible in traditional music theory to assign a reasonable function value to any chord as an expression of a given tonality. The function values usually are limited to simple chords, such as major triads (like {c, e, g}), minor triads (like {c, e, g}), etc., and some tetrads. But already chromatic triads, like {c, c, d}, are problematic. This might be one reason that classical harmony was given up by Schoenberg and others in the first decade of the twentieth century. You see here that harmony is not only a given constant of theory but could be used to construct new meaning! Schoenberg’s ‘solution’ to quit harmony altogether with his dodecaphonic approach in 1921 looks quite confusing here. We shall come back to this topic in tutorial 13 on serialism. The second wall of value domains is a bit harder to soften. In common language, we have a rich repertory of meaning, such as the real world’s objects, ideas, emotions, etc. They do all offer contents. But with music, the semantical domains, such as the traditional Western triple T, D, S, are extremely poor. Exercise 40 Try to draw a (even fictitious) graphical image of a musical composition with (tonality related) dominant, subdominant, and tonic values of its chords. Compare with the same semantic image for a real world English sentence. Are you satisfied with this kind of semantic control of music? There are a number of straightforward enrichments of such semantics. To begin with, one can add some more refined values. This is quite common in Western harmony. For example, one adds the value Dp, dominant parallel, which would be taken by the minor triad at e in C-major, see Figure 9.3. Another method would consist of allowing several values at the same time for a given chord. This only makes sense if we attribute weights to these distributed values (for example, 30% D, 20% S, 50% T in G-major). This is the fuzzy logic

9.6 How Can We Extend Opened Walls?

53

approach, where truth can have any value between truth and falsity. We have realized this fuzzy approach in our software RUBATO for musical analysis [74]. So softening this wall a bit is possible, but we are still left with the main question of whether we really have those harmonic values that we want to address when creating music. Is harmony a question of three values: T, D, S (with respect to tonalities)? Fig. 9.3. Dominant The third wall, the aesthetic question in harmonic triad (left) and domfunction, is the hardest. What is beauty? What is truth? inant parallel (right) If Keats were right, why then is it so difficult to talk in C-major. about logic to musicians and composers? Why then is any logical discourse in music so critically reductive? Which composer would ever reduce his/her composition’s meaning to logical categories? Although Schoenberg said, “Musik soll nicht schm¨ ucken, sie soll wahr sein,1 ” he created much more than truth values. The point is that between ornamentation and truth values there is the most important thing: beauty. I would prefer to claim, “Music should be neither ornament nor truth but beauty.” At this point we see that there must be harmonic values to be taken other than logical ones of simple truth or falsity. We have at least softened this constituent of the wall: Let us search for value domains that are more susceptible to expressing beauty than abstract logic. And also look at and listen to other music systems, such as the Indian Raga music with its mela scales or at the Korean traditional music, or at the Tuvan throat singers’ overtone-based system, for example.

9.6 How Can We Extend Opened Walls? The first wall: Rigidity has been softened in order to define tonality via harmonic functions. How could we extend this wall? We could just take a set of functional assignments to chords in order to define higher structures, such as tonalities. We need not reproduce known tonalities, as they are usually derived from scales. We may define a new tonality by a creative construction starting from given Riemannian tonalities. For example, we may take two tonalities, C-major and F-major, and their Riemannian definition that assigns values C-major(X) and F-major(X) to a chord X. We may then define a product tonality by the formula C-major×F-major(X) = (C-major(X),F-major(X)). Here the harmonic function values are pairs of classical values. So, for example, we would have C-major×F-major({c, e, g}) = (T,D). This yields a bitonality function, and its values are a combination of values for each tonality that are already known from Riemann theory. Exercise 41 Try to construct other generalized tonalities using given classical Riemannian tonalities. 1

Music should not ornament but be true.

54

9 The Harmonic Aspect

The second wall, abstract values, could be extended by a new approach to functional assignments. We have in fact seen that the normal functional value in Riemannian theory is to give every chord a value without looking at values of other chords. This defines, for example, all chords with tonic value just by checking each chord and seeing whether it has the value T or not. If it has that value, we get the set T(Z) = {X|Z(X) = T} of all tonic chords in tonality Z. This is a bottom up method: define T(Z) by the values of single chords. But we could start on top and then go down, by first defining sets of chords VZ describing a tonality Z in an abstract way, without specifying values of single chords first. For example, take a piece of music that has a score and a recording. Listen to the piece and start finding a pitch t you think is the “tonal center” of the piece. Listen until you believe that the tonal center has been changed. Take all the chords until this moment of change and give them the functional value X(t). Then all these chords have obtained their value not because of their individual properties, but because you put them into the ‘bag’ of chords expressing that tonal center t. It does not matter how you conceive the property of “being a tonal center,” it is just a feeling, but this works. Then you go on in the piece with the second tonal center, etc. This is a typical top-down procedure. And it could happen that some chords have two or more values in your system. Suggest how to define a functional value in such a case. Feel free to invent whatever you believe is reasonable! Exercise 42 Listen to Hindemith’s Ludus tonalis, prelude in E, to see how he creates the harmonic function using the tonal center E. Exercise 43 Listen to Lutoslawski’s Two Studies for Piano, I, and look at the end of the piece to see how he builds creatively the cadence. It is not T, S, D, T in the conservative harmonic functions, and the harmonic functions are his own creations. Remark 8 This is also the strategy in the mathematical theory of counterpoint [77], where we do not first say which intervals as individuals are consonant or dissonant and then define the set of all consonant intervals, but we first determine the set by a special set property and then say that an interval is consonant because it is member of that set. This is opposite to the classical acoustical consonance theory where consonance is first given for individual intervals, for example by Euler’s gradus suavitatis function. See our tutorial on counterpoint in chapter 11 for a more detailed discussion of this topic. This means that you, the composer, are free to define harmonic values by describing sets of chords that you consider having a determined harmonic value. The value is defined by the set of chords and not by a technical property to be checked for single chords. And if you want to to define a tonality in your own way, you have to propose sets VZ of chords that you think share a property V that defines an aspect of your concept of a tonality Z.

9.7 Final Step: Testing Our Extension

55

Exercise 44 Try to construct other generalized tonalities using this top down approach. The third wall is the most exciting to be extended. We are asked to introduce values that might be more akin to beauty than those abstract Riemannian T, D, and S and their variants. Let us first think about the nature of these values. As opposed to the common language semantics, musical semantics has no external references. Harmonic functions do not refer to external meaning; they do not mean anything from the outside—works, an apple, love, temperature, desire. They can only have values internal to the music system. So if we review T, D, and S as internal values, T refers to the tonic of a tonality, the basic pitch c of the C-major scale, for example, and D refers to the interval of a perfect fifth above the tonic, a position that is thought to create a force toward the tonic in a final harmonic situation as described by the cadence T → S → D → T described above. And opposed to D is S, it is a fifth below the tonic, or a perfect fourth above, and it can help open the harmonic pathway when starting from the tonic. So we see that these values are derived from a simple aesthetic idea. If we use this opening of the aesthetic wall, we may think of harmonic function values as structures describing beauty in the harmonic movement. Yes, but what is beauty here? Is there a general basis for beauty? Is the movement D → T a general fact of beauty? Isn’t it just a conventional movement that has gained its prominence by a virtually infinite repetition? We do not have to decide upon these questions, but you, the young composer, may now extend the wall by introducing structures that you consider to be beautiful. In other words, why don’t you define beauty by naming and listing musical harmonies that you consider beautiful. This would mean that you create your own harmonic semantics of beauty, the meaning of your own language. This means to create internal reference structures that you consider beautiful, without any further reference, just your own vocabulary of beauty. If you apply the top-down strategy described above, you may then introduce sets of chords that you define as carriers of music contents of your language. This means that you can make beauty, not just applying given schemes of more or less formal and abstract aesthetics.

9.7 Final Step: Testing Our Extension Exercise 45 Build sets of chords that you consider elements of an aesthetic harmonic content. Try to write compositions (also in your own notational system, this does not matter) that express statements about your harmonic vocabulary. Exercise 46 Build sets of chords following the classical Riemannian approach. Try to compose following this scheme. Test the difference of success as compared with your own harmonic principles.

56

9 The Harmonic Aspect

These testing exercises are not meant to create easy decisions, but they should invite you to think about how a harmonic function might affect the way you create musical works.

10 Melodic Aspects

Melody is a central aspect of musical creation. In most popular music, the melody is the only characteristic feature that is retained and memorized by everybody. This is certainly also due to the fact that the human voice is the first of all valid musical instruments, and that melody relates to music for and by human voices. Knowing this dominant role of melody, many instructors stress composing cantabile (in a singing manner) and then also performing as if one were singing—even on instruments (such as the piano) where this approach is somewhat artificial in view of their mechanical construction (the piano being more a percussive than a melodic instrument). Mozart was in fact famous for his “oily flowing” pianistic attack. In this tutorial unit, we want to investigate the creative potential of composing melodies. Melodies have two roles: 1. They are complex motivic shapes in themselves, and 2. They play a role as germs for the dramatic story they may tell as a composition’s theme. Before discussing these roles, we have to stress that the invention of good melodies is still one of the most difficult compositional tasks. There is no theory guaranteeing good melodic creations, and there are no computer programs; it seems that this task is the hardest in all directions of musical creation. The lack of theory is not only for composing good melodies, but even more elementary: In contrast to harmony, there is not even an established theory of melodies. Rudolf Reti’s classical work The Thematic Process in Music [100] is a first heroic approach to the topic, but it suffers from ambiguities (Reti does not even believe that precise concepts can be built in this field) and even contradiction (for example, Reti’s concept of identical themes is not even transitive, i.e. theme A can be identical to theme B, and theme B identical to theme C, without A being identical to C!). For a more rigorous new approach to the theory and its computerized implementation, we refer to the theory of musical motives [77, chapter 22] and its software implementation [77, chapter 41.2] as developed by Chantal Buteau, Guerino Mazzola, and Oliver Zahorka. G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 10, © Springer-Verlag Berlin Heidelberg 2011

57

58

10 Melodic Aspects

This means that the present tutorial unit (like some of the other units) is more a strategic catalyzer than a set of recipes to find those melodies that make you rich.

10.1 What Is Your Open Question? As a musical structure, melody involves, above all, onset time and pitch. See Figure 10.1 for an example, the melody of the famous Beatles song Michelle. But it is not only a structure in time and pitch. It also has a meaning like

Fig. 10.1. The first eight-measure sentence of the famous Beatles song Michelle.

chords, but this meaning is radically different from a harmonic function. It is not coincidence that melodies are strongly correlated to lyrics, with short texts that create a human story. Melodies have the potential of stories. They are musical germs of entire tales to be developed in the musical composition. One can take parts of such melodies, short motives, and encounter them all over the composition. One may find them in similar forms, deformed or transformed, but still acting as placeholders or delegates of the original ‘mother’ melody. This sounds familiar as long as we focus on the linguistic metaphor. But musical stories are usually not loaded with extramusical contents; melodies and motives are not words and therefore cannot create novels and stories similar to common language1 . This is a big problem and in fact our open question: How can melodic structures create meaning in musical compositions that takes over the function of a story in common language? 1

Although lyrics refer to extramusical stories, the musical quality of a melody is independent of the lyrical thread.

10.2 Let Us Describe the Context!

59

10.2 Let Us Describe the Context! The context of melody and its associated open question are easily described once you look at a melody and try to explain it to someone. Exercise 47 Focus on the above melody from Michelle and try to explain the melody in detail to a fellow student. Use whatever technical tools or limbs to do so. You will immediately realize that this melody relates to a movement in time. It is a kind of gesture that you might describe by a movement of your hands or by dancing it while you sing the melody. You will also observe that this entire movement could be called a shape, or what psychologists call a gestalt. In fact, psychologist Christian von Ehrenfels introduced gestalt theory using a melody as a model [38]. He characterized gestalt by two attributes: A gestalt can be transposed (or more generally: transformed) without changing its identity, and it is super-summative, i.e. it is more than the sum of its parts. In our case this means that a melody is more than the collection of its single notes. And we should add to the Ehrenfels criteria that a gestalt is also invariant under small deformations. When you make a gesture, it is not so important that you move exactly the same every time; perception is quite tolerant. Otherwise we could not even recognize letters that are written in different fonts, let alone in different handwritings! This first bunch of contextual topics is what one calls paradigmatic, and it pertains to associative fields of concepts or objects. Observe that the paradigm can be either a transformational one (transposing a melody) or a topological one (namely, deforming a melody to a similar shape; topology is the science investigating deformations). To this paradigmatic context we have to add another one, namely the fact that if you tell a story, be it in common language or in music, you have to place plot, your melodic units, in time, simultaneously or one after the other. This is the syntagmatic axis, the timeline on which a text is displayed. Let us look at a classical situation of such a story: a fugue, as shown in Figure 10.2. In the first two measures, you have a melody (the fugue’s dux, in the alto voice), which is answered by a second appearance of the melody in measures 3 and 4 (the fugue’s comes, in soprano) and transposed a fifth upward from the dux. Then a second comes, in bass, appears on measure 7. And so on. The composition is a story displaying this initial melody’s shape in time and pitch space (Attention: We have more than just the time axis in musical syntagm, we also have pitch!). Putting these two perspectives together, the melodic context is paradigmatic and syntagmatic, and we have to investigate the distribution of melodies in this two-dimensional field.

60

10 Melodic Aspects

Fig. 10.2. The classical situation of a musical story: a fugue. In the first two measures, you have a melody (the fugue’s dux, in the alto voice), which is answered by a second appearance of the melody in measures 3 and 4 (the fugue’s comes, in soprano) and transposed a fifth upward from the dux. Then a second comes, in the bass line, appears on measure 7.

10.3 Find the Critical Concept! Melodies can occur just as nice ornaments in pitch and time, but their power is only fully developed when they bear a potential of ‘telling a story’. As such, they are a kind of germinal structure that unfolds in the course of the composition, they become omnipresent like a story’s dominating subject2 . The concept that is loaded with this potential is called the composition’s theme. Beethoven is famous for having worked out the thematic potential. In music theory, his method is called motivic-thematic work. In intuitive terms, one could call this the art of telling a story as a musical emanation of the composition’s theme. Exercise 48 Listen to Beethoven’s Piano Sonata op. 2, No. 1, in F-minor, first movement, to see how he unfolds the motif in a motivic-thematic way into his entire music. 2

It is a challenging question whether, conversely, a story in common language is also in its intrinsic quality conceived as a musical tale and less a message about extramusical contents.

10.4 We Inspect the Concept’s Walls!

61

Exercise 49 Listen to The Beatles’s Michelle entirely and try to describe the story that is developed around the main theme shown in Figure 10.1. The difficulty of a theme is to understand what its potential is, how it can unfold its forces to shape the composition’s syntagmatic display. If the theme were just that assembly of notes in the time-pitch space, it would be a simple thing, but this is not how it must be viewed as loaded with the story’s subject’s power. This makes clear that the theme’s wall is its identity: What is the theme more than that structural entity? What makes the theme to what it is as a theme?

10.4 We Inspect the Concept’s Walls! As we already stated in section 10.2, a theme is what psychologists call a gestalt, a shape that has three characteristics: It is more than its parts (supersummativity), it can be transformed, and it can be deformed without losing its identity. Therefore its wall is this complex set of attributes that define it as a gestalt. But why is this a wall? The first reason is that it is not clear how much more than its parts it is. And also: What are these parts more than the theme’s notes? Second: If the theme can be transformed, what is a transformation, and how can a transformation be realized in the composition? Do we just have to recognize that there is an original structure X in the score and then somewhere later in the score is its transformed version f (X) by transformation f ? Third: How much can we deform a structure X such that its deformed version X  is still recognizable? Example 1 Let us look at the Bach example in Figure 10.2. The dux theme appears again as comes 1 in measure 3. This one is a transformation f (dux) of the dux, namely by two measures in time, and then by a perfect fifth upward (c → g) in pitch. Here the comes 1 is easily recognized as being a transformed version of the dux since playing later and somewhat higher is an easy task for human cognition. If Bach had chosen a more complex transformation instead (for example, a rotation or a shearing), we would not recognize the transformed version so easily—if at all! But the comes 1 is not only this transformation, it is also a deformation of f (dux). The fourth note is a g in the dux but is not a fifth higher, a f (g) = d, in comes 1; Bach takes the perfect fourth higher c instead. This is not very different from the expected perfect fifth, so it does not destroy the overall shape. There is a simple reason for this deformation: Bach would like to have a consonant interval with the e of the alto voice. The note d would not meet this requirement, as we would have a dissonant minor second d − e. But then Bach could also have chosen the c one octave higher, with the same consonance argument. However, the shape would then be different, and the deformation not really acceptable, because the movement would be ascending instead of descending at the fourth note.

62

10 Melodic Aspects

When we state the theme as a gestalt, we should be aware of the fact that a theme is not just there like an object of nature, but it is created by humans. And it is created as a gestalt, not as a set of uncorrelated notes. It is an organism that grows from the composer’s constructive effort. This creative work is then reflected in the composition’s development. The composition’s tale is also a reflection of this constructive genealogy. This means that when we discuss the theme’s wall, this is also a wall for the creation, not just for the compositional usage or theoretical analysis of a theme. After all, a composer should be completely aware of the theme’s ‘anatomy’ when working out a thematic composition3

Fig. 10.3. The second group of four notes (over the lyrics “these are words that”) is a deformation of the retrograde of the first group. The group of four notes in measure 4 plus the first of measure 5 (over the lyrics “go together well”) are in a peak shape that reappears in a shortened version, like a ‘tail,’ (transposed in time) for the last three notes.

Example 2 As a small example about the theme’s anatomy, let us reconsider the Beatles song Michelle from Figure 10.1. The theme starts with a four-note motif over the lyrics “Michelle ma belle.” It is a horizontal movement into the peak note d and then descending to a. The next four notes in measure 3 are of similar shape but in inverted time direction: ascending from g to the peak c and then descending to a horizontal pair of gs. Therefore, one could say that the second group of four notes (over the lyrics “these are words that”) is a deformation of the reflection (in time, a retrograde) of the first group at the vertical axis. The group of four notes in measure 4 plus the first of measure 5 (over the lyrics “go together well”) are in a peak shape that reappears, like a ‘tail,’ (transposed in time) for the last three notes (over the lyrics “my Michelle”). We even observe a correspondence between the lyrics that parallels the motivic one. 3

There are also athematic compositions, but we do not discuss them here.

10.5 Try to Soften and Open the Walls!

63

Exercise 50 Try to construct a simple theme by use of the above types of transformations and deformations. Try to make it sounding interesting. Make first a drawing and then the corresponding musical shape.

10.5 Try to Soften and Open the Walls! What we should have in mind now is to open this wall in order to be able to build, handle, and decompose themes as if they were living body gestures of sound. The aesthetic aspect of such themes is less our concern than the dynamical management we can think of. Here is the limit of our theory of melodies, but we believe that the techniques we shall describe are quite powerful. First, let us look at the wall of the theme X’s parts. What can we do there? As we have seen in the above Bach and Beatles examples, we can take subsets of a theme as parts. We could envisage all of the theme’s parts and consider the powerset 2X = {Y ⊂ X} of the theme set. This is a large repertory of parts; for example, if X has 8 notes, the powerset has 256 elements, starting with the singletons {x} for the theme’s notes x and ending with the whole theme set X. In the above Beatles theme, we had just four out of the 216 = 65, 536 possible subsets. So the first opening would consist of considering all possible subsets of a theme, or, when constructing it, considering any combination of subsets of notes to build the whole theme. So far this is quite primitive, as we cannot make more than just select subsets. But there is more, namely the entire set-theoretical machinery of set unions, intersections, and complements, and this is what is known as Booelan calculus: Take sets, unite them, intersect them, look at complements of subsets of a large set. OK, now you, the musically interested student, would probably react against such “abstract nonsense,” right? We understand, since such mathematics is not really what you think is helping your creativity. But be assured, this is wrong for two reasons. To begin with, famous composers, such as Iannis Xenakis, have used this method of Boolean operations on sets of notes (also known as sieve calculus) with great success, e.g. in the famous composition Nomos Alpha. Second, the Boolean calculus is nothing but the formalism necessary to describe and manipulate sets of notes regarding common notes or shared notes. There is a beautiful simple example of how this can be understood in five minutes. Take one octave of the scale of C-major, X = {c, d, e, f, g, a, b}, a set of seven pitches. Look at the seven triadic chords that you know from elementary music theory, namely: I = {c, e, g}, II = {d, f, a}, III = {e, g, b}, IV = {f, a, c}, V = {g, b, d}, V I = {a, c, e}, V II = {b, d, f }. The Boolean calculations of the intersections among two or three of these triads is very important in harmony because it tells which chords have common tones, and then we may use this information to connect chords by common tones in voice-leading theory. Arnold Schoenberg called this connection between triads the harmonic band. What Schoenberg did not know is that when you look at all the pairs

64

10 Melodic Aspects

or triplets of these seven chords that have a non-empty intersection, you get a fascinating geometric picture:

Fig. 10.4. The harmonic band of C-major is a M¨ o bius band. Two triads are connected by a line if they have one or two notes in common. Three triads are connected by a triangular surface if they have a note in common. For example I and V are connect by a line, whereas I and II are not. And III, V, V II are connected by a triangular surface since they have the note b in common.

Exercise 51 Draw each triad as a point on a piece of paper. Connect two of them by a line if their intersection is non-empty (e.g. I is connected to III since the tones e, g are common). Draw a triangular surface among any three of these triads if their intersection is non-empty (e.g. I, III, V are vertices of such a triangle since they have g in common). Check out what this geometric figure looks like! (The solution is in Figure 10.4. It is a M¨obius band!) Therefore the Boolean operations on subsets of the theme tell us a lot about the common pitches (if we forget about time) or times (if we forget about pitch). This approach is a kind of musical geography: The parts are like charts of an atlas, and their overlaps tell us how geographic charts are connected. You see that if you select as charts just the single note sets, you get a quite trivial geography: only isolated singletons, no connection. The connectivity of your theme’s “charts” is a sign of how these parts are connected with one another. So it is important that composers and improvisers should look for these subset possibilities and try to construct interesting intersecting configurations in their musical creations. Exercise 52 Listen to the melodies of your favorite music. Is it just an amorphous sequence of pitches? The composer utilizes the submotives of the theme

10.5 Try to Soften and Open the Walls!

65

in an interesting way. We have seen this in Beethoven’s Piano Sonata op. 2, No. 1, F-minor, and in The Beatles song Michelle ma belle. Example 3 Take four chords such that their Boolean intersection geography looks like the surface of the Earth’s globe! Can you do this? One simple solution is to take a tetrachord, like X = {b, d, f, a}, and to look at its four triadic subchords defined by omitting one of the four notes of X, i.e. Xb = {d, f, a}, Xd = {b, f, a}, Xf = {b, d, a}, Xa = {b, d, f }. We then see that any three of them have a note in common but all four have no note in common, and therefore the intersection geography looks like a tennis ball that is covered by four triangular pieces (see Figure 10.5).

Fig. 10.5. The intersection geography of a tetrachord that is covered by its four triadic subchords looks like a spherical surface.

Now, let us look at the wall’s aspect relating to transformations. We have seen in the Bach example that transposition in time and pitch is a common transformation. In the Beatles example we also recognized reflection at a vertical line, the retrograde transformation reversing time direction. To give you an idea of how to make transformations, work in the plane whose x-axis is time and whose y-axis is pitch, where you might draw semitone distances as equal size. If you draw a melody (or a part thereof) as a point set, the points will all have pitch values in that grid defined by semitone distances. Take that plane and think of a common transformation thereof. Translation by a multiple of time units and/or pitch units (like a perfect fifth, seven semitone steps) moves that melody into another position whose pitch is still in that pitch grid. And if you make the reflection at a vertical line, the pitches will still be in that pitch grid. But what about a rotation? See Figure 10.6 to understand this situation.

66

10 Melodic Aspects

Fig. 10.6. A transformation of a melody (blue circles) in the semitone-quarternote grid can create a ‘melody’ (red squares) that is no longer in this grid. For example, the rotation by 45o produces pitches that are no longer in the semitone grid.

Exercise 53 Try to draw an image of the rotation of a melody by 45 degrees in counter-clockwise direction. What do you observe? Yes, you see: It can happen that a rotated note no longer has a pitch value in that grid of semitones! So we are confronted with this problem: Either only perform transformations that keep pitch in that semitone grid, or else we have to shift the rotated pitch down or up to the next semitone value! And this is a deformation. This means that deformations are a natural tool helping us stay in the pitch domain that makes sense in the usual notation system. Of course, the same necessity can arise in the time domain if the onsets or durations fall out of a grid of time quantization after such a rotation. Summarizing, we see that transformations may be a strong tool but may also endanger the parameters of sounds: Pitch may fall into values that are no longer standard in the usual pitch domain. Let us now focus on the wall of deformation. We have just seen that deformations may help us stay in a given parameter domain. But the question here is another one. If we are allowed to deform parameters, what is the amount of deformation that we can accept while conserving the original shape? To what extent can we accept to have the deformed shape in sufficiently near neighborhood of the original one? We want to deform, but not ‘too much.’ This aspect of nearness is totally different from the transformational situation. A transformation may move the original shape quite far, whereas deformation deals with proximity and similarity. This is the problem of this opening of the wall of a theme.

10.6 How Can We Extend Opened Walls? The three wall extensions are built upon Boolean operations, transformations, and deformations of thematic material, essentially operating in the plane

10.7 Final Step: Testing Our Extension

67

of time and pitch but, in a more general setup, also on other melodic parameters such as duration or loudness. These operations are not only paradigmatic, but also cover the syntagmatic distribution of thematic components, since the overall theme may be viewed as a union (Boolean) of a number of translated (transformation) parts. The theme is then covered by parts that are results of the three operation types. Whereas these methods can be a priori applied in any order and quantity, there are a number of measures to take to keep the result transparent to the listener. It may be important to the composer to communicate what has been done in order to tell the story in a way that is perceived and understood without having to Fig. 10.7. The three basic operation types of delve into deep analytical ef- melodic creativity. forts, which, after all, are not the primary message of a composition. For Boolean operations, clarity may be achieved by separating parts into different voices or making them disjoint, without overlapping parts. For the transformations, it is important to create a certain evidence of the correspondence between transformed parts, like in the Beatles piece, where the retrograde of the first motif is played right after the original motif. For the deformations, it is important to keep the distance short between the original and deformed shapes. For example, in a normal context, it would be confusing to deform a melody by raising or lowering pitches by an octave.

10.7 Final Step: Testing Our Extension As a first test of the above methods of melodic creativity, we would like to present two examples of such constructions from our own compositional works. Example 4 In this example (see Figure 10.8), we look at the main theme of Joomi Park’s composition Black Summer, see chapter 23 for a thorough discussion of the CD Passionate Message where the composition is included. Part A is a minor second descending interval. Part B is an extension of part A. Part C has the same interval property as B but puts the lowest note in the middle. Part D starts like B, but the third note is one octave higher. Rhythmically speaking, the theme is divided into a double sequence of three eighths (the first eighth of the first sequence being given in the initial chord), followed by equally long (2 × 3 = 3 × 2) three two-eighths groups. This is a

68

10 Melodic Aspects

perfect example of using Boolean operations (the parts!) and transformations between these parts to build the global anatomy of the main theme. Joomi develops the main theme as follows (see Figure 10.9). The second appearance T’ of the theme in the treble clef is a fifth higher than the original theme T. The third version T” of the theme is a locally inverted shape, with ascending contour. This Fig. 10.8. The main theme and means that the four parts A, B, C, D of its four components in Joomi Park’s the original are inverted and yield four new composition Black Summer. parts in the same order in version T”. The shape Tp is derived from the parts C and D of T”; it refers to it in its inverted shape and intervallic content (minor ninth). The shape Tp’ is the same as parts C and D of T”, and it is repeated again in Tp”’, while interweaving the Tp and Tp”.

Fig. 10.9. Joomi Park’s development of the main theme.

Example 5 This example stems from Guerino Mazzola’s CD Synthesis [73]. It is the composition’s main theme. It was used in all four movements. In the first movement, the theme was played on percussion instead of pitched instruments. In the fourth movement, it appears twice in the beginning, played by electric bass. The theme is shown on top of Figure 10.10. The lower part of the graphic shows a grid with horizontal onset times in multiples of eighth notes, and semitones on the vertical axis. The theme is shown in red points. The theme is a union of twenty-six three-element motives. They are quite overlapping. Each of the three-element motif is shown at the bottom in blue. The number of each motif is the number stemming from a classification of all three-element motives

10.7 Final Step: Testing Our Extension

69

Fig. 10.10. The main theme of the composition Synthesis is shown on top. The lower part shows a grid with horizontally onset times in multiples of eight notes, and vertically in a grid of semitones. The theme is shown in red points. The theme is a union of 26 three-element motives. They are quite overlapping. Each of the threeelement motif is shown at the bottom in blue.

that has been achieved using methods of mathematical music theory. We do not delve in these technicalities here. Suffice it to say that every three-element motif is the transformation of exactly one of these twenty-six motives. The transformations include translations, rotations, inversion, etc., from the theory of musical transformations, but see [77, chapter 11.6.3] for details. Therefore, this theme is, in a certain sense, the most general theme as it is composed from all possible three-element motives. Exercise 54 The following composition exercise is to complete a melody based on the given materials. The students are supposed to look at, play, or sing it

70

10 Melodic Aspects

and choose one or two motives to complete one melody. Try to compose another melody from your completed melody, using transformations and deformations.

Fig. 10.11. Four motives to be completed to a theme.

Look at the four motives in Figure 10.11. These are the motives: 1. 2. 3. 4.

B-major, light and oily A-minor, lyrical E-minor, 5/8 time signature, dance-like atonal, strong expression

Examine carefully what property/possibility the given material has. Try to compose out of it, developing from the original, keeping the initial ideas, and also contrasting the original. Try to apply transformations, too, such as inversion. See the examples of The Beatles’s Michelle, and Beethoven’s work in the last example of this chapter (see Figure 10.13). Try several different directions once you have composed. For example, when you have composed an ascending melody, then try to compose a melody with the same given material but descending, and so on. As discussed above, Boolean operations are important to imagine the localized possibilities and their connections. Also try different contours, different leaps, different articulations. Think about all options to create a good melody, and choose the best. Example 6 As a last example, we want to mention that Beethoven, in his famous Hammerklavier sonata, op. 106, used only one basic theme, a chromatic zigzag of a minor third, as shown in Figure 10.12. All his motivic and thematic

10.7 Final Step: Testing Our Extension

71

Fig. 10.12. Beethoven’s germinal zigzag theme in op. 106. (The formula e6 G means the temporal shift of G by six time units.)

work is derived from transformations of parts of this germinal theme. One example, a vertical shearing of the zigzag, generating a melodic unit (Allegro, measures 75-76), is shown in Figure 10.13.

Fig. 10.13. A vertical shearing transformation of the zigzag in Allegro, measures 75-76.

11 The Contrapuntal Aspect

Counterpoint is perhaps the most critical topic when it comes to the search for creativity. In fact, counterpoint is strictly codified and taught as a sort of catechism of classical compositional discipline in polyphony, the art of combining several voices to a balance interplay. In its most formalized shape, counterpoint has been described in Johann Joseph Fux’s famous treatise in 1725, Gradus ad Parnassum [44]. It was written in Bach’s time but consciously refers to Fux’s idol Giovanni Pierluigi da Palestrina (February 2, 1526 (or possibly February 3, 1525) - February 2, 1594). In the foreword, Fux even stresses his conservative position: “Why should I be doing so (writing about music) just at this time when music has become almost arbitrary and composers refuse to be bound by any rules and principle, detesting the very name of school and law like death itself.” This last sentence could have been written in our time where everything seems to be possible. Our challenge will be to inspect the deep structure of this theory to discover its inherent potential for a creative extension. In view of the role of counterpoint pedagogy in music schools, this seems to be an important enterprise. Counterpoint is commonly understood as a frozen knowledge that at best can serve as a gymnastic exercise in compositional rigor, but never as a tool that one would actually use for contemporary composition. It is amusing to see that even progressive present music theorists appear somehow shocked if any attempt to bring counterpoint to new horizons is undertaken. See [118] for such a case, where the mathematical models of generalized counterpoint designed and programmed by Guerino Mazzola, Daniel Muzzulini and Jens Hichert [77, Part VII], Julien Junod [60], and Octavio Alberto Agust´ın Aquino [3] have been attacked in defense of classical counterpoint as a perfect catechism for eternity. This situation refers to a core problem in music that we have already pointed out in chapter 5: Are composers required to follow eternal rules of music theory or are we still in that early stage (the Planck time) of human thought where the final laws are not found yet? We hope that the students of G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 11, © Springer-Verlag Berlin Heidelberg 2011

73

74

11 The Contrapuntal Aspect

composition will be inspired to think about this fundamental question when working through this tutorial unit.

11.1 What Is Your Open Question? Let us get started with the simplest situation in Fuxian counterpoint (see Figure 11.1). We are given a melody line, called cantus firmus, c.f. Classically, it is composed according to the style of Gregorian chant, but this is not important for our concerns. The contrapuntal assignment is to invent a second melody, called discantus, to be added to c.f. and fulfilling certain rules. In our example, we have the simplest case, called first species, where over each c.f. note we position a discantus note of same duration. The contrapuntal rules give us constraints of how to position these notes. There are two parts of this rule system. The first part is completely rigid: It allows us to set a discantus note over the c.f. note only if the interval (number of semitone steps) from the c.f. to the discantus note is a consonant interval. This means that this consonance can be either a prime (or unision) (0 steps, same pitch), a minor third (3 semitones), a major third (4 semitones), a perfect fifth (7 semitones), a minor sixth (8 semitones), or a major sixth (9 semitones). We may also use intervals derived from these when adding an octave (12 semitones)—for example, the octave (0+12 = 12 semitones), the minor tenth (3+12 = 15 semitones), etc. All other intervals are called dissonances, and they are strictly forbidden. In our example, numbers above each interval are the steps in the C-major scale, i.e. 5 stands for the fifth (the fifth note above e), 8 for the octave (the eighth note above c), etc. The second of the

Fig. 11.1. The contrapuntal assignment is to invent a second melody, called discantus to be added to cantus firmus (c.f.) and fulfilling certain rules.

contrapuntal rules prescribes which intervals may succeed which intervals. In this rule system, one distinguishes between perfect consonances (prime, octave, fifth) and imperfect consonances (thirds, sixths). Further, different movements from one interval to the next are described: direct motion: Both voices ascend or descend; contrary motion: One voice ascends while the other descends, or vice versa; oblique motion: One voice moves while the other remains on the same pitch. Here are the fundamental rules as stated by Fux:

11.1 What Is Your Open Question?

75

1. From one perfect consonance to another perfect consonance one must proceed in contrary or oblique motion. 2. From a perfect consonance to an imperfect consonance one may proceed in any of the three motions. 3. From an imperfect consonance to a perfect consonance one must proceed in contrary or oblique motion. 4. From one imperfect consonance to another imperfect consonance one may proceed in any of the three motions. Essentially these rules are rule one and rule three, and the others just allow for any motion. In particular, one is not allowed to move from a fifth to another fifth by rule one. This is the famous “forbidden parallels of fifths” rule. One may also summarize these rules in a single rule: Any motion to a perfect consonance must be contrary or oblique. Exercise 55 Take the same cantus firmus as in the example of Figure 11.1, but compose another discantus according to the above rules. These rules are very formal, but they intend to produce a double gesture of cantus firmus with discantus that alternates between perfect and imperfect intervals. In fact, once you are at a perfect interval (prime, octave, perfect fifth), there is not much of a choice to move to another perfect interval, therefore you want to go to an imperfect interval, where the choice is larger. But again, what we experience in music is not these formal properties, but rather a well-shaped pairing of voices, one that has a dynamic power. The rules seem too be the formal result of a principle of interacting forces. But what is the force guiding this interaction? It is somehow related to the inner life of consonances, but it is unclear how this happens. Our open question therefore might be this: What are the forces that drive the contrapuntal movement of voices? Remark 1 Let us stress the situation in regard to musical creativity that we meet with the formal contrapuntal rules. There is no valid artistic or scientific behavior that would just accept formal rules without asking for their generating thought and impulse. When we look at Fux’s rules, they are totally empty, and there is no reason to follow them except for historical imitation of Palestrina’s style. But this cannot be of interest to anyone who takes the responsibility for what is constructed. It is paradoxical, but it seems that it is exactly the absence of responsibility that has favored the dissemination this musical catechism (like any catechism disseminates as a package of empty forms of indoctrination), and it is not coincidence that the question-answer style between master Aloysius and student Josephus of the gradus is also the style of catechism. We are not claiming that Fux’s rules are wrong, but we strongly ask for their generative content since no creation can ever be accomplished on the formal surface. Creativity must always delve into and extend meaning and content. Please refer to section 19.1.2 for an in-depth presentation of this characteristic of creativity in our approach.

76

11 The Contrapuntal Aspect

11.2 Let Us Describe the Context! The context of our question is multifaceted. On the one hand we recognize the melodic context: Counterpoint deals with a multiplicity of melodic voices. We have seen the cantus firmus and discantus voices here, but the general setup deals with three, four, or more voices. Second, the intervallic aspect refers to harmonic considerations. Consonances and dissonances are harmonic functions on intervals. They are not Riemannian (no T, D, S values here), but have two values κ (for consonant) and δ (for dissonant). We do not consider more complex chords, only intervals. And it is a strict theory that attributes to six intervals exactly one value. And that value is also invariant under octave extension, i.e. consonances are defined on pitch classes modulo octave, this is the so-called octave equivalence. We might therefore define the set K = {0, 3, 4, 7, 8, 9} of consonant intervals (of pitch classes), whereas the complement D = {1, 2, 5, 6, 10, 11} contains the dissonances. We use here the domain Z12 of the twelve chromatic pitch classes with semitone steps, which might be best represented by the common clock circle that has twelve hours (0, 1, 2, 3, . . . 11, Fig. 11.2. The sets K (red 12=0 (!)), and where every hour represents a key squares) and D (blue cir- on the piano, up to octave. Then K and D unite cles) of consonances and dis- to Z12 , i.e. K ∪ D = Z12 and have no interval sonances. They cover the in common: K ∩ D = ∅ (see Figure 11.2). This set of all twelve intervals setup in counterpoint is quite exceptional because of pitch classes, contain six the perfect fifth (7 semitones) is consonant as we intervals each and have no expect it from Pythagorean tradition, but the percommon elements. fect fourth (5 semitones) is dissonant, which contradicts the Pythagorean tradition and the acoustical theories about consonance altogether. Counterpoint is not an acoustical theory since the simple frequency ratio 4:3 for perfect fourth is not an argument for consonance. We learn that counterpoint is a highly symbolic theory that is independent of acoustical reality. It is a construction for the sake of composition of melodies, not for frequency ratios. The allowed consonant intervals are a kind of gestural movement from c.f. to discantus, and the entire discantus melody could be understood as the final position of a big sweeping gesture starting at the c.f. melody. The common understanding of counterpoint is that its Latin etymology, “punctus contra punctum,” point against point, means the interval’s note ‘points’ are opposed to each other. The discantus is then viewed as a ‘vibrational’ deformation of the c.f. It is straightforward that such a vibration requires the consideration of the forces of ‘elasticity’ that relate c.f. to discantus.

11.5 Try to Soften and Open the Walls!

77

11.3 Find the Critical Concept! In music, the movement of voices and their interplay express a dynamic exchange of forces that creates the interesting connections of parts not as static entities, but as poles spanning and being spanned by a force field. This is a remarkable phenomenon in musical syntax: Although its components look like static structures, they are in fact the visible part of an entire system of tension and relaxation. We therefore suggest to focus on the critical concept of a contrapuntal tension and relaxation. Before delving into the details of this investigation, we should remark that the idea of musical forces has been forwarded by famous theorists, such as Arnold Schoenberg. He often speaks about forces between chords in his classical treatise [107] on harmony (nothing dodecaphonic, just classical harmony!). He even uses the metaphor of sexual attraction and repulsion between chords. Of course, this is utterly anthropomorphic, chords certainly have no sexual impulses whatsoever, but it is not a far-fetched idea at all that chords are more than static structures and by this added property create a dynamic force field of harmony in a composition. In mathematical music theory [77], we have described a model of tonal modulation, i.e. the change from one tonality to another, that is based upon a force concept derived from certain symmetry transformations. This model is capable of simulating Schoenberg’s modulation theory, which he described in [107].

11.4 We Inspect the Concept’s Walls! When we talk about tension, the first question that arises is about its direction: between what and what does the tension span? The first answer might seem to be obvious: It spans between the two voice gestures, and this is what common understanding of counterpoint suggests. But then, why should this be a tension? We have only consonances that move the c.f. to its discantus. What kind of tension can this be? The second wall is the question about the nature of such a tension: What is creating the force between the tensed parts? It is also a wall because in view of the first wall, we do not see any kind of tension within the consonances.

11.5 Try to Soften and Open the Walls! The first wall looks quite hard. The only direction that is visible is the gestural movement from c.f. to discantus. This is due to the fact that the classical construction of counterpoint begins after the c.f. melody has been built. But let us look more carefully at a composed counterpoint in the sense of a gestural movement. We have the c.f. gesture, and this one is taken as a starting point of a gestural movement toward the discantus gesture. But if we look more

78

11 The Contrapuntal Aspect

carefully, this “hypergestural” construction (gesture of gestures) consists of a sequence of tiny intervallic gestures. We first build the c.f. gesture and then successively move in each interval to the discantus gesture, see Figure 11.3, where this movement is indicated by a vertical arrow.

Fig. 11.3. Two interpretations of counterpoint as a hypergesture. The first is a hypergesture of melodic lines from c.f. to discantus. The second, a hypergesture of intervals, is more aligned with to the original contrapuntal thinking.

But this is not mandatory. One may also view the resulting counterpoint as being generated by a hypergesture that moves horizontally from left to right. And this one would then take each interval gesture and move it to the next interval to the right. This alternative is a very simple special case of what in gesture theory is known as the Escher Theorem: One may always exchange the order of generating gestural movements, see chapter 24 for a detailed account on this far-reaching principle. Is this reinterpretation musically reasonable? Historically speaking, it is, since it has been shown that the concept of punctus contra punctum was not the vertical opposition of voices, but the opposition of successive intervals. In other words, the concept of a point (punctus) in this contrapuntal approach was the entire interval! Each interval is thought of as a kind of ‘thick’ point, a point in the contrapuntal space. This is completely natural. The contrapuntal idea takes intervals and moves them around. This is not in contradiction to the Gregorian construction of the c.f line, but it tells us that the tension has another direction: The movement proceeds from interval to interval, although the intervals are all germinating from the c.f. notes. Does this opening help approach the second wall: the nature of the contrapuntal tension and relaxation? The question now is no longer about a tension between c.f. and discantus notes, but between successive consonant intervals. We are looking at the set of consonant intervals and move around in this set. But can there be any kind of tension on this seemingly compact set of consonant intervals? After all they are all consonant. This is true, but we know that there are intervals that are more consonant than others: Some are perfect, the others are imperfect. It is interesting that in the historical development of contrapuntal theories, imperfect intervals have also been called dissonant; see [102] for a detailed account on this terminological finesse in the history of counterpoint! The question about this second wall would then be whether the set of consonant intervals bears something dissonant and whether

11.6 How Can We Extend Opened Walls?

79

some intervals might be infected by a dissonant character, and then moving from perfect to imperfect intervals would be a hidden change from consonances to dissonances. So think about consonance and dissonances less as given qualities, but as properties you may redefine in your own creative context. For example, a minor second and the tritone are “consonant” in the system no. 78 to be discussed below. In fact, many contemporary listeners don’t feel uncomfortable with the tritone.

11.6 How Can We Extend Opened Walls? The opening of the first wall refers to what can be done for the second wall. We are confronted with the set K of consonant intervals. We are searching for forces that might explain the tension between successive consonant intervals. The next question is therefore about the nature of such a tension. Obviously, this is a deep problem since it regards the very definition of consonances and dissonances. How are they characterized? Is there any chance to conceive of such a tension from a deeper understanding of the dichotomy of intervals defined by splitting them into the complementary sets K, D? We have already seen that there is no acoustically valid definition of consonances in counterpoint, since the perfect fourth is dissonant, whereas the perfect fifth is consonant. Also observe that no acoustical theory of consonances would allow for a strict separation of consonances from dissonances; there are only degrees of consonance in acoustical theories. Let us look how these theories would define consonances. They would define a function (like Euler’s gradus suavitatis function) that would be evaluated for each interval and yield more or less high ‘sonance’ values. But this is not the only way to define K! One may also define the whole set instead of first collecting its members. There is, in fact, a beautiful way to do so and to arrive at the set K. The solution looks like this: We look at all splittings of the ensemble Z12 of all intervals (of pitch classes) into two six-element complementary sets (X, Y ), i.e. X ∪ Y = Z12 and X ∩ Y = ∅. But how do we find our exact candidate (K, D)? We have a first very important property of this splitting namely that there is exactly one symmetry that maps each element of K to an element of D, and vice versa, and this symmetry is A(x) = 5x + 2. It is called autocomplementarity symmetry. Exercise 56 Verify that the formula A does indeed map K onto D, and vice versa. And check that there is no other symmetry (a function of the shape f (x) = ax+b with all variables in Z12 ). For example, we have A(7) = 5×7+2 = 37 = 1 in Z12 , the perfect fifth is mapped to the minor second. This formula means that every consonance is turned into a dissonance, and vice versa.

80

11 The Contrapuntal Aspect

In Mathematical Music Theory [77, chapter 30], it has been shown that there are essentially1 only six such interval dichotomies, called strong dichotomies. Here they are, together with their unique autocomplementarity symmetry; their number relates to the classification theory in [77, chapter 30]: 1. No. 64, Δ64 = (I, J) = ({2, 4, 5, 7, 9, 11}, {0, 1, 3, 6, 8, 10}), A64 (x) = 11x + 5. 2. No. 68, Δ68 = ({0, 1, 2, 3, 5, 8}, {4, 6, 7, 9, 10, 11}), A68 (x) = 5x + 6. 3. No. 71, Δ71 = ({0, 1, 2, 3, 6, 7}, {4, 5, 8, 9, 10, 11}), A71 (x) = 11x + 11. 4. No. 75, Δ75 = ({0, 1, 2, 4, 5, 8}, {3, 6, 7, 9, 10, 11}), A75 (x) = 11x + 11. 5. No. 78, Δ78 = ({0, 1, 2, 4, 6, 10}, {3, 5, 7, 8, 9, 11}), A78 (x) = 11x + 9. 6. No. 82, Δ82 = (K, D) = ({0, 3, 4, 7, 8, 9}, {1, 2, 5, 6, 10, 11}), A82 (x) = 5x + 2. Besides the last dichotomy, which is the classical Fuxian consonance and dissonance dichotomy, we have another remarkable one: number 64, the first; its consonant part is precisely the set of proper intervals of C-major from the tonic! This one is therefore called the major dichotomy. But why choose the last one for the classical consonance and dissonance concept of counterpoint? There is a beautiful geometric reason: It is the one in which the images A(x) of intervals x have the largest number of third intervals to be connected to each other. This means that A82 throws x farther away than every other autocomplementarity function. For example, the fifth x = 7 is mapped to A(x) = 1, the minor second, which is two minor thirds away. Also, the distances among all six consonances in K in terms of number of thirds between any two of them is minimal; they are grouped as tightly as possible, but see [77, chapter 30] for details. Therefore, the classical consonance-dissonance dichotomy can be chosen (among the six strong dichotomies) for its geometrically extremal properties. This is all OK, and we are viewing five other dichotomies on which one could potentially focus. But is this game helping us understand the nature of contrapuntal tension? We still have not understood how we could import the dichotomy of consonances vs. dissonance into the part K of consonances as suggested by the idea of perfect/imperfect consonances. The situation is really dramatic: On the one hand, we have to move from consonance to consonance, and on the other, we would like to move from consonance to dissonance. This is a plain logical contradiction. It is, as long as we view the entire setup as a configuration of sets of intervals. But we should see from the very definition of these strong dichotomies that they are effectively defined by their autocomplementarity functions. This 1

This means that any such dichotomy of intervals is isomorphic to one of these six under a symmetry of the intervals, see [77, chapter 30] for these technicalities.

11.6 How Can We Extend Opened Walls?

81

is what we should really take care of. The sets as such are of secondary relevance. Can we try to move from a given consonance ξ to another consonance η as if the latter were a dissonance? This would mean that we have to construct a limiting line between ξ and η that turns the latter into a kind of dissonance. Exercise 57 Try to find ways to view such a limiting line as a separation between consonances and dissonances. Exercise 58 Think about the concept of consonance and dissonance in your music. Didn’t you just try to avoid the traditional concepts when creating unheard music? Why don’t you define your own concept of consonance and dissonance? The major seventh could be a consonance, depending on how you treat it in your music. Yes, you, our model student, are right: Why not just transform the consonance-dissonance dichotomy (K, D) such that parts of D overlap with K? And such that η, but not ξ falls into the transformed part of D? Let us make a picture of such an idea (see Figure 11.4). To the left you see the two halves K, D covering the interval set. The transition from one consonance to another would be the problem when living inside K. But we could, for example, add a tritone (6 semitones) to K and to D. That would yield Δ82 +6 = (K +6, D +6) = ({0+6, 3+6, 4+6, 7+6, 8+6, 9+6}, {1+6, 2+6, 5+ 6, 6 + 6, 10 + 6, 11 + 6}) = ({6, 9, 10, 1, 2, 3}, {7, 8, 11, 0, 4, 5}), and we see that the image of minor second 1 is the perfect fifth 7! So 7 appears as a transformed dissonance (see right half of Figure 11.4). If we had the consonance ξ = 3, then the consonance η = 7 would be a transformed dissonance, i.e. a consonance that is on the other side of the transformed dichotomy (K + 6, D + 6), while 3 is a transformed consonance. In this way we may move from one consonance to another consonance but move from the ‘imaginary’ dichotomy’s consonant part K + 6 to its dissonant part D + 6. In our example this would enable us to move from the minor third to the perfect fifth, a movement that we observe in our very first example in Figure 11.1 from the third to the fourth interval. The idea here is that we simulate a separation of consonances from dissonances by a ‘deformation’ of the (K, D) dichotomy that would create imaginary consonances and dissonances within K. We would understand this deformation as the result of a force action (the transformation we used above!) upon (K, D) to make it produce the imaginary dichotomy. In the theory of this approach, it can be shown that such deformations in fact strongly relate to what physicists call “local symmetries,” which are responsible for force fields. Good, but is this a reasonable approach? Does it produce the rules that Fux has given to us in his catechism of counterpoint? The answer is yes, it does. In particular it produces the forbidden parallel of fifths and no other forbidden parallels, see [77, chapter 31.4]. Summarizing, we have proposed an extension of the wall of tension by local symmetries, which transport the dichotomic tension of (K, D) into K. But we

82

11 The Contrapuntal Aspect

Fig. 11.4. We may move from one consonance to another consonance but move from the ‘imaginary’ dichotomy’s consonant part K + 6 to its dissonant part D + 6.

have much more than this interpretation of tension in terms of symmetries of intervals: We now have five other strong dichotomies, Δ64 , Δ68 , Δ71 , Δ75 , Δ78 on which we can try to compose contrapuntal music! This type of creative work is not one of direct compositional work, but one of the invention of new strategies to compose contrapuntal music based upon other concepts of interval consonance. We retrieve the classical theory but also five new contrapuntal worlds in which to be creative.

11.7 Final Step: Testing Our Extension Let us first make a small example of a first species counterpoint on another dichotomy. We are taking the major dichotomy Δ64 and allowing ‘consonances’ to be the intervals in its I part. This yields a composition as shown in Figure 11.5.

Fig. 11.5. We take the major dichotomy Δ64 and allow ‘consonances’ to be the intervals in its I part. This yields the composition as shown.

It is evident that such an extension of contrapuntal worlds provokes a number of questions and also a number of new tools that might be helpful

11.7 Final Step: Testing Our Extension

83

for composing in these new worlds. To begin with, we have a software that was developed by Julien Junod [60] to compose contrapuntal works in all six worlds. Second, one may ask about the relationship between works in different contrapuntal worlds (the worlds associated with the six strong dichotomies). Junod has succeeded in solving the following problem: Suppose that you are given a counterpoint composition X1 according to the rules of contrapuntal world W1 . How can you deform X1 to a composition X2 that follows the rules of contrapuntal world W2 ? He has systematically investigated all possibilities. And now his software can take any nice Bach counterpoint (well, it is still species one, but the idea is clear) and deform it into a composition of any other world. We refer to his dissertation [60] for technical details.

Fig. 11.6. Left: the strong dichotomy of class No. 78, together with its unique autocomplementarity symmetry. The ‘consonances’ are the six red pitch classes. Bottom: The numbered intervals of the score of Figure 11.7 are shown at the bottom.

Another test would be to look for modern composers and to see whether they have been working with dichotomies other than the classical ones. Let us just look at one, to which this book is closely related: Joomi Park’s composition Black Summer, which is included in our CD Passionate Message discussed in the case study of chapter 23. It has been created without any conscious reference to the contrapuntal worlds described above. It is therefore remarkable that she uses the dichotomy of class nr. 78 in her interval selection. The left hand of this homophonic composition is dominated by five of six intervals from the ‘consonant’ half {1, 3, 5, 6, 9, 11} of the dichotomy (see Figure 11.6). Let us look at the score as displayed in Figure 11.7. These intervals occur in an agreeable and relaxed way as if they were normal consonances. The

84

11 The Contrapuntal Aspect

Fig. 11.7. The intervals of the left hand are belonging to the ‘consonant’ half of the strong dichotomy No. 78. The numbers of the intervals relate to Figure 11.6.

traditionally consonant unison in the last line then surprisingly appears as a tension! It plays a dissonant role. And it is remarkable that it is ‘resolved’ to a series of minor seconds in the last measure, since the (consonant) minor second is the interval that corresponds to the unison under the autocomplementarity symmetry of the dichotomy! In view of these examples, we urge the student to work in one of the non-classical strong dichotomies to create new contrapuntal compositions with a different flavor of the tension between ‘consonances’ and ‘dissonances’.

12 Instrumental Aspects

After a series of abstract sound-oriented topics of creativity we want to consider a practical aspect of music: musical instruments and how they are played by humans. We give an in-depth description of the physical theory of instruments in the theory chapter 21, and especially in section 21.1.2 devoted to technical question regarding the making of sound. Here we focus on the creative rather than the descriptive aspect: How can we be creative when we play an instrument? It might have been observed by the attentive reader that we are looking at humans playing instruments, not computers, player pianos, or other machines. This is not by coincidence, and the reason is that we should first learn to understand what making music is about before we delegate this (still poorly understood) phenomenon to machine-made imitations. But be aware: Our limitation is about those who operate instruments, not the instruments themselves. Instruments can be fully computerized—however designed for human use beyond pushing buttons. See the section 17.3 for more thoughts about computer models of creativity.

12.1 What Is Your Open Question? It is well known, and Beethoven and Schoenberg are no exceptions here, that performers are commonly understood in their ‘waiter’ role of ‘serving’ the music sound, which was conceived and written on paper by those ingenious composers; there is no more, and no less than this delivery. They are executors of orders given by higher instances, much like the MIDI (Musical Instrument Digital Interface) slave paradigm has realized making music as electronic slaves (see 21.3.2 for details on MIDI). The reality is very different. To begin with, classically important instruments, such as the family of violins, are hard to play; the musician has to go through a long and hard educational process until a reasonably agreeable sound can be produced. Second, the shaping of sounds is extremely important G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 12, © Springer-Verlag Berlin Heidelberg 2011

85

86

12 Instrumental Aspects

to expressing the contents of the scores in a significant way. Playing the score literally as written is like eating a deep-frozen pizza. The instrument is not merely a machine that upon pushing some dead buttons produces sounds as prescribed by the score. This is wrong even when playing piano, where one is inclined to think of the keys of being such sterile buttons of sound command. Although pushing a single key with a determined fingertip velocity might be a trivial action, the connected operation of the hand’s movement upon the piano’s keyboard cannot sound intelligent unless the hand connects and integrates the single keystrokes in a quasi-dancing gesture. Hitting the keys is only the moment where the dancing hands touch base on the ‘instrumental stage.’ Playing an instrument is only valid if we view it as a complex interaction of the musician’s body gestures with the instrument’s user interface. We are far from understanding this interaction in the concrete making of music. We do not know about the resonance dynamics between the musician’s gestures and the musical invention, be it in composition or in improvisation. Therefore, it is a good choice to pose this open question: To what extent does the instrumental interface define music?

12.2 Let Us Describe the Context!

Fig. 12.1. Neanderthal flute excavated in Slovenia.

Fig. 12.2. Digital drumpad from Yamaha.

The context of this topic is triple. First of all, we are dealing with a physical device, the so-called instrument, which generates acoustical sound events by a specific and mostly sophisticated action of human body limbs (hands, feet, lips, arms, legs). The inner life of such an instrument is an extremely important and variable material configuration, spanning from simple mechanical constructions (such as the Neanderthal bone flutes excavated in Slovenia, see Figure 12.1) to extremely complex electro-acoustical devices such as a digital drumpad (see Figure 12.2). Second, this instrument’s output is an expression of a system of signs, called a composition1 . A composition is highly symbolic and is viewed as a 1

We disregard here improvisatory constructions to make the open question more profiled.

12.3 Find the Critical Concept!

87

mental entity whose instrumental realization is not even necessary; the composition is understood as existing independently of the instrumental realization. Third, this context includes the performer’s body, whose complex gestures act upon the instrumental interface but are not merely abstract commands to produce sound mechanically. This context is highly artistic and essential for the production of a valid performance. There are musics in which gestures are even the essential part and do not rely on any kind of written composition, such as free improvisation or traditional oral music cultures such as the traditional Korean song genre Pansori, performed by a singer with percussion accompaniment. Exercise 59 Describe these three contextual components for different musical traditions, such as hip-hop, rap, and Indian music. Consider different epochs and cultures.

12.3 Find the Critical Concept! Although the complexity of playing an instrument is known among performers2 , it is communicated exclusively on an oral basis, and this culture is not communicated to composers and theorists. We therefore want to stress a critical concept that contains all those disconnections between instrumental performance and what one could call ‘paper music’ of the scores. The critical concept we would like to choose looks harmless (and this is one of the tricky points of walls!), namely to play. The concept is critical because it seems so obvious that playing is what an instrumentalist does. However, the concept is ambiguous: It means to play an instrument, but also to play a composition. But these two targets of playing are totally different. How can this splitting be understood as one and the same thing? It can be done if we recall that there must be someone who plays the composition on a specific instrument. Although the action of playing focuses on the compositional objectives, it is the agent of this action, the human performer, who gives the whole enterprise a reasonable shape. But this agent is not only hidden in the verb “to play,” it is also downsized to a person who plays a game, an individual who leans back and games with some instrument and composition somewhere ‘down there’. The connotation of a gaming activity brings up the social role of a performer: He/she is just a kind of (hopefully) agreeable gamer, and the historically well-known low status of performers might be related to this (dis)qualification, whereas the composer is perceived as being a divine creator. Exercise 60 Compare the activity of playing the violin to playing soccer or chess. What are commonalities, what are the differences? 2

. . . and even more when it should be creative.

88

12 Instrumental Aspects

12.4 We Inspect the Concept’s Walls! From the above we learn that there is a wall in the critical concept of playing, namely the disconnection between the two aspects: playing an instrument versus playing a composition. The essence of this wall is the hidden agent connecting these two perspectives. How is it hidden, what is there more than in a soccer game? The common access to this wall is that the musician plays a given composition, that set of creatively compiled symbols, on an instrument to produce the sounds symbolized by the composition’s score. This is the deliverer role of a performer: to humbly bring the composer’s dignified creation to the audience.

12.5 Try to Soften and Open the Walls! In order to soften the wall of disconnection, we should first of all recall the genealogy of a score. Historically and also for a good number of composers, the score is not the first creation, but a symbolic projection of instrumentally generated gestures. Western music notation is an abstraction from neumes, which signify gestural contents, small movements that ultimately became casted into rigid note symbols. We are not alone to have characterized the score as a repertory of frozen gestures, in this characterization of Western notation. We are joined by authorities such as Theodor Wiesengrund Adorno, Roger Sessions, Renate Wieland, and Manfred Clynes, the latter having even claimed that “score notation kills music” [89, chapter 14]. And a number of acclaimed composers, among them Beethoven, who shaped many compositions from his improvisation at the piano, do create music from their own anatomically driven performing action much more than from abstract symbols [68]. Exercise 61 Without seeing a score, listen to legendary pianist Vladimir Horowitz playing Scriabin’s Po`eme op. 69, No. 2. Imagine how the notation would look. Listen again, now with the score. Are you surprised about the simplicity of the notation? And there is even no time signature change. The piece itself is short and shows a small number of musical ideas. Think about how interestingly Horowitz unfroze the score by not following any clich´e but actively directing the interpretation of the composition, which would sound “flat” without such expressive shaping. This a major reason why the audience enjoys Horowitz’s performance so much. Exercise 62 Play Scriabin’s Po`eme op. 69, No. 2 with a music software (Finale or Sibelius, for example) that only plays the written symbols. Note the difference to a performer’s rendition. This means that the waiter metaphor is wrong insofar as the composition is the paper projection of an originally gestural creation. Not all composers are aware of this fact, and they might even disregard (consciously or uncon-

12.6 How Can We Extend Opened Walls?

89

sciously) the gestural origin of score notation and compose directly from score symbols. This is not forbidden, but it guarantees a musical success as much as combining arbitrary deep frozen ingredients would yield a good meal when unfrozen3 . It is known that some composers of the so-called New Complexity style effectively disregard the playability of their frozen symbols and turn the evident deficiency into a compositional strategy (see Figure 12.3). We do not condemn this approach, but it is symptomatic of a culture of seemingly dead and disembodied utterances, a culture where works titled Mouvement (– vor der Erstarrung) f¨ ur Ensemble [64] or Pray for Death [37] become fashionable. If you only pursue the complexity of the scoring, for example, a sixteenth subdivision simultaneously with a nineteenth subdivision, then don’t be surprised somebody already did it before you. Once the relationship to the composition that is being played is reconsidered, the role of the instrument radically changes. It is no longer the mechanical interface that makes available some ‘knobs’ and ‘buttons’ to output a planned sound object, instead it plus a fundamental role in the making of music. The sound you produce is only the last member of a long and complex chain of actions and reactions upon the instrumental interface. Once the performer no longer has the waiter’s obligation to move a determined message of symbols to sound, the question of the inter- Fig. 12.3. A typical contemporary complex face’s nature and function arises. music score. We are dealing with an emancipation of the instrumental interface from its symbolic drivers.

12.6 How Can We Extend Opened Walls? The emancipation of the instrumental interface does not mean to put aside scores and start everything from scratch on the instrument. But it suggests that we might reconsider the perspectives offered by the interaction with the instrumental interface before we go back to reproduce abstract symbols. To begin with, we should reconsider the interface as an expressive surface, as a stage, where the musician dances instead of simply pushing buttons. Let us give two examples of such an interpretation. The historical example is the traditional Japanese Nˆ oh theater (see Figure 12.4). The stage of this art is not 3

Pianist Mitsuko Uchida in an interview on her performance of Schoenberg’s Piano Concerto, op. 42 mentioned that Schoenberg does not consider performative instrumental aspects.

90

12 Instrumental Aspects

only a passive surface, it is an instrument because it is built above several big drums that are hit by the actor when he stomps on the stage’s floor. The actor is a drummer, his dance a drummer’s action. A similar stage instrument, the floor xylophone XALA, was invented by Swiss musician/dancer Ania Losinger (see Figure 12.5).

Fig. 12.4. Nˆ oh theater stage with drums below main stage (below Shite).

Fig. 12.5. Ania Losinger dancing on the floor xylophone XALA.

It is not true that such a stage configuration is only realized for these large dimensions where humans dance, because we encounter the stage situation also for common instruments, such as the piano or the violin, and, of course, the drum set. Cecil Taylor, the monstre scar´e of free jazz piano, has stressed that he tries to imitate a dancer’s leaps with his hands. And playing the violin is such a complex interaction with the instrument’s body that it is not exaggerated to call it a dance gesture (see section 21.1.2 for details). All of this goes far beyond what is traced on a score’s symbolic language. It is not an ornamental action added to the ‘real’ music, but it defines music as such. The sounds that are associated with such dancing expressivity are not the music, but its acoustical outlet. One could even state that music deals with sounds among others, and that the dancing reality is the comprehensive statement of this art. The score’s symbolic reduction must be unfrozen and deployed into a dancing gestuality in order to make something that deserves the name of a work of music.

12.7 Final Step: Testing Our Extension To test these extensions, we recommend that the student take a score and perform it in a dancing movement, that he/she try to dance the composition instead of focusing on the simple sounding surface. Exercise 63 Choose a composition you like and listen to it from a dancing point of view. Try to imagine the dancing movements of the piece. Choose a simple and familiar composition you like and play it as a dancing expression.

12.7 Final Step: Testing Our Extension

91

Exercise 64 Think about the restrictions of the notation system. It is straightforward to play a score following all of its restrictions. But think about how you could actively realize the score’s performance as your own creation. Example 7 In Asia classical music education has produced a huge number of good professional performers. They are definitely well represented on the global stages, but the critical point is that their education usually has one ideal way of performing that everyone is eager to reach: Standardized posture, demanding techniques, and even pianistic attack. We have to be aware that this kind of education may endanger the arists’ performing personality. We should be aware that listeners actually enjoy the presence of the artist’s unique individuality. Have a look at one of the videos showing extremely individualist pianists Vladimir Horowitz, Glenn Gould, or Fazil Say. Do you enjoy their nice delivery of the score or their own personality in performance. It doesn’t sound the composer’s piece anymore, but has become theirs. Let us conclude with an example of a different approach to performance and composition as conceived in the dance paradigm. Example 8 This example thematizes the question of designs of new musical instruments following the approach of a dancing creation. There are many such instrumental interfaces, mostly using gestural tracker systems that translate to sound output. Let us just mention, for example, the work of Andrey Smirnov at the Theremin Center of Moscow State Conservatory, or the work of Jared Bott, James Crowley, and Joseph LaViola Jr. exploring 3D Gestural Interfaces for Music Creation in Video Games [16]. Refer also to New Interfaces for Musical Expression, also known as NIME, an international conference dedicated to scientific research on the development of new technologies for musical expression and artistic performance [93].

Fig. 12.6. Musical instruments are played through several gestural interfaces. Clockwise from top-left: drums, guitar, violin, theremin. Figure from [16].

92

12 Instrumental Aspects

Let us just recapitulate an interface for musical composition, the BigBang rubette developed by Florian Thalmann, which is discussed in the Boulez case study in chapter 25, section 25.5. Besides the classical input of musical compositions via MIDI files, his interface allows for graphical input of note objects by the mouse; one can directly draw sets of notes. And then such material can be manipulated with most general geometric transformations defined by three-finger gestures on a trackpad. It is also possible to squeeze and stretch such music data in nonlinear form, for example compressing the beginning of the composition and expanding it successively toward its end. This means that we can manipulate musical objects in a direct intuitive way of gestural interaction. Our recomposition of Boulez’s Structures pour deux pianos as described in chapter 25 has used these tools of gestural creation.

13 Creative Aspects of Musical Systems: The Case of Serialism

In this tutorial we want to think about dealing creatively with a given musical system. There are many such systems—the traditional Western tonal system is one of them, and other cultures have their own systems, such as Indian Raga with its mela scales, or the traditional Korean musics like Pansori, Pungmul, Sanjo, Nongak, etc. with pentatonic scales. In what follows, we want to investigate a system that is an ideal candidate for such a reflection: the serial composition technique. It is an ideal candidate for our tutorial because it has been invented from scratch; it has no dark and complex historical roots and can be viewed as an experiment in musical composition systems. Moreover, the serial system is particularly simple and explicit. Its rules are clear, and it is operated more like a game than a deep musical thought. It has inspired musical composition during the first two-thirds of the twentieth century but has also paralyzed essential aspects of music, such as new directions in harmony or generalized tonality. Serialism was invented in pitch dimension under the title of dodecaphonic composition technique by Arnold Schoenberg around 1921 and realized in his first dodecaphonic composition, the Waltz for Piano, op. 23, No. 5. Later, the dodecaphonic idea was extended to all other parameters— duration, loudness, and attack—and then called serialism. The motivation for Schoenberg’s invention was a deep dissatisfaction with the systems of tonality and associated harmony in the first two decades of the twentieth century. Composers had transgressed the system of classical harmony; its theory was no longer applicable and left composers in a chaotic field of unoriented creativity called “free atonal music.” Schoenberg invented his dodecaphonic compositional technique as a theoretical ‘counterpoint’ to the principles of tonality—in particular, he wanted to ‘emancipate’ dissonances, which had been degraded from Pythagorean theory to classical counterpoint as standardized by Johann Joseph Fux. Let us describe the dodecaphonic idea before we delve into creative manipulation of such a system. Schoenberg’s idea was to give all pitches of the chromatic scale the same chance, as opposed to tonal music, where the tonic and dominant play a strong role against the other pitches. As he did not add G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 13, © Springer-Verlag Berlin Heidelberg 2011

93

94

13 Creative Aspects of Musical Systems: The Case of Serialism

any other conditions, he postulated that we have to select these twelve pitch (classes) each once, in any of the possible 12! = 479, 001, 600 orders. And such a selection is called a 12-tone or dodecaphonic row. For example, the first row in Schoenberg’s own compositions was the row of the Waltz for Piano, op. 23, No. 5, it shows the pitch sequence G = (1, 9, 11, 7, 8, 6, 10, 2, 4, 3, 0, 5), with 0 ∼ c, 1 ∼ c, . . . (see also Figure 13.1). For every composition one has

Fig. 13.1. Arnold Schoenberg’s first dodecaphonic row for his Waltz for Piano, op. 23, No. 5, published in 1923.

to make a choice of such a row, which is then called “Grundreihe” or “tone row.” For probably historical reasons, Schoenberg then permitted generation of a paradigmatic field of derived rows by use of the group F of all 48 transformations f that were allowed in classical counterpoint: 12 pitch transpositions, 12 pitch inversions, 12 retrogrades, and 12 retrograde-inversions. This generates a set F.G = {f.G|f ∈ F } of 48 derived rows. An example of the original row, together with inversion, retrograde, and retrograde-inversion is shown in Figure 22.1. Exercise 65 Calculate the set F.G of the 48 derived rows for our example shown in Figure 13.1. This is nearly all of the system! Schoenberg allowed distribution of any collection of such derived rows on our composition, and the temporal sequence

13 Creative Aspects of Musical Systems: The Case of Serialism

95

of row entries was quite elastic: One may also display a subsequence of notes in a row as a chord. Figure 13.2 shows an example from Schoenberg’s op. 33.

Fig. 13.2. An example of a dodecaphonic composition from Arnold Schoenberg’s op. 33.

We see the tone row G, starting at (transposed to) b, then its retrograde inversion KU , starting at a, then the same KU , but starting at a, then the retrograde K, starting at e. Observe that the first row G has chords, so from that representation of the row, the original order of G cannot be retraced since the order of the notes in a chord hides the order in which they have been placed on the chord. This problem is also the case for the general question of retracing the original tone row from the composition. If we are not given the patchwork of rows but only the resulting set of notes, it is often impossible to find out the patchwork, and then even knowing the patchwork, it is often not possible to find out which is the original tone row. In fact, any of the 48 paradigmatic variant could be the original one. Besides the a priori impossibility1 to retrace the construction from the resulting composition, Schoenberg also has not given any additional rules for how to distribute such derived rows. Anything goes—no harmonic or contrapuntal or rhythmical rules are required. They may be played in temporal overlapping as shown in our example in Figure 13.2. There is no syntactical rule in dodecaphonism. And this is a significant difference to the tonal system, where 1

By analytical activity, but even more when listening to a performance of dodecaphonic music.

96

13 Creative Aspects of Musical Systems: The Case of Serialism

harmony also prescribes certain rules of syntagmatic order, e.g. the cadential order T, S, D, T. Perhaps to compensate for this totally chaotic syntagmatic display, huge attention has been given to the internal structure of a tone row. For example, a row may show all proper intervals when stepping from one pitch to its successor. Such a row is called “all interval row.” Dodecaphonic rows have been completely classified, see [39] for a reference. Dodecaphonic composers sometimes were so proud of their rows that they even would patent them (Schoenberg did so!). The composers thought that their rows were hiding a mystery of inner beauty that, hopefully, would be revealed in the course of their deployment in compositions as built from the 48-element row matrix derived by transpositions, inversions, retrogrades, and retrograde-inversions. In serialism, the idea to organize pitch in rows was later generalized to other tone parameters and was conceived in 1947 by French conductor, composer, and writer Ren´e Leibowitz. In 1952 and 1961, Pierre Boulez composed a famous piece, Structures pour deux pianos, I/II [17, 18], where he not only prescribed a sequence of twelve pitches, but also of twelve durations, twelve loudness values, and twelve pianistic attack modes. See our case study relating to this composition in chapter 25. In the following tutorial unit, we somewhat deviate from the previous presentations in that we shall not only present a creative aspect of a theory but also question the theory, namely the serial system, in the sense of transcending it in its totality. But we shall not just reject the serial system and replace it with still another one like Schoenberg did with the tonal system. We want to help students open their minds to new directions without necessarily having to rebuild a system to compose new music.

13.1 What Is Your Open Question? It is undeniable that music history and cultures all over the world have invented systems of musical thought and musical composition. And it is also unquestionable that these systems have helped generate a big number of musical compositions complying with these frameworks. In philosophy, systems have also been constructed and rejected. It has become a major challenge in philosophy to overcome such systematic approaches and to take philosophy beyond systems. In music, we also want to thematize this approach and to ask whether musical systems are a necessary condition for composing music with a thorough discipline. So let us ask this open question: What can be the role of a musical system for the composition of valid musical works? Exercise 66 Describe some musical systems that come to your mind. Don’t worry about precision or technical vocabulary, just use your natural language to specify them in your understanding. For example, the system of traditional

13.2 Let Us Describe the Context!

97

Korean music has pentatonic scales with many sliding techniques producing microtones in performance.

13.2 Let Us Describe the Context! Let us collect the ingredients that constitute musical systems. 1. They are all built on a set of available sound events, be they defined by pitch collections (such as scales in tonal music) or rhythms or melodic repertories (such as series in dodecaphonism). 2. This material can be manipulated following a set of rules (such as the selection of chords in tonal music or transformation of series in dodecaphonism). 3. These rules must be built according to a defined logic and be logically consistent, i.e. contradictory rule conflicts should not be possible. For example, the contrapuntal rules must be defined in such a way that the composer is never led to dead ends in the contrapuntal construction, i.e. to situations where the continuation of a contrapuntal construction is impossible because of too strong constraints. 4. The rules must be of a certain simplicity to enable constructions of compositions with a reasonable effort. Simplicity also means transparency. A rule system that can only be controlled by a computer is not what we aim at; there is not substantial delegation of musical creation to external devices and authorities. 5. These rules must be communicative in the sense that compositions that follow these rules make them evident to the audience. Rules that are beyond communication are possibly interesting to the composer in his/her intimate work, but the system has to solve the problem of transferring them into the musical utterance. 6. These rules must be flexible enough that musical creation is possible and even boosted. They must be catalyzers of musical invention and not only directives that kill the composer’s fantasy. It is like with traffic rules: They enable maximal freedom within the logic of a working traffic system. 7. The system must help create musical meaning, not just forms. This is in accordance with our principle of creativity as a generator for new meaning in music. We have chosen to discuss the serial system in more detail here because it is very explicit in its structure, in particular with regard to the possible production of new contents. It was not grown from long historical processes of evolution, but introduced as a creation initiated by one man in a very precise moment and for very precise reasons. Schoenberg even believed that this system would guarantee German music a dominance for the next hundred years! So this invention was conceived as a strong motor for future musical creation. It is a bitter irony that its Jewish inventor was exiled from Nazi Germany in 1933.

98

13 Creative Aspects of Musical Systems: The Case of Serialism

Exercise 67 Listen to Schoenberg’s Waltz for Piano op. 23, No. 5 published in 1923. Does the composition inform you about the six ingredients of a musical system described above? Play the composition’s row shown in Figure 13.2. Try to hear the row or one of its transformations when listening to the waltz.

13.3 Find the Critical Concept! Within a musical system, there is a core instance that may be called the primary motor for the construction of musical works, and this is the set of rules, together with the primary material that is fed into these rules’ action. Let us therefore select the concept of system rules as the critical concept. The system’s rules play a mysterious role. On the one hand, they enable what the composer might want to construct; on the other, they direct and restrict his/her production. It is the admitted or hidden hypothesis of a system’s architects that these rules may foster creative production. In terms of contemporary cognitive science, one may restate this attitude as a construction of artificial musical intelligence. It is assumed that such rules encode processes that are responsible for well-formed and ideally interesting musical output. In the case of serialism, the rules are very simple: Take a series and generate its 48 derived series by musically standard transformations inherited from the successful tradition of contrapuntal composition that had matured since Palestrina’s early days in Western art music. And it is argued that the basic material, the series, has also been built following a generative principle that guarantees a balanced display of all pitches available in our standard semitone chromatic repertory. The first problem of this approach is the creation of meaning. How do we step over from such formal constructions to new contents? This must be seen from the condition that all these serial devices do not rely on any of the known semantics. How can a row set up contents? And why do the 48 paradigmatic variations thereof help build a story in the composition? A story of what? What is the thematic power? Is the series a theme? And if yes, how does the theme unfold in the course of the composition? Is this unfolding communicated to the audience in any perceivable way? Is it a secret story that can only be understood in an intense introspection while listening to this music? These questions are extremely important to the composer—and ultimately also to the audience. Can he/she be creative while applying such rules? What is the space of freedom that serial construction offers? To answer these questions, we have to inspect the walls of the critical concept. What are the limits, and what is the potential of these rules when working as a creative composer?

13.4 We Inspect the Concept’s Walls! When we are applying a set of rules, we think of the situation as an automatic mechanism that could be delegated to a computer. In serialism, our model sys-

13.4 We Inspect the Concept’s Walls!

99

tem, we would program the computer to generate any series and then calculate the 48 variations. We could then do anything with this material. The theory does not add further rules, so any random walk would do in the spreading of such variations over the score. The same algorithmic approach can be made for counterpoint, harmony, and other systems. In such an approach, the subject which applies the rules has been eliminated. The rules have become formulas that yield output upon any materially admissible input. This is the approach used by American composer Tom Johnson, who has devoted much of his compositional efforts to get rid of the individual and subjective layer of composition. His string quartet Formula is typical for this approach: Each of the eight movements is based upon a mathematical formula. The wall of the rule concept is exactly this seemingly unavoidable constraint: to apply the rules. The wall properties of this situation are characteristic for walls: They are so strong and omnipresent that even the question about their necessity and function seems absurd. They are part of the definition of the concept. A rule is there to be applied, what else? But this is exactly the moment where we have to step back and look at the complete context of musical composition. To reduce compositional activity to apply a rule system is a huge step. What happens in this process? Are we playing an empty game, or is the system so strong and intelligent that we are dispensed from rethinking everything once we have understood the logic and power of these rules? It could be the same as with mathematical formulas: Once we understand a formula, we can implemented it in a calculation program and do more interesting things than unwrap the formula each time we have to use it. Let us recall that one of the greatest successes of metro-nome inventor Johann Nepomuk M¨ alzel was his chess automaton (see Figure 13.3), where he cheated everybody with a machine that could play chess better than most humans. His career ended abruptly in Cuba. He died from alcohol poisoning on the seaway back to the U.S. shortly after it was discovered that he was hiding a chess-playing dwarf in the machine’s innards. This episode nour- Fig. 13.3. M¨alzel’s chess automaished many dreams of artificial intelligence ton showing a Turkish puppet playing seemingly machine driven chess in the arts. It is a typical example of peoand the dwarf who was in reality ple’s dreams of a machine system that cre- playing the game. ates something intelligent and artistically valid.

100

13 Creative Aspects of Musical Systems: The Case of Serialism

13.5 Try to Soften and Open the Walls! The formulaic approach to composition by a system of formal rules is a style of depersonalized activity. The agent is hidden if not eliminated. The rationale is manifest: The rule system is supposed to be powerful enough to produce intelligent and interesting musical structures. But can formulas take over the roles of composers? Let us look at a very classical formula: the cadence sequence TX → SX → DX → TX of tonic, subdominant, and dominant chord functions in a given tonality X. If the essence of the cadence formula were this sequence, any realization thereof by a set of chords would do. But this is exactly what does not guarantee any interesting musical output. In tonal music one could argue that voice-leading rules must be added to the cadential formula, and so forth. But is that a sufficient rule set for interesting music? It is not. We are always facing this situation: The set of rules is exhausted, and we still do not have interesting music. In serialism it is the same situation, and even simpler: How could any random distribution of row variations on the score generate interesting music? We have known a number of sophisticated computer-driven systems of accompaniment in improvisation, e.g. [5]. They are defined by refined mathematical formulas, but it is always a failure. Our experience is that the formula congeals music rather than nourishing it. To be clear, we neither defend nor attack the claim that a rule system can produce intelligent music. It is something different that fails. And this is hidden in the meaning of the verb “to apply,” which we identified as the wall of the rule system. It is obvious that this keyword reminds us of the wall in the tutorial on instrumental aspects in chapter 12. Recall, that wall was about the nonconnection between the two aspects: playing an instrument versus playing a composition. When we face the wall of applying a rule in this chapter, we have a similar conflict: We just bring the formula to the audience. We are waiters of the formula and hope the client will appreciate the precooked meal. And we also had the critical concept “to play” in chapter 12, and here we have a similar one, “to apply”! In our analysis of the instrumental aspect, we could resolve the wall of “playing” by a reconstruction of the composition’s original movement, which is a dancing gesture that is performed on the instrumental stage. Is there a similar solution here? Can we go to the rule system in its early state where it was, like the neumes, not a dignified authority, but a living organism, so to speak? If “playing” could be resolved by “dancing on the instrumental stage,” then the rule system’s execution resembles the application of the theater’s screenplay. If we may dance the composition, should we then also choreograph it instead of applying the screenplay’s prescriptions? Recall that in dance, choreography is also beautifully known as dance composition. In other words: Could we replace the rule system’s screenplay with a choreographic autonomy?

13.6 How Can We Extend Opened Walls?

101

Exercise 68 Think about the difference between a playwright and a choreographer of a dance. Who is responsible for what? Could you compare the author’s written play to a musical system’s symbolic rule system, while a choreographer is more concerned with the the system’s embodied performance?

13.6 How Can We Extend Opened Walls? Let us try to extend the wall of rule application with the last section’s question on choreography or dance composition. What does a rule system look like if we choreograph it (with autonomy) instead of applying it faithfully? First of all, we have to appropriate the system; it must become ours instead of us being its slaves or delegates. This means, in particular, that its components must also be given a meaning that we cover with full responsibility. We (the composers) cannot refer to contents that are charged to others—all of it must be ours. If, for example, we use a row of pitches in serialism, it must be ours, meaning that we know and are fully responsible for the row’s pitch sequence. We must know why we have chosen this row and no other, and we must express this choice musically, not only as an abstract and private thought. It must be created and presented like a theme in the melodic process, full of anatomical shape of which we know(!), and ready to be unfolded in the composition’s screenplay. Otherwise it is exactly like a composer who says “I use a tonal system” but the audience cannot realize any tonality. There is no Platonic sky (the ‘refrigerator’ of deep-frozen gestures) where abstract ideas are deposited; we have to make them move here and now. Second, and this relates to the first point, we must communicate the system’s substance in the compositional work. This is why no formula can be applied as is; we have to unwrap it and to unfold its ‘innards’ in the compositional result. This is a delicate point because we do not want to be the delegates of an absent divinity. We follow Aesop’s sportive imperative: Hic Rhodus, hic salta! (Here is Rhodos, jump here!) This means that all of the content we are thinking about when composing should, if ever possible, be materialized as a concrete musical substance, notes, pauses, whatever. A theoretical system of thoughts is not what we show in choreography, we show the musical shape and its body. This is a hard and exhausting requirement, but we have to meet it, otherwise there is no opening of the rule application wall. Choreography means to drag to the sensible surface of musical realm those system rules and materials. Let us look at the situation in serialism to illustrate these points. To begin with, when we present the original tone row X, we have to make sure that it is perceived and fully understood by the audience. It is absurd to play it just once and then go on to whatever variated versions. We have seen in Figure 13.2 the first presentation of the row in a compact shape even containing chords. This

102

13 Creative Aspects of Musical Systems: The Case of Serialism

is definitely not the choreographer’s approach. It is nothing but Schoenberg’s “private music”2 world. It is not trivial and annoying at all to insist on X before stepping to other things. A row is a complicated object, and its transformations are also tricky variations of the original. Similar to a tonality, it needs a cadential statement: Here I am, have a firm look at me. We could, for example, play X several times in different time modes, or different dynamics, or different instruments, etc. We also could blow up the single notes to multi-instrumental big sounds with a temporal unfolding in long durations. Exercise 69 Listen to Gy¨orgy Ligeti’s beautiful composition Lux Aeterna for such long-lasting sound effects. Exercise 70 Listen to Pierre Boulez’s Structures pour deux pianos, a prototype of a serial composition, beginning of the first part, to see how one should not present rows in serialism. Exercise 71 Suggest other ways of making the tone row X more approachable in the beginning of a serial work. When we play a first transformation of the tone row, why should anyone recognize this one as a variant of the original? A composer can, of course, refute such a transparency, but why? There must be a good reason for doing so. Perhaps does the composer attempt to install a seemingly independent second row? But if he/she does not want to create this independence, then all efforts must be made to communicate this relationship. Otherwise it is like a modulation in tonal music but the composer has no idea how to modulate. If the second row is just a transposition, this is easy, because everybody recognizes transpositions as transformed versions, but it is not easy to perceive dependency when the row is inverted or retrograded. Exercise 72 Try to compose the inversion of tone row X in such way that the relation to the original can be perceived by the audience (and not only by analytical efforts). The problem relates to the idea of developing the theme (the original tone row X in our case) into a story or choreography, as you prefer. We want the listener to say: “Aha, now they step over this variant, I hear the correspondence, wow, so interesting that the original can appear embodied in this new shape.” It must be a pleasure to listen to the second shape. The pleasure can only occur if we deploy the transformational process into the musical material, and not only in our heads. 2

He is famous for his anti-audience and anti-communicative elitist attitude.

13.6 How Can We Extend Opened Walls?

103

Example 9 Here is an example of communicating row inversion by decomposing the original row in a sequence of small two- to three-note motives X1 , . . . Xk and then playing after each Xi its inversion I(Xi ) with a second voice/instrument before proceeding to the next unit, so we get the sequence X1 , I(X1 ), X2 , I(X2 ), . . . Xk , I(Xk ). After that we could play the entire inversion I(X) at once and proceed with this one. If the inversion of the first row is the meaning of the second row presentation, then please, communicate this meaning. The story must be choreographed and danced, not only conceived.

Fig. 13.4. The four measures 124-127 in op. 106, Allegro, showing an inversion that is responsible for the following modulation from G-major to E-major.

Fig. 13.5. Left: pitch-onset points in measures 124-127; right: all pitches transposed to one octave, showing that the first two measures are inverted at g (equivalently tritone d) to yield the third and fourth measures.

Example 10 Let us give a second example of materializing musical thoughts. Beethoven was a master of such techniques. See Figure 13.4 for the score’s shape of our example. If we look at the pitch-onset points described by this part, we see in Figure 13.5 that Beethoven composed an inversion-symmetric configuration around pitch g (equivalently tritone d), which maps the G-major scale to the E-major scale. Summarizing our extension, we may say that we have to complete the rule system by choreographing it for the sake of communicative embodiment of

104

13 Creative Aspects of Musical Systems: The Case of Serialism

materialized musical gestures. It is not so much the system per se that is put into question, but its musical realization. Nevertheless, we have to relativize the significance of a system in favor of the possibilities it offers and the options we have to make music out of abstract rules. In this sense, the serial system is a particularly weak corpus, because it lacks the most important features for creating musical compositions.

Fig. 13.6. The modulatory process induced by an inversion at d in measures 11-12 in Guerino Mazzola’s sonata composition L’essence du bleu.

13.6.1 Another Extension We have focused on the composition realization of the serial system without asking for the extension of the system as such. Let us do this now to complete the perspectives opened with this discourse. The serial system is built upon two components: (1) the initial material, the tone row X, and (2) the group G of symmetry transformations that are applied for the creation of the 48 variants of X. One may break up this catechism with two initiatives. They result from the fact that X and G have no intrinsic connection, they are totally independent choices. This means that we can replace X with any initial material Z and also, independently of this choice, replace G by any system H of transformations or deformations. Young composers are, in fact, often annoyed with having to follow the classical serial rules. This means we used the serial system only as a raw material and not as a system, like paint and canvas to make a painting. They now can free themselves from these constraints: On the one hand, they may use a row X because is offers a wide perspective on pitches that must be used. This can help invent interesting melodic structures that are incorporated in a tone row. But they may then use their own system H to create derived structures. Or they may generalize the row X to a non-dodecaphonic structure and then use the given group G to generate its derived specimens. Or

13.7 Final Step: Testing Our Extension

105

they may quit altogether these two standards and apply a generalized system H to a generalized initial structure Z.

13.7 Final Step: Testing Our Extension We have applied Beethoven’s modulatory idea in our sonata composition L’essence du bleu [79]. Figure 13.6 shows the modulation process from Cmajor to B-major in measures 11-12. This compositional part is built around the inversion symmetry around d. Exercise 73 Try to compose a dodecaphonic series X and make it evident to the listener in your score. Then, take a transformation thereof and add it to the score so that the transformation becomes evident. Exercise 74 Think about the dodecaphonic system as a system in your music. How does it affect your piece when composing differently from your previous approach. Think this time about the dodecaphonic system as a raw material. Then it is not your system. What do you need to add when composing. Compose each case and find out what is more suitable to your compositional style. Exercise 75 Read Fred Lerdahl’s critique of serialism in the article Cognitive Constraints on Compositional Systems [66]. Do you agree with his opinion?

14 Large Form Aspects

The topic of large forms is a culmination of our previous tutorials: All of them converge to this one, as can be seen from the leitfaden (Figure 3.1). This means that large forms use all of the preceding aspects: systemic, contrapuntal, and instrumental—of course including their own predecessors. But what are large forms? We have dealt with small forms: melodies, harmonies, rhythms. Large forms are combinations of smaller forms to express added meaning to the contents given by small forms. We stress “smaller” here in the sense that a large form is a combination of already defined forms, but these forms may also be large forms (well, not so large, but not ‘atomic’ either). The idea is a recursive one: We start with really small ‘atomic’ forms: elementary melodies, harmonies, rhythms. We then build combinations thereof, and then combinations of these combinations, and so forth. The complexity of these constructions depends on the complexity of contents the composer wants to build. But the essence is that musical contents result from such multiple interlocking of parts. To get a concrete idea of forms, we first look at two simple examples of large forms of the same size: the period and classical developmental sentence (see Figure 14.1). They are considered to be prototypes of a non-atomic form. Both are usually eight measures long and are built from two-measurelong phrases. Phrases are also not atomic, but they constitute minimal closed statements, similar to a linguistic sentence. In both sentences, you see the first phrase built from motives a and b. The second phrase in the period is one motif, c, whereas in the developmental sentence, the second phrase is a nearly literal repetition of the first phrase with transformed and deformed motives a , b . This repetition is characteristic of the developmental sentence type. In the period, this repetition with transformed and deformed a , b is postponed to the second half of the period (the consequent, following the first part, the antecedent), while the phrase c first develops the motivic statement before the initial phrase is repeated. To understand the multiple interlocking method in large forms, let us look at the sonata form. This one is considered really large in this chapter. AccordG. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 14, © Springer-Verlag Berlin Heidelberg 2011

107

108

14 Large Form Aspects

Fig. 14.1. The period and the classical developmental sentence with their phrase parts a, b, c etc.

ing to the standard concept of a sonata form, the movement contains four parts, to be played in temporal succession: exposition, development, recapitulation, and coda. Each of these parts is itself a highly complex large form, which we do not further investigate here. Rather, suffice it to say that the form is a rhetorical one: The exposition presents a theme, possibly together with secondary subjects; the development unfolds the theme and its consequences, the recapitulation comes back to the theme after the insights of the development have been exposed; and the coda concludes the sonata form. Each of these parts has its own parts and parts of parts down to single periods, phrases, and motives. The sonata form’s parts may also include complex harmonic processes such as tonal modulations. For example, the Allegro, the first movement of Beethoven’s op. 106, is a sonata form and displays eight modulations, starting and ending in the sonata’s main tonality B-major. The sonata form is, however, not the global form of a sonata. Often, as a standard, it is just the first Allegro movement’s form, while other movements—a slow Adagio movement, then a dance movement Scherzo, and finally the faster Finale movement (often again in sonata form)—follow. The four-movement sonata is frequently used in string quartets and symphonies. Observe that sonatas with four movements have only one or two movements in sonata form. But we should be aware that composers have always transcended given form norms when they wanted to express a content with a special flavor. For example, in Sonata, op. 109, which we discuss in the case study of chapter 26, Beethoven has written three movements, the third being a big sequence of six variations of a single theme! We are not going to describe the entire classification of large forms. Their types are numerous; one counts more than sixty of them, varying according to their historical position: medieval, Renaissance, Baroque, classical and romantic, or contemporary. We are more interested in the creative potential when

14.1 What Is Your Open Question?

109

dealing with large forms in the sense of how much more they are then a mere accountability of extended musical structures.

14.1 What Is Your Open Question? When looking at the classification of large forms, it seems to be an accountability subject. We see parts A, B, C,. . . and then, for example, the twelve-measure Blues form ABA, or the classical song form of jazz compositions: AABA, with “bridge” B, each part having eight measures, etc. But this is only the trivial external syntax of these forms. It has no deeper musical content and could as well be used for common language texts. The essence of large forms is not their syntactical schemes, but what content is generated under the surface of such a configuration. For a composer, however it is important to set up a balance between the form and the contents he/she wants to express. A form could be compared to a dress that expresses the body’s shape. Both the dress and the body have to correspond to each other. A composition can be overdressed by a form that by far passes the poor contents. And vice versa: A poor form can blur a body’s complex shape. But the dress-body metaphor is also somehow dangerous because the form not only expresses contents, but it also creates them. Let us explain this with an example from poetical creations in language. If a verse rhyme form ABAB is created, we may have “God” at the end of the first A, and then “bankrupt” at the end of the second A. This rhyme position of the two words creates a correspondence that may set up a correlation of contents, like “God = bankrupt,” say. So the formal position of these two word contents adds a content and does not merely describe given contents. This phenomenon has been called poetical function by linguist Roman Jakobson [58]. He considers two axes: first the axis of paradigms, i.e. associative fields of concepts. For example, the paradigm of “car” would contain “automobile,” “van,” etc. The second axis is the syntagm, i.e. the position of concepts within the thread of a text, like the position of “car” in the sentence “Claudia’s car is yellow.” Jakobson’s theory is that any poetical effect is generated by “projecting the paradigmatic axis to the syntagmatic axis.” This means that he is addressing the distribution of associative fields of concepts in the text’s thread. Rhyme poesy is one example of this theory. The idea is that this arrangement of paradigms creates a body of contents that is richer than the sum of the paradigms of its parts. It is not about two independent contents designed by “God” and “bankrupt,” but about the increase of contents setting “God = bankrupt” because of their syntagmatic position. In this sense, the construction of large forms is not only a distribution of given contents, but the anatomy of a form also creates added content values by the relationship of the paradigms defined by their positions within the formal architecture. In other words, the dress is not only a passive expression of the body, it also defines the body. Therefore, we suggest that the open question

110

14 Large Form Aspects

here is about the relationship between the external shape of a large form and the content of a composition written in such a form.

14.2 Let Us Describe the Context! The context of this topic relates to strategies of larger musical content creation. How can we compose musical works that compile complex aggregates starting from atomic structures? This is definitely a topic in music semiotics, the theory of music signs. The semiotic context is more precisely that of semantical production in the sense of Jakobson’s poetical function: What is the mechanism of poetical contents creation in musical large forms? Observe that we cannot transfer linguistic premises one-to-one to music. There is a difference here in that linguistic contents are given from common contents, it is not necessary to explain the common meaning of the words “tree,” “love,” “eternity” in a poem. In music, this first condition fails, and we have to apply radically different strategies in content generation. This difference is dramatic: We cannot rely on externally created contents, so the musical form must take the full responsibility for whatever it contains. But observe that the contextual question also has a historical dimension— every semiotic system has its diachronic evolution—and this may be responsible for contents. In language, every word has its etymological development—for example, “discipline” stems from Latin “dis-capere,” “to cut into pieces,”— that is a strong historical root for the word’s content. In music, we also have this weight if the musical language of a composer and his/her audience share a strong historical reference. For example, in Western culture, nobody can cite Beethoven’s main motif of his Fifth Symphony without evoking the Beethoven reference. But in the musical environment of one of the numerous small Indonesian islands, this may be without any semantic charge (meaning). Such a historical reference need not stem from a musical community. One can also create a ‘historical’ reference within a single work by repeating a motif so often that the listener ends up recognizing the motif as a returning base point. Michael Cherlin [24] has related this to the idea of memory in music, and he distinguishes three types of memory: reference within a work, reference between different works, and reference to topics outside works. The latter case is the exception: We are given a content of human life and compose as an expression of such content. For example, Gy¨orgy Ligeti admitted that many of his musical networks stem from an early recurrent dream he had where he was captured in a fetid, nauseating spiderweb.

14.3 Find the Critical Concept! We have seen that large forms deal with the creation of content in music. Couldn’t we just take this concept, “musical content”, as the critical concept

14.4 We Inspect the Concept’s Walls!

111

of this tutorial unit? We could, but this would not hit our main concern in musical creativity. The critical concept is not content as such but its expression in a poetical musical communication, so let us call the concept meaningful expression. Let us also explain this choice. We stress expression here, not the meaning. We do so because the creation of content is not the critical aspect; it is a semiotic operation that can work if done in a logically clean way. Accordingly, Schoenberg, in his book on principles of composition [108], refers to two criteria for a good large form: logic and coherence. The criterion of coherence, if it is not a logical one, must be an aesthetical one, dealing with the refined connection between parts in a large form. Schoenberg adds that composing a large form is not comparable to a child who juxtaposes wooden cubes; the composer creates a living organism, not just a collection of separated material units. But the logic of this organism is not sufficient, it needs to be complemented by an elegant connectivity. And we should add that this connectivity should be a movement between parts, like a living body that is not just a collection of dead of limbs but lives by their smooth interaction. The composer has to become aware that his/her composition has to express its contents; it is not sufficient (as Schoenberg believed1 ) that music is true, but it must also express truth in an aesthetically valid way. One could call this requirement a rhetorical one, one that tells you something that is true ‘anyway.’ But this is not what composition is about. It is about embodiment, about making and not merely representing. Unlike mathematics, there is no abstract reality where the musical truth is deposited, so that the composer has to pass it to the audience. We saw already in our tutorials on melodic and systemic aspects that the waiter’s role of serving a precooked meal is not what music is about. The truth of a large form’s logic can only be realized in its embodied communication: in short, in its poetical expression. In his famous definition of musical contents in [49], Eduard Hanslick states: Der Inhalt der Musik sind t¨ onend bewegte Formen2 . This provocative claim unites the form and the content.

14.4 We Inspect the Concept’s Walls! What can be a wall of meaningful expression? The concept has a mysterious aspect because the content is deeply intertwined with its formal aspect. It describes a seemingly paradoxical situation in that the two components, contents and form, are combined in a circular reference of seemingly complementary constituents: content  form. If a composer wants to compose in a creative way, this circle must be resolved. It is a hard wall, since circular configura1

2

Musik soll nicht schm¨ ucken, sie soll wahr sein. Music should not ornament, it should be true. Musical contents are sounding moved forms.

112

14 Large Form Aspects

tions often have a vicious character, and vicious circles3 are the dead end of a thought—it destroys itself in an impossible configuration. Our hope is that this circle is not vicious!

14.5 Try to Soften and Open the Walls! In order to understand the nature of this circle wall content  form, we have to investigate the nature of contents and then see what are the general construction principles of contents. For a more in-depth discussion of concept architecture, please see section 21.3.6. If we look at how a concept is structured, it becomes clear that most concepts are understood by some kind of reference to other concepts. For example, if you talk about a musical motif, this refers to a set of notes, and a note refers to two (or more) coordinates: pitch, onset time, etc. So talking about a motif refers to other concepts. This implies that the content of the word “motif” refers to the content of the word “note,” and this refers to the contents of the words “pitch,” “onset time” This is the general principle: You refer to other already-conceived concepts, and then to a method to derive the new concept from these antecedents—in this case, a set of notes, and for each note a list of coordinates: pitch, onset, and time. This example is typical: concepts refer to (1) other concepts, and (2) specific building methods starting from these other concepts. So the contents of a concept are constructed from contents of already-built concepts and the architectural methods to step from these preconceived concepts to the new one. Contents are built referencing other contents; they are recursively constructed from other contents by well-defined reference ‘arrows.’ In other words, contents are given by formal schemes to refer to other contents. They are a system of arrow links to other contents. Contents are a formal arrangement of other contents. Of course, this looks hopeless because then we have to ask for the referred contents to understand the new concepts. The latter may also refer to their own antecedent contents, and so forth. The unanswered question is: Will there be an end in such referential recursion? Before answering this question, it is important to acknowledge that content heavily relies on formal constructions, and it is not just an amorphous fact. In section 21.3.6, we present all types of recursive reference architectures in conceptual constructions. Their number is very small, essentially only three types: collections, lists, and libraries. In the above example of a motif, it is constructed as a collection (set) of notes, and a note is a list of its pitch and onset coordinates. 3

Vicious circles are logical constructions that lead to contradictions because of an impossible self-reference. For example, the barber saying, “I shave all men in this city who do not shave themselves,” is such a circle: If he shaves himself, he would be one of those who do not shave themselves. If he does not shave himself, he should shave himself. There is no solution to this logical problem.

14.6 How Can We Extend Opened Walls?

113

There are two answers to the unanswered question: First the reference may end in some ‘atomic’ concept, whose content is elementary for our understanding. For example, if we look at the motif concept, pitch and onset refer to numbers. And we may assume that these concepts are somehow ‘atomic’ to the music theorist. The second answer is that a reference can also be an autoreference or circularity: a reference to itself. For example, the Fourier series of a sound, describing its partial tones, is defined as a list consisting of (1) the fundamental sinusoidal wave, and (2) the overtone series, and this latter one is again a Fourier series! But see section 21.3.6 for details. This discussion of conceptual architectures makes clear that contents are built from • the form of referential architecture and • either atomic first concepts or circular reference (autoreference). This lesson can be applied to understand the circular wall content  form of meaningful expressivity. In fact, the first component of contents is a form! Therefore, the circular wall can be understood: Form is one half, so to speak, of content. If we can manage to understand the second half for large forms in music, we have solved the problem.

14.6 How Can We Extend Opened Walls? Coming back to the musical situation of large forms, the critical concept of a meaningful expressivity, and the wall content  form, our analysis of conceptual architecture teaches us that we may first have to see what is the formal “referential” half of the concept of a large form. All large forms are defined by the poetical function setup where paradigmatic units are distributed on the syntagmatic axis: They describe a list of parts, such as AABA, or the four sonata movements. So the content of such a form is defined by the contents of its parts, together with the content that is generated by their mutual reference, such as the double repetition of A in the song form, or the repetition of the exposition in the sonata recapitulation. Although such repetitions need not be literal, they pertain to the same paradigm in Jakobson’s sense, but the precise way must be described by opening the form’s architecture of these parts. We see that this way of understanding concepts is completely analogous to what we saw above: The method relies on continued recursion. Before we discuss in detail recursive depth perspectives, we should reconsider the question of how content is generated by the formal display of parts. The most elementary situation is repetition. Why should repetition generate contents? First, the long repetition in time creates a firm grid, to which other musical structures refer. Second, it creates a regularity for a dancing trance movement. Third, a repetition changes the unit’s perception.

114

14 Large Form Aspects

In the case of repeating A once in the song form before stepping over to the bridge B, it makes you listen differently to its content. You may become aware of relations within A that a single listening would not have shown; it expresses a fact to you, the listener. This is useful for understanding the bridge B: We perform a major change of material, e.g. a tonal modulation, and therefore have to be given the reference to A as a starting point of the musical process. Finally, repeating A a second time after the bridge brings us back ‘home’ to the basic structure, which we only recognize as such because it had been played twice before the bridge. In the case of the sonata’s recapitulation movement, this is a repetition, but far from simply doing the same again. After the developmental movement, the original theme looks totally different. We know its ‘inner life’ and therefore cannot play and listen to it again as in the exposition. Also the tonal structure can look quite different from the exposition’s one. It is a second look at the main theme, but it includes and applies all the added knowledge. These are some examples showing how added content is generated by the simple repetitive process within Jakobson’s paradigmatic fields. But the harder problem is the last step when running through recursion: first and atomic concepts or circular reference4 . What can we expect in musical forms when we step down to the very beginning? The atomic concepts can occur. This is a characteristic feature when we build our forms upon motives or rhythms that are known from music history and/or culture. For example, Bach’s motif B-A-C-H (b − a − c − b) is often cited, e.g. in Sonata No. 2 for violin and piano by Alfred Schnittke. The frequent citations of jazz and other musics in Schnittke’s Symphony no. 1 are also famous (e.g. Tchaikovsky’s B-minor Piano Concerto, Johann Strauss Jr.’s Blue Danube Waltz, Chopin’s Second Piano Sonata, etc.). Or Arnold Schoenberg’s citation of the popular song Oh, du lieber Augustin in his String Quartet, op. 10. Here the creative action of a composer is to build the formal framework where these semantic ‘atoms’ are embedded and act as motors of added content. The tricky point however is not the original content of these atoms, but their staging as constituents of enriched content. This may typically be achieved via transformations and/or deformations (as discussed in the melody tutorial in chapter 10) as mechanisms that generate new paradigmatic representatives of the atom or of of a derived form. Whereas the atomic approach could be called ‘music about music’, the second type of first concepts, circular reference, is more substantial in that it does not ‘borrow’ contents from elsewhere. Music that follows this constructive 4

In an architecture of concepts, an atomic concept is one that comes first without any further explanation, like “God” in religion, or “negation” in logic. A circular reference is given, for example, if you look up in a dictionary “point.” It usually defines it as, “A point is an element of a space.” And if you look up “space”, it says, “A space is a collection of points.” It does not create any contradiction, it is a nonvicious cycle of first concepts.

14.7 Final Step: Testing Our Extension

115

approach is often called “absolute music” because its contents are not linked to external references. A fugue is a classical example of such a construction (see section 10.4). The motif of departure is the dux. It is associated with a number of comes variants, which answer the dux as representatives of the same paradigm: They are transformed and/or deformed derivatives of the dux. The form of the fugue sets up a dialogue among these basic shapes, and the content is built by their autoreferential character. This example is fascinating because it simulates a quasi-literary dialogue without having to build upon external semantics. The dialogue makes sense without preconceived contents! Perhaps is it true that literary forms are also not so strongly dependent upon the given contents, but more on the form’s shape (we already mentioned this possible fact in a footnote of section 10.3). An extension of this idea is the construction of a musical drama, which imitates the literary drama with musical ‘subjects’ instead of real world persons. Beethoven is famous for having composed such dramas of absolute music. His music is dramatic in that it tells a complex story of musical motives and gestures without any extramusical references. The body of his music is a drama, and that drama is dressed in a complex form of references between musical subjects in such a way as to express a meaning of unfolding forces and energies. Therefore, our extension of the circular wall content  form is the attempt to build an inner-musical drama. Although the concept of a drama seems to suggest a temporal development of a story, this interpretation is a too narrow. A musical drama might as well be a development of perspectives of a given basic subject. In Beethoven’s op. 109, which we discuss in detail in the case study of chapter 26, he unfolds the drama as a sequence of variations of a basic structure. This spatial display offers a variety of perspectives upon a subject, whose function is to deepen the understanding of one and the same basic idea. It is an iterated view upon a given object. It communicates the expressive contents of a strong germinal structure and meets the dramatic character of the large form in its intense investigation as a force field of musical contents. It meets the principle that understanding is created by variation of perspectives, like with a sculpture whose shape is understood when one walks around it and looks at the object from all important points of view. Remark 10 If you want to learn more in depth about Beethoven’s deformation of standard large forms, yielding new meaning, please refer to our case study of the six variations in the third movement of Sonata op. 109 in chapter 26.

14.7 Final Step: Testing Our Extension To understand dramatic constructions in absolute music, let us look at some beautiful expamples:

116

14 Large Form Aspects

Example 11 Listen to Beethoven’s String Quartet Grosse Fuge in B-major, op. 133 and try to describe its dramatic overall shape. Example 12 Listen to Scriabin’s Sonata op. 72 Vers la flamme and try to describe its dramatic overall shape. Exercise 76 Try to set up a thematic germ and compose a dramatic form thereof, using a system of formal parts where you express the germ’s contents in its variety of paradigmatic variations by transformations and/or deformations. Exercise 77 Listen to Beethoven’s String Quartet No. 9 in C-major, op. 59, No. 3, second movement. This movement is written in sonata form. Look at the score and find out where the recapitulation starts. Think about how he treats the recapitulation concerning the first and second themes. Do you recognize the second theme in the recapitulation? If not, think why he had to adjust its conservative role.

15 Community Aspects

This tutorial unit and the next one are a bit different from the preceding ones because these two touch less on the music and more on the human conditions that define the composer’s existence: community and commercial aspects. But we believe that they are nonetheless important to understand the shape music gives to composers’ lives as its creators. Community aspects are a hard topic. Cooper Moore, a relatively successful free jazz pianist and instrument constructor, has, in a recent interview with bassist William Parker [95], stated: “Although we say there’s community, there’s not community.” And he adds that his main concern is paying the rent. We will come back to the latter, but it is by no means an exceptional statement about community. All composers and creative musicians keep stressing that the only thing they care about is their own music—the personal, individual, and exclusive dedication to their music. So should we just cancel the topic and join the lonely genius’s one-man world view? If we recall Schoenberg’s paradoxical Verein f¨ ur musikalische Privatauff¨ uhrungen, his Company for Private Music Performances founded in Vienna after World War I, we would definitely resign from community discussions. Fortunately, there is more. These statements are strong, but if we analyze them, they are not against community as such but express a deep doubt about the community’s relationship to the music that these creators are inventing. They wish to have a better community, after all they are not antisocial, they are simply disappointed. Even the famous and successful free jazz pianist Cecil Taylor, who is known for his uncompromising style and for his extremely critical judgment of community in music, admitted that in his most lonely periods when he had no gigs at all (and was forced to wash dishes and earn money as a cook) he played his unheard rehearsals at home as he would perform for a virtual audience. He had to simulate the communication with music lovers. The conflict is a very clear one: On the one hand, you, the composer, want and must create your own new world of music, and as such you are—by definition—lonely because these new things are only conceived and imagined by G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 15, © Springer-Verlag Berlin Heidelberg 2011

117

118

15 Community Aspects

your individual work1 . On the other hand, you absolutely need recognition by humanity, by the humans who have brought you up, who are your friends and fellows. This latter is an often unacknowledged desire, but it persists—despite some heroic claims of absolute autonomy. The present tutorial unit deals with this tension, and we hope it is able to give you some hints to overcome its existential challenge.

15.1 What is Your Open Question? God must have been very lonely when he created the universe. A composer is not a god, but he/she ideally creates a new small universe. And this makes the creator lonely. But unlike God, the composer has a community of humans (and sometimes also animals) who might be addressed to appreciate this creation. The tension of this situation is that the autonomy of a composer’s works and the appreciation of the human community are not a priori unified and may put each other into question. Cecil Taylor rightly stated, “Nobody asked me to play the piano.” Nonetheless, it is not acceptable to just let things go by without trying to find a solution or at least understand the duality of positions. The open question therefore is about the options that are available to negotiate between these two perspectives: the composer’s musical universe and the community’s appreciation. Let us put it this way: How can the composer’s right to create his/her own musical universe be reconciled with the community’s right to appreciate or refuse that musical universe?

15.2 Let Us Describe the Context! The historical context of musical composition is anything but isolated from community aspects. On the contrary, music was for a long time created exclusively for its social function, be it in religious or social ceremonies or for military targets (execution, march, heroic hymns). For example, in China during Mao’s cultural revolution, music was strictly channeled to its socialist function to support workers’ moral. Also, in certain restrictive present societies, music is still forbidden as if it were a destructive and immoral drug. In modern societies, one might add musical community functions such as Muzak in stores, elevators, and other social environments; entertainment music for dances, parties, and similar fun events; music for media such as films, games, or news background; and music as acoustical furniture. The pretension that music should be an individual world without essential community reference is relatively new and might go back to the Renaissance 1

This is not absolutely true, since creation of new works can also occur in community contexts, such as collective improvisation or technology-supported social music creation, see [114] for the latter. We shall come back to such perspectives later in this chapter.

15.3 Find the Critical Concept!

119

and its discovery of the individual position as an existential analogy to the discovery of central perspective in the visual arts. This does not question the autonomous claim of a composer, but it opens the view to a long tradition of music as a social art, as a functionally determined activity. Therefore, relating one’s composition to community concerns is not artificial and also makes us understand why communities have a tradition in their relationship to music. We have seen in the previous tutorial units that the creation of music in an autonomous individual universe essentially involves creation of new contents, which may be seen as extensions of the given music-semiotic system. If a composer creates such an extension, it is important to address the communication of such extensions to an audience that is familiar with the given system and that may understand the new contents from their artistic expression. This semiotic aspect was not critical as long as musical creation was an utterance in a given musical language, and where new contents were extremely rare. But in present times, when a composer creates his/her own new world, it is no longer guaranteed that any community would understand the content of such new expressions that lack any social acceptance. In other words: If you create new contents, the language that you use might be so far-out that nobody can understand it. You would then have to make extra efforts to create and bring to life a community that is familiar with your newspeak. The community context of musical creation is not only “here and now”, one should be aware that one’s compositions may be understood only after a change of civilization or culture (synchronically) or epoch (diachronically). If a composer is aware of these coordinates, it might be easier to make the targeted community more precise. For mathematicians this is a completely standard situation because mathematical research is usually only digested by a larger community after several decades, if not centuries. This mathematical attitude (which is strongly supported by the scientific community) could be a model for advanced composers, who would then be urged to build an elitist huis clos community of researchers in music. Schoenberg’s Verein f¨ ur musikalische Privatauff¨ uhrungen was such an initiative, and the famous Darmst¨ adter Ferienkurse are also conceived in this avant-garde spirit. The major difference from mathematics however is that the progress that would be achieved in such new music elitism is by far less demonstrable than for mathematics. A mathematician who was invited to a concert by a friend who wanted to offer him a nice evening was asked after the concert, “How did you like it?” His answer: “It was OK, but what was demonstrated after all?”

15.3 Find the Critical Concept! The critical concept for the open question about the composer’s musical universe and the community’s right to appreciate might be relevance, the problem of how important such creation could be, and for whom this matters.

120

15 Community Aspects

The critical concept is very general, but this is fine since the problem is fundamental: It regards the problem of why one should spend one’s life composing new music in light of the fact that there is already so much music, and there are so many more or less standardized ways to build music according to given rules and traditions. In the sciences, mathematical progress is objectively needed; the relevance is obvious. It is even argued among theoretical physicists [96] that any far-out theory in mathematics will sooner or later become a pillar of physics. Of course, Cecil Taylor is right: Nobody asked him to play the piano (in that way). He can do whatever he likes. And so can every composer—one-man universes are not forbidden. They are, however, kind of ridiculous if the artist does not care about the consequences of his/her creations. Schoenberg was also right: Musical composition of large forms is not a child’s playing around with wooden cubes, and we should add that it is not the infantile gaming of a child either.

15.4 We Inspect the Concept’s Walls! Relevance is a delicate word, because it is loaded with instinctive rejection: Who cares? And aren’t we lacking any concrete answer when we deal with the arts? Can the mathematical situation be of any help here? One of the key walls of “relevance” is that it is a serious affair. What is relevant must be taken seriously, it is neither a simple game nor a joke. And this wall is again (like many others we have encountered in this tutorial) hard since it is tied to the very definition of the critical concept. The key point of being serious is that the affair is objectively important, we must recognize it as such and deal with it in an adequate way. There are many relevant affairs that are not taken seriously—until it is too late. Hence we are asked to behave in a serious way, and this is asked of the composers and the community alike.

15.5 Try to Soften and Open the Walls! So far, the wall makes an impenetrable impression: Is there any means to open this moral attitude? Or are we simply thrown back into outdated ethical categories? To be serious is a communicative category: You care about your work, but you also care about its reception, about transferring the complete message for which you are fighting so passionately. Seriousness is about dedication and precision. We do not believe that it can be achieved in an exclusive conversation with yourself. You need a minimal distance to what you are doing to escape narcissistic love. It is well known that there are always strong issues that escape the auto-conversation, and that communication always strengthens your inner dedication and message. And the community you choose should also

15.7 Final Step: Testing Our Extension

121

be the result of a reflection about whom you take for serious and who takes you for serious.

15.6 How Can We Extend Opened Walls? Yes, seriousness is about dedication and precision. We have to open the wall of seriousness and ask ourselves what exactly we want to say, and to whom we want to communicate this concern. We have seen highly talented young composers who were not capable of saying one single word about why and how they are working. This failure must be avoided at any cost to be successful as a creator of works that are relevant to humanity and not only to your own private one-man universe.

15.7 Final Step: Testing Our Extension Exercise 78 Think about why you want to compose your works and to which community you want to communicate them to reach a mutual understanding of what you are creating. Exercise 79 Read the paper Who Cares if You Listen? by Milton Babbitt, High Fidelity (Feb. 1958). (Babbitt called this article “The Composer as Specialist.” The original title was changed without his knowledge or permission by an editor at High Fidelity.) This is an article about the problem of difficult music for the general audience. Babbitt compares this music with nuclear physics which is also too difficult for the general public. What do you think about this article? Exercise 80 Imagine your composition is being played in the Carnegie Hall, or any place you like. Imagine that you are the audience, enjoying your piece. Does it cause a doubt or pride to you? does it sound like a one-man world or does it say anything to the audience?

16 Commercial Aspects

When you have put all the ingredients together, from community to large form, you are confronted with the most annoying question: Where will I be positioned in the local or global marketplace? After all, you want or have to make a living out of your profession as a composer. Of course, you can become a teacher or even a university professor of composition and then you have also answered the survival question, but this is not our concern here. We want to think about earning money as a composer, not as a teacher. If you look at the composers’ economic situation, you find that since all have to compose, they must try to survive, whatever situation they may be thrown into. Cecil Taylor washed dishes, drummer Sunny Murray drove a taxi, Conlon Nancarrow was just wealthy, Beethoven was moderately paid for the publication of his works, pianist Cooper Moore’s most important question was how to pay his rent, and bass genius Sirone never resolved his money struggle. Some commercially successful contemporary music composers hate to talk about marketplace, but they know about their advantageous commercial position and simply refuse to talk about that aspect of their profession. Yes, the topic is nasty, and no universal solution is at hand. This is why we should spend some time thinking about creative answers to the money question.

16.1 What Is Your Open Question? Why is the commercial aspect of musical composition so annoying? A quick answer would be that because music is about beauty, passion, and highest spirituality, it is absurd to connect it to the down-to-earth question of making a living. Composers might prefer to live in the spirit of the Bible (Matthew 6:26): “Look at the birds of the air, that they do not sow, nor reap nor gather into barns, and yet your heavenly Father feeds them.” But this attitude is inadequate. Composers do sow, and they do it to a community of their choice. So why shouldn’t they gather? Gather recognition G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 16, © Springer-Verlag Berlin Heidelberg 2011

123

124

16 Commercial Aspects

and income as citizens of a civilized humanity? Why should money only be earned from selling arms, real estate, and animal food? We have seen in tutorial 15 on community aspects that the serious relevance of a composer’s work in his/her communication with a community may establish a relationship of (mutual) appreciation. Why should this appreciation not be reflected in commercial value? But our question is somewhat more delicate: Is the aesthetical value of a musical work beyond any commercial valuation? And conversely: Is every commercial compensation of musical works their automatic degradation to merchandise? So let us envisage this provocative open question: Why shouldn’t money be earned from selling beauty?

16.2 Let Us Describe the Context! The context of commercial aspects of serious musical composition is immediately related to abusing highest human goods—in other words, as if it were “Love for Sale,” to cite one of Cole Porter’s jazz standards. The commercial system is in fact defined by an exchange of goods, of objects you can put onto a shelf to await purchase by a consumer. We have seen in tutorial 12 on instrumental aspects that music—if conceived in its full expressive power—it is a fully engaged dance of humans on the instrumental stage, which is incompatible with detachment and consumerism. How could such an intense existential utterance be deposited on a shelf like a labeled consumer good? Our context is also related to the gigantic music industry, where composers (called songwriters) are the first germ in a complex and brutal machinery between music publishing, performing, and recording, see [57] for an excellent reference. Although we understand every composer’s disgust, a closer look at this environment is important. You have to know about copyright for music works (©) as opposed to music performances (℗), etc. You have to know the copyright protection companies, otherwise you will be a loser whenever you sign a contract with one of those labels or publishing companies. In the music industry, a composer, even if he/she enters with good intentions, will very probably be squeezed until all ideals are eliminated and Cole Porter’s song becomes reality. So the context is that the factual environment is an industrial chimera while the kernel concern of a composer is to create living beauty. It could not be more split.

16.3 Find the Critical Concept! In this split world of commercial and aesthetical perspectives, there is one common concept that challenges all of them, and this is value. What is the value of what you are creating in a musical work? Is there any answer to the question of creating values here that might be valid in both systems? We suggest that our critical concept is this one: value.

16.5 Try to Soften and Open the Walls!

125

16.4 We Inspect the Concept’s Walls! It is about values when we confront the creation of musical beauty in a composition with the problem of making a living. In these values there is one common aspect, namely that both are about life: Composing is the innermost value of life and making a living is about life in a very basic sense. The wall that we have to face is the value for life. This wall unites the entire paradox of high artistic aspects of life and its low everyday aspect. This split view makes the theme so annoying: The high perspective is what the composer desires, the low one is what must be suffered. During Mao’s cultural revolution, they killed intellectuals (where anybody who was reading literature except Mao’s red booklet was qualified as an intellectual) or sent them to the farmers to learn about “real life.” It was an ideology that wanted to destroy the “high” bourgeois aspects of life—everything had to be low. A composer had to compose music for workers, not those “useless intellectual and decadent Western artifacts”; see the interview with Chinese composer Gen Gan-ru in [95] for more details. The rigidity of this wall exists in the pair of opposites, “high” vs. “low.” The question is whether we could resolve this tension without destroying one of them in the manner of Mao’s cultural revolution.

16.5 Try to Soften and Open the Walls! To begin with, the polarity “high” vs. “low” of life values is derived from an implicit gravitational force. Low is where you fall when you lose control over your body. High is where you jump when you are in full control. So what is that gravitation of life values, against which Goethe’s Faust was fighting when the uht, den angels welcome him with the promise1 : “Wer immer strebend sich bem¨ k¨onnen wir erl¨ o sen.”? It is the force of death against which every plant shows its triumph when growing towards the sky. But there is more than biology—it implicitly includes a divinity at which our high aspirations are targeted. There is a perfection and fulfillment toward which we are striving. For such a plan there is nothing more disturbing than everyday life’s contingencies. Perhaps should we spend a few minutes on divinities. After all, it’s them who create the gravitational paradigm. The plan is to approach them and ideally be united. Every composer, when he/she creates that mini-universe of compositions, implicitly portrays a god’s creational act. So let us question this attitude. Do we have to portray God when we compose? Or is there an alternative to this “divinity neurosis” that would not send us to Mao’s farmers? 1

He who strives on and lives to strive/ Can earn redemption still.

126

16 Commercial Aspects

16.6 How Can We Extend Opened Walls? Recall that we already had a similar situation when discussing instrumental performance in tutorial 12. It was about the performer playing a composition as if he were a waiter serving a dignified plate from the heavens. And we then suggested replacing this ignoble and musically destructive role with the musician’s embodiment of the composition in the role of a dancer, who interacts with the instrument as a stage and not as a sound-generating command station. Our alternative in the present situation would then be to think about replacing the divinity paradigm with one where we do not need to ‘jump to infinity’ to achieve our compositional goals. Essentially, we could try to apply the same strategy as described in the instrumental context: embodiment instead of reference to divinities. Take the composition inside ourselves and fill it with the intensity of our lives. Becoming a composition with all our vital energies, being filled with our work as opposed to setting them down like a mini-universe creation. This being filled with our composition is a promising display: We have replaced the polarity of “high” vs. “low” with a new one: “inside” vs. “outside.” We are identified with the composition, and our skin is the composition’s skin, so to speak. Why is this inside-outside paradigm a creative extension of the wall? Because now there is a natural interface between work and everyday life: the skin. There is an interface that before did not exist; there is no reconciliation between high and low. But the skin’s paradigm offers an exchange of energies and ideas—its vibration can transmit messages from inside and also from outside. We are all of a sudden provided with a sensitive channel between the life of a work and the work of life.

16.7 Final Step: Testing Our Extension This metaphor is not only beautiful, it really means something in the understanding of one’s compositional work and one’s life environment. The work’s pulsation can and should be felt in life, and the life’s pulsation can and should be felt in the compositional development. Let me make a short example. Example 13 The Beatles, Bob Dylan, Witold Lutoslawski, and Iannis Xenakis modeled lives that balanced the values of the aesthetical inside and living outside experiences without having to sacrifice one of the two sides of life. Their music is highly artistic and beautiful, and also became a part of our life. Exercise 81 Think about how these artists or any of your favorite musicians developed their career. Could they have become your favorite musicians if they didn’t consider any of the commercial aspects and therefore didn’t have many chances to approach you?

16.7 Final Step: Testing Our Extension

127

Exercise 82 Think about the pulsation in your own life as a musician and person of a living society.

Part III

Theory

17 Historical Approaches

Historically, creativity was reserved for divine instances, which accordingly were also called creators. Humans were never endowed with the genuine capability of creating anything. In particular, when composers started being recognized as constructors of musical works in the Renaissance (as opposed to Pythagorean musicians who simply were reflecting eternal creations, not making new works), their creative actions were understood as inspirations, literally: getting some spirit from somewhere, i.e. being driven and shaped by a divine instance. Therefore, musical compositions were not human creations but simple outputs generated via the divine origins. Beethoven’s famous statement, “Musik ist h¨ ohere Offenbarung als alle Weisheit und Philosophie,” confirms this understanding of musical creation: It is a revelation, not an invention of the composer. When we speak of creativity today, we often refer to an ability that is beneficial in all sorts of domains: economics, politics, art, education, science, or technology, for instance. Even in everyday life we are often faced with situations that require us to find creative, non-evident solutions. If we, for instance, forget to buy a crucial ingredient to complete a dinner recipe, we might have to improvise with whatever is available in the kitchen. Or when we don’t know a word in a foreign language, we might have to come up with a creative way to circumscribe what we intend to say. There are countless situations in life where we are not even aware that we are being creative. But what exactly is creativity, and how can we describe what happens when we are being creative? A recent definition given by Sternberg and Lubart reads: Creativity is “the ability to produce work that is both novel (i.e. original, unexpected) and appropriate (i.e. useful, adaptive concerning task constraints)” [112, p. 3]. This definition well reflects today’s notion and is abstract enough to be applied to numerous domains of creative activity. Still, who is it who can be creative? What do we mean by ‘work’, and what is the product? Where and in what situation can someone best be creative? In what context does something have to be novel in order to be perceived as creative? Common answers to such questions are broad and often contradictory. This section G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 17, © Springer-Verlag Berlin Heidelberg 2011

131

132

17 Historical Approaches

attempts to create a selective but multifaceted picture of recent and historic views on creativity. We first investigate where our concept of creativity comes from and how it developed over time. Then, we give an overview of early and recent psychological research as well as contemporary characterizations of the creative process. Finally, we briefly discuss the special case of musical creativity and examine previous research in the field.

17.1 The Concept of Creativity through (Western) History The urge to be creative has been central to human life and culture for tens of thousands of years, and it can be traced back to the early days of human civilization. It is common today to view the evolution of the concept of creativity as closely connected to art history. We know that already in the period of the Upper Paleolithic, almost 40,000 years ago, and presumably even much earlier1 , humans created objects, paintings, or musical instruments. Surviving artifacts such as the Venus of Hohle Fels, the cave paintings of Chauvet, or the Hohle Fels flute are undeniably linked to an activity that we would call creative today. The former two are clearly symbolic expressions of a human perception of the world, and their appearance hints at an interest in the aesthetic. The latter implies the existence of an early form of music, an abstract way of expression requiring interest in experimentation. Unfortunately we are unable to find out what exactly the motivation behind the manufacture of such items was. Even though we consider them works of art today, it is likely that they were created in a context significantly different from the context a present-day artist works in. They might have been expressions of a higher sense of creativity and meant to worship a higher creator, as was the custom thousands of years later.2 Throughout human history, religion and mysticism played a great role in activities that we would consider creative today. Creative acts were often analogized with the creation of the world by a higher authority, or at least put in context with the prevailing worldview, as we know it of the earliest civilizations known to us. At the same time, individuality and innovation were appraised as secondary, if at all. In ancient Egypt, for instance, it was not innovation that was central to creativity, but repeated ritual actions. These actions were perceived as preventing chaos and keeping the world rolling [6]. In ancient China, creativity was understood as a long and slow process in which not a single being, but numerous individuals are involved, much in the fashion of the Darwinian view of the evolution of nature. The goal of this process was again not innovation, but the transformation of what already exists [70]. Finally, in 1

2

The arguable oldest surviving sculptural object is said to be the Venus of Tan-Tan, about 400,000 years old [11]. It is of course equally possible that they were perceived to be as individualistic and innovative as we perceive a contemporary work of art.

17.1 The Concept of Creativity through (Western) History

133

Taoism and Buddhism, a seeker has to avoid goal-directedness, concentration, and especially self-consciousness and submit to the void and indeterminate in order to be able to transform and create. It is taught that one cannot want to be creative, but one has to yield to creativeness [54, p. 25-6]. Today’s Western concept of creativity is dramatically different from Egypt’s repetitive and ritual, China’s collective and long-term, and Buddhism’s selfless and passive notions. However, our notion is the product of barely more than a hundred years. In the early days of Western civilization, the concept of creativity was much closer to the ones of the aforementioned ancient civilizations. Even though the concept of creativity might have originated in artistic creation, it is often not associated with human creation at all. For a long time, God was seen as the only possible creator, and every creation was seen as emanating from him. But what do the words ‘creation,’ ‘to create,’ and ‘creativity’ mean, and where do they come from? It is important to note that all these words are far more recent than all the notions just described. In fact, it is not before the nineteenth century that the term ‘creativity’ appears. Etymologically, the word goes back to the Latin term creare (to create, to beget, to choose). Before that, the Greeks used the word poiein (to make) and limited its usage uniquely to a description of the work of poets, who were seen as people who ‘bring to life a new world’ based on their own personal perspectives. Still, a poet’s creative ideas were believed to be directly determined by the gods through the mediation of muses. Creative poets were seen to have been chosen by the gods to realize their poems [34, p. 16]. Artists3 , such as painters, sculptors, or even musicians, were merely supposed to skillfully imitate earlier works while following strict laws and rules in order to reflect the perfection of nature [115, p. 244-5]. It was a common belief that nothing new was possible in art. Nevertheless, artists were seen as individuals and they were publicly honored for their achievements [34, p. 20]. Closely associated to creative work was the Greek concept of genius, which stood for a mystical fortune and giftedness on the verge of madness. This concept reappeared in the nineteenth century and still exerts a considerable influence today [2, p. 18]. The Romans saw poets and visual artists as equally entitled to create according to their own will. They even had two verbs for creation, creare and facere, as well as terms for creating (creatio) and creator [115, p. 246]. In Roman culture, we first find the notion of imagination and inspiration being important in connection with a created artwork. This came close to today’s notion of the concept, and it took many centuries after the fall of Rome until the Western world had regained a similar valuation of individuality. In the Middle Ages, under the influence of Christianity, the meaning of the noun creatio was narrowed and uniquely interpreted religiously. It came to refer only to God’s act of “creation from nothing” (creatio ex nihilo). Neither art nor poetry nor any other human creation was perceived as a product of 3

Poets were not counted as artists by the Greeks.

134

17 Historical Approaches

a creative act. As in Greek antiquity, an artist should imitate nature, and even poets were now limited to imitation and compliance with strict rules [115, p. 247]. Every work of art was supposed to be created in honor of God, and the individual artist had no significance at all [34, p. 22]. If an artist was unusually gifted, his talent was perceived as originating from an outside spirit [2, p. 18]. The age of Renaissance brought with it the notion of creativity in art as we understand it today, with innovation being a crucial aspect. Architects, painters, and musicians were seen to ‘invent what is not’ (Paolo Pino), ‘realize their vision’ (Michelangelo), or ‘shape a new world, a new paradise’ (Federico Zuccari). A composer was defined as ‘one who produces new songs’ (Johannes Tinctoris) [115, p. 248]. Artists again found personal acclaim and increasingly worked under patronage independent of the Church. In the seventeenth century, the word creare started to appear again in connection with poetry and later with other art. Artists were said to ‘create anew,’ ‘in the manner of God’ (Maciej Kazimierz Sarbiewsky), and art was seen as “the completion of nature, as if it were a second Creator: It completes nature, embellishes it, sometimes surpasses it” (Baltasar Graci´ an) [115, p. 248]. In the Age of Enlightenment, creativity slowly became a more fundamental part of art theory. Writers began to both make the distinction between genius and mere talent, and demystify the concept of supernatural genius. They maintained that there was a potential for genius and talent in everyone and that it was highly dependent on the political atmosphere and social context whether it could be exploited [2, p. 22]. William Duff considered genius to be based on a combination of imagination, judgement, and taste [34, p. 25], which could only occur in total social freedom [2, p. 22]. Talented artists merely “followed the path blazed by the geniuses” [34, p. 29] and were highly dependent on formal education, whereas genius seemed to come out of nowhere [2, p. 22]. Nevertheless, a significant amount of diligence was seen as necessary in order to benefit from genius or talent. There were also skeptics of the concept of creativity. French philosophers Diderot and Batteux claimed that the human mind cannot create, but merely recombine, magnify, or diminish what already exists. “Even monsters invented by an imagination unhampered by laws can only be composed of parts taken from nature” (Batteux) [115, p. 249]. In the nineteenth century, art was fully acknowledged as an act of human creation. The terms creative and creativity were finally introduced and quickly disseminated. They were frequently used, but still only in connection with art: The only humans able to create were artists. The meaning of creation shifted, especially with the emergence of new explanations for the workings and the evolution of the world, such as Charles Darwin’s. It became irrelevant that something was created “from nothing;” what was important was only that it was novel [115, p. 251-2]. The association of genius and madness reappeared, and it was widely acknowledged what emotional tests creative personalities had to go through [54, p. 30-1]. William S. Jevons, for instance, defined genius as being “essentially creative” and he also believed that creative thinking to con-

17.2 Creativity in Early Psychology

135

sist in “divergence from the ordinary grooves of thought and action,” which anticipated theories of divergence of more than half a century later [10]. On the edge of the twentieth century, the concept of creativity was finally generalized and began to be used for other domains than art, e.g. the sciences (Jan L  ukasiewicz) and nature (Henri Bergson) [115, p. 249]. The aspect of novelty is central to the concept to this day.

17.2 Creativity in Early Psychology With the emergence of experimental psychology in the late nineteenth century, it was no longer just the concept of creativity that was investigated and described, but the creative process. Psychologists and philosophers began to describe what precisely happens when someone is being creative. The beginning of the psychology of creativity was characterized by two main convictions about the nature of creativity, corresponding with the two major psychological movements at the time: the associationists and the Gestaltists. The associationists, under the lead of Sir Francis Galton, who was inspired by his half-cousin Charles Darwin’s groundbreaking theories, claimed that creativity consists of associating things or ideas in new and different ways. The associationists believed that the unconscious could be made conscious through association of thoughts. Galton described this process as follows: “Ideas in the conscious mind are linked to those in the unconscious mind by threads of similarity” [34, p. 28-9]. The Gestaltists, in turn, explained the creative process as seeing a Gestalt, a mental pattern or form4 , in a new way, i.e. from a different position or in a different situation [34, p. 30]. Max Wertheimer, for instance, a later Gestaltist, proposed that a new point of view on the whole of a problem is more effective than rearranging its parts. Such a change of point of view is often perceived as uncomfortable and stressful. For Wertheimer, this is the reason why people have a tendency to become rigid and stifle their creativity to avoid problems [34, p. 33]. In the early twentieth century, the creative process was associated with problem-solving, and there was a conviction that insight, the outcome of the process, is usually sudden and passive. Graham Wallas was one of the first to describe the creative process precisely. His model contains five stages [122]: • preparation: gathering of information, focusing on form and content • incubation: stopping to work on problem, maybe give up, letting information sink into unconscious mind; gathering more information proves counterproductive • intimation: feeling that an idea is on its way • illumination: having a sudden insight, perhaps an unnamed thought, not necessarily a clear solution; might lead back to more conscious work 4

The elements of a Gestalt form complex interrelationships that go beyond mere association, which makes the entire Gestalt greater than the sum of its parts.

136

•

17 Historical Approaches

verification: trying out and checking a solution

This five-stage model is the most influential characterization of the creative process, but it has turned out to be difficult to verify, for it is very unlikely that someone can be creative in an artificial research context. Another question discussed by early psychologists was about the origin of creative ability or genius. Sir Francis Galton dealt with it and he maintained that creativity and especially genius are inherited genetically. William James, in turn, suggested that a person’s environment and upbringing have the greatest influence on creativity [34, p. 32]. Many later psychologists followed James’s path and began to look for triggers and favorable aspects. The psychoanalytically oriented psychiatrists were convinced that creativity is highly connected with defense mechanisms against a state of frustration about an unfulfilled need (deficiency need). Sigmund Freud, for instance, claimed that creativity is established by a traumatic experience undergone in the first five years of life and then buried in the unconscious mind. He saw it primarily related to sublimation, which is artistic pursuit due to a lack of sex drive. Further notable psychoanalysts saw creativity being based on other kinds of defense mechanisms, e.g. regression of an adult to an ego-less, child-like state out of frustration (Ernst Kris), or compensation for feelings of inferiority in another area (Alfred Adler) [34, p. 36-9].5 The humanist psychologists opposed the view of Freud and his followers and proclaimed that creativity comes from being needs rather than deficiency needs. According to Abraham Maslow, when lower-level needs—i.e. physiological needs, safety, love, and esteem—are fulfilled, a person pursues her being needs. These include self-actualization needs, i.e fulfilling one’s personal potential, and aesthetic needs, i.e. understanding the world. The former are closely linked to creativity, and self-actualized people are prone to witness a peak experience, a state of unselfish ecstasy. Similarly, Erich Fromm claimed that creativity stems from the need to raise above one’s instinctive nature. Another humanist psychologist, Carl Rogers, listed the personality traits supportive of creativity as follows6 : • • •

openness to experience ability to evaluate situations by one’s own standard ability to experiment and engage with unstable situations

Rogers claimed that someone who possesses these traits is likely to be in excellent psychological health. Fromm proposed five additional but related necessary personality traits:

5

6

Only Otto Rank claimed that creativity is a realization of one’s will, an integration of one’s personality, and a reaction to a healthy environment, i.e. a family with supportive parents. These trays are also psychologically advantageous properties for the creativity process that we describe in chapter 19.

17.3 Creativity Research in Recent Years

• • • • •

137

capacity to be puzzled and surprised ability to concentrate objective knowledge of self ability to accept conflict and tension willingness to let go of security

All these theories and attitudes worked toward a clearer picture of what creativity is. Nevertheless, it was not until 1950 that creativity research was pursued in a systematic way with clearly defined experiments for verification.

17.3 Creativity Research in Recent Years Scientific study of creativity is often said to begin with the work of Joy Paul Guilford in 1950. He pointed out how neglected the topic of creativity was in psychological studies and initiated a rapid emancipation of systematic creativity research. His main idea was that creativity should be studied in everyday subjects instead of exceptionally creative individuals. Before him, several notable scholars had attempted to do systematic research, but with a strong focus on genius. Inspired by Galton’s and James’s empirical research, early twentieth-century psychologists such as Lewis Terman and Catharine Cox performed historiographic studies of individuals deemed geniuses by estimating their IQ based on their biographical and autobiographical data as well as their social contexts [2, p. 26-7]. A major consequence of these studies and later follow-ups was that developmental and family factors are more influential on creative abilities than intelligence and that at an IQ higher than 115, creativity and intelligence are more or less uncorrelated [2, p. 28]. Guilford proposed that there is no such thing as a general intelligence. He divided mental capacities into 120 factors, which could all be more or less distinct in individuals. Each factor is dependent on one or several of five operations: divergent production, convergent production, cognition, memory, and evaluation. Divergent production, which is an important part of today’s view on creative production, encompassed different contents (figural, symbolic, semantic, and behavioral) and products (units, classes, relations, systems, transformations, and implications). Guilford implemented psychometric tests to find products on the 120-factor matrix and concluded that the results originating in divergent production were by far the most significant [7, p. 14]. Based on Guilford’s work, E. Paul Torrance developed the famous Torrance Tests of Creative Thinking that rated multiple responses to problem-solving tasks according to fluency, flexibility, originality, and elaboration (detailedness). Typical tasks were asking as many questions as possible about a given drawing, or listing as many unusual uses for an everyday object, such as a cardboard box [112, p. 7]. In the 1950s and 1960s, research focused on the creative personality in a wave of a “new version of individualism” [2, p. 28]. Psychologists of the time believed that the best way to study creativity is to analyze individuals and

138

17 Historical Approaches

their motivations and goals as well as their internal cognitive representations. Traits similar to the ones listed by the humanist psychologists were found to be crucial for creativity. Later, the topics and subject backgrounds broadened and included research on creative types or styles and everyday creativity. In the late 1980s, researchers led by Mih´ aly Cs´ıkszentmih´alyi and Dean Keith Simonton began to incorporate social and contextual environments to assess where and how creative work could best be performed [104, p. 24]. With his systems view, Cs´ıkszentmih´ alyi claimed that whether or not something was perceived as creative work was highly dependent on and often determined by the context, or the field, in which it was produced. Inspired by this, scholars have started to investigate group creativity and interactive creative processes. They claim that “creativity cannot always be defined as a property of individuals” [104, p. 25] but as the property of an complex social network where members each contribute their parts. Other contemporary scholars have focused on the cognitive process underlying creativity and created models and even computer simulations of it. The Geneplore model, for instance, instead of confirming Wallas’s five-step model, assumes that there are two phases, the generative and the exploratory phase. In the former phase, the subject creates mental representations, which are then processed in the latter phase (via retrieval, association, synthesis, transformation, analogical transfer, and categorical reduction) [42]. The BACON software, to cite a further example, was designed to rediscover scientific laws through heuristics; it found Kepler’s third law of planetary motion from simple numeric astronomical data [65]. More achievements in the research of the cognitive process will be presented in the next section. There are many other directions and theories in the study of creativity, and we will not be able to give a full overview. The universal applicability of the word ‘creativity’ today inspires many scholars to do interdisciplinary studies and adopt ideas from other domains. Sternberg and Lubart, for example, draw from economics and say that creative individuals are willing to buy low (invest in an unpopular idea) and sell high (earn respect for the idea) [62, p. 30]. Simonton created a model for creativity that is inspired by evolutionary biology. To his mind, ideas are created in the subconscious mind and then purposefully retained by the conscious mind if they prove viable [62, p. 37]. This can be generalized so that society as a whole would be seen as a filter for viable ideas. Creativity would then be the force that quasi-randomly produces ideas that are integrated into culture if they prove viable. It would be the analogous force to genetic mutation in Darwin’s natural selection. This idea is, for instance, brought up by Cs´ıkszentmih´alyi, who refers to Richard Dawkins’s concept of memes, the informational units of selection in cultural evolution [33, p. 7]. Creativity would thus take the function of a generator of new memes. Yet, many scholars today have trouble accepting the assumption that generated ideas are principally random. There are, in fact, good reasons for such a critical attitude. First of all, reduction of creativity to randomness does not answer the problem of what creativity is, it just denies it in a killer argument and delegates it to

17.3 Creativity Research in Recent Years

139

the scientific concept of randomness, which is nothing but a nice rephrasing of ignorance. Second, creativity is a highly spiritual achievement of intelligent beings, which is never realized by random mutation, otherwise Picasso would not have created one masterpiece after the other. In contrast, Darwinian mutation is purely random without any spiritual dimension. Creativity is not a casino affair. It is remarkable that creativity was a divine process in ancient times, and now in certain theories, creational divinity has been replaced by randomness (like Darwinism replaced divine creation in biology). In their summary of theories of creativity, Kozbelt et al. bring to attention that it is not divergent thinking alone that counts for creativity but also a certain amount of moderation that ensures that the resulting ideas are not irrelevant to the problem [62, p. 20]. In Guilford’s terms, convergent thinking is equally important for creative results as divergent thinking is [62, p. 32]. This brings us back to the definition cited at the beginning of this chapter, where appropriateness was a requirement for something to be considered creative. In the context of this book, we think it is suitable to stick with this definition. The criterion of appropriateness is met in our own approach in sections 19.1.3, 19.1.4 when we search for the critical concept and then identify its walls in order to open new perspectives of creative extensions.

18 Present Approaches

18.1 The Creative Process Today Since we are proposing strategies and tools for being creative, it is necessary to briefly focus on contemporary theories that deal with describing the creative process. Before giving an overview of such theories, we will come back to the questions asked in the introduction to this section and attempt to give presentday answers to them that also reflect our view presented in this book. 18.1.1 The Four P’s of Creativity Instances of creativity are traditionally described using the four P’s of creativity: process, product, person, and place.1 The first P, process, is the most important in connection with this book and will be described in the next section. The second, product, deals with the outcome of the creative process, which can be just about anything: the conception of a car, a mathematical proof, a practical trick, a sculpture, an unexpected joke, or in our case, a musical composition or improvisation. We have seen in the historical overview that today, creativity is applied in all conceivable fields and thus everything that is both novel and appropriate can be seen as the product of a creative process. Person, the third P, and especially personality, as depicted earlier, was the main subject of most research and theory in the first two-thirds of the twentieth century. It is evident that there are differences in creative capacity from individual to individual, but in this book we assume that anyone can potentially be and often is creative. It merely depends on the context in which the creative activity takes place as well as the context from which the creative product is evaluated. If a person finds an alternative way to open a door if the key does not work, or if an unsuccessful deer hunter finds a way to get sated eating berries, we could consider the respective solution to be creative in the 1

Some recent publications presented extensions of these and added persuasion and potential [62, p. 24], which will not be treated here.

G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 18, © Springer-Verlag Berlin Heidelberg 2011

141

142

18 Present Approaches

subject’s context. In our case, we think that a musician with any background can be creative, at least in his/her own context. Musical works are, however, often conceived to be valued in a greater context so that the musician has to know the context very well to be successful. The fourth P, place, deals with the environment in which a person can best be creative. We have already touched on this question in the theories of Galton, James, Freud, and Maslow, for instance, who were convinced that a person’s environment, such as her upbringing and family or her state of wealth, had the greatest influence on her creative potential. Generally, creativity seems to flourish when “there are opportunities for exploration and independent work, and when originality is supported and valued” [62, p. 25]. In musical production, there are often strict external constraints, such as a movie that the music has to fit, a certain theme or performance context for which it is commissioned, or simple time constraints. However, originality and experimentation is almost always valued to a certain degree. Studies in place and process are often very closely connected, and we will thus come back to it in our discussion of the first P, process. 18.1.2 The Creative Process Theories that deal with the creative process are bound to understand the mental mechanisms that occur when a person is being creative. We have already discussed Wallas’s five-stage model (see section 17.2), which is still one of the most popular today. However, many scholars have criticized its linearity and have allowed for recursion, where each of the stages can be reached multiple times, in different orders, and on different levels of abstraction [62, p. 31]. Some theories, instead of proposing a simple stage model, suggest component mechanisms, where prior knowledge and information as well as motivation play an important role in guiding the creative process. Mark A. Runco and Ivonne Chand, for instance, suggest a two-tier model (shown in Figure 18.1) in which the first tier represents a set of three basic skills, the controlling factors involved in the creative process: problem finding (identification, definition, etc.), ideation (ideational fluency, originality, and flexibility), and evaluation (valuation and critical evaluation). A second tier shows two additional contributing factors, knowledge and motivation, which are highly dependent on the primary factors. Motivation is, for instance, both dependent on ideation and evaluation [101]. One of the components of this model makes it significantly different from Wallas’s: the problem-finding component, which replaces the preparation stage. Many contemporary theories of creativity put a problem at the center of the creative process. Such a problem can be well-defined, such as a specific puzzle or winning a game with specific rules, or ill-defined (admitting many ‘good’ solutions), such as painting an aesthetically pleasing picture or writing an avant-garde string quartet. Now who defines such a problem or where does it come from? Among the theories that answer these questions, there are two

18.1 The Creative Process Today

143

Fig. 18.1. Runco and Chand’s component model of the creative process.

major categories opposing each other. The first one, problem solving, argues that in a creative process a problem is usually constructed by the creator, and if it is ill-defined, it can often be deconstructed into well-defined parts [62, p. 33]. Scholars proclaiming this view usually see creativity as a rational phenomenon and claim that both constructing a problem and deconstructing it need a significant level of expertise. John R. Hayes, for instance, found that great composers typically need to be trained for at least ten years before they create their first masterpiece [50], a rule that applies to creative achievements in many fields. However, expertise is only one of the factors that influence creativity and hardly ensures it. The advocates of the problem-finding view criticize exactly this and argue that the problem-solving view does not explain how creative persons really construct their problems. Cs´ıkszentmih´ alyi and Getzels, for instance, observed how more-creative individuals spent more time with exploratory behavior lacking clearly defined goals, which, however, typically lead to more success in their activities. The problem-finding view is more oriented toward describing the subjective experience of the creative individual and manages to explain instances of creativity independent of expertise. However, some scholars refute the problem-finding view by arguing that the subjects’s exploratory behavior might well be described with standard subconscious problem-solving routines. Cognitive psychology has contributed a lot to theories concerned with the creative process. There, scholars often deal with one of the stages or components and go slightly more in depth with the mechanisms. Many studies from the past decade have shown that conceptual combination, associating disparate sets of information, play a great role in ideation. Such sets are often only com-

144

18 Present Approaches

binable on a highly abstract level and only intelligible in a so-called metaphoric logical rather than literal way [62, p. 32]. Galenson suggests that in art there are two different types of creative personalities, the creative processes of which differ significantly: the “seeker” and the “finder” [45]. The former typically struggle and experiment guided by a mostly aesthetic motivation and no clear goals, and develop rather steadily. Finders, in turn, make detailed concepts and clearly define their goals before starting to work and are more prone to making radical conceptual innovations throughout their lifetime. Systems theories, such as Cs´ıkszentmih´alyi’s, describe the creative individual embedded in a large and complex social system. Seen from this perspective, the individual creative process loses its significance and is extended to a process where several individuals and groups interact. Evaluation, for instance, is taken over by a group of experts, and problem finding can be an indeterminate interaction between several creators. In other words, the main creative process does not merely take place on the level of a person, but also on the level of the place (the environment). Cs´ıkszentmih´ alyi does not ask “What is creativity?” but “Where is creativity?” [31, 33]. He sees creative ideas as the product of a system with three components: a domain or culture with symbolic rules, a person that brings new ideas into a symbolic domain, and a field of experts that validate these ideas. None of the components can be missing, even though there might be variations. The field of experts, for instance, has a different meaning in everyday creativity than in eminent, so-called big C, creativity, the acts of which are significant to all mankind. Anyone might be able to understand the originality of an everyday joke and would thus be part of the field of experts, while it takes a lot of expertise to understand the inventiveness of a symphonic work. Another major contribution of Cs´ıkszentmih´alyi’s to our notion of the creative process is his concept of flow. Flow is the state of mind in which a person is equally challenged and successful when performing a specific task. The person loses both his/her sense of time and self-consciousness and is completely immersed in his/her activity, experiencing pure enjoyment [30]. Cs´ıkszentmih´alyi discusses each of the factors necessary to reach the state of flow during creative activity, most notably the clarity of one’s goal, a good amount of self-judgment, the urge to face challenges, and the ability to avoid distraction [33, p. 113ff]. It turns out that these are the same factors that generally describe creative personalities now and in the long term. Generally, flow seems to accompany the creative process, and the more creative a person, is the more she experiences it in her activities. Finally, Margaret Boden, who has ideas very similar to the ones of the authors of this book, finds three different types of creative processes, which she bases on the three ways in which one can be surprised [12, p. 3]. The first type, combinational, is closely related to the associationist view and comprises unfamiliar combinations of familiar ideas. The exploratory type consists of the exploration of a conceptual space, i.e. a certain thinking style such as a theory,

18.2 Musical Creativity

145

a fashion style, or a musical genre. This exploration results in a new idea within the same space, which can nevertheless be of an unexpected nature. The third type, transformational, means the extension of a conceptual space to include a previously impossible concept or idea [13, p. 72].2

18.2 Musical Creativity For the final part of our short investigation of previous work on creativity, we now turn to musical creativity. There is hardly a domain where achievements are mystified as strongly as in music. This attitude makes the fields of music composition, improvisation, performance, and even listening distinctly hard to study. Composers who refuse to speak about the making of their works, performers who enact an artificial personality, and listeners who merely pretend to understand are omnipresent. The study of musical creativity suffers under these circumstances and is rather underrepresented in creativity studies.3 Nevertheless, many of the findings from general creativity studies and theories can be applied to the domain of music. In this section, we will give a short overview of studies specifically dealing with musical creativity. First of all, what do we mean by “musical creativity”? Generally nothing more than being creative in one of the various fields and activities in music. Or reconsidering Sternberg and Lubart’s definition of creativity given above, it is the ability to produce work that is both novel and appropriate in the domain of music. Again, this work can be a composition or improvisation; an interpretation, cover version, or remix; a recording or production, an instrument with a certain conception of sound, or even a way of listening to a specific work. In our book, we typically deal with creativity in the fields of composition and improvisation, but we also provide examples from other fields. In Western music, such a work or product indeed has to be novel in a certain way in order to be valued by an audience, and it is straightforward what this means. It should incorporate unheard melodies or old ones in an unheard context or realization, a new conception of sound, an innovative form, new catchy rhythms, etc. But what does it mean for a musical work to be appropriate? Bj¨ orn Merker explains this as the necessity of a “command of craft and grounding in a musical tradition” [90, p. 25]. Similarly, Simonton found in his research that compositions that are generally perceived as creative do depart from prevailing norms, but not as much as innovative compositions that are not widely acclaimed do [109]. Even though creativity itself was not of much interest until recently, as seen in the previous sections, the domain of music was connected to it from the beginnings of theoretical investigations in creativity. In 1931, for instance, Julius Bahle wrote a letter to thirty-two composers, including Luigi Dallapic2 3

This is analogous to the “opening of walls” in our model. The only area where musical creativity is a crucial topic is music education, which is not subject to this chapter.

146

18 Present Approaches

cola, Ernst Krenek, Arthur Honegger, and Arnold Schoenberg, with the intention to find out more about how they approach the process of composing a new work [8]. His evaluation of their answers was based on the work of the German psychoanalysts, especially Freud and Kris. He assessed how much the composers’ choice of material, their productive experience, their affective and emotional life, and their obligation to work affected the composition process. Especially worth mentioning is a question about how easily dreams could influence musical composition, again inspired by Freud. Many, especially new music composers, answered that a lot of their inspiration would come from dreams. Much like Bahle’s work based on psychoanalysis, every direction of creativity research has had its application in musical creativity. Carl E. Seashore’s investigations, for instance, are closely connected to the historiographic studies of Terman and Cox. He investigated biographies, diaries, and letters of famous and prodigious composers to find out how they proceeded in music composition. For example, he found out that Mozart used to actively retain musical ideas he liked, which then more or less automatically transformed into a whole composition [105]. Of course, this creates more questions than it answers and it merely describes Mozart’s subjective experience of his activity, which appears to be of a highly passive nature. The majority of the compositional process seems to take place subconsciously. Many exceptional composers describe it this way. Paul Hindemith, for instance, said that the imagination of musical ideas, i.e. a short melody, is the basis of the process of composition. The remainder is fulfilled by the ability to clearly envision a finished composition. In Hindemith’s opinion, this ability is what makes the ‘real’ composer, and the highest goal for a composer is the convergence of this vision and the final materialization [99, p. 42-3]. Karlheinz Stockhausen describes his compositional activity as a conscious prolongation of such flash-like visionary moments [99, p. 56].4 In their studies, both Bahle and Seashore clearly focus on the creative process. This applies to most research in musical creativity. The general type of question is almost always: How do composers compose, or how do performers perform? As the examples show, such questions are exceptionally difficult to answer, especially when looking at the musician’s subjective experience. Interest in the process of music creation is constantly growing, especially in connection with specialized theories of creativity. David Cope, for instance, presents computational models of the creative process. His software Emmy emulates the styles of composers such as Beethoven, Mozart, Mahler, and Prokofiev by recombining fragments of original compositions saved in a database [28]. Cope does not consider its results as original, but nevertheless as creative, since they originate from a creative process [29, p. 33]. Now, the creative process of Emmy cannot, however, be regarded as actually creative, since it merely applies the compositional rules provided by Cope [53]. As is often the case in computer 4

Nicholas Cook creates an interesting connection between these flash-like visions of genius composers and Schenkerian analysis in [26].

18.2 Musical Creativity

147

modeling of creativity, there seems to be a confusion between processes that create something new and real creative processes in our sense, where the result is essentially creative. Starting in the mid-twentieth century, scholars increasingly dealt with the creative personality question as well as the psychometric assessment of musical creativity, again influenced by major trends in creativity research. Some applied the findings of Guilford and Torrance and designed psychometric tests, such as the Musical Aptitude Test [36] or the Seashore Test of Musical Ability [20]. Others published detailed studies describing the personality traits of gifted musicians, such as [46] or [110], which do not differ significantly from the ones described in Section 17.2. A distinct case is Sawyer’s theory of group creativity. There, person is not a single person, but a group of interacting musicians. In an improvised jazz performance, for instance, for Sawyer “the whole is greater than the sum of the parts” [104, p. 73]. Sawyer notes that the only product of an improvisation is the creative process itself. He names this kind of creativity improvisational creativity as opposed to compositional creativity, where there is a product distinct from the process [103]. Furthermore, Sawyer says that improvisation is generally based on problem finding rather than problem solving [104, p. 73]. One of the most important questions when looking at a specific domain of creativity, such as music, is in what respect the product differs from products in other domains. The fact that music is a performing art makes it significantly different from many other domains. Musicians create in real time in front of an audience. The musical product can range from a faithful rendering of a work, to an embellished interpretation, to an entirely unplanned free improvisation. Scholars have analyzed and compared such products, e.g. alternate takes of jazz solos, and come to the conclusion that even improvisation is to a large degree built on “smoothly and innovatively combining or sequencing phrases or motifs from memory while conforming to structural constraints” [90, p. 27]. Charlie Parker, for instance, is regarded as a formulaic improviser, which means that he acquires countless patterns and formulas for different harmonic situations and applies them spontaneously. At the speed at which Parker played, it would have been impossible to compose new music from scratch [94].5 For performative musicians, Merker proposed four different kinds of products differing in novelty and preparation, as shown in Figure 18.2. When asking for the role of place in musical creativity, we come back to Hayes, who coined the ten-year window of activity of composers until they write their first masterpiece (see section 18.1.2). Hayes, for example, pointed out that Mozart’s first seven piano concertos did not contain any original music, but consisted of gathered and rearranged material from works of other composers [50]. This is not as strictly the case for early works in other categories, but the composer still closely followed the models of earlier composers. 5

The concept of formulaic improvisation was used in improvising software, e.g. in [59].

148

18 Present Approaches

Fig. 18.2. Merker’s space of musical performance products.

Mozart’s first ten years of musical experience thus consisted of immersion in the works of others by means of composition and performance. This is what Howard Gardner calls “creative mastery,” one of his four forms of extraordinariness (or creative intelligence6 ) [46]. Ericsson et al. come to the same conclusion for expert performance in classical music [40, p. 233]. The same ten-year rule also applies to jazz performers, as Robert W. Weisberg shows. Even the greatest improvising musicians say they have learned from older musicians by playing back their solos from records note for note [123, p. 236-7]. And it applies to musicians and songwriters in popular music, such as The Beatles. In the band’s early history (1957-62), 90 percent of the songs that entered its repertoire were cover versions [123, p. 239]7 . We should however note that the ten-year rule is nothing else but the necessary condition to get acquainted with a certain cultural context of the art that is at stake. If you want to write a Chinese poem, you first have to learn Chinese. Every higher culture needs this type of basic education. But this is far from describing sufficient conditions for creativity. In fact, a considerable number of those who have the necessary education in a determined field of knowledge or art never produce creative works. With respect to the ten-year rule, we refer to our discussion of a creative pedagogy (chapter 20). It is a pedagogy where the student’s creative attitude is a central concern, while traditional education often casts students into a rigid framework that prevents creativity. 6

7

“Kreative Intelligenz,” as adapted in the German translation of the book Extraordinary Minds. Another question in connection with place is the importance of a supportive family. Several studies found it to be extremely rare that a composer reaches high standards without early support and encouragement [56, p. 435].

19 Our Approach

This chapter presents two versions of our approach to creativity. First (section 19.1) we discuss this approach from a semiotic point of view. This is also the more accessible presentation since it does not rely on mathematical formalism. It is however connected to the second version (section 19.2) which relies on category theory. This means that both versions have corresponding components, but the mathematical one shows them as processes that are formalized in terms of functors.

19.1 Approach to Creativity: A Semiotic Presentation We shall now discuss the compound creativity process. At the end of this chapter, section 19.2, we shall present a processual list summarizing all characteristic steps in this process. 19.1.1 The Open Question’s Context in Creativity Our approach to creativity starts with a context, where an open question to be answered takes place. This means that creativity is not an absolute action but must be related to a specific context. For example, if a monkey discovers a new way to grasp a banana in a difficult place, this is creative for that animal, but very probably not for humans. Or if somebody reinvents something that has already been invented, but without knowledge of this inventor, then this is creative in this person’s “context.” We need to specify such contexts before identifying creativity. We propose that such a context be a semiotic, i.e. a system of signs with their syntactics, semantics, and pragmatics. This approach guarantees that the creation adds meaning to a given context—it is not only a new abstract element that is created, but it means something new in that system. Let us make this clear on the creation of the Appassionata Sonata, op. 57, by Beethoven. Mathematically speaking, or, more precisely, in the language of deG. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 19, © Springer-Verlag Berlin Heidelberg 2011

149

150

19 Our Approach

notators and forms1 , all sonatas in a given notational score system are simply points (elements) in a big powerset of denotators. So in this mathematical system, a sonata is nothing new: All sonatas are already given in that abstract mathematical realm. Evidently, this is not the argument that one would accept when looking at this sonata as a musical creation. What is creative here is not the identification of op. 57 as a point in a mathematical space, but rather it is the added meaning to the understanding of what is a sonata, among others, that counts here. It is also the historical moment of this creation, that counts. It refers to the state of the music as a semiotic system in that very moment (1804-1806) of the first Viennese period, where it changed a number of insights in to what music-theoretical structure, process, and (above all) gesture could be. In particular, this contextual setup is not related to finiteness or combinatorial conditions. Even if the total range of possible sonatas within a given score notation system is finite, the semiotic extension of the sonata concept is the relevant criterion for creativity here, i.e. creativity requires the introduction of a new sign to be added to a given semiotic. However, it is not characterized by this condition only; the extension of a semiotic is a necessary but not sufficient condition for creativity. We now add the following specifications. 19.1.2 Motivation for a Semiotic Extension The second specification of the concept of creativity is that the added sign is introduced with a specific purpose, not just for any old reason. It matters why this new sign is added. And it matters how this is accomplished. Let us first focus on the reason why a sign is added. Our approach to this criterion is that the new sign should provide the solution to a given open question or problem2 that has been identified within that given semiotical context introduced above in section 19.1.1. Let us look at a typical example of this situation. Albert Einstein, when dealing with the special relativity theory, could answer open questions of physics at the beginning of the twentieth century by the extension of the concept of time. The essentially singular Newtonian time was extended to a multiplicity of times in inertia systems that are related to one another by the Lorentz transformation. The creation of Einstein’s new time concept was an extension of the semiotics of physics by a new sign: the multiplicity of times in inertia systems, together with their Lorentz relation. It might seem too restrictive to impose the answering of an open question upon the semiotic extension in creativity. In the typical context of musical composition or improvisation, why should there be any question to be answered, when acting in a creative way? Isn’t this a condition that hampers artistic freedom in creativity? If it is understood that this freedom means that anything 1 2

See section 21.3.6 for these concepts. Stravinsky [113] conceived musical creativity as a type of problem solving.

19.1 Approach to Creativity: A Semiotic Presentation

151

goes, then yes, it hampers it, and rightly so. In fact, artistic creation should occur with a consciousness of a certain necessity to conceive and realize this work of art. If freedom then means that the artist comes to an insight of such a necessity, we might agree. Although one cannot prove such a position, it is however a strong criterion for quality in artistic creation. Evidently, this point of view is quite standard in scientific contexts, but we contend that it is equally advantageous to apply it to the artistic creativity. In the theory of group flow as conceived by Keith Sawyer [104], flow is defined as an equilibrium between two components: shared structures and specific goals. Sawyer describes free improvisation in music as being in flow if there are few shared structures and there is only an intrinsic goal, i.e. no extrinsic goal to be achieved as it may happen in a group of computer programmers that has to solve a specific problem. Sawyer then comments that in free improvisation the first half of performance time might be spent inventing a problem that may be solved in the second half. Although this setup seems to relate to our approach of starting from an open question in a creative activity, it is fundamentally different. Let us explain this. To begin with, in our approach to creativity, one does not have to invent a problem to be solved, because there is already a given open question. So at least from the creativity point of view, we are only dealing with this action when Sawyer’s problem has been introduced. And we do not agree with Sawyer that this invention of a problem is what group flow in free improvisation achieves, but see [85] for details. The most important difference in Sawyer’s approach is his idea that half of the performance time would be driven not by any creativity but just finding a problem. Such music would be bare of any aesthetical quality. In reality, the open questions that we address here are present from the beginning and need not be invented or found. If Coltrane gets off the ground with “My Favorite Things,” the first task is to unfold the main theme into a manifold of variations. The question then is to conceive the main theme as a critical concept/sign of Coltrane’s context, and to open its walls to unfold its potential in rhythm, harmony, and motivic neighborhoods. One could also say that the basic question is to explore the initial data (theme, rhythm, whatever is thrown into the staging dynamics) with respect to their walls in the sense of what has not been played yet in the neighborhood of this initial data. So there is no problem to be invented; the open question is given from the very beginning. 19.1.3 The Critical Sign Once we are given the semiotic context (section 19.1.1) and have identified the open question backing the creative action (section 19.1.2), we may proceed to the first operational action upon this context, namely identifying the concept(s) that is (are) responsible for the present open question. This is a sign in our semiotics, but it may not be a singular one. It might be that there is a series of such critical concepts (signs). In this sense we just select a first critical concept

152

19 Our Approach

to start the creative process. Let us make an example. In the discovery of the three Kepler laws (first law: planet orbits are ellipses, the sun being in one of their foci; second law: the surface covered by the connecting line from the sun to a planet in a time unit is constant for that planet; third law: the orbit time t and the semi-major axis a of a planetary ellipsis define a number t2 /a3 which is constant for all planets), the open question Kepler was confronted with was the failure to model planet trajectories as perfect circles, i.e. having constant distance from the sun. Here, the critical concept is the distance of a planet to the sun. Once this concept is questioned, it must be replaced by some other “distance” concept. But even then (supposing this problem has been solved), the shape of the trajectory—that now is no longer a circle—can be quite arbitrary. The second law replaces the geometric distance with the surface covered per time unit, i.e. with a more abstract “distance.” The first law then replaces the concept of a perfect circle with the more general elliptic trajectory shape. 19.1.4 Identifying a Concept’s Walls To this point, the procedure was not very specific. This is what one would always do to answer an open question: Locate it in a reasonable context and identify the critical concepts. We now present the step that we believe is more specific for creativity. It deals with the question of how the critical concept may be investigated. It is assumed that the concept is not working well, and that it should be—in some way—replaced by another concept. A concept is defined by some of its components and by the way, it refers to these components. For example, in Einstein’s time problem, the time concept is that of a real number, plus some other properties. Or for Kepler’s laws, the distance is a geometric length in an Euclidean space. So it seems obvious what the concept’s anatomy is. In the case of Einstein’s problem, one could question the properties given by a real number, or in Kepler’s situation, one could question the Euclidean distance and try to replace it with another metrical distance construction. In fact, in the case of time, the astrophysicist Stephen Hawking has questioned the real number quality of time and replaced it with a complex number to get rid of mathematical singularities for the cosmic Big Bang 14 billion years ago. But the problem of those properties is that they might be quite hidden to the normal understanding. We therefore call such subtle properties that one might easily overlook the concept’s walls. They are properties that are so evident that one would not even become aware of them in the first naive approach. Let us look how this happens in Einstein’s problem. In Einstein’s situation, time was the Newtonian time that, except for trivial gauging and origin transformations, was one and the same for every point of the universe. It was just the divine time that was shared by all beings of this world.

19.1 Approach to Creativity: A Semiotic Presentation

153

In the description of time, there is one property that one would not even think of in the first moment: the fact that time is a singular concept. There are no two times, at least not besides the above trivial gauging and origin transformations. So what is critical here? The point is that the singularity of time is a very deep assumption: We all have the one and only time. Of course, one would answer, I also have only one weight, only one shoe size. It would be absurd to assume I have more than one weight. And this is exactly the critical point: The wall character of singularity of time is that it is identified with the time concept in such a way that putting it into question would look like putting into question the very essence of the concept. Let us look at the example of the Post-It adhesive by 3M’s Spencer Silver in 1968. He was in search of a super adhesive and never succeeded, but instead synthesized an adhesive that did not glue but could be removed easily. In 1974, a friend of Silver, Art Fry, discovered the advantage of having such a weak adhesive when adding small notes to musical scores in church. Here the critical concept is “adhesive.” It seems mandatory to have the characteristic attribute of the concept, which is “glueing.” That this might be a wall seems impossible. However, it is exactly this characteristic that turned out to be a wall against the introduction of a new class of adhesives! Here the opening of the wall meant that we question the “glueing” property. This is a binary property: either it glues or it doesn’t. But this binary logic turned out to be too restrictive in Silver’s situation. Could glueing be extended to a broader spectrum of logical possibilities? Why not think of the property in terms of a fuzzy logic: glueing by x percent? In fact, this extension solved the open question and opened the success story behind 3M’s Post-It. Before we investigate singularity of time, we should stress the perhaps deepest problem in the creative process: The identification of walls, supposing that they are not even given in any explicit definition and understanding of the critical concept(s). We may have to invent such a wall property before investigating its possible role in the given open question. It is as if we were looking at the concept of a number and all of a sudden had to think about a number’s color. Of course, the chance to invent an absurd/useless wall property is very high. So the real art would consist in finding walls that are hidden at first sight, but open up an essential new perspective on a given concept. This art could be misunderstood as a combinatorial task of just going through a list of possible properties and checking their chance of being a wall to the given critical concept. However, it would not help to go through any such list since its size would be far too large to answer the open question. But the approach of using a list is also not what the solution could offer. It is not about formal properties, but about semantics: What is the signification of a determined property for the given concept and its problematic position. For the time being, we cannot make this process more explicit, but want to have it named and memorized. Let us look at the property “time in singular.” What could one question here? The straightforward action would be the logical one: to negate the wall

154

19 Our Approach

property. So the negation of “time in singular” would be either “no time at all,” “time in plural.” We would have to ask whether it makes sense to have not one but several times. Time would then be a plural concept. But how might this be? Would we have a masculine and a feminine time? Or ten times, or none, or what? We learn here that it is not easy to act on an identified wall. Brutally breaking down the wall of a singular time is not the answer of the open question since the logical method does not take care of the specific ‘anatomy’ of the critical concept. So let us look at this more carefully. 19.1.5 Opening a Wall and Displaying Its New Perspectives After the first properly creative action of identifying a concept’s wall property, the second creative action consists of creating the open space that can be conceived once the wall is broken. Logically speaking, the breakdown of a wall simply reads as its negation: In Einstein’s situation, it reads like “there are not only one but several times” (even none is possible. . . ). This is of no use so far. The problem is not the brute negation but an extension, which ideally includes the old situation (which was after all not totally useless!). And here lies a further subtlety of creation, since the negation must be refined to a constructive extension of a conceptual landscape. It is the same situation as with Kepler’s second law: If the distance is not the right one, what kind of distance do we need? It is not about negating the distance concept all together, but about extending it in a constructive way. Perhaps could one think of another kind of “distance.” Not the usual one, but one that is of a different nature yet is still some kind of distance. In the Keplerian situation, it turns out that, yes, the distance in terms of the surface covered by the vector from the sun to a planet in a time unit is the wall’s extension. It generalizes the same quantity as the previously conceived circle-shaped trajectory of planets moving at constant velocity. It unites the connecting vector from sun to planet and the velocity the planet has when displacing that vector. Let us look at another example taken from musical creativity. In his first years as a jazz pianist, Cecil Taylor, now recognized as the leading pioneer in free jazz piano music, used to perform in his individual way. His thoroughly percussive attack of keys shocked jazz and classically oriented experts. They agreed that Taylor was obviously incapable of playing the piano in an acceptable way. This acceptable way was defined by the great tradition of piano music, spanning from Mozart’s “oily” fingering realizing the cantabile paradigm, to Liszt’s style that was described by his pupil Valerie Boissier as: “M. Liszt’s playing contains abandonment, a liberated feeling, but even when it becomes impetuous and energetic in his fortissimo, it is still without harshness and dryness. (. . . ) He draws from the piano tones that are purer, mellower and stronger than anyone has been able to do; his touch has an indescribable charm.” From this normative perspective, Taylor’s approach to the piano was unacceptable. You cannot play the piano in this way. In classical music education, it is still believed that there is an ideal way of playing the piano aesthetically.

19.1 Approach to Creativity: A Semiotic Presentation

155

The critical concept here is “playing the piano.” And its wall could be called “cantabile paradigm.” Its negation by Taylor is, of course, not of any use if we simply view it as a logical negation since anything could work in the generic negation. The wall can only be opened if we consider the “cantabile paradigm” as a variant of a multitude of paradigms applied to the way of attacking the piano keys. In this multitude, the percussive paradigm, which opposes the cantabile norm, answers the question raised by Taylor’s piano playing. It is not about connecting discrete pitches to a fluid melody, but about shaping onsets as points in a rhythmical space, where pitch reduces to a secondary parameter. This change of perspective might have been the most difficult one in the given wall configuration: European music tradition had defined pitch as being the most important singular parameter of sound, whereas onset was only used to organize the order of appearance of pitch. Onset was just a logical parameter, something like a tick to position pitch for a melodic contour. This change of perspective is a very powerful technique of wall opening. Let us state its backing principle in intuitive words here. We shall come back to it in detail in chapter 24 under the title of the “Escher Theorem.” What we are dealing with here is a kind of hierarchy of compound attributes. In our case, it is the hierarchy of primary versus secondary sound parameters. Pitch is primary, onset or loudness are secondary. This means that when dealing with a sound event, it is a pitch that has onset, loudness, duration, and possibly also a sound color. But it is above all a pitch. The Escher Theorem deals with hypergestures, which are, in a very intuitive description, “gestures of gestures.” The Escher Theorem then allows us to permute the order of gestural components in hypergestures. So, for example, if the hypergesture is a circle of line movements, the permuted hypergesture would be a line of circle movements. So the circle perspective “circle of. . . ” would be replaced by the line perspective “line of. . . ” This change can also be applied to sound parameters. A “pitch with onset. . . ” can be viewed as an “onset with pitch. . . ” So the Escher Theorem enables us to change the perspective of a compound attribute. This is much finer than the simple logical negation because it takes care of the given anatomy of conceptual walls. It does not destroy them all together, but handles them in a different way. 19.1.6 Visual Representation of the Wall Paradigm The idea of conceptual walls as described above uses the wall concept in a purely abstract way. Although we spoke about opening a wall, for example, there has been no substantial use of this metaphor qua metaphor. When applying the wall paradigm, we should try also to offer a visual representation of these ideas that might help manage such abstract concepts in a manner that enables objects to represent concepts in a reliable way. The delicate point here is that the context, where concepts and their walls live, are quite general semiotics, where any kind of connotational, metatheoretic, and motivational imbrication (the Baboushka principle) might apply.

156

19 Our Approach

Such a complex configuration does not allow for simple, say 3D, spatial representation of concepts.

Fig. 19.1. Walls that limit rooms can be used for visual representation of creativity process steps. The semantics of a concept may be represented by stepping from one room to a neighboring room.

We nevertheless suggest a simple visualization of concepts, namely as rooms within a complex building. The building might not be representable, but a single room representing a concept might help (see Figure 19.1). In this visualization, a wall might be represented by a wall of that room, and the wall’s opening might be visualized as a door in that wall. The advantage here is that opening a wall might lead you into another room, so that the semantics of a concept will look like navigation through a configuration of connected rooms. It is also advantageous to think of such conceptual configurations as spatial ones and to imagine the elimination of a wall as an opening of that room to new spatial perspectives defining larger rooms. This visualization strategy also fits into the description of concepts in the denotator formalism since denotators have components that are also denotators and could be visualized by walls pointing to denotator components as adjacent rooms. This room visualization is, however, not representative of the underlying semiotic structures, because a room does not decompose into a sign-theoretical three-component object with expression, signification, and content. The only representation of such semiotic linkage can be realized when stepping from a room into an adjacent room that represents a property of the former room and thereby helps access hidden contents (or semantics). The room metaphor is mainly efficient when thinking of concepts as limited spatial configurations whose limits are given by defining properties, namely the room walls and what lies on the other side of such walls. So the visualization strongly suggests that a concept is limited as a space of thoughts and that those limitations could be questioned and possibly eliminated.

19.2 Approach to Creativity: A Mathematical Model

157

19.1.7 Evaluating the Extended Walls Once we have constructed an extended wall concept, the creative procedure is clear: The extended wall is applied to the given open question. If the question is answered, we are done; otherwise we go back to the wall search stage described in section 19.1.4 and start over. In the successful case, the given semiotic context has been extended and is ready for a further creative investigation or practical application. Despite having technically achieved closure, one should always stop the creative cycle at this point and reflect on the deeper lesson about why this extension was successful and how one might apply the insights to a future question that asks for creative solutions.

19.2 Approach to Creativity: A Mathematical Model In the following discourse we shall relate the previous theoretical model of creativity to a less generic context, namely mathematical category theory. A similar model can be exhibited, but it is one that has obtained a deep significance in the development of mathematics since category theory began to influence algebraic geometry, eventually leading to topos theory, which now is the foundation of geometry, logic, computer science, and physics. This model is based upon the core construction that led to topos theory: namely, the famous Yoneda lemma. This lemma is an easy mathematical proposition, its proof is just two lines of mathematical arguments. But its consequences are huge and deep. Let us first state the lemma in its technical form. If X, Y are two objects in a category C, then there is a map that associates with X, Y their set-valued functors @X, @Y , i.e. contravariant functors C → Ens with values in the category Ens of sets. And it associates with every morphism f : X → Y a natural transformation @ : @X → @Y . The Yoneda lemma states that this map ∼

Hom(X, Y ) → N at(@X, @Y ) from the set Hom(X, Y ) of morphisms from X to Y to the set N at(@X, @Y ) of natural transformations from @X to @Y is always a bijection. This means in particular that X and Y are isomorphic if and only if @X and @Y are so. This implies that one may look at the functors @X instead of the objects X of a category. This looks quite abstract and harmless, but it has deep implications. The lemma first of all implies that one may look at the functor @X in order to fully understand X since the functor’s isomorphism class reflects the class of X. But looking at the functor means this: We replace the abstract object X with a system of sets, namely one set A@X = Hom(A, X) for each object A of the category C. We call A also an address of the functor. Why? Because A@X is the set of all perspectives (= morphisms) looking at X, so one intuitively

158

19 Our Approach

stands on the address object A and looks at X. The address A is where one stands when looking at X. Yoneda’s overall statement is then that X is fully understood (that is, its isomorphism class is known) if one knows what X looks like from all possible addresses. So the first step toward understanding X is to select all addresses A and to look at X. Of course these addresses are not independent but also have address change morphisms g : B → A that connect them. For such an address change g, we then have a set map g@X : A@X → B@X that sends the perspective f : A → X to the perspective f ◦ g : B → X. In most cases it is not necessary to inspect all addresses to understand X. For example, if C = Ens, then the singleton address A = 1 = {∅} is sufficient to obtain the isomorphism class of X since the set 1@X is in bijection with X, so the cardinality of X is the same as 1@X, the set of perspectives of X viewed from the singleton address 1. The inspection of addresses A to help understand X is a major effort in category theory. For example, in algebraic geometry, the points of a scheme X are often chosen to be algebraically closed fields, the C-valued points being the most prominent address selection. Suppose we have exhibited a system of characteristic addresses (sufficient for characterizing X up to isomorphism). Suppose that this system also specifies transitional address changes. This is formally described by (1) a diagram D in C that contains address changes gijk : Ai → Aj with (2) the perspectives fi : Ai → X from these addresses to X (compatible with the address changes). Then if the category has colimits (in particular if it is a topos), we may step over to the colimit colim(D) of the diagram and its canonical morphism f : colim(D) → X induced by the perspectives fi . Clearly, this latter morphism in some sense “summarizes” the characteristic perspectives fi and represents the understading of X in a single perspective from the “big” address colim(D). Since the colimit address is so powerful, the next step would be investigating its own functor @colim(D). This means that our process starting from X is now completed and may restart from the big colimit address. This procedure looks very formal, but it is completely analogous to the general semiotic creativity process described above3 . Let us trace this analogy. To begin with, the open question must be defined. In the categorical setup, this would be the question of understanding the behavior of X. For example, if X = R, the problem would be that all squares of real numbers are not negative, x2 ≥ 0. This is also equivalent to the fact that there are quadratic polynomials without real roots, such as x2 + 2. Next, we would identify the semiotic context of the question. Here, we would exhibit a category C, and then identify X as the critical object in C. The next procedure is the inspection of the critical concept’s walls. This means to inspect the general behavior of the critical concept and to look for its characteristics. In the mathematical context, this means to investigate the object’s functor @X, namely to look at X from all the addresses as explained above. In the case of X = R, we would look at a polynomial 3

And it in fact inspired the construction semiotic variant.

19.3 The List of the Creativity Process

159

ring R[x], and its quotients R[x]/P by irreducible polynomials P . The perspectives4 would be just the embeddings R → R[x], R → R[x]/P . The address changes would be the quotient morphisms R[x] → R[x]/P . If the wall inspection leads to a repertory of critical properties, these are represented by our diagram D of characteristic addresses. They allow us to understand the critical concept/object X. The opening of X by this diagram D enables an extended understanding of X, and this is what is represented by the big address and its arrow f : colim(D) → X to X. So we would have extended the naive context X by this big new colimit object. We should stress that the question heres is not to have an object X, or any other being a member of the given category, but to have a technical access to understanding this object’s behavior.

19.3 The List of the Creativity Process To conclude this chapter, we give a summary of the creativity process in the form of a list to which we refer in this book: 1. Exhibiting the open question 2. Identifying the semiotic context 3. Finding the question’s critical sign or concept in the semiotic context 4. Identifying the concept’s walls 5. Opening the walls 6. Displaying extended wall perspectives 7. Evaluating the extended walls

4

Arrows are reversed here, but in algebraic geometry, this is canonical via the prime spectra of these rings.

20 Principles of Creative Pedagogy

The concept of a creative pedagogy as an approach to creative teaching was introduced by Andrei Aleinikov in 1989 [4]. We summarize its theoretical and practical aspects and discuss its realization when introducing our concept of creativity. We conclude with the special situation of creative pedagogy in musical education, where not only the methodological aspect of this pedagogy relates to creativity (which would be the case in general education in chemistry or physics, for example), but also the creative activity of a composer or improviser.

20.1 Origins of Creative Pedagogy Following Andrei Aleinikov’s definition [4], creative pedagogy is defined as: Creative pedagogy that includes educational influence on the learner for acquisition of certain study material (subject) [as pedagogy in general] and differing from the above by the fact that in order to achieve higher efficiency of learning, the pedagogical influence is provided on the background of centrifugal above-the-criticism mutual activity in which the learner is raised from the object of [pedagogical] influence to the rank of a creative person, while the traditional (basic) study material is transformed from the subject to learn into the means of achieving some creative goal, and the extra study material includes the description and demonstration of the heuristic methods and techniques. This is an educational methodology, not education to become creative. The educational field can be any science or art. Learning is meant as achieving a creative goal. The learning process is different from the traditional one, and this new approach to shaping classes is called “creative transformation.” The learner is no longer an “object” of pedagogy, but becomes a creator in the field being taught. G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 20, © Springer-Verlag Berlin Heidelberg 2011

161

162

20 Principles of Creative Pedagogy

This methodology is the one that is needed in the twenty-first century for the development of a new social class, the Creative Class [43]. This class fulfills the growing need for experts in the continuous renewal and extension of our knowledge society. A variant of creative pedagogy has been implemented in a science called TRIZ (a Russian acronym), and translated to “Theory of Inventive Problem Solving.” Boris Zlotin and Alla Susman [128] describe TRIZ as being invented in Russia by Genrich Altshuller in the 1940s. It is based on the study of evolutionary patterns in various systems of technology, science, and organization. Methods for creative solutions of problems were developed and have been summarized as follows [128]: 1. The logical analysis of a given system and its problems. This enables users of TRIZ to understand the essence of a problem and to reveal the nonobvious contradictions that hinder problem solving. 2. The application of a special knowledge base that includes the most effective methods of problem solving, along with examples of how these methods are used. 3. The means to overcome psychological inertia in the process of problem solving. It is remarkable that this list resembles our concept of creativity in the following sense: Similar to our setup, the list starts with a problem, together with a system where the problem is located, corresponding to our semiotic where the problem is situated. In the list, the essence of the problem has to be found. We would see this as corresponding to our critical concept. Further, they have to reveal the non-obvious contradictions that hinder problem solving. In our setup this would be the inspection of the concept’s walls. Next, they apply special knowledge bases for effectively solving the problem. In our setup this would mean to open the walls and to find conceptual extensions that eventually would solve the problem. The last item in their list is of a psychological nature, and we would attribute it to the problem of finding walls since becoming aware of critical walls also depends upon the sensitivity to the concept’s hidden limits. An important point in this TRIZ approach is its relation to language. It is rightly remarked that children are naturally creative, but that the infusion of language with its normative logical nature builds a strong obstruction to creativity, since creativity is “paralogic” (wording by the TRIZ authors). In our approach, we do not specify the language but the semiotics, which is an extended representation of what the TRIZ authors think about language. In our approach, the effect of a successful creative action is always an extension of the given semiotics, which in terms of language means extension of the language. Clearly this is a big effort against the language’s normative power. The inability to imagine an extended language (or: The inability to conceive that a language can be extended) is due to this normative framework, and it

20.2 Applying Our Concept of Creativity to Creative Pedagogy

163

is clear also that children1 who are not yet casted by the language’s norms can be more creative and will often lose this ability as they become immersed in the language’s rigid status quo. The TRIZ authors then also criticize the inevitable battle that characterizes the relationship of teachers and students—the battle of rigid norms against creative extensions. They propose replacing this battle relationship with one where a teacher assists students in their quest for self-perfection.

20.2 Applying Our Concept of Creativity to Creative Pedagogy Our concept of creativity is built as a multistep process as described in chapter 19. Summarizing, its steps are: 1. 2. 3. 4. 5. 6.

Exhibiting the open question Identifying the semiotic context Finding the question’s critical sign or concept in the semiotic context Identifying the concept’s walls Opening the walls and displaying its new perspectives Evaluating the extended walls

Essentially, we are dealing with an extension procedure that starts in a given semiotic system and adds extended conceptual perspectives so that we obtain an extended semiotic context. If successful, this creation solves the original problem. This process is very akin to what is constructed in algebraic Galois theory, which starts with a polynomial phrasing of an open question and then finds the answers by an extension of the original field. To make this more understandable from an elementary point of view, we want to show how complex numbers C are “discovered” in a creative process, starting from the very beginning with natural numbers N. So let us start with the natural numbers N and seek to solve the equation a + x = b, a, b ∈ N. The answer to this question around the critical concept of the linear equation a + x = b is the group of integers Z. In this domain, a new critical concept is given by the equation ax = b, a, b ∈ Z, a = 0. The extension of Z that answers this question is the field of rationals Q. In this domain, the critical concept is that of a Cauchy sequence (ai )i∈N , which in general does not converge to a rational number. In order to obtain numbers to which all Cauchy sequences converge, we construct the field of real numbers R. Finally, in R the critical concept is a polynomial equation a0 + a1 X + . . . an X n = 0, which may have no solution in R. The 1

We should be aware that children usually do not create masterworks. This is due to the fact that they have neither the maturity of a professional instrumental performer nor are they competent in the complex knowledge about music theory and notation. They simply live in a context where such creations are out of reach.

164

20 Principles of Creative Pedagogy

extension of R to the field of complex numbers C answers this question. And it is remarkable that each extension is found by taking the critical concept as a type of new element of a larger structure; in other words, the very concept that is responsible for the problem also yields the solution! For example, R is obtained from a structure whose elements are the Cauchy sequences over the rationals! And C is obtained from R by considering all the polynomials with real coefficients. This means that the walls provide the doors that offer the solution. Teaching along the idea of creative pedagogy means that we teach a field of knowledge by successively extending a given semiotic context around a critical concept. Learning then means to create a solution following our six-element process of creativity. The pedagogical work consists of presenting the problem, working out critical concepts, searching and inspecting their walls, inventing wall extensions, and finally investigating the potential of such extensions as solutions of the given problem. This procedure puts the students into a situation of being creative, building their knowledge through creative activities. Whether these are successes or failures doesn’t matter, because it is important that the students experience these evolutionary dynamics in the building of a knowledge architecture. The minor miracle in such a work of conceptual extension is that very often the solution is found directly from a precise and clever restatement of the walls, like in arithmetic as explained above! It must be stressed that all these extensions are not purely formal (without content), we really add signs to the given semiotic context, i.e. expressions and contents. This must be taken into account when working out the detailed operations. It is also important to understand the operational character of this type of creative work. We do not rely on psychological categories in the identification of creative activity; creation can be checked without psychology, merely by following the thread of those six steps described above. Of course, it is also true that such an activity may be difficult or even fail because of psychological obstructions. But the explicit “flow chart” of creativity should be of great advantage when asking ourselves where we are located in the adventure of such creative work.

20.3 Creative Pedagogy for Musical Creativity As already recognized by its inventor, creative pedagogy can be applied to any field of knowledge—it is a teaching methodology, not a field of knowledge to be taught. However, when special fields are at stake, the general methodology may be difficult to apply if not problematic. The field of musical invention is such a delicate situation since composition and improvisation are creative activities. When it comes to teaching these fields by creative pedagogy, the tension between methodology and teaching object arises: Teaching creatively

20.3 Creative Pedagogy for Musical Creativity

165

musical creation looks somehow circular. So let us investigate what kind of tension we are dealing with in this case. In the tutorial of the practical part of this book, we have given a number of examples for creative pedagogy (see chapter 3). To begin with, we should think about how to teach musical creativity creatively. Musical creations always have two components: a set of material, such as a set of notes, motives, chords, instruments, etc., and a set of operators that act upon this material, such as Boolean operators, permutations, harmonization, contrapuntal rules, ways to play an instrument, etc. Teaching musical creativity then means to teach creatively how to generate or select material and how to generate or apply operators upon selected material. Both activities, relating to material or operators, can be taught creatively in the sense of applying the creativity “flow chart” to a critical instance of material or operators. For instance, one may question the concept of an instrument and add new instruments by extension of defining properties. The history of the keyboards and of the violin family are excellent examples of such a case. Or one may question the way of playing the piano. The addition of the thumb to piano-playing techniques or the percussive style of piano technique added by Cecil Taylor to the Mozart-Liszt tradition are examples. But also a new way to compose a sonata2 might emerge from a creative questioning of the sonata concept. This means that composing with given rules and materials without any creative process is not what we would like to teach in such a class. Such uncreative processes are just rule-based mechanical activities and not proper compositions. It is evident that such mechanical activities play an important role in the music industry, and there might be a computer program for such algorithmic applications, but this is not what would be taught under the title of creative activities in a composition class. Observe however that the writing of such a program might very well be a creative action, but the produced samples are not, at least not in the semiotic system of the applied rules and materials. Let us give an example. From mathematical music theory, we know that counterpoint can be composed in six so-called worlds according to the six possible dichotomies of the twelve interval pitch classes modulo octave into a “consonant” and a “dissonant” half [77]. The construction of ‘exotic’ counterpoint compositions other than those referring to the classical Fuxian dichotomy of classical consonances—prime, thirds, fifth, and sixths—have been implemented in the RUBATO Composer software by Julien Junod [60]. In this system, an exotic counterpoint would just be a sample of the computer’s program. But if a composer came up with such a composition by questioning the classical Fuxian dichotomy and extending the concept of consonances/dissonances and counterpoint rules, this would define a creative action. In Junod’s program, the creative work had been achieved by the extended counterpoint theory from 2

As Beethoven did with his late sonatas.

166

20 Principles of Creative Pedagogy

[77] and by Junod’s creative writing of such a sophisticated program, applying creatively sophisticated techniques of graph theory. So far we have focused only on composition, but we should also consider improvisation in a creative pedagogy for musical creativity. In the compositional context it is clear how to apply the process of creativity, since one has plenty of time to think about walls and their extensions. When improvising, the process seems to become impossible because improvisation takes place without any time to reflect in a quiet atmosphere. However it is possible to anticipate certain musical constructions, such as motives, chords, or rhythms, and to call them up while improvising. But there is another creative strategy, namely to grasp a critical concept in the making, i.e. to focus on a musical structure being played and to call up its walls in the making. This boils down to focus on typical gestures being performed to realize such a musical structure—a rhythm, a motif, etc. In this moment one would recall Jean Cavaill`es’ insight that “understanding is catching the gesture and being able to continue.” One would take the given gesture and continue it with a new gesture, and, most important, not just mimic it but continue with another gesture that resonates from the first one. The conceptual extension of walls here is realized as a resonance of gestures, quite in the spirit of cultural evolution built upon the psychological DNA paradigm of mirror neurons as suggested by Vilayanur S. Ramachandran [98]. This type of real-time process will enable an improviser to be creative without having to step back for a meditative reflexion. 20.3.1 Conceiving Our Tutorial in Creative Pedagogy for Musical Creativity In the practical part II of this book, we have developed a tutorial applying creative pedagogy to teach musical creativity. The idea behind the tutorial is to implement the above techniques and strategies to educate a student to become a creative composer/improviser. We start by the most elementary situation of absolutely unshaped sonic and instrumental material. Everything that could be imagined as sound or a sound generator would be admitted at the onset. This is the totally chaotic original position. We would then successively introduce extensions of critical concepts in this manifold of sound material or devices. Critical concepts will emerge when the chaotic variety cannot present any interesting statement or utterance. In the tutorial we have organized a tree of successively enriched conceptual extensions, starting with shaping time in rhythm, or pitch in scales, etc., and thereby giving the student all the power to shape his/her compositional strategies and tools. The overall development in the tutorial would show the student how to generate a successively more sophisticated composition when applying the sequence of conceptual extension. It might seem contradictory to state that successively enriched creative activity with a sequence of conceptual extensions would generate a more and more non-chaotic, orderly composition. One would rather expect the opposite in musical creativity: the movement to increasingly

20.3 Creative Pedagogy for Musical Creativity

167

wild compositions by use of such conceptual extensions. It is a common misunderstanding that musical creativity tends to produce wild works. The famous conductor Nikolaus Harnoncourt once remarked, “The highest creation is the one which can express its thoughts in a very limited context, such as a small chamber ensemble, not a Wagner orchestra.” Creativity is about applying the finest choices to build new works, much like the existence of the human species is only possible in the extremely refined terrestrial environment of the universe. In our tutorial, the creation of a musical composition will be generating an intelligent output that expresses all the richness of the creative path of the tutorial’s logical tree. If wild compositions emerge, they will be wild only on their surfaces; the internal logic will be a richer one, but possibly difficult to see without a sophisticated culture of understanding music. Creativity is always a taming act of the chaos, never its progressive extension. We shall illustrate this fact in the case study of Beethoven’s third movement in op. 109, (chapter 26), where the wild trill surface of the finale in the sixth variation is the disciplined and precise expression of a cadential symmetry.

21 Acoustics, Instruments, Music Software, and Creativity

This chapter is about technological issues of music and therefore might be more interesting to those who deal with technology, for example in computer music, studio recording and postproduction, or new electronic instruments. However, this chapter is not a standard exposition of acoustics and music hardware and software as may be found in textbooks about music technology, such as [55, 81]. We shall discuss those contents but with emphasis on their potential roles as tools for musical creativity. This chapter is also conceptually demanding and may be used as a reference for those who want to apply principles of creativity to technological environments. We do in fact believe that technological tools are essential for the expansion of creative processes only if they are understood as autonomous devices and not just as realizations of given paradigms. Recall that the free jazz pioneer, saxophonist, and composer Ornette Coleman used the violin in a very creative way in his free jazz performances, but only because he completely disregarded the standard usage. And John Cage created new ways of playing classical instruments, such as the organ in his composition “ASLSP,” where a new pipe is installed every period of several years for the total duration of 639 years!

21.1 Acoustic Reality Acoustics deals with the physics of sound. Sound is generated by a sound source, typically an instrument or human voice in musical contexts (see Figure 21.1). This sound source acts on the molecules of air and makes them produce a variable air pressure (around the normal pressure of 110.130Pa, Pa = N m−2 , at sea level) that propagates through space, is redirected by walls and objects, and then reaches the human ear, which is sensitive to such pressure variations. The auditory nerves in the cochlea in the inner ear then conduct the sound input to the auditory cortex and other sensitive afferent brain centers, where G. Mazzola et al., Musical Creativity, Computational Music Science, DOI 10.1007/978-3-642-24517-6 21, © Springer-Verlag Berlin Heidelberg 2011

169

170

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.1. Sound is generated by a sound source, typically an instrument or human voice in musical contexts. This sound source acts on the molecules of air and makes them produce a variable air pressure that propagates through space, is redirected by walls and objects, and then reaches the human ear, which is sensitive to such pressure variations. The auditory nerves in the cochlea in the inner ear then conduct the sound input to the auditory cortex and other sensitive afferent brain centers, where different properties of the sound information are perceived and processed for the human cognition.

different properties of the sound information are perceived and processed for the human cognition. This basic setup can be taken as a germ for musical creativity. But what would be a problem here? There are many! To begin with, let us first blank out the instrumental source structure providing the sound, so we may focus on the complex trajectory of the sound from the source to the ear. In the realization of sounding works of art, the problem of controlling the trajectory is crucial. Historically, this relates to room acoustics: How can we shape the music halls in such a way that performed works can be heard in an optimal way? This original problem has the sound trajectory (from source to ears) as its critical concept. What are the sound trajectory’s walls? A subtle wall is the assumption that sound travels through air with the given speed and follows the room’s physical walls and other objects until it reaches the human ears. This wall specifies a given travel mode. Sound travels through air, and it does so without any further influence. It is just “given physics.” This is the hard part of this wall: What could we do against given physics? Let us see how this is feasible. To begin with, we can question the path. Why does

21.1 Acoustic Reality

171

the sound have to go through air, be reflected at walls, and enter the human ear? Couldn’t we imagine other pathways? A first opening of the wall would be to ask whether a direct pathway is mandatory. And it is not. There are a number of technologies to split the trajectory into a multistep pathway. The sound can be piped into a microphone system and then distributed to a number of loudspeakers. This has been done not only for simple reasons of acoustical quality in music halls and dance clubs, but also to redefine the output in an artistic way, for example when a multiplicity of loudspeakers is part of the composition and may enable migration of sound among several loudspeakers. Another interruption of the sound trajectory might be saving the sound on a tape or digital soundfile. This relay station can then be used to reconstruct the sound and apply any kind of sound processing before forwarding the sound to the audience. Finally, following some ideas from Venetian polychoral style of distributed choirs, one may also rethink the role of the music hall’s space, extending use of the stage to the entire hall (including audience space and even the music hall’s bar) and setting up musicians to play anywhere. 21.1.1 First Sound Anatomy The structure of musical sound waves is described in Figure 21.2. They consist of longitudinal waves, i.e. the pressure variation of air moves in the direction of sound propagation away from the sound source. If they carry the pitch, the variation of pressure is periodic, i.e. a snapshot of the pressure shows a regularly repeated pressure along the sound’s expansion in space. The time period P of this regularity, as shown in Figure 21.2, defines the sound’s frequency f = 1/P . For example, the chamber pitch a1 has frequency 440Hz. And going an octave higher means to double frequency; therefore, the octave a2 above a1 has frequency 880Hz. The wave’s pressure amplitude A (relative to the normal pressure) is perceived as loudness. For example, the minimal loudness that can be perceived by the human ear at 1000Hz is A0 = 2.105 N m−2 . Loudness for amplitude A is then defined by l(A) = 20. log10 (A/A0 )dB, where the unit dB is Decibel. What could be a creative endeavor in this elementary sound description? A sound is a wave, starting at an onset time, lasting a certain time (its duration), and during that time showing the amplitude and periodicity discussed above. Taking this kind of objects as an element of music, however is a problem. To begin with, it is not clear why these attributes should hold during the entire sound. Why should amplitude be constant? A sound may well increase its loudness. And then, why is the pitch constant? What about glissandi? And finally, where do we find access to the sound color, the instrumental characteristic? Putting all these questions together, the critical concept here is the parameters that are used to describe the sound object. They are all too simplified and incomplete to cope with musical reality. The walls in this situation are the

172

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.2. Standard parameters for sound with frequency (pitch).

parameters used to describe sound. Are these parameters inevitable—does sound has to have one pitch, one loudness, etc.? And where is sound color hidden? Isn’t periodicity for pitch too simple to catch all of the sound’s anatomy? Of course, the wall character of these parameters is supported by the correspondence with notation of sound in scores. Onset, duration, loudness, pitch, instrument name—all of this is described in classical score notation. But only when comparing these forms to the real sound events does one become aware that these walls are not necessary and that opening them can create richer musical compositions. We shall come back to such options in the discussion of sound synthesis methods, such as Fourier, FM, wavelets, and physical modeling. But one big wall has not yet even been mentioned: the fact that sounds must be fabricated. Sounds are not there by themselves; they are made by some agent. And this is the theme of the next section. 21.1.2 Making Sound It is perhaps one of the most intriguing walls in the theory of music that sounds are not abstract entities defined by mathematical parameters such as onset, duration, loudness, and pitch. They are not even defined by mathematical sound parameters but by the way the variation of sound pressure is realized as a physical process. Sound must be made, not mathematically but physically. It is not astonishing that music theory does not take care of this elementary fact since human understanding usually starts from huge abstraction. For example, numbers and their arithmetics are abstracted entities from real ob-

21.1 Acoustic Reality

173

ject assemblies. One takes five apples and adds three apples, but arithmetic yields eight apples independently of whether these objects are apples, bananas or planets. Music theory is about very strict abstraction. Usually, in harmony one disregards duration, loudness, and instrumental color and only focuses on pitch, or even pitch classes modulo octave. How then was it that making sound became a strong paradigm beyond abstraction? Until the first half of the eighteenth century, instruments were still placeholders for voices; any instrument would do if it could play the pitches required. But around 1750, also the moment when the string quartet emerged with Haydn and Boccherini, the voices were required to express increasingly complex structures. The abstract note sets were set in such a complexity that their sensual expression required more and more sophisticated sound bodies. So the complexity of abstraction enforced a corresponding complexity in expression. And vice versa: The increase in expressive power of the family of violins, and the richer color spectrum of the piano, enabled increasing complexity of compositional structures. What is this wall about? It is about the intimate relationship between abstract musical structures, of counterpoint and harmony, say, and their expression via a sounding body. This relationship has never been recognized as the central topic of orchestration, so it is a wall in the best sense of our theoretical approach: invisible at first, and impenetrable. How could abstract music structure be related to these very practical instrumental objects? Even Beethoven argued that this “stupid violinist” Schuppanzigh was not his concern if he could not play his supreme tones. The philosophy of this resonance of structure and instrumental expressivity is a creative extension of what it means to play an instrument. What is the point of this creative extension? It means that a musician’s movements on an instrument that merely cope with the required abstract note parameters limit not only the musician’s bodily richness, but also the the potential expressivity required by advanced compositions. In other words, the creation of instruments is not just a technical process for the realization of abstract tone parameters, but an activity in its own right. And the richness of instrumental interaction is a field of primary musical interest. Making music is a rich dance of the musician’s limbs on the complex interface of the music instrument. Figure 21.3 shows three methods for the production of regular sound pressure variations (inducing pitched sound), using clarinet, flute, and trumpet. Each method creates pressure variations in different ways. With the clarinet, the air vibration is generated by the vibrating reed. With the flute, the air vibration is generated by an alternating air stream moving into the flute or outside. With the trumpet, the air vibration is generated by vibration of the musician’s lips. Accordingly, the technical basis of musical expressivity varies strongly. So it is definitely not about producing the abstract pitch here.

174

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.3. Three methods for the production of regular sound pressure variations (inducing pitched sound), using clarinet, flute, and trumpet.

Although these examples are evident messages about the body’s expressivity in the making of music, the really complex case is the violin family, whose typical representative, the violin, is shown in Figure 21.4.

Fig. 21.4. The violin’s anatomy and its cross-section.

21.1 Acoustic Reality

175

The violin, as it had been developed to its ideal shape by the families of Stradivari, Amati, and Guarneri in the first decennia of the eighteenth century, is a highly complex mechanical device. Essentially, its sound is produced by drawing a bow across one or more of the five strings, whose horizontal vibration is transmitted by the bridge to make the belly plate—and, via the sound post, also the back plate—vibrate vertically. The vibrational shapes on these plates are extremely complex and can be classified following investigations of Ernst Chladni by so-called modes (basic vibrational states), which are visualized in Figure 21.5. The fabrication of the plates is a highly artistic process. For example, the quality of these plates is checked by a number of bending actions with the artisan’s hands. This gestural access to the instrument is complemented by a rich repertory of playing modes, such as plucking the strings, knocking the belly plate, etc. Also the bowing can be executed on different locations of the violin’s strings and with different movements, such as chopping, shuffle, col legno, lour´e, sautill´e, etc. In other words, the violin is a gestural universe, and it is an interface that requires intense training and sensitivity.

Fig. 21.5. Chladni’s resonance modes describe a secret of a violin’s quality.

This characterization is also valid for the other instruments of that family. And it is the basis of the dominating role of string orchestras, and especially of the string quartet. The development of the string quartet has proved that the gestural extension of the violin family’s sound production universe has become the driving creative force for a compositional approach that does not theorize, but simply applies the technical tools of the instruments as compositional methods.

176

21 Acoustics, Instruments, Music Software, and Creativity

Summarizing, we have learned that making sound on a sophisticated instrument is not just a complicated way of letting abstract sound parameters take place in acoustical reality, but it is also a highly creative interface between structural and expressive universes. And these universes have no hierarchical relationship with each other; on the contrary, they set into action a deep resonance of extensions in structure and performative expression. Following these lines of thought, orchestration should include a theory of this resonance and answer the questions concerning adequacy of an orchestra for a given composition, and vice versa (the adequacy of a composition for a given orchestra). It should deal in a scientific way (!) with categories of questions such as why a recorder quartet cannot adequately perform a Beethoven string quartet, or why a simple song such as Happy Birthday to You should not be performed by a Wagner orchestra. 21.1.3 Fourier The instrumental extension of abstract sound parameters is complemented by descriptions of sound colors independent of the instruments that produce them. Or rather: using very simple devices that act like ‘atomic’ instruments, whose combination yields a complex sound. Among these attempts, Joseph Fourier’s theorem, discovered in research about heat conductance around 1800, provides a first model. The atomic “instruments” he uses are vibrating masses m attached to a fixed point at distance x by a spring with spring force constant k, and moving along a fixed line. Newton’s differential equation for the force acting upon m is m.d2 x/dt2 = −k.x. The solution of this vibrating system is a sinusoidal function x(t) = (k/m) sin(t). Recall that pitch of a sounding periodic air pressure vibration w(t) is proportional to the logarithm of its frequency f = 1/P eriod. Fourier’s theorem then states that w(t) can be expressed in a unique way as a sum of sinusoidal functions, or, so to speak, as a sum of those atomic instruments given by masses and springs (see Figure 21.6). Uniqueness means that for the given frequency f of w(t), the amplitudes A0 , A1 , A2 , . . . and the phases P h1 , P h2 , . . . are uniquely determined (and called amplitude and phase spectrum, respectively). The nth component function An sin(2π.nf t + P hn ) is called the nth partial or overtone of w(t). Before we delve into the creative investigation of this setup, let us describe how the Fourier theorem is related to realistic sounds. We have to see that the wave function w(t) is all but natural. In reality no such infinitely lasting regular air vibration can occur. The relationship to realistic sounds can be seen in Figure 21.7. If a singer sings “laaaa” at a determined pitch, the pressure variation around the mean pressure looks like a bundle as shown on top of Figure 21.7. The bundle can be described by its envelope, i.e. the locally maximal pressure variations (shown at the left bottom), and by a periodic excitation of pressure, limited by the envelope, and shown to the bottom right.

21.1 Acoustic Reality

177

Fig. 21.6. Joseph Fourier’s theorem, discovered in research about heat conductance around 1800, provides a first model of sound as composed of ‘atomic’ sound shapes: sinusoidal functions.

This combination of envelope and periodic function yields the realistic sound. In a systematic building process as shown in Figure 21.8, this means that the periodic wave w(t) is superimposed with the envelope H (normed to duration and maximal excitation both = 1), and then shifted and squeezed by a so-called support to yield onset, duration, and amplitude of the real sound. In technological modeling of this building process, the envelope is often represented in a very simple shape, following the so-called ADSR (Attack Decay Sustain Release) model as shown in Figure 21.9. Although this seems to describe realistic sounds quite faithfully, it turns out that instrumental sounds are more complex in that the partials are given independent amplitude envelopes that enable the sound to have variable overtones as it evolves in time. Figure 21.10 shows such a situation for a trumpet sound. The graphic displays envelopes for each partial, and we see that higher partials have lower and shorter envelopes, meaning that the amplitudes of these partials contribute only a short time and to a lower degree to the overall sound. The display of these temporally variable overtone envelopes is called chronospectrum. Fourier’s theorem and the description of sounds by overtones are firm scientific facts, as a mathematical theorem cannot be deemed false once its proof is correct. Why can such a fact be a problem for musical creativity? To begin with, its facticity seems to condemn everybody to its unrestricted acceptance. It is an eternal truth. And can an overtone-related analysis— although not precisely true [77, Appendix B.1]—in the inner ear cochlea’s Corti

178

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.7. When a singer sings the syllable “laaaa” at a determined pitch, the pressure variation around the mean pressure looks like a bundle as shown on top. The bundle can be described by its envelope, i.e. the locally maximal pressure variations (shown at the left bottom), and by a periodic excitation of pressure, limited by the envelope, and shown to the bottom right.

Fig. 21.8. This combination of envelope and periodic function yields the realistic sound.

organ be taken as a strong argument for the universal presence of the Fourier decomposition of a sound? The wall here is precisely the mathematical truth of this theorem. Nobody can even think of putting the theorem into question. This theorem is the critical concept. But the creative softening of its wall(s) can be accomplished when we think about the justification of this theorem: We are analyzing a sound, and

21.1 Acoustic Reality

179

Fig. 21.9. The envelope is often represented in a very simple shape, following the so-called ADSR (Attack Decay Sustain Release) model.

Fig. 21.10. A trumpet sound. The graphic displays envelopes for each partial, and we see that higher partials have lower and shorter envelopes, meaning that the amplitudes of these partials contribute only a short time and to a lower degree to the overall sound. The display of these temporally variable overtone envelopes is called chronospectrum.

Fourier tells us how to do so. But analysis is not synthesis. The wall softens if we question the exclusivity of analysis. After all, composers are interested in synthesis rather than in analysis. The point here is whether this formula is the end of an insight or the beginning thereof. If it is just about understanding a periodic function, it is the end, but if it is about putting together atomic instruments (the sinusoidal functions) to form a compound instrument, it is a new thing. We could construct instruments by adding sinusoidal functions, provided with their individual amplitude envelopes. And we do not have to

180

21 Acoustics, Instruments, Music Software, and Creativity

add sinusoidal functions with frequencies that are multiples of a fundamental frequency. This latter generalization is called inharmonic partials. Summarizing, there is no need to follow the original formula, but we can generalize it and generate compound sounds with the original building rule but more flexible. This is precisely what spectral composition discovered [47]. They were interested in composing sounds, not abstract symbols. And the synthetic interpretation of Fourier’s formula was the technical tool they used to shape the sound’s anatomy as a living body. 21.1.4 FM, Wavelets, Physical Modeling We now discuss some prominent sound synthesis methods which complement Fourier’s traditional approach. They also prove that Fourier’s decomposition is not the only one that one might conceive in the description of a sound’s ‘anatomy’. 21.1.4.1 Chowning’s FM Synthesis Fourier’s formula has not only provoked more general compositional approaches, such as spectral music, but also more realistic sound representation for the technology of sound synthesis. The first of these innovations was introduced by John Chowning in 1967 at Stanford University and was called FM for “Frequency Modulation.” His invention was patented and had a remarkable commercial success with Yamaha’s DX7 synthesizer since 1983 (see Figure 21.11). The creative idea was again to review the critical concept in Fourier’s theory. It was critical in the context of sound synthesis technology since the production of reasonable sound colors is described by a huge number of overtones and of corresponding envelopes. Whereas the spectral music creativity focused on the extension of sinusoidal components in Fourier’s formula, Chowning questioned its arithmetical construction. This was a delicate wall since addition of sinusoidal functions for the partials is hard to open. Should one just negate addition, and then how should it be replaced or embed by another operation? Should one multiply sinusoidal waves? Or apply any other operation? There are infinitely many such candidates, mostly unattractive because of mathematical properties. And the problem was not only the addition as operation but also the number of summands necessary to produce interesting sounds. Chowning’s idea was highly creative in that he succeeded in replacing addition with another operation, which also allowed him to massively reduce the number of sinusoidal components. As seen from Yamaha’s list of components used in DX7, only six components are used in this technology. How can we build complex sounds with only six overtones? The idea focuses on the meaning of the wall “addition of sinusoidal partials.” The original formula for a wave w(t) as shown in Figure 21.11 takes a fundamental wave and changes its values by the values of a second overtone,

21.1 Acoustic Reality

181

Fig. 21.11. John Chowning’s invention of “Frequency Modulation” (FM) in 1967 at Stanford University.

and then this sum has values that are in turn changed by values of a third overtone, etc. Chowning’s idea was to have the influence of higher overtones not on the values of the preceding overtones, but on their time arguments. Instead of having time in a linear expression 2π.n.f.t + P hn , Chowning allowed a nonlinear correction of the time argument, the nonlinearity being defined by the overtone function! This means that instead of adding to values, his new synthesis formula adds to arguments! The result is shown in the formula W (t) in Figure 21.11, where the second overtone value is added to the argument of the first overtone, and where the third overtone value is added to the argument of the second overtone. The overtone distortion of the time argument is called a modulator, and it in facts modulates the frequency, in the sense of a time function distortion of the frequency-related linear argument. In Chowning’s approach, it was also permitted to add several overtone distortions to the linear time argument. The resulting architecture shows a number of sinusoidal functions (called carriers) that are altered by other sinusoidal functions (called modulators). Yamaha calls such an architecture an “algorithm.” Yamaha’s DX7 displays 32 such algorithms. Some of them are even their own modulators. Of course this is possible on the technological realization since output and input values of these sinusoidal “boxes” are shifted by elementary time units.

182

21 Acoustics, Instruments, Music Software, and Creativity

21.1.4.2 Wavelets

Fig. 21.12. A wavelet function g(t) of time. It is a sinusoidal function cos(wt) that has been modified by the envelope function exp(−t2 /2).

Although FM synthesis was a huge step in the reduction of sinusoidal components and the richness of sound colors realized in consumer technology, the FM formula (what Yamaha calls an algorithm) is still far from realistic. Because it is infinite like Fourier’s formula, the sinusoidal components are not limited functions. There is a canonical way to correct this, namely by envelopes. And this is exactly what was done in the approach of wavelet functions. Figure 21.12 shows such a wavelet function g(t) of time. It is a sinusoidal function cos(wt) that has been modified by the envelope function exp(−t2 /2). Although this is still an infinitely extended function, its values converge fast to zero by the exponential envelope. In wavelet theory, there are many such wavelet functions, but all have the property of showing the shape of a timely short wave “package,” and this is why such functions are called wavelets. They play the role of sound atoms, much as sinusoidal waves in Fourier theory. Whereas sinusoidal waves are modulated by amplitude, frequency, and phase, wavelets are modulated by a time delay argument b and an expansion argument a to yield the deformed wavelets ga,b (t) = g((t−b)/a). Figure 21.12 shows a number of such deformations that are placed in the plane of a and b.

21.1 Acoustic Reality

183

The creative component of this theory is to take one single sinusoidal wave ‘atom’ g(t), limit it by a strong envelope, and vary its time position and duration according to a whole two-dimensional parameter set (a, b). The open question is then whether this two-dimensional variety of deformed and shifted wavelet atoms is capable of representing a given sound function s(t). The answer is positive, and it can be stated as follows: Define a coefficient function  1 S(b, a) = ( √ ) g a,b (t) · s(t)dt. a This function replaces the Fourier coefficient calculation. So we have an entire two-dimensional function, not only that discrete set of amplitudes and phases of the Fourier formula. But like with Fourier, this new function enables the reconstruction of the original sound wave by the followig formula:  s(t) = Qg ga,b (t) · S(b, a)dadb, where Qg is a constant depending on the chosen wavelet type g. An example of the coefficient function is given in Figure 21.13, see [63]. Here the gray value represents the numerical value of the coefficient function. The octaves in this musical example are visible as dark coefficient values.

Fig. 21.13. An example of the coefficient function. The gray value represents the numerical value of the coefficient function. The octaves in this musical example are visible as dark coefficient values.

21.1.4.3 Physical Modeling The last example of refined sound modeling is a radical change of paradigm. In Fourier’s theorem, Chowning’s FM, and the wavelet methods, we always

184

21 Acoustics, Instruments, Music Software, and Creativity

dealt with sound representation. We have to recall however that instrumental technology, as discussed above for the violin family, has always been a radical answer to the question of sound synthesis. It is an answer of sound production, as opposed to the more or less abstract representation by mathematical formulas. With the development of powerful personal computers, the instrumental approach has been recreated on the level of software simulation of entire instruments. It has become realistic to not only create an “instrument” built from a series of sinusoidal waves a la Fourier, but also to simulate complex physical configurations on fast computers. This approach has three styles, namely: •

The Mass-Spring Model: The software simulates classical systems of point masses that are connected by springs and then outputs the sound wave resulting from the movement of such mechanical systems. A famous example is Chordis Anima developed by the Institute ACROE Grenoble, which was presented at the ICMC (International Computer Music Conference) in 1994. • Modal Synthesis: Modeling of different sinusoidal components that are experimentally determined, e.g. Modalys-Mosacic at IRCAM (Institut de Recherche et Coordination Acoustique/Musique) [92]. • Waveguide: It models propagation of waves in a medium, such as air or string. This is implemented, for example, at Stanford’s CCRMA (Center for Computer Research in Music and Acoustics), and a commercial version is realized on YAMAHA’s VL-1. Physical modeling is not just a simulation of physical instruments in software environments. By the generic character of such software, the parameters of the physical instruments, such as string length, diameter, material elasticity, material density, and so forth, can be chosen without any reference to realistic numbers. For example, physical modeling can simulate a violin with strings of glass and extending to several miles in length. So the critical concept being the musical instrument with its physical limitations, the software simulation in physical modeling now enables opening all the walls of realistic physical parameters. It is like a reconstruction of physical devices with fictitiuous parameters. A fascinating example of such a simulation has been implemented by Parry Cook in his singer synthesis software [25]. The software’s functional units, simulating a human singer’s physiology with throat, nose, and lips, is shown in Figure 21.14. This means that the creative action here is the opening of instrumental parameters by transferring the sound production to computer software, where the physical properties of the instrument are only used as data to calculate the output. This means that we are only interested in the sound output of the instrument, not its real physicality. This seems to be a huge extension of instrumental technology, but one should observe that physical modeling deals only with the instrument as such, not with the instrument as an interface between human gestures and sounding

21.2 Electromagnetic Encoding of Music: Audio HW and SW

185

Fig. 21.14. Parry Cook’s singer synthesis software.

output. In other words, the entire virtuosity of musicians on their instruments must be reconsidered when building interfaces capable of letting humans play such physically modeled sound generators. This means that creativity can, on the one hand, expand one concept—in this case, that of a musical instrument as a sound generator—but on the other hand also create new problems for the management of such expanded concepts when relating to other concepts of the given context—here the musician’s interaction with these new instrumental devices. To put it simply: It is not sufficient to create a new fancy instrument by physical modeling, but you also have to consider how to play the instrument. Moreover, to preserve the information about your creation, you would have to develop a new appropriate notation (analogous to the traditional Western notation, for example) and then also an educational system that teaches musicians how to play such notated information.

21.2 Electromagnetic Encoding of Music: Audio HW and SW 21.2.1 General Picture of Analog/Digital Sound Encoding The electromagnetic encoding of music is the trace of an incredibly rich history of creative acts. We want to follow this pathway and highlight the most important milestones.

186

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.15. The general scheme of electromagnetic encoding of music. It shows that this encoding technology inhabits a physical reality and mainly realizes a movement from the poietic original code given by the music’s sound waves to a neutral niveau level of communicative transfer, where the message is encoded in either analog or digital fashion.

The general scheme of such encoding is shown in Figure 21.15. The scheme shows that this encoding technology inhabits a physical reality and mainly realizes a movement from the poietic original code given by the music’s sound waves to a neutral niveau level of communicative transfer, where the message is encoded in either analog or digital fashion. “Analog” means that the original waveform is mapped onto a similar waveform imprinted on a media, and “digital” means that the original waveform is transformed into a sequence of numbers according to a more or less refined algorithm. The neutral niveau is in the expressive layer of the semiotic system; it has no meaning as such, but needs to be decoded in order to give us back the original musical sound content. Observe that “content” here means just the sounding wave, not the deeper symbolic or psychological sound signification. Technically speaking, we have the arrangement shown in Figure 21.16. The physical content is recorded by a set of microphones (short: mics), and sent to an analog mixer that merges the mics The mix is then piped to a analog or digital encoder, which then— after potentially mixing on the digital material—outputs its analog decoded wave to a second analog mixer for voltage output to a system of loudspeakers. Although the microphone and loudspeaker technology looks marginal in the music encoding process, it has been a major concern to obtain optimal input from or output to the acoustic level. Figure 21.17 shows the three major

21.2 Electromagnetic Encoding of Music: Audio HW and SW

187

Fig. 21.16. The physical content is recorded by a set of microphones (short: mics), and sent to an analog mixer that merges the mics. The mix is then piped to a analog or digital encoder, which then—after potentially mixing on the digital material— outputs its analog decoded wave to a second analog mixer for voltage output to a system of loudspeakers.

Fig. 21.17. Three major mic types: moving coil dynamic, ribbon, and condenser.

mic types: moving coil dynamic, ribbon, and condenser. While the technologies defining these devices are evident, their use in music recording studios or for concert amplification is subject to sophisticated discussions that also reveal personal tastes and ‘ideological’ positions concerning the characteristic quality of the sound. It is remarkable how much effort has been invested in creating

188

21 Acoustics, Instruments, Music Software, and Creativity

better loudspeaker systems or better mics and marketing such systems under the title of creative technologies. Figure 21.18 shows the typical anatomy of a loudspeaker device.

Fig. 21.18. Typical anatomy of a loudspeaker device.

The concept of a microphone and loudspeaker as technological surrogates for ears and instruments has a more creative implication for the technology of sound recording and reproduction, however. The critical concept here is the ear/mic. The wall here is the singular (like with Newton’s time, see section 19.1.2). Why should we have only one recording device, or two like in the human ear configuration? The answer (confirming the wall character) is that human ears are also just one (pair), so humans do not need more than a corresponding mic (pair) to record music. This paradigm of high fidelity is also used in audio label advertising (e.g. the excellent Cadence Jazz Records): “We guarantee the most faithful real music recordings.” But the wall can be softened by asking: Where do you listen when hearing a musical performance? Are you simulating a concert hall position, and where in such a hall? The question opens a variety of answers: One can sit in an ordinary seat, also creating a variable listening according to where the seat is located, and in which hall one wants to be seated. But the variety of answers is much larger in that it is also possible to listen to the music as a musician, really near to some instrumental configuration (within the string choir, for example, or next a drum, etc.). Which is the ‘real’ listening position? In studio technology,

21.2 Electromagnetic Encoding of Music: Audio HW and SW

189

this is solved by a heavy intermediate process involving the installation of a large number of mics across the orchestra, recording all these tracks and then mixing them while stepping to postproduction. The multiplicity of such artificial ears is even potentiated by positioning some twelve mics across the drum set, for example. Ideally one would wish to have mics everywhere in the orchestra’s space. Of course, this would presuppose a new mic technology that is capable of tracing an infinity of mic locations and merging them to a postproductive synthesis. But it is clear that this opening of the hearing concept redefines musical reality altogether: It is no longer evident what is the audio output. The variety of local hearing inputs enables a very creative reconstruction of the typical final stereo output. On the other hand, thinking of the concept of a loudspeaker as a surrogate of instrumental output also has a wall in that it is thought in analogy to the two ears as two sources, or four in quadro setups. But similar to the theoretically infinite multiplicity of mics, there is also the option of an infinity of loudspeakers distributed in any manifold within the hall or living room of audio reproduction. This infinity might be realized as a surface mic, i.e. a surface that vibrates in a locally variable way. A technology using an infinity of hearing positions could be realized in headphones whose output depends on its position in space and a bluetooth wireless input of sounds that had been recorded and/or edited to represent spatial positions in the orchestra’s recording space.

Fig. 21.19. The overall analog and digital encoding variants.

The overall analog and digital encoding variants are shown in Figure 21.19. For analog encoding, we have the options of a mechanical LP and an analog

190

21 Acoustics, Instruments, Music Software, and Creativity

magnetic tape. For the digital encoding, we have three options: (1) the optical technology of a CD, where the data is engraved in a glass material (see below for the details); (2) the magnetic technology realized by DAT (Digital Audio Tape) or Hard Drive technology; (3) the electric (semiconductor-based) technology with volatile (RAM, Random Access Memory), semi-volatile (memory sticks), and non-volatile (ROM, Read Only Memory) technology. 21.2.2 LP and Tape Technologies, Some History

Fig. 21.20. The classical analog encoding and its historical highlights.

Figure 21.20 shows the classical analog encoding and its historical highlights. To graphically represent sound on discs of paper, a machine using a ´ vibrating pen (without the idea of playing it back) was built by Edouard-L´ eon Scott de Martinville in 1857. While this phonoautograph was intended solely to visualize sound, it was recently realized that this depiction could be digitally analyzed and reconstructed as an audible recording. An early such phonautogram, made in 1860 and now the earliest known audio recording, has been reproduced using computer technology. In 1877, Thomas Edison developed the phonograph, which was capable of both recording and reproducing the recorded sound. There is no evidence

21.2 Electromagnetic Encoding of Music: Audio HW and SW

191

that Edison’s phonograph was based on Scott de Martinville’s phonautograph. Edison originally recorded on wax-coated tape while Scott de Martinville used soot coated glass. The version Edison patented at the end of 1877 used tinfoil cylinders as the recording medium. Edison also sketched (but did not patent) recording devices using tape and disc recording medium. This phonograph cylinder dominated the recorded sound market beginning in the 1880s. Lateral-cut disc records were invented by Emile Berliner in 1888 and were used exclusively in toys until 1894, when Berliner began marketing disc records under the Berliner Grammophone label. Berliner’s records had poor sound quality; however, work by Eldridge R. Johnson improved the fidelity to a point where they were as good as cylinders. Johnson’s and Berliner’s companies merged to form the Victor Talking Machine Company, whose products would come to dominate the market for many years. In an attempt to head off the disc advantage, Edison introduced the Amberol cylinder in 1909, with a maximum playing time of 4 minutes (at 160 rpm) to be in turn superseded by the Blue Amberol Record whose playing surface was made of celluloid, an early plastic that was far less fragile than the earlier wax (in fact, it would have been more or less indestructible had it not been for the plaster of Paris core). By November 1918, the patents for the manufacture of lateral-cut disc records expired, opening the field for countless companies to produce them, causing disc records to overtake cylinders in popularity. Edison ceased production of cylinders in 1929. Disc records would dominate the market until they were supplanted by the compact disc, starting from the early 1980s. 21.2.3 The Digital Approach, Sampling Philips and Sony have used digital encoding since 1982 upon recommendation by Herbert von Karajan. It is remarkable that his acceptance of the CD quality (characterized by a 20 kHz upper frequency limit in the Fourier spectrum) was decisive for these companies, although Karajan’s age (around 50 then) was not ideal as a reference for faithful sound perception. The hardware display of a CD is shown in Figure 21.21. The CD’s laser reads the land and pit levels engraved upon the polycarbonate carrier as bit sequence of zeros and ones. The transformation from the analog soundwave to the digital representation on the CD is explained in Figure 21.22. The wave is quantized in two ways: First, the sound’s amplitude is quantized by 16 Bits1 . This means that we are given 216 = 65, 536 values (0, 1, 2, . . . 65, 535) defined by the binary integer representation b15 215 + b14 214 + . . . b1 21 + b0 20 , bi = 0, 1. Second, the (quantized) values of the wave are only taken every 1/44, 100th of a second, i.e. the sample rate is 44, 100 samples per second. In total, this gives 635 MByte, 1 Byte = 8 Bit CD capacity for one hour stereo recording. We shall see in the next subsection that this allows for partials up to frequency 44, 100/2 = 22.05 kHz. The human ear is known to perceive up to 20 kHz. 1

A one-Bit quantization would allow for just two values, 0 or 1.

192

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.21. The hardware display of a CD. The CD’s laser reads the land and pit levels engrained upon the polycarbonate carrier as a bit sequence of zeros and ones.

Karajan, in his fifties, could very probably not hear more than 15 kHz, so for him the quality of the CD was perfect. A young human however would have asked for higher resolution. In recent technology, a sample rate of 96 kHz with amplitude quantization of 24 Bit is being envisaged. This looks like the endpoint of a long development of sound conservation and transfer, for which the 200 billion CDs sold by 2007 is a good argument. However, the creative argument comes from the critical concept: the container unit of music, the CD. What is a wall thereof? First, its material part, this disc, why should this be the container? And second, why should music be transferred using such a hardware container? Because the Internet has defined the global reality of information transfer since the early 1990s, the answer to the above questions has become straightforward: Of course music data transfer can be accomplished via Internet. This transfer would use the electromagnetic digital technology that we display in Figure 21.23. And then the unit of transfer would no longer be a collection, such as those 635 MByte containing a small number of musical compositions. This answer is, however, only theoretically valid because those 635 MByte are far too much to be transferred in reasonable time by the present Internet performance. We shall see in the following sections that the solution of this performance problem has been an odyssey from rejection of patents and disinterest of industry to the celebrated integration of new ideas in the now commonly known MP3 format.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

193

Fig. 21.22. The transformation from the analog soundwave to the digital representation on the CD.

21.2.4 Finite Fourier Analysis So let us first recall the Fourier formula for a sound function w(t) of frequency f and look at its structure:  An sin(2π.nf t + P hn ). w(t) = A0 + n≥1

It has two drawbacks: 1. In real life, a (non-zero) function is never strictly periodic. In particular, it cannot last forever, and we cannot know it except for a finite time interval. 2. It is not possible to recognize the function’s value at every time; this would be an infinite task. In real life we deal with functions that have a finite interval of definition, and we can measure only values for a finite number of times. This is, of course, not what must be offered to a Fourier setup. The solution of this situation is as follows. We are given a function w(t) that is only defined in a finite interval of length P . No periodicity is presupposed for this function—it could be a short sound recording of 0.5 seconds (see Figure 21.24). To turn this data into a Fourier situation, we have to create a periodic function from w(t). This is done by continuing the function by copying it infinitely to intervals of same length P to the left and to the right of the original interval such that the period of

194

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.23. Transfer modes of analogue sound using the electromagnetic digital technology.

Fig. 21.24. To turn general sound data into a Fourier situation, we have to create a periodic function. This is done by copying the function infinitely to the left and to the right of the original interval such that the period of this continuation is P by definition.

this continuation is P by definition. This creative solution starts by opening the wall of nonperiodicity of the given function. It is not periodic, yes, but we are not interested in anything else but the representation of this function in its original domain of definition. So whatever we have outside does not matter. And the solution is to take the continuation of the function by shifted copies,

21.2 Electromagnetic Encoding of Music: Audio HW and SW

195

so the result is periodic and coincides with the given function in its original domain. The second problem is dealt with by measuring only finitely many values of w(t) in the original interval (see Figure 21.25). It is related to a technically reasonable procedure where the function values are measured in regular time periods Δ. This Δ is called the sample period. The frequency 1/Δ is called the sample frequency. The total sample period P is a multiple of the sample period, P = N Δ = 2nΔ, which means that we assume the multiplicity of the sampling action is even, N = 2n. This will be important later in section 21.2.5, when we discuss Fast Fourier calculation.

Fig. 21.25. Measuring only finitely many values of w(t) in the original interval.

So the situation is this: We have fabricated a periodic function with frequency f = 1/P , and we have to find a Fourier representation that coincides with all the values measured in the 2n sample times. This is a mathematical problem relating to a Fourier formula that satisfies the preceding conditions. To solve this problem, we use the goniometric equation  a. cos(x) + b. sin(x) = A. sin(x + arccos(b/A)), where A = a2 + b2 . Using this equation in the Fourier equation for the overtone expression Am sin(2.πmf t + P hm ), we can write the Fourier equation at time t = rΔ:  w(rΔ) = a0 + am cos(2π.mf.rΔ) + bm sin(2π.mf.rΔ) + m=1,2,3,...n−1

bn sin(2π.nf.rΔ). The 2n equations for r = −n, . . . n − 1 are linear in the 2n unknown coefficients a0 , am , bm , bn . It can be shown, using the so-called orthogonality relations

196

21 Acoustics, Instruments, Music Software, and Creativity

of sinusoidal functions [80, section 38.8], that these equations have exactly one solution. This mean that we can find exactly one Fourier formula that solves these conditions. The formula has its maximal overtone at frequency nf = N f /2 = 1/(2Δ), which is half the sampling frequency. This result is known as: Theorem 1 (Nyquist’s Sampling Theorem) Given a wave w(t) defined in a period P of time as above, if we measure its values every Δ sample period, dividing the period P into N = 2n intervals, then there is exactly one set of coefficients a0 , am , bm , bn in the above finite Fourier expression for all sample times rΔ, r = −n, . . . n − 1. The maximal overtone of this expression has index n = N/2, half the sample index N . Its frequency is half the sample frequency nf = N f /2 = 1/(2Δ); it is called the Nyquist frequency. This result applies to the CD sampling at 44.1 kHz. The Nyquist frequency is half of it, 22.050 kHz, as already announced above. Let us make another simple example: Suppose we have a wave of period P = 20 seconds and a sample frequency of 80 Hz. This means that Δ = 1/80 seconds and N = 80 × 20 = 1, 600. The fundamental frequency is f = 1/P = 0.05 Hz. How many overtones do we have in the finite Fourier analysis? We start with the fundamental at f , and go to the overtone with index 1, 600/2 = 800, having frequency 800 × 0.05 = 40 Hz. The Nyquist theorem is the connection between general mathematical Fourier theory using infinite sums and infinite information about the waves, and the practical theory that is based upon finite information about the limited extension of the not-periodic wave function in time and the finite number of practically possible samples of this function. This finite Fourier theory does solve the conflict addressed in the beginning of this section. But it does not solve the question about the time needed to calculate the Fourier coefficients if the sample number N is large. For example, if we are given the CD sampling rate of 44, 100 per second, and if we have a period of P = 10 minutes, we get 44, 100 × 600 = 2.6460 × 106 equations, and the calculation of its solution involves an order of the square 7.001316 × 1012 of arithmetical operations. This is beyond reasonable calculation power. In the next section, we shall solve this major problem. 21.2.5 Fast Fourier Analysis (FFT) Fast Fourier Transform (FFT) is an algorithm that enables faster calculation of Fourier coefficients in the finite Fourier theory described above. This approach however is not easily accessible by the classical description of Fourier’s formula using sinusoidal functions. We have to prepend a short discussion of using complex numbers to restate the formula. Then we shall be able to understand the beauty of FFT.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

197

21.2.5.1 Fourier via Complex Numbers

Fig. 21.26. The complex representation of sinusoidal functions.

In the discussion of the Nyquist theorem, we have restated the Fourier repesentation of the periodic wave function w(t) in the shape w(t) = a0 + a1 cos(2π.f t) + b1 sin(2π.f t) + a2 cos(2π.2f t) + b2 sin(2π.2f t) + . . . This representation must be replaced by a much more elegant representation involving complex numbers. Recall that a complex number is just a vector 2 x = a + i.b √ in the plane R of real numbers (with coordinates a, b ∈ R), where i = −1, and besides the usual coordinate-wise addition, we have a multiplication defined by (a + i.b) · (u + i.v) = (au − bv) + i(av + bu), using i2 = −1. This enables a complex representation of sinusoidal functions as shown in Figure 21.26. The idea is to represent the sinusoidal values as the two coordinates of an arrow of length one rotating around the origin, and where time is parametrized b the angle of the arrow with the x axis. The deeper advantage of this reduction of sinusoidal functions to rotating arrows is that this arrow’s head satisfies Leonhard Euler’s famous equation cos(x) + i. sin(x) = ei.x  n where ez = n≥0 zn! is the exponential function. The advantage of this exponential representation is that the exponential function has a beautiful property: namely, it transforms addition of arguments into the product of values: ez+w = ez · ew . This is the basis of all apparently complicated formulas for sinusoidal functions. The creative idea behind this Eulerian result is to (1)

198

21 Acoustics, Instruments, Music Software, and Creativity

connect cos(x) to sin(x) in the complex vector, and (2) to use the complex arithmetic for the combination of such complex vectors. The relationship between sinusoidal functions and the exponential is this: cos(x) = (eix + e−ix )/2 sin(x) = (eix − e−ix )/2i. This is used to restate the Fourier formula in terms of exponential expressions as follows:  w(t) = cn ei2π.nf t n=0,±1,±2,±3,...

with the new coefficients cn relating to the old ones by a0 = c 0 , and for n > 0, an = cn + c−n bn = i(cn − c−n ). With this data, the above finite Fourier equation for w(rΔ) looks as follows: In order to keep ideas transparent, we only look at the special case of a sound sample of period P = 1 from t = 0 to t = 1 , such that Δ = 1/N and rΔ = r/N for r = 0, 1, . . . N − 1. We may then write  cm ei2π.mr/N . wr = w(rΔ) = w(r/N ) = m=0,1,2,3,...N −1

Why are the negative indices gone here? They are not, but they are hidden in the equation ei2π.mr/N .ei2π.m(N −r)/N = ei2π.mr/N +i2π.m(N −r)/N = e0 = 1, which means

ei2π.m(N −r)/N = ei2π.(−m)r/N ,

where −m is a negative index! Also, cN −m is the complex conjugate to cm since am , bm are all real numbers. Therefore we have a total of N/2 independent complex coefficients, i.e. N real coefficients as required from the original formula. This enables us to represent the sample sequence with the vector w = (w0 , w1 , . . . wN −1 ) ∈ CN in the N -dimensional complex vector space CN . In this space, we have a canonical scalar product of such vectors u, v (similar to the high school formula) u, v =

N −1 1  ur v r . N r=0

21.2 Electromagnetic Encoding of Music: Audio HW and SW

199

We have exponential functions em = (em (r) = ei2π.mr/N )r=0,1,2,...N −1 and their scalar product is em , em  = 1, em , eq  = 0, m = q, which means that they define an orthonormal basis of CN (see Figure 21.27). These relationships are the orthogonality relations mentioned above. These functions replace the sinusoidal functions in the complex representation. Every sound sample vector w = (w0 , w1 , . . . wN −1 ) can be written as a unique linear combination (see Figure 21.27)  cm em , w= m=0,1,2,3,...N −1

and the coefficients cm are determined by the formula cm = w, em  =

1 N



wr e−i2π.mr/N .

r=0,1,2,3,...N −1

This writing in the complex vector space, using the exponential basis is the

Fig. 21.27. An orthonormal basis of CN .

Fourier theorem for such finite sums. It is much more elegant than the original one using sinusoidal waves, and the calculation of the Fourier coefficients (essentially the amplitude and phase spectra) is very easy. We hope that the reader can appreciate this beautiful simplification of the theory through the step to the complex numbers.

200

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.28. The FFT trick: To split the vector w = (w0 , w1 , . . . w2N −1 ) ∈ C2N into its even and odd parts w+ , w− .

21.2.5.2 The FFT Algorithm The FFT algorithm was invented to calculate the Fourier coefficients cm faster than in the quadratic growth algorithm that is given by the solution of the above orthogonality relations. The FFT was discovered by James W. Cooley (IBM T.J. Watson Research Center) and John W. Tuckey (Princeton University and AT&T Bell labs) and published in 1965 in a five-page (!) paper, “An Algorithm for the Machine Calculation of Complex Series” [27]. It has become one of the most cited papers of all times. It is remarkable that the algorithm was already discovered by the great German mathematician Carl Friedrich Gauss in 1805 for astronomical calculation, but it was not recognized because he wrote the paper in Latin. (This might be amusing, but it also teaches us to share creations in languages that are communicative. It is interesting that many great creators do not care about this and even avoid any straightforward communication of their findings. The psychology of such hidden creativity has not been written to this date.) Compared to the quadratic growth N 2 defined by the orthogonality relations, this FFT algorithm grows with N. log(N ). The idea is very simple but ingenious. The trick consists of taking the sample vector w = (w0 , w1 , . . . w2N −1 ) ∈ C2N and splitting it into its even and odd parts w+ , w− as shown in Figure 21.28. Supposing that we have calculated + + + − − − − = (c− the coefficient vectors c+ = (c+ 0 , c1 , c2 , . . . cN −1 ), c 0 , c1 , c2 , . . . cN −1 ) with a growth F ourier(N ), we can calculate the coefficient vector c of w using r − the formula cr = (c+ r + e(N ) .cr )/2. This latter formula permits a calculation F ourier(2N ) ≤ 3 × 2N + 2N + 2.F ourier(N ) = 2.F ourier(N ) + 8N . This is a recursive formula that is used to show that the growth can be set to N. log(N ). The detailed mathematics here is not so important; the question is rather how this problem could be solved in such a simple way. What was the critical concept, what was the wall? The critical concept is clearly the formula for the calculation of the coefficients cm . Nothing suggests that this formula consists of sub-information that can be calculated separately and yield the desired values. Once we are told that such a procedure is possible, the solution is not difficult. The solution is just the splitting of the wave w into its even and odd parts.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

201

Then, once the even and odd coefficients have been calculated, the connecting r − formula cr = (c+ r + e(N ) .cr )/2 yields immediately the required result. So the creative part of this work is not the mathematics, but the unbelievable fact that everything can be solved recursively from even and odd parts. 21.2.6 Compression In order to appreciate the quality of MP3 compression, we have to briefly discuss the concept of compression in computer file formats for digitized audio data. Such formats contain not only the audio-related data, but additional technical information about the encoding method of the raw data, and data concerning the composer and other poietic information. This additional information can be complex according to the actual method. We shall see for MP3 that the encoded data stream is in itself a virtuosic achievement invented to enable playing a piece starting at any time moment of the piece2 . The additional data strongly depend on compression methods invented to reduce the saved data with respect to the original audio data. By definition, compression for a determined file format is the ratio of two quantities: original data/file data. For example, if the original data is a 10-minute CD and the file needs to be 10 MB, what is the compression? Answer: 44.1 × 103 Hz × 16 Bit × 2 channels × 10 min × 60 sec/107 × 8 Bit ≈ 10.58. There are two types of compression: • •

Lossless compression. This means that the original data can be completely reconstructed from the compressed ones. Lossy compression. This means that the original data cannot be completely reconstructed from the compressed ones.

Here are two lossless compression methods also used in MP3: •

•

RLE = Run Length Encoding. It consists of replacing a Bit sequence (such as black and white pixels in a black-white TV screen) with the sequence of numbers of uninterrupted occurrences of a determined Bit. Huffman. This method is described in Figure 21.29. We start with a number of values, here six (step A), and list their frequencies. Label the two least frequent values 0,1 (step B), then add the frequencies of the two least frequent values and define a new value * (step C). Start over with the new values and frequencies, append the 0.1 to 12:* at the end of the algorithm (step D). And here are two examples of lossy compression:

• Analog to Digital Conversion (ADC). From analog to CD with 44,100 Hz and 16 Bit amplitude digitization. This has an infinite rate. 2

Remember that Fourier’s coefficient calculation needs all time moments!

202

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.29. An example of Huffman compression.

•

Quantization. In the 16-Bit representation of amplitude, the five last Bits are neglected. So, for example, 1011 1100 0101 1110 is transformed to 1011 1100 0100 0000 and then shortened to the 11 relevant Bits left, 1011 1100 010.

21.2.7 MP3, MP4, AIFF Figure 21.30 shows the best-known digital audio file formats, together with their compression types. Let us discuss MP3, the most sucessful format. MPEG (Moving Pictures Experts Group) is the code name for the standardization group ISO/IEC JTC1/SC29/WG11 (Int. Organization for Standardization/International Electrotechnical Commission). It was created in 1988 to generate generic standards for encoding digital video and audio data. MPEG-1 is the result of the first work phase of the group and has been established in 1992 as standard ISO/IEC IS 11172. It contains Layer-1, Layer-2, and Layer-3, which means three operation modes with increasing complexity. By MP3 one denotes Layer-3 of MPEG-1. MPEG-2 Advanced Audio Coding (AAC) is the result of the second work phase. In enhances Layer-3 in many details. We are not going to discuss this phase here. The development of these compressed audio formats goes back to research by Dieter Seitzer since 1960, who at that time worked at IBM, and his student Karlheinz Brandenburg, who is above all responsible for the psychoacoustical compression methods. It is remarkable that Seitzer’s patent was rejected in 1977; however it was awarded in 1983 but was then suspended because of lack of interest from the industry! MP3 is above all based on research and development

21.2 Electromagnetic Encoding of Music: Audio HW and SW

203

Fig. 21.30. The best known digital audio file formats, together with their compression types.

by Brandenburg at the Fraunhofer Institut f¨ ur Integrierte Schaltungen (IIS) in Erlangen. It is an open standard, but it is protected by many patents (more than 13 US patents, and more than 16 German patents). We shall discuss legal aspects at the end of this topic. MP3 includes these options: • •

•

Mono and stereo, in particular joint stereo encoding for efficient combined encoding of both stereo channels. Sampling frequencies include 32 kHz, 44.1 kHz, 48 kHz; for MPEG-2 also 16 kHz, 22.5 kHz, 24 kHz; and for MPEG-2.5 (Fraunhofer-internal extension) also 8 kHz, 11.5 kHz, and 12 kHz. The compression’s bitrate (= the Bits traversing the audio file per second) goes from 32 kBits/sec (MPEG-1) or 8 kBits/sec (MPEG-2) up to 320 kBits/sec. For MP3 the bitrate can even vary from frame to frame (a frame is the unit package in MP3, we come back to it later) and, together with the so-called bit-reservoir technology, allows a variable as well as constant bitrate.

Besides compression MP3 has the advantage of being platform-independent. This is also a strong reason for its popularity. Let us now look at the MP3 encoder chain (the decoder works symmetrically, we don’t discuss it for this reason). Figure 21.31 shows the original overall flowchart of audio information. 1. From the left, the digital datastream defines

204

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.31. The MP3 encoder chain with its core Perceptual-Audio-Coding device.

the system’s input; the quantity is given by 768 kBit/sec ≈ 48, 000×16 Bit/sec. In the region 2, a FFT processing of the digital signal is applied, yielding a number of Fourier coefficient banks (filter banks—we discuss below what is a filter, but essentially it just selects a range of Fourier coefficients). We cut the spectrum 0 − 20 kHz into 32 subbands of 625 Hz each (32 × 625 = 20, 000) for 1/40 sec. windows. We then use MDCT (Modified Discrete Cosine Transformation, a variant of FFT) to split each 625 Hz band into 18 subbands with variable widths, according to psychoacoustical criteria. We then get 576 = 18 × 32 “lines.” Region 3 is the heart of the MP3 compression, covering 60 percent of the compression. It operates by elimination of physiologically superfluous information following the so-called Perceptual-Audio-Coding Model (PAC). PAC has three components: PAC 1—hearing thresholds; PAC 2—auditory masking; PAC 3—temporary masking. PAC 1, hearing thresholds, uses the fact that there is a loudness threshold under which the human ear does not hear sounds. The curve is shown in Figure 21.32. Therefore we may eliminate Fourier components (overtones) with amplitudes below these thresholds. PAC 2, auditory masking, uses the fact that for a frequency component there is a curve under which no other component can be heard. Figure 21.33 shows the situation for 2, 4, and 8 kHz components. PAC 3, temporary masking, uses the fact that for every sinusoidal frequency component of frequency f and loudness l, another subsequent component cannot be heard below the given curve of loudness in time, because the

21.2 Electromagnetic Encoding of Music: Audio HW and SW

205

Fig. 21.32. Hearing thresholds: One uses the fact that there is a loudness threshold under which the human ear does not hear sounds.

ear needs some time to “recover” from that first component’s perception. This is even true before (!) the given one, because the perception needs to be built up. Figure 21.34 shows the curve. In point 4 and 5 of the flowchart, we have the lossy quantization and lossless Huffman compressions discussed above—they make up the remaining 40 percent of compression. The final output is shown in region 6, where the MP3 frames are also built. We discuss them now. To begin with, the MP3 file is headed by a MP3 file identifier, the ID3 tag. It has 128 Bytes and its information is shown in Figure 21.35. After this header, a number of frames is added. Here is the concept: A frame is an autonomous information package. This means that all encoding data is provided within every frame to enable playing a file from any given time onset. A frame’s duration is 1/38.28125 ≈ 1/40 sec. This enables virtually continuous playing for humans. Each frame has these parts: •

a 32-Bit header indicating the layer number (1-3), the bitrate, and the sample frequency; • the Cycle Redundancy Check (CRC) with 16 Bits for error detection (without correction option) but frame repetition until correct frame appears;

206

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.33. Auditory masking uses the fact that for a frequency component, there is a curve under which no other component can be heard.

Fig. 21.34. Temporary masking uses the fact that for every sinusoidal frequency component of frequency f and loudness l, another subsequent component cannot be heard below the given curve of loudness in time, because the ear needs some time to “recover” from that first component’s perception.

• •

12 Bits for additional information for Huffman trees and quatization info; a main data sample block of 3,344 Bits for Huffman-encoded data.

Each frame has a 32-Bit header containing the information shown in Figure 21.36. The frame structure is given as shown in Figure 21.37. Each frame has a header and a reservoir of 3,344 Bits. However, the reservoir technique allows information to be placed in reservoirs that are not filled up yet. So data for block n might be saved in block n + 1. This is called the reservoir technique.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

207

Fig. 21.35. The MP3 file identifier, the ID3 tag, has 128 Bytes.

Fig. 21.36. Each frame has a 32-Bit header containing the information shown here.

There are two important formulas regarding frame capacities for MP3. The fixed data is this: 1. number of frames/sec = 38.28125 2. maximal audio data capacity per frame = 3,344 Bit/frame 3. number of frequency bands = 32 The first formula is this: The maximal bitrate 3,344 Bit/frame × 38.28125 frame/sec = 128 kBit/sec guarantees CD quality. And the second one is this: (44,100 time-sample/sec)/(38.28125 frame/sec) = 1,152 frequency-samples/frame guarantees CD quality. And this yields 1,152/32 = 36 frequency-samples/band. Observe: 625 Hz/band/38.28125 Hz = 16.3265 frequency-samples/band. We have overlapping info, but this is OK to minimize measurement errors.

208

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.37. The MP3 frame structure.

Some performance values are shown in Figure 21.38. Let us add some remarks regarding the joint stereo encoding: MP3 implements the Joint Stereo Coding compression method, which is based on these two principles: •

•

Mid/Side Stereo Coding (MSSC), where instead of taking the left and right channels (L, R), we use the equivalent data (L + R, L − R)—since L and R are usually strongly corrupted, the difference is quite ‘tame’. Intensity Stereo Coding (ISC), where the sum L + R and the direction of the signal are encoded (replacing the L − R information).

This coding method also uses the fact that the human ear is weak in localizing deep frequencies. Since the direction is detected by phase differences that are difficult to retrieve for deep frequencies, they are encoded mono! Let us end the discussion with some commercial information: The license rights of Fraunhofer IIS are represented by the French company Technicolor SA, formerly Thomson Multimedia. Here are the figures: • • •

0.50 USD per decoder 5.- USD per encoder 15,000.- USD annual lump sum

This means that an enterprise that sells 25,000 copies annually of the encoder software, pays 25,000 × 5.- + 15,000.- = 140,000.- for the first year and then 15,000.- in annual fees for every successive year.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

209

Fig. 21.38. Some MP3 performance values.

21.2.8 Filters and EQ In our discussion of finite Fourier methods, we had encountered that clever trick that transforms a non-periodic function in a time interval of length P (the period) into a periodic function by simply juxtaposing the shifted function to the left and right of the original function. This might be clever if we have a finite time interval of this function. But what happens to the function if it is extended to infinity and is not periodic? Then that method fails.

Fig. 21.39. Letting a finite frequency go to zero without losing the formal power of the original theory.

Mathematicians have however been able to solve this problem by a very adventurous idea: Why do we have to suppose a finite period for our theory? Is Fourier only feasible for finite periods? In other words, why couldn’t we try to extend the theory to infinite periods? This is a typical situation of a mathematical problem of creativity. The critical concept is the Fourier sum given from a P -periodic function w(t). The wall is the index of the partials, which is a discrete quantity. The problem here is to ask for a Fourier formula whose overtones are not only reasonable for finite, but also for infinite periods. This means that we are looking for a Fourier formula for frequency f → 0. This is a very mathematical situation! It is about reshaping a given expression or concept until its shape is such that a seemingly indispensable limitation is absorbed in a more general setup.

210

21 Acoustics, Instruments, Music Software, and Creativity

In our context we are fighting against the problem of letting a finite frequency go to zero without losing the formal power of the original theory (see Figure 21.39).

Fig. 21.40. The correspondence between the product of frequency functions and the convolution of their corresponding time functions.

The solution of this limit problem, the transition for a finite frequency to the vanishing frequency, works because the limit of a sum is, if mathematically correctly worked out, the integral, and the finite frequency converges in an infinitesimal frequency. Evidently, under this transformation, there will no longer be a discrete multiplicity of a fundamental frequency, but an entire continuous variety of frequencies that describe the Fourier formula. And here is the result. We use also the usual symbol f (t) for the function (instead of w(t)), and ν for the frequency (instead of f ):  ∞ 1 f (t)e−iνt dt. F(f )(ν) = √ 2π −∞ This function F(ν) of frequency ν is called the Fourier transform of f . It is a function of frequency instead of time. Figure 21.40 gives an overview of the power of this new Fourier transform formula.

21.2 Electromagnetic Encoding of Music: Audio HW and SW

211

Fig. 21.41. A filter is a function that alters an audio signal on the basis of its Fourier transform.

Here are the properties: F defines a linear isomorphism from the space of all time functions (right) (with some technical properties that are not interesting in our context) onto the space of all frequency functions (left). The inverse of F is denoted by F ∗ . For example, the sinusoidal function sin(ωt) is transformed into the function that is 0 except at frequency ν = ω, where it is 1. So the sum of sinodial functions is transformed into a sum of such spike functions, which symbolize the Fourier overtones with their amplitudes. Furthermore, by the inverse F ∗ , the pointwise product F · G of two frequency functions F,G ((F · G)(ν) = F (ν) · G(ν)) is transformed into the convolution (f ∗ g)(t) = ±∞ f (x)g(t − x)dx.

Fig. 21.42. The filtering process in the frequency domain for a time function f (t).

This means that we can use pointwise products in the frequency domain to cut functions. This is precisely the fact we need to understand filters! A filter

212

21 Acoustics, Instruments, Music Software, and Creativity

is a function that alters an audio signal on the basis of its Fourier transform (see Figure 21.41). The filtering process for a time function f (t) consists first in a Fourier transform F = F(f ) of f , and then we are given a filter function G(ν) in the frequency domain. It is shaped in such a way as to eliminate a number of frequencies from F(f ) by pointwise multiplication, yielding F · G, and then to transform the result F · G back to the time domain, yielding fG = F ∗ (F · G) = f ∗ F ∗ (G) (see also Figure 21.42).

Fig. 21.43. Frequent filter types.

In audio technology there are a number of special filters that are given standard names: low pass, high pass, low shelf, high shelf, band pass, notch. Their filter function is shown in Figure 21.43. The meaning of “low pass” is that only low frequencies, below a determined cutoff frequency, can “pass” higher frequencies are cut off. A low shelf filter passes all frequencies above a cutoff frequency, but increases or reduces frequencies below the cutoff frequency by a determined quantity. Etc. In audio technology, the representation and management of filters is realized in two ways: as a graphical EQ and as a parametric EQ, EQ stands for “equalization” and just means filtering using complex filter functions. Figure 21.44 shows these two types in commercial software. Let us conclude this filter topic with a practical remark. Filters are not only used as abstract devices, but also occur in very practical contexts. Two

21.2 Electromagnetic Encoding of Music: Audio HW and SW

213

Fig. 21.44. Two types of equalizers, graphic and parametric, in commercial software

simple examples are walls and electrical circuits (see Figure 21.45). If a sound is produced in a room, it propagates to the room’s walls. There, it is reflected but also travels through the walls. The sound that arrives in the rooms adjacent to the room where the sound is produced is (hopefully) very different from the original one. The walls usually act as low pass filters. This means that the sound in the adjacent room(s) contains a certain amount of only low frequencies. This is made clear when we think of the bass sound in a commercial music sound. In the adjacent rooms, you will (have to) hear the bass, an effect that can be very disturbing at times. The second example, a special electrical circuit shown in the lower part of Figure 21.45, is capable of realizing the low pass filter in an electrical setup. It takes the input as a voltage Vin and “filters” it by a resistance R and a capacity C. The capacity induces a low pass filter in that it only transmits high frequencies, and therefore the low frequencies pass at Vout . The cutoff frequency is 1/2πRC. It should be remarked that the use of electrical circuits to simulate acoustical configurations is a powerful standard method in acoustics, especially acoustics of musical instruments. It serves as a methodology of representation in acoustics, but also as a means for the construction of acoustical effects on the level of electrical sound hardware technology.

214

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.45. A special electrical circuit shown in the lower part of the figure is capable of realizing the low pass filter in an electrical setup.

21.3 Symbolic Formats: Notes, MIDI, Denotators It is not true that musical objects are just acoustical and that anything that is used to notate music refers to these acoustical objects. This misunderstanding was virulent in the middle of the twentieth century when the electronic musicians discovered the precise structure of acoustical sounds, above all in the German school centered around the Studio f¨ ur elektronische Musik where Herbert Eimert tried to downsize all musical notation to the acoustical one, arguing that its precision would make all other notations superfluous. Thinking in symbols that are not just short writings for acoustical data has a long tradition in musical thought. The difference between symbols and acoustics is described by performance transformations. Scores are symbolic data; they are not acoustical. For example, a quarter note can have any physical duration, depending on the tempo chosen for that performance. In John Cage’s famous composition “ASLSP” (as slow as possible) [21], the total duration is 639 years, so the physical duration of notes in a score is totally relative. Notes live in a symbolic space, which is independent of the physical performative realization. In the following sections, we shall discuss a number of relevant symbolic notations: Western notation and the performance problem, then the main technological format, MIDI (Musical Instrument Digital Interface), and denotators, our own universal format implemented in the RUBATO software [74]. 21.3.1 Western Notation and Performance We do not have to recapitulate the history or the elementary properties of Western notation, but we should observe some of its symbolic prerequisites. To

21.3 Symbolic Formats: Notes, MIDI, Denotators

215

begin with, the staff system allowing for positioning note heads on or between the five lines (and lower or higher ledger lines) is encoding the diatonic scale from C and the third distances (between lines and line interspaces). This diatonic scale can be used to define alterations several semitones downward by s, or upward by s. It is however not clear from this notation what is the basic tuning of such pitch information. The diatonic scale as well as the alterations can relate to 12-tempered, just, or any other tuning. Also, the alteration is not simply an indication of the resulting lowered of heightened pitch, but also of the movement (vector) from the original pitch to the altered one. Relating to the time dimension, the notation of onset and duration is somewhat ambiguous because the onset is only fixed by the bar lines and, more relatively, for notes pertaining to chords. The other onsets must be determined from pauses and relative positions. While onset and duration are independent parameters, they are symbolic, and thus their physical values are quite difficult to be determined. One information in this regard is the tempo, which encodes the inverse derivative of physical time as a function of symbolic time, see [89, chapter 6] for details. However the tempo only encodes the onset of the beginning of a note, not its end, the offset. This can be encoded using the formula offset = onset + duration. But this is only valid if there is no articulation to be performed, articulation is a thoroughly symbolic information that has to interpreted by the performer in order to obtain a physical representation—or if the tempo curve is well defined between the onset and the offset in the score. Other symbolic time information might be ambiguous, such as the fermata. Its tempo-related information is not well defined; it is a symbolic sign much as the pitch position of notes is. If we look at dynamical information, absolute and relative, then we have the same situation, but even more dramatic since there is little agreement about what an mf should mean in terms of dB. Also are these symbols not precise in the sense that they designate one determined physical value as pitch would be. Dynamical signs are more like intervals of dynamics, and even overlapping ones, as there are situations where some parts in mf could have the same loudness as parts in f. All these score symbols are parts of a symbolic language where sound parameters are expressing more or less huge abstractions from physical reality. The music theory dealing with the relationships between these symbols and their physical (instrumental) realization is called performance theory and has been an intense research topic in traditional investigations as well as in contemporary research involving quantitative methods and specifically computer-aided models, see [89] for a recent documentation. The creative aspect of this symbolism is a very sophisticated one, and it has evolved in the course of the historical development of notation. What is the general problem leading to the development of such a notational sign system? Writing score symbols has never meant to represent all of the music; it was always understood that musical notation is an abstraction. So the problem was to find symbols of abstraction from the sounding reality of music. It is interesting that the Western culture of music has converged to the concept of

216

21 Acoustics, Instruments, Music Software, and Creativity

a composition by means of its trace in notated form. A composition is written, but it is not the entire music. Notation does not pretend to represent the sounding music but only certain aspects that either have a theoretical relevance or are meant to be reproducible. Therefore the creative wall inspection here for the critical concept of “notation” is about the parametric representation of the sound to be symbolized. The original solution of this problem was to encode the sound by gestural signs, the (cheironomic) neumes (see Figure 21.46).

Fig. 21.46. Neumes above text from Gregorian chant.

The function of a score to grasp the gestural rendition was then more and more reduced to the essence that could be reproduced and was thought to be essential in its memorization. The invention of neumes was a first-order creative act since these symbols could be deformed ad libitum (gestures are elastic!) until their substance was cast in the graphical shape needed to express the thought’s essence. The subtlety of neumatically driven abstraction, leading to the present notation, is that even after this huge geometrization, notes are still representing gestures. Following a modern insight by the author, and also confirmed by Theodor W. Adorno’s writings about performance [1], they are “frozen gestures.” Performers know that playing a composition from a score means to thaw these frozen gestures until they recover their original “temperature.” This means that whenever we deal with a Western score, we implicitely think about its gestural kernel. This is the creative solution of the notational problem: The notational symbols always represent compact gestures. Although they look like symbols, they can and must be opened to show their gestural essence in a successful performance. In the course of the history of notation in musical composition, the successive enrichment of notation is a known developmental path. Viewed superficially, it reflects a successive integration of performative commands into symbolic representation. This surface, however, is only functional for the performer, because in reality, it drives toward the completion of the gestural essence of notation, i.e. notation is always a vector toward performance by gestures— nothing more, nothing less. This also means that the direction of this vector

21.3 Symbolic Formats: Notes, MIDI, Denotators

217

shapes what is not written, namely the performative action, which we will identify with improvisation in chapter 22—namely all that is not preconceived (and fixed in notation) when sounding music is played. Summarizing, Western notation is not symbolism for its own sake, but a wrapping and compactification of performative gestures. However, understood as a surface of forms, it can be taken as a playground for composers, who manipulate the notational symbols as if they were without performative substance. This is definitely a creative usage because the performative substance can also act as a wall against formal manipulation. And it is also historically relevant that notation was first motivated by theory, not only as a mnemonic device. The symbolism for theory is essential for the compositional strategies when they are driven by knowledge bases. Obviously, we have a schism here between a theory that is built upon abstraction, such as parameter spaces and transformations dealt with in mathematically structured theories, and performative gestuality. Only in the last decade has it become possible to think about music theory as a coherent body of forms that treat abstraction and gestuality using the same basic formalism, see [84, 87]. 21.3.2 MIDI: What It Is About, Short History MIDI was officially introduced in January 1983 at the NAMM (National Association of Music Merchants) with its “MIDI 1.0 Specification”3 . It replaced the first sketches of a “Universal Synthesizer Interface” from 1981, which had been presented by Dave Smith and Chet Wood at the 70th Convention of the Audio Engineering Society. It is interesting that it was no longer the academic Audio Engineering Society but the thoroughly pragmatic community of music merchants who remained at the wheel. Also, no music-theoretical community was involved in this development, but of course they later complained about the MIDI’s deficiencies. Standardization of music formats had never been a topic of music theory before mathematical and computational music theory started developing universal standards, see [77, 124, 125, 126]. It is important—and we have stated this already in section 21.2.5.2 regarding the FFT algorithm already discovered by Gauss, but only communicated in Latin—that creative actions be communicative. It is not reasonable to hide creativity or to abstain from participation in highly active specialized communities. In the following sections we describe MIDI communication and the structure of MIDI messages. These messages are exchanged between any two: computers and/or synthesizers. The general functionality of MIDI is shown in Figure 21.47. Making music is a threefold communication: We usually have a (1) score whose frozen gestures are being “thawed” to (2) gestures, which act on an instrumental interface and thereby produce (3) sound events. The map from score to sound events is the objective of performance research. The gestural 3

The MIDI Specifications are accessible through the MIDI Manufacturer’s Association at http://www.midi.org.

218

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.47. Making music is a threefold communication: We usually have a (1) score whose frozen gestures are being “thawed” to (2) gestures, which act on an instrumental interface and thereby produce (3) sound events.

interaction with an instrument (process from the left top position to the right top position in Figure 21.47) is where MIDI has its main functionality. The movements of the human limbs (hands for keyboard players) are encoded and then communicated to a synthesizer that produces corresponding sound events. MIDI is therefore essentially a simplified code for human gestures interfacing with an instrument. A second functionality of MIDI is the transformation of instrumental gestures to a Standard MIDI File such that the performance can be replayed later. Standard MIDI files can also be transformed to (electronic files for) Western scores, and vice versa. But the essence of MIDI is that it encodes commands of gestural nature, although in a very simple setup as we shall see in the following discussion. It is nota coincidence that machines that understand the MIDI messages are called slaves in MIDI jargon. MIDI just tells an agent (mimicking a musician) to do simple gestures at a defined time and with a specific key and instrument. It was probably the most ingenious creation in the MIDI code to refrain from abstract symbols and to encode the simple gestural music-playing action instead. The music industry was not interested in higher symbols but in a communication code that would help musicians play electronic instruments

21.3 Symbolic Formats: Notes, MIDI, Denotators

219

when performing onstage or in a studio. Again, we see here the schism between codes and thoughts in theory vs. performance. 21.3.3 MIDI Networks: MIDI Devices, Ports, and Cables

Fig. 21.48. MIDI messages are Bit sequences, and the ports connecting MIDI devices have three types: IN, THRU, OUT.

MIDI communication is done via MIDI cables. They accept MIDI messages at a Out port and pipe them either to an In port, where a machine uses the message, or to a Thru port, which makes the message available to an Out port of the same machine for further messaging to other machines. The three port types are shown in the lower center of Figure 21.48. The MIDI messages are Bit sequences sent at a frequency of 31, 250 Bits per second (= Bauds) with a current of 5mA. Equivalently, this is 1/32 of a MegaBaud. This means that the transfer of one Bit needs 32 μsec. Originally, such transfer was strictly serial (one Bit after the other), but nowadays there is also parallel data transfer. MIDI is a semiotic, and the MIDI Specification connects the expressions given by Bit sequences to their symbolic signification (see Figure 21.49). But we should be precise here: The contents of the MIDI messages are not directly musical. They just decipher the Bit sequences and yield information that a synthesizer could understand, but it is the synthesizer’s semiotic (!) that must transform the message contents to musically meaningful information. We will clarify this when looking at the message structure.

220

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.49. The semiotic role of MIDI Specification.

21.3.4 MIDI Messages: Hierarchy and Anatomy A MIDI message consists of a sequence of words, each having a length of 10 Bits (see Figure 21.50). These words are delimited by a zero startbit and a zero stopbit, and it takes 320 μsec per word, which means roughly a third of a millisecond. MIDI messages are sequences of a few words with a fixed content anatomy. The first word is the status word, and the status character of the word is initiated by the first Bit: the statusbit one. The word encodes two

Fig. 21.50. The MIDI message anatomy.

information units: (1) what is being played, encoded in three Bits; and (2) who is playing, encoded in four Bits. We will come back to these contents later. The second portion of the message is composed of a number of words.

21.3 Symbolic Formats: Notes, MIDI, Denotators

221

They are all—on the Byte (= 8 Bits) inside their start- and stopbits—initiated by the databit zero. These words encode information about how the actions, as defined by the status word, are played.

Fig. 21.51. The MIDI message types.

Let us first make the example shown in Figure 21.50. Here the first three Bits 001 encode the action of pressing down a key (Note-ON, the note’s onset action). The following four Bits 0101 tell us which “musician,” called channel in the MIDI code, is playing the Note-ON action; here it is channel 5, written in its binary representation 0101. With this information, the data words tell the musician which key to press down: The seven Bits in the first data word, namely 0111100, correspond to 60, the middle C on the piano. The second data word shows the seven-Bit binary number 1111001, corresponding to 61, and meaning “velocity.” Here we see the gestural approach: “Velocity” means the velocity with which a keyboard player hits key number 60, and this is a parameter for loudness. The data words can specify any number in the interval 0 − 127, so the velocity value is a middle loudness. The total time for such a message is three times 320 μsec, roughly one millisecond. This is not very short, and it can in fact happen that complex musical information flow can be heard in its serial structure. As announced above, these numbers are still quite symbolic. The pitch associated with a key number is not given in the MIDI code, so it is up to the synthesizer to interpret a key number in terms of pitch. And velocity must also be specified by the synthesizer in order to become a loudness value. This is a typical situation of a connotative semiotic system (see chapter 2): The MIDI Specification only decodes the Bit sequence, and then this result is taken as an expressive unit pointing to the acoustical contents via the synthesizer’s signification engine. The messages are divided into different categories (which are then encoded in the statusbyte). These categories are shown in Figure 21.51. There are two big categories of messages: channel and system messages. The former relate to single voices or their collaborative action, where the latter concern the entire system and are encoded by the statusbyte starting with 111 after the

222

21 Acoustics, Instruments, Music Software, and Creativity

Fig. 21.52. Some channel voice messages.

initial status index 1. Channel messages of voice type relate to single musicians (channels), while channel mode messages relate to the channels’ collaboration. Figure 21.52 shows some channel voice messages. You recognize the 1 as the initial Bit of the statusbyte. The action type is 1000 for Note Off, the databyte starts with the mandatory zero and then has the 128 possible values indicated by kkkkkkk. The two databytes define Key number and Velocity, as in our example above. Program Change, for example, describes the instrument that is going to be played. This is a typical situation where the meaning depends upon the synthesizer’s settings. For the so-called General MIDI standard, these numbers have a fixed meaning, 0 being reserved for acoustic grand piano and 127 being assigned to gunshot. The system messages are of three types: System Exclusive for messages that are reserved for specific information used by the industrial producer, like Yamaha, Casio, etc.; System common messages for general system information, such as the time position in an ongoing piece; and Real Time messages for time information (e.g. the timing clock that produces the ticks, see below about ticks). 21.3.5 Time in MIDI, Standard MIDI Files MIDI has a special treatment of time. It is not a physical time format, but counts time in multiples of ticks. The value of a tick is variable. Usually a tick means a 24th of a quarter note. But the physical duration of a tick must be defined on a special header. In the Standard MIDI Files, this header is called meta information and it tells us about the tempo that relates ticks to physical durations. Standard MIDI Files save MIDI informations in order to play a recording back on a MIDI device. The Standard MIDI File Formats define the syntax of such files. Let us look at a typical Standard MIDI Files to see what this syntax looks like in a concrete piece of music (see Figure 21.53).

21.3 Symbolic Formats: Notes, MIDI, Denotators

223

Fig. 21.53. A typical MIDI standard file to see how this syntax looks in a concrete piece of music.

In the first column of the file’s table, we see the delta time, which indicates in multiples of ticks the time delay between successive messages. T