Perspective, Space and Time in Music Notation

Underlying Assumptions

September 2000

This paper was originally written for the Symposium on Systems Research in the Arts, which was held as part of InterSymp' 2000, (The 12th Annual Conference on Systems Research, Informatics, and Cybernetics), at Baden-Baden, Germany from 31st July - 5th August 2000. It is published, on paper, in their proceedings.
The complete program for the Arts Symposium is described at their website. This was intended to be a multi-disciplinary event looking for common ground between several of the arts, including music, architecture, interior design, dance, theatre and the visual arts.
While concentrating on my specialised knowledge of music notation, this paper therefore tries to deal with levels of abstraction which may be useful in any of these fields.

A Social Perspective
The mensural notation of music
Space: A Copyist's World
Time: A Composer's World
Software: Replacing the Institutions


Perspective, in the generalised sense of a scheme by which local phenomena can be described in more detail than those further away, plays a part in many aspects of the arts and sciences. There is even a perspective of knowledge itself (we are all increasingly uncertain at the limits of our specialised knowledge), and this has to be taken into account in any attempt to achieve interdisciplinary progress. The context within which I work is therefore described, so that my points of view can be better understood.
Romantic attitudes to music notation have shaped the institutions which currently dominate music, and I argue that new software solutions will force them to change or become irrelevant.
Localness and context are crucial to understanding both the spatial (material) and temporal (information) aspects of music symbols. In space, local hierarchies of symbols are created by clustering (legibility criteria). In time, each music symbol has globally definable parameters whose precise value depends on a local context. Their global meanings (default values) constitute a style.
This paper concludes with a proposal for music editing software with which music symbols and their meanings can be separately edited and stored. Scores and performance practice are separately recordable.


music notation; perspective; space-time; matter-information; institutions; software


Writing is a basic technology. Without it, neither harmony nor counterpoint would have developed in music. Without it we would have no mathematics, no software, no literature. It is therefore quite scandalous that the crisis in music notation, which has been acute for about a hundred years, has not been resolved in spite of many helpful insights coming from the sciences. Musicians are victims of the wider cultural crisis which goes under the name of “The Two Cultures” (Collini and Snow, 1998; Brockman, 1995).
Awareness of the perspective of knowledge enables me to say what I think, as clearly as I can (with a passion which is perhaps proper only for an artist), while being very open to ideas which may prove more powerful. It should be borne in mind, that errors are the means by which knowledge grows, but that the rarest ones (the ones most worth finding) are those closest to the foreground of one's personal experience. For this reason, and because you will better understand what I am saying if you know something about who I am, I shall begin with a short description of my personal context.

A social perspective

In 1972, I left music college as a composer wanting to tackle the notation crisis facing written music, but highly suspicious of the institutions controlling art. My reservations about institutions were (and are) very similar to those described by the physicist Julian Barbour (Barbour, 1999), and like him I deliberately looked for a job which would keep me close to the problems I was interested in, while allowing me to develop independently. My first work experience was as a free-lance copyist for Universal Edition in London, but I soon moved on to become Karlheinz Stockhausen's principal copyist in 1974. This was a lucky chance: Here was an employer from whom I could continue to learn, and who was prepared to let me buy the time I needed for my own work with the money I earned. We still enjoy a fruitful symbiosis.
As Stockhausen's copyist for the last 26 years, I have been observing the institutions on which composers depend, at close quarters, without becoming directly dependent on them myself. I have to say that my worst suspicions have been confirmed over the years. The institutions have done nothing to help advance our knowledge of notation, which is the fundamental technology on which writers of music (and hence they themselves) depend. Not only do composers leaving college today face the same problems that composers faced when I left college, they are facing a crisis which has been with us for about a hundred years.
The current cultural schism between science and the arts has a long history, but becomes acute at the beginning of the nineteenth century. The Romantic movement marked a parting of the ways, and is the assertion of the importance of mind in the face of a too successfully materialistic science. Absolutism underlies the attitudes of both art and science at that time.Footnote 1 More recently, it has emerged that the possession of different points of view, combined with the readiness to modify those points of view, is a necessary part of the evolving culture in which we live. There is a relativity of knowledge which arises out of the fact that individuals each have their own perspective.
The crises in Romanticism and science at the beginning of the 20th century are not unrelated. Absolutism and Romanticism are the root causes of both the current crisis in written music (Ingram, 1985), and more serious social disasters which have happened during the last hundred years (dictatorships). The institutional failure to grapple with the problems of music notation has happened partly because the members of those institutions have thought that composers can be left to do the thinking. They have confused local knowledge (the knowledge of individuals) with global knowledge (what we all agree about), and both of these with absolute knowledge (god-like knowledge).
Science is nowadays more about information than material, so the ground rules have changed and Romantic art has lost its raison d'être. We all now agree that mind is somehow different from matter. Romantics have always been antipathetic to science, and have always tried to pretend that they are independent of the social structures on which they in fact depend. It is therefore not surprising that these structures have been neglected and are becoming ever more irrelevant as the world moves on.
Computers (institutionless machines which process information) are increasingly dominating our social context, taking over many of the functions previously carried out by groups of people. By removing the fierce competition for scarce institutional resources, they are removing the need for the rivalry which has characterised Romanticism. Currently, successful composers tend not to talk to each other very much, but given the perspective of knowledge, specialists must talk to each other and the rest of society, and they must do so informally. Internet software is currently revolutionising the way this can be done.Footnote 2
Notice that the institutions controlling art were not consulted about the invention of the internet, and that it poses insoluble problems for many of them. In particular, the laws of copyright, which generate vital revenue for many institutions, are in a mess because they too confuse the levels of material and information. Owning an object is not the same as owning the information it represents. In fact, ownership of information is a contradiction in terms.
I do not expect the institutions which have grown up around the dogmas of art, suddenly to become extinct. More likely (as in the case of religious institutions, which had similar problems during the Renaissance), those which fail to reconnect themselves to the real world will just struggle on with a mixture of resignation, cynicism and hypocrisy.
Triumphant scientists should bear two things in mind at this juncture. Firstly that over-confidence in science was a prime cause of our cultural schism in the first place, and secondly that their victory has been won at the price of concentrating attention on information rather than material. The domain of meaning used to be reserved for the arts.
Art and science are both, like children's play, ways of investigating reality. The complexities of the world “out there”, and the ways in which we try to understand it, should not be underestimated. Poetry too is about truth, beauty and insight. Of all the arts, music has always been most closely linked to the sciences, so it is probably a good place to look, for ways in which our two cultures might be reconciled.

The mensural notation of music

In painting, the simplest perspective has one fixed vanishing point, which in the plane of the picture shows where one should look to find the most distant thing represented. The vanishing point simultaneously determines an ideally fixed configuration for the viewer's eye, because the geometry approximates the projection of the outside world on the retina of an eyeball looking in a particular direction. Objects close to the eyeball are given more space on the canvas, and can therefore contain more information than those further away. One looks at the largest objects to find the most information per object. But nothing moves. This perspective excludes time. Perspectives with more than one vanishing point allow observers to behave more naturally in real time by exploiting the imperfections of peripheral vision to let the eyeball change direction. Each vanishing point corresponds to a timeless configuration of objects, including the eye of the beholder.
Standard music notation has evolved to describe events rather than fixed configurations of objects. It is a spatial perspective which allows symbols to grow in physical size according to the amount of information they carry rather than according to the meaning of that information. There are simple, “atomic” symbols like noteheads, but also a whole hierarchy of increasingly complex objects, which are created by clustering the symbols at lower levels.

The notation has evolved for maximum legibility, and is especially good at creating blocks of information which can be read at a glance before moving on. As with multi-focus spatial perspective, the focus of attention moves over the page in complex jumps. But in music, the timings of these jumps are related to the state of the reader's memory, the need to update it, and to the meaning, legibility and variable complexity of the symbols involved.
There is, of course, a link between the events described in this symbolic notation, and the events one can record on a machine. A “Chord” is a symbol for an event, and an event has an envelope which can be displayed on a graph in which one dimension of space is used to represent physical time. But it must be emphasised that we are dealing here with time stored in two different ways. These neighbouring levels of representation correspond exactly to the distinction which can be made between information and material. (Failure to come to terms with this distinction was the root cause of the collapse of 20th century music notation, and we have seen that the failure to distinguish between material and information is behind the crisis in the arts institutions...) On the one hand, we have time stored in the performer's volatile short-term memory, on the other, time which has a fixed relation to the spatial configuration of some physical object. The physical time is not the one we experience. Experienced time is always related to memory, even when we are listening to a clock ticking.
If I understand him correctly, Julian Barbour (op. cit) wants to show that physical (material) time is a redundant concept at the quantum level, because it is replaceable by sophisticated geometries. I am not a quantum physicist, but this idea resonates with my experience of space-time notation and the situation in ordinary mechanics. There is a fixed relationship between the spatial configuration of any physical object, and the time we perceive that it articulates. I would not be surprised if the only “time” we need as a separate concept, is the one which arises as the result of the interpretation of symbols (information). My working hypothesis, is therefore that symbol interpretation (intelligence, meaning) probably creates time and space out of space-time. I will be able to describe a software interface between music notation and mechanical space-time after I have said a little more about the software I currently write.Footnote 3

Space: A copyist's world

As a copyist, my job is to create scores which are as legible as possible. I need simple, highly flexible solutions to the purely graphic problems of music notation. Commercially available music software is generally inadequate or overly complicated in this area, because programmers are under pressure to provide solutions to widely perceived problems, and these problems involve sharing their customers' preconceived ideas about music notation (the unresolved 20th century notation crisis affects the software which can be conceived and sold). Also, much software is complicated by becoming involved at too early a stage in the complexities of real-time performance.
My approach is therefore to use a powerful general-purpose graphics program (FreeHand), and to develop plug-in modules to perform the specialised operations I need. It is not necessary to reinvent the wheel. Riding piggy-back on a program like FreeHand not only gives me access to its powerful functions, but also automatically ensures maximum integration into the wider context of software development (file formats, internet etc.).
Tools, of the sort I am developing, first have to analyse the graphic elements on which they operate, creating larger scale objects which behave in characteristic ways. Writing these tools means accurately describing the symbols of music notation in a programming language (I use C++), and understanding the details of the graphic contexts in which they arise. This approach also has the advantage that I can proceed cumulatively, solving the urgent, low-level problems first, and reusing the code in more complex tools later. The strategy for writing the code mirrors the cumulative levels in the symbols themselves.
The lowest level object for which I have had to write code, is the Chord (which I define as having zero or more noteheads). Chords occur in left-right sequences on a Staff, and correspond to events in before-after sequences in perceived time. This is the only fixed correspondence between perceived space and perceived time (meaning) to exist within this symbolic typographical system.

Chords are part of the general hierarchy of complex symbols, and are built by accumulating elementary graphical objects (Noteheads, Stems, Flags, Dots, Accidentals, Accents and other Auxilliaries). Typically, the core of a Chord is a Notehead which defines the context in which further elements can be added. Stems and Flags are attached directly to the Notehead, and other elements have their place in the areas to the left, right, above or below. Accidentals are found only to the left of a Chord, AugmentationDots to the right, so these are unambiguously attached to a particular Chord in a left-right sequence. In contrast, StaccatoDots can be found either above or below Chords, because the vertical spacing of Chords is completely unrelated to their meaning. Vertical proximity is, in the standard notation, only related to legibility.
(Since about the 15th century, multitrack scores have been consistently notated on parallel staves - before that, the tracks were notated in separate blocks on the page. Here again, legibility of the contexts created between the tracks is the only reason for notating the staves in parallel. Legibility, in this case, promoted the development of harmony and counterpoint.)

Time: A composer’s world

In the widest possible sense, every artist expresses meanings. Some of the meanings which composers express are labelled by the symbols they use in scores. The symbols are not themselves the meanings. Even at the lowest level, identical symbols mean different things to different composers. Nobody has a monopoly on the StaccatoDot.
Becoming an expert composer requires a lifetime of practical experience, and the creation of a personal “performance practice” or “style”, defining the relation between the symbols in the scores and real time. Historically, individual institutions have only been able to provide this for very few composers. Our current institutions are not unusual in imposing some rather special requirements on the composers they support. Generally speaking, pieces of new music are only ever played once, so experience is gained in proportion to the number of new pieces produced, and the standards of performance style are usually low (when measured against the standards achieved in pieces for which a tradition of performance practice already exists). Composers who are successful within the traditional system of institutions are therefore prolific, have to be content with unsatisfactory performances, and have no time to tackle the theoretical problems besetting the notation which has been in use since the middle of the 19th century. The expectation, that performers be able to communicate composers' meanings without having learned their style, stems from a time at which styles were more nearly universal, and derives from the Romantic attitude in which symbols are thought to have absolute meanings from which one simply deviates by means of expressivity. This attitude underlies not only the problem of the stagnating orchestra repertoire, but also the predatory nature of institutions which support young composers for a few years, only to discard them when they have lost their novelty value.
It is a composer's prime responsibility to express what (s)he means as clearly as possible at all levels. In rehearsals this requires both large amounts of time, and performers who are already well acquainted with the style which is being developed. With the best will in the world, one cannot expect our current institutions to provide these essentials, and the stagnation of musical style in the late 20th century is the result.

Software: replacing the institutions

Our best composers are straining to work outside the traditional institutions, often being associated with particular performers who specialise in performing their work. If they were given software with which they could develop a personal syle, this could be used to create end products (e.g. electronic music), and/or to create demonstration recordings for teaching performers how to interpret their scores (shortening rehearsal time).
Here is a possible design for such software: Professional sound editing programs currently use a form of space-time notation because it is easy to translate a mechanical recording into a space-time diagram. In fact, both the mechanical recordings and the visible diagrams are space-time configurations whose time can be extracted by scanning the data continuously. Changes made in the diagrams can also be made to produce predictable temporal results because they can easily be linked to the machine's underlying configuration.
Space-time notation is, however, difficult to read. It is much more difficult to locate perceived events in this notation than it would be using the common music symbols which have been developed for that purpose. It would be nice if we could add an additional user interface to such programs, displaying a symbolic representation of the sounds. The problem here, is that the standard (Romantic) notation cannot be used in its present form, because it assumes the existence of an absolute tempo in the music it represents. Its graphical and temporal aspects cannot be cleanly separated.
We need a more abstract notation, in which the rules governing the relations between the symbols are strictly graphical (spatial) in nature, so that they remain legible, while their meanings (the relations to the underlying space-time) are free to be defined elsewhere. I described the underlying concepts for such a notation in (Ingram, 1985).
It is possible to convert a space-time diagram into my abstract notation using purely spatial information. This is a process of spatial chunking, in which all temporal information is lost.

In order to convert the symbolic level back to space-time, the temporal information has to be replaced. So the spatial and temporal information can be separately encapsulated in two distinct software modules which a user could edit separately. The spatial module contains symbols (the score) and the temporal module (the performer) connects the score to space-time.

Issuing an “edit” command for a particular symbol in a score (a Crescendo for example) would open a window containing various controls (including space-time diagrams) with which its meaning (its relation to space-time) can be edited. Every symbol, at any level in the symbol hierarchy, has a characteristic set of parameters which could be edited in this way.
There is a perspective of definitions for each symbol. For example, a Sforzato sign must have a default meaning (a particular envelope curve) in the style, so that music can be edited directly in the score. But the meanings must be modifiable at lower levels of context too. It is easy to imagine that tweaking a local definition might affect the general definition somehow. As in ordinary language, the meanings of symbols change depending on how we use them in various contexts. Fuzzy logic would seem to be a good strategy to use here (we are concerned with the degree to which a local symbol definition conforms to a general definition), but other approaches could also be tried. There will be clearly understood interfaces to the symbolic and space-time levels, so various kinds of interpretation module could be programmed, allowing this project to evolve.
The “style” is, as in everyday music making, the sum total of the settings for the general definitions of the symbols in use. A style is comparable to what performers know about playing a particular piece, before sitting down to play it. In a particular performance, the symbols have to be given local values, which differ more or less from the default values depending on the local context. If there were ready-made style libraries which could be edited by teaching them to play other music, then style would begin to develop again. Developing a style means learning the performance practice in the existing library - which is what composers and performers have always done.
Two final remarks on this hypothetical software:
1) It is the user's responsibility to define exactly what the symbols mean. The programmers of such software, would of course have to ensure that conventional default meanings are available, without restricting their development unduly. There are well understood techniques for doing this, including the consultation of a number of different potential (expert) users to find out what they mean by particular symbols. This could also be done by attending rehearsals, and/or comparing scores with historic recordings. The strict separation of graphical and temporal aspects of symbols is the key to the fundamental design and maintenance of this software. There is nothing mysterious going on.
2) Performance practice is here being stored as information, separate from the graphic information in the score. There are implications for both Artificial Intelligence research, and the recording industry.


Human intelligence is the intelligence of individuals. It involves the identification of relevant, specialised, local knowledge, and the convertion of this into global knowledge for use by other individuals. Software may never be able to do this, but it is already assimilating global knowledge (the knowledge on which we all agree) - and is therefore replacing institutions. The idea that software may someday achieve absolute (god-like) intelligence is a mistaken Romantic fantasy.


Footnote 1: It is my contention, that German idealism (the academic philosophical tradition which begins with Kant and Hegel) has at its roots the conception that it is possible to understand the world without taking communication or the perspective of knowledge into account. Idealists think that the world would be generally explained if clear descriptions could be given of the terms and symbols within their own minds. (It is also assumed that these descriptions are somehow finite and well behaved.) This kind of idealism is now bankrupt, because there can be no universally agreed, precise meanings for such local symbols. The world “out there” is all we have in common, and is the final arbiter of untruth.

Footnote 2: “Informally” here means using a language which is generally comprehensible, avoiding the stricter forms (jargon) used when talking to other specialists in the same field, even though the underlying levels of comprehension may be very complex. Specialisation is important, because there would otherwise be nothing interesting to say at the surface, public level. As in the programming language C++, there are levels of language apropriate in different situations: The public level can be understood by many people, irrespective of their particular specialisations. The protected level contains vocabulary peculiar to a specialisation (jargon) so that communication can be efficient between the people interested in a particular domain. The private level of language is necessary for freedom of thought within individual minds - not only are misunderstandings inevitable if one tries to voice one's private thoughts (especially to strangers), but social disasters are also inevitable!

Footnote 3: software and time: Like a mathematical proof, software has a logical, sequential correctness which can be ascertained by reading it through, interpreting the symbols to make sure that their uses are consistent. This requires both perceived time and experience. Experience also involves perceived time, because the meanings of the symbols have to be learned in the first place. Experience is the prerequisite for intelligence, and is a recursive, evolutionary process. I remember being rather disappointed in my first computer when it took time to execute programs. I had imagined that this would be quasi instantaneous, like switching on an electric light! Intelligence requires time.
Also like a mathematical proof, software has a timeless sense in which it is correct “all at once”. This sense no longer involves intelligence, but is the level at which the software is simply part of space-time. Having a timeless aspect means that an infinite number of physical instantiations can be produced. The timeless aspect is the crucial advantage that software has over institutions.


Barbour, J. (1999) The End of Time; Oxford University Press. Julian Barbour's website is at

Brockman, J. (1995) The Third Culture (pp. 17-21 ); Simon & Schuster. John Brockman maintains the website of The Edge Foundation, including contributions by several of the authors in his book “The Third Culture”.

Collini S. and Snow, C.P. (1998) The Two Cultures; Cambridge University Press

Ingram, J. (1985) The Notation of Time; originally printed in Contact magazine No. 29, spring 1985