Currently outlining a blog post that may or may not materialize. Working title: "Reflections on Gestured Sound". It's my attempt to focus some of the concepts I've been developing that center around the notion of "gesture".

There are three sections: concept, implementation, realized structures.


the concept: "gesture" is fancy name for a control signal. a set of coordinated, synchronized gestures are used to manipulate a sound object. This produces a "gestured sound". A well mapped moving gesture will produce something that also sounds like movement... a sonic gestalt. gestalts work together to produce a perceptual cue that our brains register as a musical "note".

The origins of a music "note" are well... notes. Notes get interpreted by human musicians, which in turn produce (body) gestures on musical instruments. Gesture here aims to crudely emulate this layer of musical performance with computers.

· · Web · 2 · 0 · 3

Implementation: a Gesture in practice is a glorified breakpoint line generator with variable slopes.

A common problem with lines is that many instances will eventually drift apart. What do? Synchronize them to a master clock. This has lead to something I built called the "gesture signal generator".

A specialized clock signal known as the "conductor" drives the timing of one or more GSGs, and programmed like a breakpoint line generator with timings relative to the conductor signal. When the conductor changes tempo, the GSG will automagically adapt without needing to actually know the tempo itself. It turns out I re-invented an EE concept known as a "phase-locked loop". This, plus an interpolator, and some way of iterating breakpoint values, is the core primitive that makes my Gesture possible. In other words: breakpoint line generators that are impervious to drift accumulation over time.

Realized Structures: With the Gesture Signal Generator, everything is a signal, and can be thought of as a topology such a DAG or signal flow chart. This leads to interesting things.

A few constructs I've invented so far include Temporal Weight, Temporal Skew, Modulated Conductors, Articulated Gestures, and Gestured Metronome with Segmented Ramp.

@paul Sounds interesting! Is this in conversation with the stuff by Mazzola & collaborators? Sounds like you're thinking along similar lines (though I've never properly worked through Mazzola)

@phetre never heard of this before. To be honest, I'm having a hard time wrapping my head around all that math. But it looks interesting!

I guess you could generously call my efforts "applied DSP": I needed better lines for my sound objects, here's what I rigged up so far.

@paul Sounds good. I don't understand the Mazzola stuff either (yet; I haven't given up hope!), but I suspect there might be a similar intuition: gestures as mappings from the unit interval? Not sure. Anyway, can't wait to see your blog!

@paul The idea sounds really cool, but I don't think I understand it that well (I don't have much background in signals). In this case, is gesture some combination of timbre, articulation, dynamics, i.e. everything that's not the actual note that musicians choose to play? or does it include the note too?

@brycew maybe this theremin performance of the swan by Clara Rockmore would help clarify things:

What's interesting about the theremin is you only have control of pitch and amplitude during a performance. No timbre control*. Yet despite these limitations, it's a very compelling performance. Why? The combined continuous gestures of the left/right hand form what we hear as notes, played with virtuosic phrasing. In other words, the note appears when all the gestures move together to articulate it.

It's reasonably straightforward to build a virtual theremin inside a computer. It's just a single oscillator with freq/amp control. Getting those controls to move in a lyrical way like how Clara does it (the "performance") is a more difficult problem, which is what my systems attempt to approach.

*: There are tone knobs on theremins that change timbre, but they are something you set before the performance not during.

Sign in to participate in the conversation

We are an instance for discussions around cultural freedom, experimental, new media art, net and computational culture, and things like that.

<svg xmlns="" id="hometownlogo" x="0px" y="0px" viewBox="25 40 50 20" width="100%" height="100%"><g><path d="M55.9,53.9H35.3c-0.7,0-1.3,0.6-1.3,1.3s0.6,1.3,1.3,1.3h20.6c0.7,0,1.3-0.6,1.3-1.3S56.6,53.9,55.9,53.9z"/><path d="M55.9,58.2H35.3c-0.7,0-1.3,0.6-1.3,1.3s0.6,1.3,1.3,1.3h20.6c0.7,0,1.3-0.6,1.3-1.3S56.6,58.2,55.9,58.2z"/><path d="M55.9,62.6H35.3c-0.7,0-1.3,0.6-1.3,1.3s0.6,1.3,1.3,1.3h20.6c0.7,0,1.3-0.6,1.3-1.3S56.6,62.6,55.9,62.6z"/><path d="M64.8,53.9c-0.7,0-1.3,0.6-1.3,1.3v8.8c0,0.7,0.6,1.3,1.3,1.3s1.3-0.6,1.3-1.3v-8.8C66,54.4,65.4,53.9,64.8,53.9z"/><path d="M60.4,53.9c-0.7,0-1.3,0.6-1.3,1.3v8.8c0,0.7,0.6,1.3,1.3,1.3s1.3-0.6,1.3-1.3v-8.8C61.6,54.4,61.1,53.9,60.4,53.9z"/><path d="M63.7,48.3c1.3-0.7,2-2.5,2-5.6c0-3.6-0.9-7.8-3.3-7.8s-3.3,4.2-3.3,7.8c0,3.1,0.7,4.9,2,5.6v2.4c0,0.7,0.6,1.3,1.3,1.3 s1.3-0.6,1.3-1.3V48.3z M62.4,37.8c0.4,0.8,0.8,2.5,0.8,4.9c0,2.5-0.5,3.4-0.8,3.4s-0.8-0.9-0.8-3.4C61.7,40.3,62.1,38.6,62.4,37.8 z"/><path d="M57,42.7c0-0.1-0.1-0.1-0.1-0.2l-3.2-4.1c-0.2-0.3-0.6-0.5-1-0.5h-1.6v-1.9c0-0.7-0.6-1.3-1.3-1.3s-1.3,0.6-1.3,1.3V38 h-3.9h-1.1h-5.2c-0.4,0-0.7,0.2-1,0.5l-3.2,4.1c0,0.1-0.1,0.1-0.1,0.2c0,0-0.1,0.1-0.1,0.1C34,43,34,43.2,34,43.3v7.4 c0,0.7,0.6,1.3,1.3,1.3h5.2h7.4h8c0.7,0,1.3-0.6,1.3-1.3v-7.4c0-0.2,0-0.3-0.1-0.4C57,42.8,57,42.8,57,42.7z M41.7,49.5h-5.2v-4.9 h10.2v4.9H41.7z M48.5,42.1l-1.2-1.6h4.8l1.2,1.6H48.5z M44.1,40.5l1.2,1.6h-7.5l1.2-1.6H44.1z M49.2,44.6h5.5v4.9h-5.5V44.6z"/></g></svg>