Follow

Working on a thing to sonify silent colour videos.

One [vcf~] per pixel with center frequency Hz determined by hue and Q determined by saturation (just using HSV for now, nothing fancy).

One 20ms delay line per column of pixels

Input to each column's filters is a weighted sum of nearby delay lines.

Output is panned to stereo according to column's position in space

Decidedly not realtime even with OpenMP parallelism on 16 CPU threads.

(though maybe if I ported it to OpenCL and used single precision on GPU and kept video resolution low...)

Smooth droney sounds.

Maybe I can make it richer with some [hilbert~] magic (given a sinusoid, hilbert~ outputs 2 sinusoids approximately 90degrees out of phase for "most" of the spectrum, allowing you to get a [phasor~] out of it, which you can then waveshape at will - I'm thinking Chebyshev polynomial recursion to get bandlimited sawtooth waves).

@paul will post some WIP later

porting to OpenCL is looking desirable because it's rendering at ~20x slower than realtime...

porting to OpenCL is looking like it will be really awkward because my code is full of stateful-objects at the moment - had to add more data storage even on CPU because the delwrite~ needs to be atomic w.r.t. delread~ and I have delread~ in multiple OpenMP threads.

If I make the delay line length the same as the block size the same as the number of samples per video frame it might work out neatly with two buffers for each delay line.

.

If I structure the algorithm code right, it should still compile as C99+OpenMP, with some preprocessor magic, which should make any debugging needed much easier...

@mathr Just slow down the time constant of the universe by a factor of around 2 and you're all set?

here's my code to go from a sinusoid v (which is output from a vcf~ ported from Pd) to a bandlimited sawtooth p using Pd's hilbert~.pd ported to C and Chebyshev polynomials of the second kind

```

sample b[2] = { v, v };

hilbert(b, &wave, b);

sample a = hypot(b[0], b[1]);

if (! (a > 0)) a = 1;

sample x = b[1] / a;

sample y = b[0] / a;

sample U0 = 1;

sample U1 = 2 * x;

sample w = 1;

for (int n = 2; n < 16; ++n)

{

w = -w;

w += U1 / n;

sample U2 = 2 * x * U1 - U0;

U0 = U1;

U1 = U2;

}

sample p = a * y * w;

if (isnan(p) || isinf(p)) p = 0;

```

not sure if b[0]/b[1] should be swapped, not sure if it matters - it'll just change the phase relation from +90 to -90 or vice versa...

https://code.mathr.co.uk/rodney/tree/6faf3d9ebdb043c79ce38d90179fe8517bf6518a check sonify.cl for the DSP, sonify.c for the control code, and sonify.sh for how to use it

@mathr How do you copypaste form PD in regards to C code?

@jayrope compare my https://code.mathr.co.uk/rodney/blob/6faf3d9ebdb043c79ce38d90179fe8517bf6518a:/sonify.cl#l28 vcf function with the inner loop of Pd's https://github.com/pure-data/pure-data/blob/eeef4ba9130d3182146927c37fa57d61bbff0f0b/src/d_osc.c#L386-L447 (Pd has a linear interpolated table lookup for cos() which is faster but less accurate; I just use the C math library which makes it simpler but slower - lines 415,419-433 of Pd's code are implementing something like `cos(cf)` and `sin(cf)`)

the process is extracting the core mathematics/algorithms, and hardcoding it in C without any of the wrapping API layers that make it useable from a patcher system

patchlore@paul@post.lurk.org@mathr I look forward to hearing (and seeing) results of this