Follow

I'm convinced that the h264 codec really is actually chaotic black magic.

I make a 25 minute generative video with a bunch of circles expanding in a rippling effect? FFMPEG crunches it into 133mb. Take those same circles and make them fall downwards in the screen? 58mb.

· · Web · 2 · 1 · 2

Experientially, I've figured that the h264 goddesses are pleased when video input is:

limited in color (using 2 colors, plus anti-aliasing)

limited in detail (vector graphics, simple primitives)

straight lines over curves (triangles and rectangles compress better than circles on average)

linear movement (falling downwards shapes is better than growing shapes)

Show thread

@paul well I have a 20 yo mjpeg low-res video playing with the video cam noise and exagerated motion vector movements, and ... it's a hell to translate correctly in h264... I'll fiddle again next days with it but it's indeed really difficult to encode without loosing the point ...
it's really a codec tested against "some" sort of imagery ...

@patchlore Your observations actually makes perfectly sense. I'm no expert on h.264 encoding, but know the basics at least. (I was working in a video conferencing company along with one of the main architects of the h.264 codec for a number of years.)

Roughly what happens is that to encode an image, it is split into 8x8 squares called macro blocks. Then each block is analysed. The macro blocks that are unchanged from the previous frame is essentially skipped, those that has moved relative to the precious frame is encoded as a displacement (motion vector), and finally the remaining blocks have to be encoded fully. (This is extremely simplified!)

So an object moving about, but otherwise stays the same can be encoded at motion vectors only. Those are really small compared to the data for a full macro block (even if the macro blocks are also very efficiently encoded.) An object changing shape is more likely to require the affected macro blocks to be fully encoded, and thus the data for the frame grows. Growing a circle will essentially change every macro block it touches, while a square or triangle will have more parts that can be coded as displacements.

@harald fascinating. Thank you for the technical insight.

I like the idea of creating content explicitly designed to play well with h264. Almost feels like a demoscene project.

Sign in to participate in the conversation
post.lurk.org

Welcome to post.lurk.org, an instance for discussions around cultural freedom, experimental, new media art, net and computational culture, and things like that.