Tech

APP OF THE MONTH: Tunetrace creates music from drawings

by Mark Teo

The object of Tunetrace is simple: As its name suggests, it converts images into music. But unlike those old Windows Media Player visualizations, the process is interactive—and downright magical. Here’s how it works: first, you snap a photo of an illustration, hopefully one you’ve sketched out yourself. Then, Tunetrace converts that photo into a skeletal structure of lines and points. Next, the app interprets that drawing, converting it into a song of beeps and boops. Tiesto it ain’t, but it’s no less transfixing. We spoke with its developer, Ed Burton, to talk about how he plans on connecting arts, craft and tech.

AUX: How did the idea for Tunetrace germinate?

Developer, Ed Burton: Together with the team at qApps [Queen Mary, University of London’s organization meant to distribute smartphone / mobile apps] we were trying to think of a way to make some aspects of computer programming accessible and fun on a mobile device. Specifically, I wanted to emphasize that programming has a playful quality.

How so?

Iteratively writing and debugging a program feels somewhat like play in that you have a safe space with a lot of freedom to invent rules and experiment. You progress by trial and error, and sometimes, the most intractable errors lead to the most surprising successes.

What’s the most complex image you’ve run through Tunetrace?

The detailed grid of pixelated text in Tunetrace’s demo video is the most ambitious drawing I’ve done so far. I did hear from someone who said he spent nearly an hour listening to the London Underground Map. The [subway] map, originally by Harry Beck, was an iconic example, focusing on topology rather than geometry. 

What kinds of drawings did you enter into Tunetrace in its testing phases—and what were some of the difficulties you encountered while developing the app?

The most difficult aspect has been the image processing to detect the topology of a drawing. When Tunetrace is processing an image, you see a process called “skeletonization,” that erodes an image until all that’s left is a spidery network of single pixel lines. The problem is that it tends to leave a very messy network full of ambiguous detail, like a [subway] map with far too many stations close together.

For Tunetrace to play the same tune when it sees the same drawing, the topology has to be as unambiguous as that [subway] map. One extra station could cause a completely different sequence of events to unfold. Judging when and how to merge a cluster of “stations” into one intersection took a lot of trial and error.

I wish it did a better job, but in an important way, the degree to which you need to discover how to draw if you want Tunetrace to give reproducible results is rather like programming. The drawings have a highly constrained grammar waiting to be discovered and explored, much like all programming languages do.

Aside from sheer fascination, do you think that apps like Tunetrace—that connect drawing and sound—can change the way we think about, visualize, or even perform music?

Music cannot help but be visualized. It unfolds into the mind. I’d prefer to think of generative music, music that emerges from a largely autonomous processes. [It’s] not so much a change, but it draws attention to something we already know—that music isn’t just a sequence of information, it’s an unbroken chain of cause and effect.

It’s something that happens. Recorded music is something that happened, in the past tense, and the sequence of sounds is the footprint left behind. Generative music happens in the present tense. It’s exciting to see the recent work of snibbestudio.com [who AUX has covered in the past] for Björk and Phillip Glass bringing visual interactive generative music to a much wider audience.

Were there any apps that inspired the idea behind Tunetrace?

There are many, I think the honourary mention should go to Sim Tunes by Toshio Iwai.

What other ideas do you have coming down the pipeline?

Later this summer, it’ll be possible to share Tunetrace drawings online, and I’m looking forward to discover what people around the world have been doing with the App. I’m working on a couple more apps this summer. One is a quirky game about interface design within extreme constraints. It’s a little like Pictionary with very, very low-resolution drawings. The other is top secret and related to Tunetrace. Ask me again in a month or two for more.

We will! So, how can you cut through the noise of app store to get your app noticed?

Gosh, yes it’s daunting to spend weeks to launch something into such a very crowded market place. To be honest, I tend make things because I want to play with them, [because] they don’t exist yet. I’d say my approach is somewhat egotistical,  [it’s about] the independence to follow what I want, rather than follow the market. Sometimes that gets me noticed, but I’m not expecting it to make me rich.