Creepy talking piano

[YOUTUBE]ZWiCUUfI7M4[/YOUTUBE]

Wasn't that just about the coolest thing evar?

Ingenious.

That's remarkable. Glad I could understand what those people were saying too

Amazing! I love it.

Freaky. It's a bit EVP?

Wow! Taking a regular piano and making human voice by computer controlled key activation - that's something I would have never thought possible.

It reminds me of those mosaic pictures where the computer selects the small picture based on its overall color value. I think something like that is happening here.

That's remarkable. Glad I could understand what those people were saying too

Here's a version with captions (it loads a LOT slower, that's why I posted the other one instead).

Yes, similar concept to what Metasynth does.

Very clever but very pointless.

if you look away from the captions it's utter {censored}.

clever. but {censored}.

Very clever but very pointless.

You think so? I see at least 1 point: it proves that dissonance isn't always dissonant to our ears:)

I can see why you'd say that but having spent a fair amount of time around the deaf, I could see getting to the point of understanding it very quickly.

Decomposing the raw sample to frequency bands that match each piano key. Then detecting the volume of each band, and sending the data to servo motors which strike the needed keys with needed velocity.

Sample resynthesis through an actual piano. I bow to the guy who thought of this!

To me this organ performance is way more impressive. This is an actual performance, someone had to figure out matching each vowlel sound by comparing it to a particular organ sound, and then actually perform the piece.

The talking piano would not be the least bit interesting if you removed the backing voice over. It would just sound like random notes. Even with the backing voice over, it still sounds like two seperate entities as opposed to one cohesive unit.

Some very clever programming and engineering to pull it off, but at the end of the day, it is the musical equivalent of painting a moustache on the mona lisa.

The piano in its natural stae, being the Mona Lisa.

Seems like alot of eggheads got to gether, put in a ton of work on it, stepped back and looked at their baby with pride, and all the passers by, stop and stare and say "Cool. But what does it DO?"

There IS no backing voice over! It's all played by piano, and the sound resembles the original voice because some of harmonic structure was preserved.

It creeps you out!

Seriously though, I think that the talking piano machine combines, compresses, and presents many symbols or metaphors for human achievement. I think it is a brilliant piece of art - done for the pure sake of it. If you pay attention (in my case to the subtitled version), the initial computer-generated re-synthesizing/sequencing of the piano was adjusted by the programmer. There is clearly human intervention required to adjust the phrasing and dynamics to improve the fidelity of the reproduced "voice".

One could ask the "what does it do?" question of just about everything hanging in every art gallery in the world, especially in "modern" art galleries. In this way of thinking the toilets in the gallery bathrooms do much more than the art.

I agree with you that the organ work is a very cool performance.

As has been suggested, I imagine this is done by spectral analysis of the source material, and then matching the spectra against known spectra from the piano it plays.

What if they analyzed a solo piano piece that way and then tried to reproduce it with their thingamjiggy? I wonder what sort of weird artifacts would be produced?

Right. The fact that someone thought a voice track was being played along with it proves it is a worthy technical achievement. The "S" sounds and other consonants in the human voice are being produced entirely by the piano, usually by quickly striking the highest keys.

It would be impossible for a human to "play" this way on a piano, and we can argue the musical relevance of something like this all day and never come to an agreement. But I think it is interesting that so many people have a negative reaction to it.

Just like Metasynth (Mac software) breaks down a sample from an image and plays it back from a bank of fixed-frequency oscillators - this does the same thing, but the piano keys act as the oscillator bank.

It's an interesting jumping-off point for discussion about how our ears perceive sound and how our brains translate it into something meaningful. I'd call it audio "pointillism".

Gus - the organ in that song is amazing, thanks for the link. Never heard it before. It would obviously take a lot of time to practice that and perform it. He's definitely hitting all of the vowel sounds, that drawbars (and filters) let you do. But it would still be next to impossible for someone to reproduce the consonants of human speech like computer/mechanical method does in the piano example.

Good question! That would be the next thing I would try if I had such a thing.

I HAVE tried vocoding things with themselves (as both the input and carrier) and the results weren't all that exciting. But who knows.

Fantastic!

Would be perfect in a Brothers Quay film.

And that piece that Gus posted was amazing too; definitely need to check this guy out.

If there is no backing track voice over, then how are the vowel sounds produced? I did not watch the entire video but I watched most of it. Are you SURE there is no voice over played underneath the piano, or is this just an assumption. Because Im not sure either, but what I am assuming is that the piano is only matching the pitch of the voice, but the voice itself is still there mixed in mith it, providing the vowel and consonant sounds.

Are you sure of this or is it just an assumption., Because if it IS 100% piano, then I would say it is alot cooler than I thought.

It IS 100% piano.

The starting sample is spectrally analyzed, broken into small "windows", and each window's frequency responds to frequency of each piano key. Amplitude of each window is constantly analyzed, and depending on loudness of each window, a different velocity is sent to be played on piano.

This is basic resynthesis. It's all the sound of the piano! Your assumption is wrong. This is the POINT of the experiment - approximating the timbre of human voice using all frequencies available on the piano.