I made a music visualizer based on MPD fifo output

There are many, many clients for MPD. Many of them are old and unsupported, but maybe someone here can help you with them anyway...
Post Reply
lagfra
Posts: 4
Joined: June 3rd, 2019, 5:39 pm

I made a music visualizer based on MPD fifo output

Post by lagfra » June 3rd, 2019, 5:59 pm

Hi everyone,

I'm looking for opinions / advice on this little thing I made:

https://github.com/gallafrancesco/mvc

It's a curses-based music visualizer written in C99 which only depends on ncurses and a C compiler. The aim was to have an extremely lightweight, self contained music visualizer which only needed a /tmp/mpd.fifo to run.

It's configurable through a settings.h header and can be built with libmpdclient if the status of the song needs to be displayed, along with the visual.

Being curses visuals fairly "primitive", it doesn't require graphical acceleration at all (can run on almost anything which runs a terminal and mpd) yet the visualization could use some improvement, especially when it comes to noise reduction and proper frequencies (or beat) detection. It works quite well though, it would be a pleasure to have your opinion on it.

Thanks!

max
Forum team
Posts: 922
Joined: January 15th, 2013, 3:43 pm

Re: I made a music visualizer based on MPD fifo output

Post by max » June 4th, 2019, 8:26 am

The FIFO output is a very bad idea; it works only if the client runs on the same machine, requires explicit configuration, consumes too much CPU and there is no format negotation, which makes it fragile.

What can we do to support visualizers without kludges like the FIFO plugin? Maybe can we implement a FFT output plugin which sends the FFT output to the client? (and do nothing if no client is connected) ... of course, this plugin should be able to listen to IP sockets, so the visualizer could run on a different computer.

lagfra
Posts: 4
Joined: June 3rd, 2019, 5:39 pm

Re: I made a music visualizer based on MPD fifo output

Post by lagfra » June 4th, 2019, 9:45 am

While I agree that the FIFO output is a bad idea, producing a FFT output is quite heavy on the CPU and I think it would be best if left to the visualizer/client. This also allow for proper customization of the data elaboration process.

I had the idea of using TCP sockets to stream data to the client, would it be feasible? I know that MPD embeds a HTTP server to stream on the network, but it seems to me that HTTP is the wrong tool for the job (new TCP connection each request, must use a HTTP client, etc).

Of course, format negotiation is an open issue.

max
Forum team
Posts: 922
Joined: January 15th, 2013, 3:43 pm

Re: I made a music visualizer based on MPD fifo output

Post by max » June 4th, 2019, 10:17 am

lagfra wrote:
June 4th, 2019, 9:45 am
producing a FFT output is quite heavy on the CPU and I think it would be best if left to the visualizer/client.
Why do you think so? With your approach, being limited to running MPD and your client on the same machine, it's going to be the very same CPU...
Doing FFT inside MPD means only a very reduced stream of data must be transmitted to the client, which helps a lot. At least one, sometimes two steps of format conversion can be omitted.
This also allow for proper customization of the data elaboration process.
That FFT output plugin could receive parameters from the client, specifying details of the desired FFT.
I had the idea of using TCP sockets to stream data to the client, would it be feasible?
Of course, why not? A TCP socket is just a transport for arbitrary data, just like a pipe is. The advantage of TCP is that its bidirectional and that it can have multiple clients. (And of course clients can be anywhere, not just on the local computer.)
I know that MPD embeds a HTTP server to stream on the network, but it seems to me that HTTP is the wrong tool for the job (new TCP connection each request, must use a HTTP client, etc).
It is an excellent tool for the job!

How is "a new connection for each request" a disadvantage? It's actually an advantage. There can be any number of clients - ranging from zero (means MPD can skip the encoder/FFT instead of wasting CPU for calculating data nobody's interested in) to one or more. The pipe (FIFO) can have at most one client, but MPD doesn't know if it's zero or one - eventually the pipe buffer will be full, which means a lot of data has accumulated there, which causes a huge latency.

"Must use a HTTP client" - that's really trivial. Send "GET / HTTP/1/1\n\n", read lines until you get an empty line, and from here on it's raw data until the TCP connection closes.

What other protocol would you suggest? The "FIFO" plugin has no protocol, just raw data, but that's part of the problem. Of course, you can invent a new container protocol, but what would be the advantage?

There's just one thing that could be better than TCP (with a HTTP subset): something based on UDP, e.g. RTP. That would avoid some of TCP's disadvantages.

lagfra
Posts: 4
Joined: June 3rd, 2019, 5:39 pm

Re: I made a music visualizer based on MPD fifo output

Post by lagfra » June 4th, 2019, 10:39 am

Doing FFT inside MPD means only a very reduced stream of data must be transmitted to the client, which helps a lot. At least one, sometimes two steps of format conversion can be omitted.
I like the idea. What you are suggesting is to build a plugin which can:

* Read data from MPD in chunks (I guess this should be done through the external API)
* Perform a FFT transform on the data chunk
* Write the result on a TCP socket
* Optionally accept some commands on the TCP socket for format negotiation (sample rate, n. samples per FFT chunk is what I'm currently using in the visualizer)

If I'm not wrong, this could also be done by performing a FFT and sending data through a 'httpd' audio output. This could require more explicit configuration than the MPD fifo solution though, so I guess it should be as transparent as possible.

I'll take a look on how to implement such a plugin (I guess this falls in the "neighbor plugins" category). Thanks!

max
Forum team
Posts: 922
Joined: January 15th, 2013, 3:43 pm

Re: I made a music visualizer based on MPD fifo output

Post by max » June 4th, 2019, 11:11 am

lagfra wrote:
June 4th, 2019, 10:39 am
Read data from MPD in chunks (I guess this should be done through the external API)
What external API? There's just one external API: the MPD protocol, but it cannot access any audio data.
I guess this falls in the "neighbor plugins" category
No! Neighbor plugins can locate nearby storages (such as file servers or USB sticks).
Maybe this FFT plugin could be implemented as an encoder plugin (encoding PCM to some sort of FFT), but then one would have to hard-code the FFT setting, because MPD uses exactly one encoder instance for all clients connected to "httpd". But in any case, having an encoder plugin would be a good start, and we could copy the code over to some new concept later.

lagfra
Posts: 4
Joined: June 3rd, 2019, 5:39 pm

Re: I made a music visualizer based on MPD fifo output

Post by lagfra » June 4th, 2019, 11:23 am

What external API? There's just one external API: the MPD protocol, but it cannot access any audio data.
My bad, I meant the internal API. I guess there's no API to directly access a data stream provided by MPD then?

I'll start with the encoder plugin for now, thanks for all the help! I'm going to keep you updated on the progress.

max
Forum team
Posts: 922
Joined: January 15th, 2013, 3:43 pm

Re: I made a music visualizer based on MPD fifo output

Post by max » June 4th, 2019, 11:29 am

When you write a plugin, remember to always compile with "-Dtest=yes", which will enable the build of "run_encoder". This program runs an encoder plugin on the command-line, reading PCM from stdin and output on stdout, without carrying around the rest of MPD. Can be helpful for development and debugging.
For the actual FFT, I suggest using an existing library, such as FFTW.

Post Reply