MB

▄▄▄·▪ ▐ ▄ ▄▄▄ . ▐█ ▄███ •█▌▐█▀▄.▀· ██▀·▐█·▐█▐▐▌▐▀▀▪▄ ▐█▪·•▐█▌██▐█▌▐█▄▄▌ .▀ ▀▀▀▀▀ █▪ ▀▀▀

Still alive, just busy. First post of the year… yay I guess.

I have been working on my fork of Navidrome that will have some audio and music analysis with the help of Essentia, the MVP is almost done. Because fuck Spotify, I am done with their bullshit. But I still want my cool data, so I am making it myself, all open source ofcourse.

Navidrome is written in Golang but most analysis libraries are written in C/C++ or Python, so gRPC was the solution because I am not writing a wrapper. My first time working with gRPC, it’s really nice once you get it all set up. But the setup can be a pain, like WTF are .pyi files. I have never seen them before, they are interface files so you have your types available (this is because Python Protobuf does some weird runtime C thing that processes the proto files or something). It has been nice to use, apart from setup. The end-to-end types are just a huge plus, you just know that they will work.

Python multiprocessing has also been a journey. Tested all types of pools but they always just deadlocked or didn’t run. In the end I just made my own worker pool with each worker having its own process. And they share a multiprocessing-safe queue for input and output. Sort of like I would do in Golang with channels instead of queues. And this dead simple approach worked first try of course, after everything I tried. But I am just happy that it works now. Still feels weird to see python use almost 100% of your CPU on all cores.

One year anniversary coming up for MB.

Today on Spotify is horrible for everybody. They killed major part of there API (since 27/11/2024), the following parts are deprecated and can’t be used by any new app.

Related Artists
Recommendations
Audio Features
Audio Analysis
Get Featured Playlists
Get Category’s Playlists
30-second preview URLs, in multi-get responses (SimpleTrack object)
Algorithmic and Spotify-owned editorial playlists

And they did this without any notice. For some people (myself included) this means that a project with months of work is just scrapped in an instant. It is really time to move of Spotify but the alternatives are not that much better, but still better I guess. I will have to look into my own audio analyzer, if I even have the compute for that…

I am happy that this blog is scraping based so I am not dependent on there API and everything just works as before. But meta-tune and my upcoming scraping project are just totally bricked.

for chunk := range slices.Chunk(allSimpleTracks, 100) { ids := make([]spotify.ID, len(chunk)) for i, a := range chunk { ids[i] = a.ID } f, err := client.GetAudioFeatures(ctx, ids...) if err != nil { return nil, err } fullTracks := make([]*spotify.FullTrack, len(ids)) for subChunk := range slices.Chunk(ids, 50) { full, err := client.GetTracks(ctx, subChunk, spotify.Limit(50)) if err != nil { return nil, err } fullTracks = append(fullTracks, full...) } for i := range len(ids) { allTracks[i] = &FullerTrack{ Track: fullTracks[i], Features: f[i], } } }

DO $$ DECLARE cols text; query text; BEGIN SELECT string_agg(quote_ident(name) || ' text', ', ') INTO cols FROM ( SELECT name FROM (SELECT DISTINCT name, priority FROM device_analyses) AS o ORDER BY priority DESC ) AS o; BEGIN EXECUTE 'DROP MATERIALIZED VIEW IF EXISTS device_analysis_summary'; query := format(' CREATE MATERIALIZED VIEW device_analysis_summary AS SELECT * FROM crosstab( ''SELECT d.deveui, da.name, da.value FROM devices d LEFT JOIN device_analyses da ON d.deveui = da.device_id ORDER BY d.deveui, da.name'', ''SELECT name FROM (SELECT DISTINCT name, priority FROM device_analyses) AS o ORDER BY priority DESC'' ) AS ct(deveui text, %s); ', cols); EXECUTE query; EXCEPTION WHEN OTHERS THEN RAISE NOTICE 'Error creating materialized view: %', SQLERRM; ROLLBACK; RETURN; END; END $$;

Penger City!!

New logo dropped!!