I made an unholy oneliner to scrape, download and upload some data for my current project. Works fine on linux, not on my Windows machine :\. It would be in all aspects better as a bash script or even a python script (cross platform). But what’s the fun in that, this is way cooler. Even though I will probably rewrite something similar in python so I can also run it on my windows machine. But it is good to keep my shell skills sharp.

seq 1 4 | xargs -I% sh -c 'echo "Fetching page %" >&2; curl -s "https://www.last.fm/charts/weekly?page=%"' \
        | pup 'td.weeklychart-name a.weeklychart-cover-link attr{href}' \
        | sed 's|^|https://www.last.fm|' \
        | xargs -I {} sh -c 'echo "Processing artist URL: $1" >&2; curl -s "$1" | pup "h1.header-new-title, div.header-new-background-image"' _ {} \
        | pup 'json{}' \
        | jq '[ .. | objects | select(.tag=="h1" or .itemprop=="image") ] as $list | [ range(0; $list|length; 2) | { name: $list[.].text, img:  $list[. + 1].content } ]' \
        | tee artists.json \
        | jq -r '.[].img' \
        | xargs -n 1 sh -c 'echo "Downloading image: $1" >&2; wget -q "$1"' _ && \
jq -j '.[] | .img |= (split("/")[-1]) \
           | @json, "\u0000"' artists.json \
		   | xargs -0 -I{} sh -c 'img=$(printf "%s" "$1" \
		   | jq -r .img); name=$(printf "%s" "$1" \
		   | jq -r .name); echo "Adding $name"; curl -s  -X POST -H "Authorization: Bearer $TOKEN" -F "img=@$img" -F "json={\"name\":\"$name\"}" http://localhost:3000/api/artists/add' _ {}