💾 Archived View for gemini.circumlunar.space › users › adiabatic › words › computing › gemini › year… captured on 2024-08-18 at 17:32:43. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Reviewing your own year in Geminispace with jq and a JSON Feed

You probably subscribe to, or at least know of, services that tell you what you did on their service the past year. Of course, they send this to you BEFORE the year is completely done, so the stats are probably somewhat bogus unless they have a dedicated “year” that goes from December Teensomething to that exact same December Teensomething.

This is Geminispace, so we roll our own summaries.

If you’re wondering what this “feed.json” is, let me get you past the Colophon page here:

feed.json

Now then. Let’s get slicing and dicing.

cat feed.json | jq '.items'

This gives me the list of items. I think.

cat feed.json | jq '.items | length'

225. This matches the number of lines matching /^-/ in my YAML source. Good. I don’t have to wonder anymore.

cat feed.json | jq '.items | .[]'

This gives me a bunch of items, as opposed to one array.

cat feed.json | jq '.items | .[] | .date_modified'

This gets me all the dates. They’re all in zulu time. Good.

printf '"a"' | jq '. < "b"'

true. Good.

cat feed.json | jq '.items | map(select(.date_modified >= "2023"))'

This gives me a plausible list of the 2023 entries. How many?

cat feed.json | jq '.items | map(select(.date_modified >= "2023")) | length'

106.

cat feed.json | jq '.items | .[] | select(.date_modified >= "2023")'

There. Now we’re working with a bunch of things instead of one list.

A filter of the form .foo[] is equivalent to .foo | .[].

Let’s try it:

cat feed.json | jq '.items[] | select(.date_modified >= "2023") | length'

This gives me a bunch of 4s. 106 of them. I can use `wc -l` to count them, but is there a way to collect them back into a list?

Yes…

cat feed.json | jq '[ .items[] | select(.date_modified >= "2023") ] | length'

…but I have to edit the beginning of this jq thing. Yuck. `wc -l` it is.

What about all the scrawlspace posts?

cat feed.json | jq '.items[] | select(.date_modified >= "2023") | select(.url | startswith("gemini://gemini.circumlunar.space/users/adiabatic/scrawlspace/"))'

`wc -l` won’t do it. Drat.

cat feed.json | jq '[.items[] | select(.date_modified >= "2023") | select(.url | startswith("gemini://gemini.circumlunar.space/users/adiabatic/scrawlspace/")) ] | length'

47.

Onward:

cat feed.json | jq '[ .items[] | select(.date_modified >= "2023") ] | group_by(.url)'

cat feed.json | jq '[ .items[] | select(.date_modified >= "2023") ] | group_by(.url)[] | {url: .[0].url, count: length}'

This gives me what I want, but unsorted.

Let’s fix that:

cat feed.json | jq '[ [ .items[] | select(.date_modified >= "2023") ] | group_by(.url)[] | {url: .[0].url, count: length} ] | sort_by(.count)[]'

And make it more compact:

cat feed.json | jq -c '[ [ .items[] | select(.date_modified >= "2023") ] | group_by(.url)[] | {url: .[0].url, count: length} ] | sort_by(.count)[] | [.count, .url]'

This gives me (remember, the year isn’t over):

[1,"gemini://gemini.circumlunar.space/users/adiabatic/start/"]
[1,"gemini://gemini.circumlunar.space/users/adiabatic/words/computing/ascending-vs-descending/"]
[1,"gemini://gemini.circumlunar.space/users/adiabatic/words/computing/gemini/put-your-capsules-url-in-your-gemtext-pages/"]
[1,"gemini://gemini.circumlunar.space/users/adiabatic/words/computing/gemini/save-gemini-articles/"]
[1,"gemini://gemini.circumlunar.space/users/adiabatic/words/reviews/books/the-programmers-brain/"]
[3,"gemini://gemini.circumlunar.space/users/adiabatic/"]
[4,"gemini://gemini.circumlunar.space/users/adiabatic/words/computing/programs/"]
[19,"gemini://gemini.circumlunar.space/users/adiabatic/scrawlspace/2023/"]
[28,"gemini://gemini.circumlunar.space/users/adiabatic/scrawlspace/"]
[47,"gemini://gemini.circumlunar.space/users/adiabatic/words/games/zelda/tears-of-the-kingdom/"]

19+28 is 47. Scrawlspace ties Zelda.

Can we make this more multiline?

cat feed.json | jq -c '
 [
  [ .items[] | select(.date_modified >= "2023") ]
  | group_by(.url)[]
  | {url: .[0].url, count: length}
 ]
 | sort_by(.count)[]
 | [.count, .url]
'

I haven’t done much programming in languages with pipe operators, so this is the nicest I can do on short notice.

⁂

You know, jq kind of sucks. I’d probably be happier with a Python or Deno notebook doing this.

Then again, --from-file is a thing.

Yeah, I’m done here. Maybe next year I’ll use counts:

nnethercote/counts

⏚

Home

Hi! I’m a one-pixel invisible tracking image! View me to let my webmaster know you’ve scrolled through the entire page!