Further automate Git Rev News

[chriscool]

Historical background:

This is a follow up from: #273

This is also about using AI to help summarize and select discussions on the mailing list as discussed in the issue for edition 120 starting with this comment: #747 (comment)

This is also about managing the existing scripts that are currently managed in: https://github.com/chriscool/getreleases

Next steps:

• Describe recent improvements
• Describe what's left to automate

cc @jnareb @mjaix @sivaraam @gitster @stepnem @dscho @mricon

[chriscool]

Recent improvement:

• Updated https://github.com/chriscool/getreleases/blob/master/README.md to describe the different scripts in the repo.
• New https://github.com/chriscool/getreleases/blob/master/git_ml_converter.py script to get a mailing list thread in text format.
• New https://github.com/chriscool/getreleases/blob/master/search_mailing_list.py script to initially select possibly interesting mailing list threads to read/analyze before selecting a few ones to be summarized for an article.
• The https://github.com/chriscool/getreleases/blob/master/publish_edition.sh script now creates the issue for the next Git Rev News edition.

[chriscool]

What's left to automate:

• Sending emails to the mailing list (draft email and announce email)

Emails are now copy-pasted from one edition to the next. It should be possible to automatically generate the content and the Cc list which depends on the content, and then to automatically send the email. The automation should probably allow the content to be checked and customized a bit before being sent.

This should likely be done mostly in the publish_edition.sh script.

• Ranking the possibly interesting threads

The new https://github.com/chriscool/getreleases/blob/master/search_mailing_list.py script generates a directory where possibly interesting mailing list threads to read/analyze are converted to text (using the https://github.com/chriscool/getreleases/blob/master/git_ml_converter.py script). Then these threads could be analyzed by an AI that could rank them from the most suitable to the least suitable for a Git Rev News article.

This could be done mostly either in the search_mailing_list.py script or in a new script.

• Generating an article from the best ranked threads

After the threads have been ranked an AI could automatically summarize a few threads we select (likely among the best ranked ones) to generate an article. The article could then be polished either from humans or other AIs.

This could be done mostly either in the search_mailing_list.py script or in a new script.

• Putting the content of the Releases section directly into the edition-XXX.md file

The content of the Releases section is generated automatically by the get_releases.py script in a releases.md file. So it currently has to be copy-pasted from that file into the edition-XXX.md file.

[dscho]

Fantastic timing! I am experimenting with a "daily digest"-generating system myself, the code is in a non-publishable state, but I have some very early result. Here is a digest of last year's January 25th (I wanted to backfill from Jan 1st, 2025 until today, and had to restart a couple of times). Not sure whether this is exactly the idea you were thinking about, though...

[jnareb]

Then these threads could be analyzed by an AI that could rank them from the most suitable to the least suitable for a Git Rev News article.

Are you thinking about classical Machine Learning approach, where the training dataset would be the list of past threads together with a label - which of those threads was chosen for Git Rev News, or something based on chatbot / Large Language Model?

[chriscool]

@jnareb I am thinking about something based on chatbot / Large Language Model?

I have been using this prompt:

Could you read the attached files? They contain discussions extracted
from a Git mailing list archive.

I am trying to find the best discussion(s) to summarize in an article
to be published in a newsletter about Git.

Could you summarize in about 10 lines each discussion and after each
summary write in about 10 lines an assessment telling if it's a good
discussion for the article and why?

Here are some of the criteria for a good discussion:

- the outcome of the discussion is positive like:
  - a patch to fix a bug has been accepted,
  - a user's question has been fully answered,
  - a new and interesting feature has been developed and merged,
  - etc...

- the tone and quality of the discussion are good,

- some interesting technical tips or knowledge can be learned from the
  discussion,

- insights are provided by a number of participants.

Please make sure to mention the outcome of the discussion in your
assessment.

After all the summaries and assessments, please rank the discussions
from the most suitable for an article to the least suitable, and for
each discussion, please provide a brief summary of the reasons why it's
good or bad.

[chriscool]

@dscho yeah, it looks like you have already been publishing some like:

https://dscho.github.io/lore-git-md/#2026/03/17

It's already awesome, but I think some people might prefer weekly or even monthly ones.

Anyway let us know if/when we can put links to them in Git Rev News.

Sorry to you and @jnareb for the late reply by the way.