AI

Summary execution

In one of our dog walking chats, Maren and I were talking about AI (I know, I know), and she was saying she didn’t really see the benefit of it in most circumstances. I was trying to be the AI pragmatist and responded that “it’s good for summarising documents”. To which Maren replied “but I don’t want that, I want to read the thing”. And Audrey has made a similar point in her newsletter pointing out that the summarised version of knowledge is “more efficient, to be sure. It’s also much, much safer.” The Apple Intelligence adverts you may have seen also make the ability to create quick summaries the main selling point.

David Wiley gives an example of how he’s “created a research assistant agent that reads through preprints each morning and identifies the latest research on the impact of generative AI on teaching and learning”. I can see how that would be useful, and I know from having a daughter in higher education, it’s useful to get a summary of research in a field that you don’t need to go into depth for. I still think summarising is useful. But Audrey and Maren have made me ask the question is it that great? First, if we just want the summary version of everything, then why are we writing all this other stuff to start with? Second, just how reliable is that summary you are getting, and will it miss the interesting nuance. Third, doing that reading, watching, listening is often actually the thing you want to be doing, the process is the point. Anyone who watches sport knows there’s a lot of difference between watching a full game and getting the highlights.

But beyond the whole “summary fetishisation” there is a more worrying trend. That is when you can only get access to the AI summary. Tom Scocca reports on how The Washington Post has (or is experimenting with) removing its archives, and instead giving access to AI trained on its archive. So much more efficient, right? Needless to say, anyone wanting to do actual research will want access to full archives, not just some bland summary of them. As Scocca puts it “No one who cared about the purpose of the Washington Post or the purpose of the Washington Post archive would have ever allowed the Ask The Post AI to be deployed. But the world has allowed the management of knowledge to be taken over by ignoramuses, and now the ignoramuses have built ignoramus machines in their own image, manufacturing non-knowledge on a scale previously unimaginable.”

You can see lots of others going the same way, we only need summaries, let the AI do the hard work. This would be a disaster for our information ecosystem. Summaries summarised into new summaries. No actual knowledge, just summaries all the way down. It’s an example of two things – first the unquestioning belief that summaries are always a good thing, and second handing over systems to AI without due care to consequences.

For fun, here is the AI summary (from Pop AI) of this post. I will say, it’s pretty good. But it lacks that Weller charm right?

________

Overview

The document discusses the implications of using AI for summarizing content, highlighting both its benefits and potential drawbacks.

Key Points

  1. AI and Summarization:
    • The conversation between Maren and the author reflects skepticism about the utility of AI, particularly in summarizing documents. Maren prefers to engage with the full content rather than a summary.
    • Audrey’s newsletter emphasizes that while summaries are efficient and safer, they may lack depth and nuance.
  2. Usefulness of Summaries:
    • David Wiley’s example of an AI research assistant illustrates the practical benefits of summarization in academic settings, especially for those who need to stay updated without delving deeply into every piece of research.
    • The author questions the value of summaries, pondering why original content is created if summaries are preferred. There is also concern about the reliability and completeness of AI-generated summaries.
  3. Concerns About Access to Information:
    • A troubling trend is noted where access to original content is being replaced by AI-generated summaries, as seen in The Washington Post’s experiment with its archives.
    • Tom Scocca criticizes this approach, arguing that it undermines the purpose of archives and the integrity of knowledge.
  4. Impact on Information Ecosystem:
    • The document warns against the over-reliance on AI for summarization, suggesting it could lead to a cycle of summaries devoid of original knowledge.
    • This trend reflects a broader issue of uncritical acceptance of AI systems and their consequences on knowledge management.

Conclusion

The document raises important questions about the role of AI in summarization, advocating for a balanced approach that values original content and critical engagement with information rather than solely relying on AI-generated summaries.

(Image – Summary judgement by Nick Youngson CC BY-SA 3.0 Pix4free)

Subscribe to the newsletter to get each month’s posts plus subscriber only content! (A new picture of my dog Teilo)

4 Comments

  • Mohamed Yaseen Kassam

    Thanks Martin for sharing this. Like you and others have too been thinking lately about how years down then line, just how much cycles of churn knowledge will have to go through in AI until it’s not really “real”, bit like ultra-process foods. For me, this line/nuance/Weller charm 😉 summaries the whole article nicely: “Anyone who watches sport knows there’s a lot of difference between watching a full game and getting the highlights.” Yet, the AI version, just casually skips over it.

    When I was a very young kid I remember, coming to the UK from a place where strawberries were not readily available, I remember when I got to tasting strawberries for the first time, being disappointed that strawberries did not taste like the Opal Fruits (old name for Starburst in the UK) sweets. We might be heading to a time quickly where younger generations may prefer summaries to the real thing!

    • mweller

      Hi Mohamed – ha, I love the Opal Fruits analogy! That is quite right. As an aside, I like banana tasting things but not actual bananas 🙂

  • Julian Prior

    “Anyone who watches sport knows there’s a lot of difference between watching a full game and getting the highlights.” That said, there’s a world of difference between, say, the way Match of the Day produces its highlights and a channel like Fubo in Canada does. The former (hopefully post-Linekar too!) has plenty of nuance, craft, skill and humanity … Fubo on the other hand is a hodge-podge of badly edited highlights where commentators are often cut off mid-sentence and goals are sometimes missed. It’s almost as if they are using an AI editor rather than a real human being…

    • mweller

      That’s a good point Julian – creating highlights, and summaries is a skill in itself. Also, to stretch the metaphor a bit further, two rival fans can watch the highlights and feel they have been biased against their team, so the summary always leaves room for more misinterpretation

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php