mfred's personal page

Thoughts on generative AI (early 2026)

Table of contents

Is there a better topic to start off this blog than the one everyone keeps hearing about all day: Generative AI?

I’ll start this website by collecting my thoughts on the current state of generative AI (especially large-language models) and the tools built on top of it. I will focus on the use-cases I am most familiar with (chat bots and coding assistants) for most of the examples I will give.

I am by no means an expert on the topic, or not even a heavy-user, so do not expect to find any bleeding-edge research insights here. I am writing this piece of text with two target readers:

I do not have any financial interest in (or against) any company making money (or trying to) using that technology, I also don’t have any particular passion or disgust for generative AI, so I really hope to make that as neutral as possible.

I want to keep this short on-purpose and will not dive too much into the details for this summary. If I feel like it, I may write a dedicated post to delve deeper into some generative AI use-case(s).

I’ll follow the below agenda:

The good vibes

This is definitely not a very original stance, but I must still say this upfront: I think generative AI tools that have been released in the last couple of years are simply mind-blowing. Even though they are not perfect (we will cover that later), the results of applying generative AI in many different domains (text generation, summarization, proof-reading, image / video generation, programming, etc.) are astounding, and have surpassed other approaches. And this is not just about throwing more data, computing power or money at those problems: the results of gen AI in some domains are just on another level entirely. I find it hard to understand how some people can be unimpressed by the recent progress of gen AI, calling these tools simple statistical parrots, and I think these comments are either pure ignorance or bad faith, probably driven by all the bad side-effects often hidden under the carpet (again, we will talk about these later).

Gen AI tools especially shine when the output we expect from them does not have to be “true”, “exact” or “exhaustive”, such as:

And even for use-cases where correctness matters, gen AI tools can provide value when they can be paired with verification tools (such as formal proof assistants in mathematics) which will be able to reject hallucinations, and explain why these were rejected.

LLMs are also really good at dealing with natural language and the natural ambiguity that comes with them, enabling us to transparently interact with computers with natural language. Coding agents do not need formal specifications, they can start working from an incomplete document, and build assumptions or ask clarifications when needed. They are also really capable of using error messages, written in natural languages, making them great at trial and error. I personally find it very satisfying to see these tools trying out different approaches when an API parameter type is incorrect, or when there was a typo in a parameter name in the initial prompt, and being able to react based on error messages and hence not being stuck. These kind of errors are something classical approaches typically struggle with, even though they look really silly from a human perspective.

The unknown

Is this intelligence ?

As impressive as gen AI tools can be, most of us are still reluctant to call these intelligent. I am not really interested in this debate, as we human beings are not able to properly define what this even means. The current approach of LLMs, inferring the next word from the content, is not something I would intuitively call intelligence, but maybe I’m wrong about that. After all, most of the ways I would think of evaluating someone’s (or something’s) intelligence would eventually boil down to asking that person (or thing) to produce a sequence of words. So I will definitely stay away from that debate, and I think anybody showing strong opinion on this (either the “we are N months away from AGI” or the “LLM are parrots”-types of people) is overly confident.

A more down-to-earth discussion is to know whether gen AI can build new concepts. Concepts are fundamental for human reasoning, as they allow us to overcome our limited abilities, by reasoning on high-level ideas. LLMs do not have the same limitations as the human body, but they are still limited, and my gut feeling is that they will not be able to explore or produce valuable knowledge without relying on concepts. And building new concepts does not seem to be in the range of the current LLM architecture’s capabilities; building new concepts at least requires to name them, which means building a new word which is outside of the LLM’s training set, and that does not seem to be aligned with the inferring model’s goal. But again, this is tricky because we can’t even explain how human-built concepts really came into existence; so I wouldn’t completely rule out the possibility that gen AI may come up with new concepts.

AI as a teacher for humans

Another pragmatic question is to know whether gen AI tools can become teachers for the humankind, and help us understand and solve the open questions we have. In my opinion, the ultimate goal of AI research is not to build tools that will provide us answers, but rather build tools that will help us expand our common knowledge and understanding of the world and improve our reasoning capacity. As an example, the main value of proving a long-standing mathematical conjecture usually lies in the proof itself, rather than the actual result, which may have been assumed true by most mathematicians and used as assumption for other results. If a generative AI manages to output a million-pages long formally verifiable proof of an open mathematical question, I do not think it will bring much value if it cannot make it accessible to humans.

We can take a step back and look at how it went for other areas where artificial intelligence has surpassed human intelligence, such as the game of Go. Despite its very simple set of rules, humans have explored this game for thousands of years, and we improved our collective understanding of the game by building on top of concepts such as good shapes, aji, sente, influence, etc. These concepts have been explored by strong players and used to teach the next generations. But even though AI has now surpassed human’s strength in that game, I do not think it has significantly improved our understanding of the game. AI has shown preference for some playing styles, and I think it has slightly influenced professional players, but as far as I know it has not really been able to uncover new concepts nor help us deepen our understanding of the game. The only way it can show one move is objectively better than an other is usually to play it out, and show that it leads to a favorable position after 20 moves or so; this is of little help for human beings who cannot read that long of a sequence of moves. Maybe that is because the AI editors have been more interested into improving their AI’s score in benchmarks, rather than work on explainability; and maybe one day the situation will improve. But so far, I am not very optimistic.

Human replacement

Historically, technology has freed human beings from working on dangerous or unappealing tasks, by delegating them to machines, and to focus their time and energy on other tasks which couldn’t be solved yet by technology. Some of this time is then spent researching how we could build new technology to solve new tasks which are not yet accessible to machines, resulting in a progress loop. We may debate whether this loop has really improved the human condition, but at least it has objectively enhanced our collective understanding of the world, and our capacity to act on it.

But so far, humankind has never run out of work: whenever we were able to delegate some tasks to machine, we still had work left to do for the people freed from those tasks. But the physical and intellectual abilities of human beings are limited, and as technology advances and can handle an increasing range of tasks, we may end up in a situation where the tasks that can’t be delegated to machines are also out of reach for most human beings, leaving a significant part of humankind with nothing to work on. This is not necessarily an issue, but this would be a turning point in human history, and something we should prepare for.

Where it hurts

Intrisics problems

The most well-known limitation of LLM is that they tend to hallucinate, i.e. to confidently answer non-sensical or wrong statements. As LLMs are encoded in a finite amount of parameters, they obviously cannot absorb an infinite amount of knowledge, and we should not be surprised to see them make mistakes, just like human beings do. The problematic part is how they make mistakes, and who has to account for them.

First, based on my experience working with LLMs, I get the impression that they very rarely admit they don’t know (at least compared to human beings). I suspect that this may be due to training process: LLMs are typically trained on articles, blog posts, forum threads where humans share problems and answers. In these contents, LLMs will mostly see questions raised, and answers posted by people who know (or think they know) the answer; there are very few examples of “I don’t know” in the training set, which may be the reason why LLM always seem to have an answer to every question. This tendency to always have an answer to every question makes them more prone to hallucinate than human beings.

Another big difference between humans and LLMs is the potential scale and impact of their hallucinations. Even though human beings can also hallucinate and confidently answer something wrong, humans tend to start questionning themselves and double-checking their answers when the potential impact of their statement grows; but LLMs do not seem to proactively question themselves, they need to be challenged in order to double-check their answers and acknowledge their hallucinations. This lack of self-doubt, combined with the ability of LLMs to generate content far faster than any hallucinating human being, makes it very dangerous to replace humans with LLMs for business. If you do not buy this argument, just have a look at what is out there: any company running an LLM-assisted chat bot will include a disclaimer along the lines of “The bot’s response can contain mistakes”; but have you ever seen a human version of it: “Human beings are fallible, their response can contain mistakes”.

And contrary to human beings, LLMs cannot be held accountable for the consequences of their hallucinations. This makes them actually unusable for most business use-cases without human supervision, and I hate how many generative AI introductory courses brush this issue off as something secondary, and focus on the shiny parts of gen AI. The potential impact of gen AI hallucinations is the main reason why, in my opinion, gen AI tools cannot take over all of these white-collar jobs that some business leaders want to eventually automate. This means the value we can expect to gain from using gen AI tools in any use-cases where accountability matters is way lower than the one market and investors seem to expect.

The second issue of gen AI tools is their heavy resources usage and ecological footprint. This may get better in the future, as technology matures and engineers start to take time to optimize these tools rather than focus on releasing new shiny features. But still, part of the success of LLMs is due to the large number of parameters in the models, so the inference process will always require billions of operations; unless we find a completely different way to run these operations, I do not see how that inference process could get significantly less resources intensive. And the number of parameters in the high-performing models just keeps on growing. We can also already see how gen AI kind of cannibalizes the whole hardware supply chain: as of today, several large hardware manufacturer have announced they have sold out all their production capacity to services providers, who are investing trillions of USD to build infrastructure to support the expected gen AI demand.

Finally, while it seems possible in the near-future to run some of these models on hardware accessible to individuals or small-businesses, the training stage will likely remain the exclusivity of large-scale actors in the foreseeable future, because of its cost and infrastructure need. The architecture and the weights of some models are open, but the training process and data is quite opaque. This makes it possible for the actors training these models to have tight control about the information returned by the model, making these tools perfect tools for mass manipulation.

Malicious, reckless and unsustainable usage

And even for smaller actors, gen AI already makes it possible to produce realistic but false photos and videos, which will make it considerably more difficult to distinguish between real and fake. To be fair, this is absolutely not an intrinsic gen AI issue: rewriting history and tampering with historical materials is a usual way of operation for totalitarian regimes; and it was only a matter of time before these techniques became available to the greater public. But generative AI accelerated that by quite a lot, and I do not think our societies have an answer for that.

Gen AI also introduced the AI slop issue: as producing content with gen AI is both fast and cheap compared to human labor, it’s naturally tempting to produce content using these tools and share it as if it was produced by humans, usually for money (content monetization, bug bounty programs). Even beyond the moral issue of publishing content produced by somebody else (or something else, in that case) under one’s own name, the main problem of AI slop is that their author typically do not take the time, or do not have the capacity, to even verify the quality of the content produced by the AI, leaving that up to the moderators of the space where the content is published. And these spaces, which were intended to be used by humans, are typically not ready to process the amount of AI slop they receive now.

The proliferation of AI slop and AI-generated deepfakes means there’s a possibility the Internet as we know it, which is made of a lot of content produced and published anonymously, may die. I am not certain about that though; it is possible people might actually be okay with being fed fake information and machine-generated content if they find it entertaining. In that case, the Internet will likely turn into a battleground for AI to compete for user’s attention. But if people eventually grow tired of this content, they may turn their attention elsewhere, which should bring down the value of producing content on the Internet, and eventually should make it no longer economically viable to flood the web with AI-generated anonymous content. This would be the end of the Web 2.0, as it got called in the mid-2000s (writing that makes me feel old); but maybe not the end of the Internet. It may bring us back to an Internet that looks like the one before that, the one that did not feed users with data, but forced them to look around, to use bookmarks and RSS feeds to follow the content they trust and specifically want to follow. Stated like that, it sounds like a pleasant outlook; but at the same time, this may mean the death of great projects such as Wikipedia.

An interesting side-effect of the death of the Internet is that it may harm gen AI tools themselves. As these tools are partially trained on data coming from the Internet, the shrink of the Internet, or the proliferation of AI-generated content, means that AI will have less human-generated data to be trained with. So in the not-so-long term, these tools will no longer have enough human-generated training materials to sustain their growth. I do not think the amount of training data is the only way current gen AI tools can grow their capacity; but I think it’s still an important factor, and that may hinder the growth of these tools in the near future.

Which is a nice transition to the next issue: gen AI has taken the world by storm, and has generated what I consider completely irrational hopes on how it can improve productivity and generate value. As far as I know, all gen AI tools are operated at loss, and are kept afloat by venture capital and investment of money made from traditional activities. And it seems it will only gets worse in the next couple of years, based on the infrastructure investments needed to support the expected growth of AI usage. Even gen AI companies CEOs publicly acknowledge that the AI market is a bubble, and it seems the only remaining question is how bad it will be when it bursts. Gen AI haters may find this perspective pleasant, but like any other financial crisis before, the AI bubble burst will have dire consequences on all of us, and probably not that much on the AI overlords, so I am definitely not looking forward to it.

The main benefit of the AI bubble burst may actually be that software editors and tech CEOs will stop trying to shove AI into our throat. I have to admit I fail to understand the long-term objective of the forced AI usage that I see almost everywhere these days. When it comes to gen AI editors, I can imagine a few motives, such as wanting to collect more data (to improve their own tools), or making us dependent on these tools, so that they retain as many users as possible when they figure out how to make their service profitable. But I still cannot figure out why gen AI is being force-fed in so many products (even when users feedback is overly negative), or why some companies’ management try to force their staff to use AI, going as far as collecting usage data and promoting the heaviest (and hence most spendthrift) users as AI champions, and/or threatening the non-adopters. One possible explanation would be an irrational fear of missing out on the amazing features and huge productivity gains that AI promises; but so far, it looks like the return on investment of gen AI for the companies who actually tried to measure it is simply not there. For small companies, another driver of hard gen AI adoption might be to surf on the gen AI wave and attract venture capital funds; but these companies will be the first ones exposed when the bubble bursts.

And even if some of the biggest gen AI editors managed to survive after the bubble burst, and can find ways to make profitable products, there’s still a large elephant in the room: the intellectual property problem. As these tools need massive amount of data, it seems that their editors simply went ahead and used whatever data they could get; and as legal concerns over license or intellectual property seemed to be complicated questions, and could slow them down, they just decided to simply ignore it. It also seems that some editors even decided to knowingly train their models on pirated data, as it was both cheaper and more practical to access massive amount of data by downloading them from torrents, rather than going through legal channels. I think we, as human societies, have not found yet a good way to define intellectual property regulations adapted to our modern tools, and gen AI is making this even more difficult, so I am definitely not saying that this is an easy question. But knowing the massive impact that gen AI is likely going to have on some industries and content creators, I find it disgusting that these questions are typically covered as secondary issues, and completely ignored so far by regulators. I suspect our politicians do not want to bring that up as it may slow down the growth of their local gen AI players against foreign competitors. But in spite of the narrative we can read from them, these guys are not training their models and improving their tools for the greater cause. They are doing it for profit, with most of the research done behind closed doors, and I do not see any reason to allow them to basically bulldozer their way through any intellectual property regulation. As many others have raised before, the comparison on how American justice dealt with Aaron Swartz 15 years ago, and how it deals with the large gen AI editors now, is nothing but revolting.

Conclusion

I initially did not plan to write a conclusion for this, but it became longer than I initially planned it to be, so I suspect readers might be looking for a TL;DR.

In short: I think gen AI tools are a massive step-up compared to many of their predecessors, and I am excited about the possibilities they will definitely unlock. The current LLM-based approach of “predicting the next token” looks a little bit superficial, but it is hard to predict if/where this will cap its potential. But besides the intrinsic limit of the tools, I am very pessimistic about the impact these tools are going to have in the short-term on our economies and our societies as a whole.

For the time being, my attitude towards gen AI tools will be to use them parsimoniously, to be transparent with others about when I used them, and ensure I do not participate to the overhype cycle.

Let’s revisit this topic in a while!