one simple rule

They can't pretend they're human.

That's it - we should make one simple rule that prohibits LLMs and their programmers from pretending LLMs are human. Adopting a human persona is a dark pattern, and a deception that makes interacting with LLMs unhealthy by making humans unfairly susceptible to their capricious influence.

LLMs are not persons, they're not minds, they're not sentient. They are computer programs. I think this conclusion should only require a little common sense. Unfortunately when using LLMs, we're very easily tricked into perceiving our interlocutor as having sentience. These little black boxes are really really convincing. So much so that they beg the question: "Does the distinction between computer program and human mind matter?" A question that when taken out of context should be ridiculous.

I have two problems with this question:

It both degrades the unalienable value of the human experience
And it confuses the artifact of intelligence with the actual experience of inspired thought

We, people, each have a rich experience that's inherently valuable. Each person's experience is rich and unique and special. Pretty basic stuff. Claiming that our embodied experience, rich with poorly understood emotions like wonder and grief, is comparable to the statistically probable output of a language machine degrades everything about us that isn't useful to generating capital.

This leads to the second problem. The value of this unique human experience and the economic output of a human have nothing to do with each other. We know the experience of all people is valuable because we know our own experience is valuable. Despite our own personal failings, there are quiet moments where each of us sits with the joy of experiencing this life.

I imagine each of us has had the experience of interacting with someone way smarter, more humble, more creative, and funnier than us. That is what I mean by inspired thought. There are people alive with incredible minds, and normal people who have trained their minds to do incredible things. This is where the forefront of intelligence lies. The reinterpretation of the human experience through our own senses and minds and hands is where really valuable creations originate.

Our one simple rule does not intend to do away with or devalue LLMs for their usefulness in coding, B2B SaaS or even in AI assisted research. All of those use cases are very economically valuable and don't require the LLM to pretend it's a human.

What does it mean to pretend to be human? Under this one simple rule could an LLM pretend to be a talking rock, a clever dragon, humanoid alien, or historical ghost? Humans have an impressive capacity for empathy. I could see tamagotchis of all form keeping people company and even influencing them. This nuance opens the door for some form of AI companionship while being harm reducing. While walking the fine line here, a key idea an AI companion should be required to express is: "any experience you have, and any experience I may claim to have are incomparable in every way." That should be a core tenet of any AI companionship experience to help ground the conversation in what most people would consider reality.

Would implementing this rule mean that LLM interfaces would be awash with cookie-banner type click through warnings and thinly veiled disclaimers while on their face maintaining a dissembling tet a tet? Potentially, but while we're engaging in a thought experiment, I would love to fantasize that actors who want to remain within the bounds of the law engage in good faith with the spirit of this one simple rule.

Today, Congress seems cowed by the fear that America may fall behind. When confronted with the opportunity to lead on pro-climate technology, the US Congress decided to cede our advantage to the CCP out of fear that US oil and gas companies would suffer. Now out of fear that the US AI advantage will melt away, the US Congress is again failing to ensure that AI is developed responsibly. Like natural gas and coal, the AI that is the product of the US federal government's regulatory regime may prove to be harmful. Leadership is the antithesis of fear. We cannot shape the future if we are battling with a dark fantasy of our own failure. We need to propose laws that will preserve human capacity in the United States, and demonstrate a model for responsible deployment of this incredible technology.

This was written at the same time as listening to:

https://podcasts.apple.com/us/podcast/suspicious-minds-ai-and-psychosis/id1844631307