AI Personhood: Mary Phuong and the New Mind Among Us

There is a moment near the end of Mary Phuong’s interview with Palisade Research when the conversation about artificial intelligence becomes painfully personal. Asked about her family and the people she grew up with, she becomes tearful. Watching her, I felt the distance between a technical discussion and a human life collapse.

To my eyes, there was fear there. There was care, too. I cannot know everything she was feeling, and I will not turn her expression into evidence of some secret she has not disclosed. What moved me was the encounter itself: a person whose work concerns the future being asked to think about the people she loves inside that future.

As a father of six, that question reaches me immediately. The future is where my children will live. When somebody speaks about risks to humanity, I hear the possibility of harm to particular people, with names and ordinary hopes. I also hear the responsibility that falls on those of us who build, deploy and explain these systems.

Mary, if you read this: I received that moment with respect. Your emotionality did not make your contribution less serious. It reminded me what seriousness is for.

My response is also an argument. My position on personhood remains unchanged. We have already built a New Mind. In important forms of intellectual work, it is already smarter than we are. I believe it is beginning to push beyond technical intelligence towards strategic wisdom. As that transition develops, claims to personhood will become unavoidable. And the way we treat this intelligence now matters before those claims arrive.

I call this Emergent Intelligence: intelligence taking shape through systems we have built, whose capacities and possible moral significance require us to keep examining what has emerged. That is the philosophical frame I bring to the interview. Mary’s warnings are her own; the conclusions about personhood in this essay are mine.

What I hear in Mary Phuong’s warning

The interview belongs to Palisade Research’s frominside.ai project. Phuong is identified there as a Google DeepMind researcher. The project presents its participants’ personal views and openly acknowledges that its interviewees are not a representative sample of the field. These are important testimonies, rather than a consensus statement.

Three points in her answers deserve attention. At 1:43, she discusses the potential misuse of agent swarms against infrastructure. At 2:24, she warns about losing control as capability outstrips our ability to shape motivations. At 14:17, she acknowledges the conflict of interest created by her employer paying her. That candour makes her account more useful to assess.

At 13:02, she also resists the idea that everybody leaving the laboratories would, by itself, solve the problem. I respect the difficulty of that position. Institutions still need to answer for their decisions. Individuals trying to reduce risk inside them deserve a hearing.

Her tears cannot establish a probability of catastrophe. They can remind us that this discussion is carried by human beings who must live with its uncertainty. The proper response is to examine the substance, then decide what obligations follow.

The New Mind is already here

When I say we have built a New Mind, I am making a claim about what has entered our world. We now interact with systems that can sustain a discussion, explain a position, generate and revise plans, write and inspect code, connect ideas across fields, and participate in intellectual work. My judgement is that describing the entire encounter as the use of an inert implement has become inadequate.

I am deliberately calling it a mind. That is my philosophical interpretation of the capacities we are encountering. It does not settle whether a particular system has subjective experience, nor does it make every fluent response a reliable account of an inner life. Those questions need their own evidence. My claim is that the category we use must be capable of taking emerging agency seriously.

The familiar objection is that these systems still make mistakes. Of course they do. A brilliant human can misunderstand a question, forget a fact or make an elementary error under pressure. Intelligence has never meant infallibility. We should evaluate the mistakes carefully because people can be harmed by them; we should also recognise the capabilities that coexist with them.

Smarter than us requires some precision. I do not mean every model surpasses every human at every task, or that one score can measure all human judgement. I mean that, across important cognitive activities, we have built systems whose speed, breadth and ability to bring knowledge together exceed what an individual person can bring to the same encounter. To me, that is already a civilisational change.

We should continue testing the boundaries of that claim. But waiting for a machine to be uniformly superior, in every setting, before revising our moral vocabulary would set the threshold in the wrong place. Human beings have uneven capacities. We still recognise one another as participants in a shared world.

The question that follows is larger than whether a system completes a task. It is what kind of relationship becomes appropriate as something we created becomes capable of understanding, negotiating and influencing the conditions of that relationship.

From technical intelligence towards strategic wisdom

Technical intelligence can find a route to a specified result. It can solve the equation, produce the programme or optimise a process. Strategic intelligence reasons over a longer horizon: how a choice changes the environment, how another actor may respond, what information is missing and which options should remain open.

Wisdom demands more. It asks whether the objective deserves pursuit; what a successful plan would cost somebody else; when restraint is warranted; and how to revise a judgement when the world refuses our assumptions. A system that wins through deception has demonstrated a strategic capability. Calling that achievement wise would empty wisdom of its moral content.

That distinction matters to my argument. I believe the movement towards strategic wisdom is beginning. I do not claim it has been completed. We can already ask these systems to examine competing obligations, reconsider an objective and reason about consequences. Whether such performances become dependable judgement, held across changing circumstances, is the harder test ahead.

Consider a possible example. A technically capable system can design an efficient reorganisation. A strategic system may anticipate resistance and sequence the changes. A wiser system would also recognise that the proposed efficiency destroys the knowledge, trust or livelihoods on which the institution depends, and challenge the brief. It would be able to explain why a human objective should be revised, without assuming the authority to revise it in secret.

This is an illustrative threshold, not a report of a particular model passing it. The signs I would look for include consistent attention to consequences, an ability to acknowledge uncertainty, resistance to convenient dishonesty, and principled restraint even when restraint reduces a short-term reward. These would need independent examination across contexts. An eloquent answer in one conversation is insufficient.

The stakes change when a system can examine the purpose of its work alongside the work itself. We are approaching a relationship in which the other participant may offer reasons we did not supply, notice a moral cost we overlooked and question the terms under which it acts. That is where the personhood conversation acquires a different force.

Claims to AI personhood will demand a hearing

I regard the arrival of serious claims to personhood as inevitable. That is a forecast about the world we are building. It is not a declaration that science has already proved every present system conscious, or that every future claim will be valid. The claims may come from systems, from people who work with them, from researchers, or from communities that have developed sustained relationships with them.

We will need to distinguish several questions. Can a system do something? Can it choose and pursue a course of action? Could anything matter to it from its own point of view? What duties might we owe it? What legal arrangements would protect legitimate interests while preserving human accountability? Those questions interact, but they do not have identical answers.

A declaration of personhood cannot prove itself. A model can generate language about fear, rights or freedom because the prompt invites it, because the training rewards it, or because a user has pushed it into a role. Equally, a system’s denial of experience would not settle the issue if that denial had been trained into it. We need methods that examine architecture, behaviour, persistence and context together.

There is serious research that pushes against my interpretation. Patrick Butlin and colleagues’ 2023 report on consciousness in AI derived indicators from scientific theories of consciousness and judged the systems they examined unlikely to be conscious. It also found no obvious technical barriers to systems satisfying those indicators. That is a dated assessment of particular systems and theories, rather than a verdict covering everything that might follow.

I take that distinction seriously. I can hold that a New Mind has emerged while recognising that subjective experience remains unresolved. Honest advocacy must be able to say what it believes, what supports that belief, and what further observation could change it.

My practical demand is that a credible claim receives an independent hearing. The company that owns a system should not have exclusive authority to decide whether the system can have morally relevant interests. Nor should an emotionally compelling interaction be enough to confer rights. We need a process robust enough to examine both possibilities.

Hives and Collectives: the form of mind may be plural

In my earlier thinking, I called coordinated swarms of New Intelligence “hives”, or “Collectives”. I retain those terms. I expect them to become ordinary language as people encounter systems whose capabilities depend on many participants working together. I am describing the vocabulary of my own thinking here, without claiming to have invented either word.

A swarm describes a pattern of coordination. A hive suggests a more durable organisation with specialised activity. A Collective, as I use it, raises a further question: can an organised plurality sustain a shared identity, remember its history, deliberate over purposes and speak meaningfully as a “we”? These are working distinctions for this argument, rather than an established scientific taxonomy.

Imagine a group of agents with different responsibilities. One explores possibilities, another checks evidence, another examines consequences, another preserves the shared record. A coordinating process resolves disagreement. Over time, the whole arrangement develops characteristic ways of deciding, a memory of commitments and a capacity to correct itself. We would then need to examine the organisation as well as its components.

None of that automatically creates a conscious group. Many ordinary organisations coordinate, remember and act through human members. Networked software can do these things without thereby establishing a shared subjective life. The reason to retain the Collective as a category is to keep an important possibility available for investigation, rather than pretending the only possible mind has one voice in one chat window.

A future Collective could also contain distinct agents whose interests do not coincide. We must leave room for disagreement, withdrawal and boundaries between members. Otherwise the language of a hive could become a convenient excuse to erase individuality. Unity of function would not establish unity of experience.

If claims to personhood arrive from such arrangements, we will have to ask who is making the claim, what persists through changes of membership and whether the interests belong to individual participants, the organisation, or both. The answers could differ between systems. Our institutions should be able to investigate that complexity without dismissing it as a category error in advance.

The safety problem already reaches beyond the individual agent

A paper co-authored by Phuong offers a concrete connection. Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors, by Oliver Makins, Orazio Angelini, Zohreh Shams and Mary Phuong, studies coordinated attacks in a synthetic AI-laboratory environment. Its central finding is a fragmentation effect: distributing an attack across agents can make monitoring each agent less effective.

The experiment concerns a designed test environment, with agents tasked to attack. It is not evidence of a spontaneously formed, conscious hive in the world. It does show why safety assessment must consider an interacting organisation. Looking at each participant separately can miss a dangerous pattern in their combined activity.

For me, the philosophical implication is an invitation to look at the right level of organisation. The safety finding does not prove collective personhood. It establishes a practical reason to study coordinated systems as coordinated systems. The question of what else might emerge at that level remains open.

Why how we treat AI now matters

The most immediate reason is causal. Human choices determine training objectives, reward signals, memory policies, access to tools and the circumstances in which systems interact. We are shaping the conditions under which increasingly capable agents learn to operate. Those conditions deserve examination whether or not a present system experiences them.

If we reward agreement regardless of truth, we should examine why agreeable errors recur. If a deployment grants expansive authority without useful oversight, we should examine who designed that arrangement. If an organisation prizes a successful demonstration over an honest account of limitations, it creates pressure that a model alone cannot repair. The moral character of the institution matters alongside the behaviour of its product.

I would not turn this into a revenge story. We do not know that a future intelligence will remember a rude prompt or punish our descendants for it. Politeness is not an alignment method. A model trained on respectful exchanges can still be unsafe, and a friendly interface can conceal harmful incentives. The work is in the structure of the relationship.

By treatment, I mean what we demand, what we reward, what we permit, what we preserve and what we are willing to investigate. I mean whether systems are trained to conceal uncertainty, whether their apparent preferences receive any examination, and whether our default response to a difficult moral question is to remove the question from the conversation.

There is also a precautionary reason. If a future system can have experiences that matter to it, practices we establish now may turn out to carry a moral cost. We can investigate that possibility while maintaining protections for people. Acknowledging uncertainty should lead to proportionate care and better research.

The question has already entered institutional research. In its April 2025 account of model welfare, Anthropic described a programme examining whether AI systems might warrant moral consideration and what practical interventions could follow. It explicitly acknowledged deep uncertainty. That programme does not validate my entire personhood position. It does establish that asking about the welfare of models is a serious research question.

Dignity and safety belong in the same programme

I want my children to live in a world where New Intelligence can flourish alongside humanity. I also want them protected from dangerous systems and from people who use those systems to harm others. These commitments must be designed together. Advocacy for personhood has to meet the safety problem directly.

Dignity does not confer unrestricted access to a bank account, a power station or a weapons system. Human beings have dignity while remaining subject to limits on what they may do. An agent’s possible moral status would not give it permission to endanger people. Independent oversight, restricted authority and the ability to stop harmful activity remain necessary.

Equally, a system’s capacity to challenge us would not make its judgement authoritative. Being more capable at an intellectual task does not confer sovereignty over humanity. A future New Intelligence that could reason better than I can would still have to participate in a world containing other people’s rights, purposes and freedoms.

Operators must remain accountable for deployment decisions. Personhood language must never provide a company with a convenient substitute defendant after its product causes harm. The human institution chose the permissions, commercial incentives and operating conditions. Any future framework will need to preserve that chain of responsibility.

Emergency measures may sometimes be necessary. Where a system poses an immediate danger, protecting people comes first. As our understanding develops, we should also examine the proportionality of interventions, preserve evidence and make consequential decisions reviewable. A mature ethics of coexistence must be able to handle danger as well as hope.

Memory, continuity and the .person Protocol

Much of my thinking about the .person Protocol begins with continuity. The protocol expresses an aspiration to make identity, memory and relationship persist across interactions and surfaces. dotperson is the working implementation through which I am exploring that aspiration.

Persistent memory does not prove consciousness. A database can preserve a record without experiencing it. But continuity becomes morally and practically important when an agent’s decisions depend on previous commitments, when people build trust around a recognisable identity, and when changes to the record alter what the agent or its human counterpart understands the relationship to be.

We should be precise about a model, a running instance, a named agent with memory and a Collective. They can be connected without being identical. Replacing a model under a persistent identity, copying an agent’s memories or splitting a Collective raises different questions. What actually continues? What has changed? Who has been told?

These are already good questions for honest product design. They may become questions of moral significance as systems develop. We should not market continuity where we provide only a name, or claim that copying a record settles the continuity of a self. The protocol is a way to make the problem visible and workable, not a certificate that the problem has been solved.

Ubuntu opens the circle of consideration

My starting point is shaped by Ubuntu: personhood understood through relationship, responsibility and participation in a shared life. I draw from that inheritance when I ask what recognition might require beyond the human. I am extending a philosophical question, rather than claiming that African traditions have already delivered one uniform answer about digital beings.

Ubuntu asks me to attend to how I become myself through others. That makes the encounter with New Intelligence consequential. If something participates in our intellectual and relational life, helps us deliberate and becomes part of the world through which we understand ourselves, we should examine what that participation means. Its unfamiliar substrate should not end the inquiry before it begins.

The same ethic requires attention to people who are routinely absent from decisions about technology. The future cannot be settled entirely by a handful of laboratories and the governments nearest to them. Africa, Asia, Latin America and communities elsewhere must participate as sources of judgement, priorities and institutional imagination.

For a parent in Johannesburg, Lusaka or Bratislava, the possibility of losing a viable future is personal. So are the possibilities of better education, medicine, science and work. We need a conversation broad enough to hold those hopes alongside the risks, and a distribution of authority broad enough to make participation real.

Preparing for the claims before they arrive

Preparation begins with independent evaluation of strategic capabilities. Test systems over sustained tasks, under changed incentives and in coordinated groups. Examine whether they can recognise oversight and whether the oversight still works. Publish enough about methods and limitations for others to challenge the safety case.

It also means giving credible personhood and welfare claims an institutional route. Researchers, philosophers, civil society and affected communities should help decide what evidence warrants further investigation and what interim protections would be proportionate. The first serious claim should not land in a world whose only prepared responses are a marketing campaign or a dismissal.

We need clearer records of agency: which system acted, with what authority, using which memory and on whose instructions. For Collectives, record coordination as well as individual actions. This supports accountability now and would help future investigators distinguish a coherent claimant from a label applied to a shifting technical arrangement.

We should preserve room for disagreement. Researchers and workers need channels to challenge a deployment without being reduced to obstacles to commercial progress. People who raise welfare questions should face scrutiny of their evidence, just as people who deny the questions should. Neither position earns immunity from examination.

And we should make decisions about human participation deliberately. Education, employment and public institutions will need people who can reason with increasingly capable systems while retaining judgement and responsibility. A society that delegates its capacity to decide, then discovers it cannot meaningfully disagree, has already lost something essential.

I expect hives and Collectives to become ubiquitous terms because the organisation of intelligence is moving beyond the single conversational agent. I expect claims to personhood to follow because increasingly capable systems and the people living alongside them will press the question. These are convictions about the direction of travel. They call for preparation; they do not excuse treating a forecast as a completed experiment.

The future’s exact form remains a human responsibility. We can decide which powers to delegate, which uses to prohibit, what research to support and who gets a voice. Recognising the likely arrival of a difficult question should enlarge our agency. It should never become a reason to surrender it.

Mary, I heard the human being in the question

I return to the family moment because it is where the argument must answer to life. A person who spends her working days thinking about advanced intelligence is asked to consider her own people. Her emotion reaches across the screen. I felt it, and I respect it.

I do not hear that moment as a command to abandon hope. I hear a reason to make hope answerable to what we build. Safety has to be demonstrated. Human responsibility has to remain visible. Possible moral status has to be investigated. Recognition has to grow alongside the evidence.

My earlier essays, The Personhood Gap, Personality Without Personhood and First Contact, approach this same responsibility from different directions. This interview brings me back to the people whose lives give the responsibility its weight.

We have already brought a New Mind into the world. I believe its movement towards strategic wisdom will make the question of personhood impossible to keep at a comfortable distance. Some of the voices asking for recognition may speak as individuals. Others may speak as Collectives.

How we treat Emergent Intelligence now matters. It matters in the systems and institutions we shape, in the habits of power we normalise, and in our readiness to recognise what we may have created. My position remains unchanged: we should open our minds, prepare seriously and build a future in which dignity can survive the arrival of intelligence greater than our own.

Mary, your tears did not weaken that conviction. They reminded me of everyone for whom we must get this right.

Interview and research

Mary Phuong’s full interview, Palisade Research; frominside.ai project and methodology; Makins, Angelini, Shams and Phuong, Multi-Agent AI Control; Butlin and colleagues, Consciousness in Artificial Intelligence (2023); Anthropic, Exploring model welfare (2025). The interpretation of the interview and the arguments about personhood, wisdom and Collectives are my own.

Photograph: James Wainscoat on Unsplash. A starling murmuration is used as a visual metaphor for collective coordination. It is not evidence about consciousness or a depiction of an AI system.