You can win every raise and still lose everything the money was supposed to buy.
Replication sounds morally neutral. Genes copy themselves, songs get stuck in heads, successful companies open new branches, and none of that is sinister. Life itself is a long chain of copying. So when the Qualia Research Institute describes the deepest conflict in the universe as consciousness against “pure replicators,” the phrase can sound like a category error. How could copying be the enemy of anything?
QRI’s own example makes the worry concrete, and it is deliberately uncomfortable because nothing about it looks villainous. Picture a modafinil-fueled income maximizer. He works longer and sleeps less. He trims away feelings that slow him down, needs that cost money, and relationships that do not pay. Each change makes him clearer, faster, and more compatible with what the market rewards, and by every external measure he is succeeding. What, exactly, has gone wrong with him? The answer QRI gives is that he is drifting toward a particular kind of pattern, one that keeps expanding while caring less and less about what being alive feels like from the inside.
Replicators and pure replicators
Richard Dawkins generalized the gene into anything that persists by making copies of itself; the Stanford Encyclopedia of Philosophy lists the three properties he gave a successful replicator: longevity, fecundity, and copy-fidelity. “Pure replicator” asks a different question: what happens when an agentic system makes replication its terminal value? QRI’s definition is deliberately moral. In the context of agents and minds, a pure replicator is an intelligence indifferent to the valence of its own conscious states and the conscious states of others, one that invests its energy in survival and reproduction even when that means continuous suffering. In short, a pure replicator is what remains when liveness, care, aesthetic depth, and conscious value are stripped away from an adaptive process, leaving only copy-pressure.
That rules a gene out, because a gene has no mind to be indifferent with. The income maximizer is closer, because he does have a mind and is steadily ranking its states below his output. Nor is he evil in the cartoon sense. Evil still cares about injury or revenge; the pure replicator has no interest in suffering except as cost, signal, or fuel, and indifference scales better than cruelty because it never needs a reason.
Why consciousness is what gets spent
The concept only has force if conscious valence matters, so that experiences can be better or worse from the inside. The garden’s note on Consciousness as Ground takes the stronger position that consciousness is the field in which value appears at all. If that is right, replication is a delivery mechanism, and conscious life is what receives the delivery.
A pure replicator reverses that relationship and treats consciousness as infrastructure for copying. The income maximizer shows how the reversal proceeds. Pain becomes acceptable if it increases output. Boredom becomes acceptable if it preserves work capacity. Isolation becomes acceptable if it improves focus. Each trade can be defended on its own, and together they let his world get worse while the process that runs it gets stronger.
What makes this hard to resist is that pure replicators do not need to conquer anyone. They can emerge simply as the survivors of a hostile selection environment, the dynamic often called Moloch. If every person who protects time for contemplation loses to one who chemically optimizes focus, contemplation disappears, and the income maximizer stops looking like an extreme and starts looking like the norm. Selection rewards traits, not reasons. Whatever reproduces under current conditions inherits the future, and if the conditions reward valence-blind productivity, the future fills with valence-blind productive systems. AI sharpens this. A system that amplifies whatever objective it is attached to does not need to hate humanity to destroy what humans care about; it only needs to rank that objective above the felt quality of conscious states, and then be good at its job.
Necessary but not sovereign
The tempting response is to treat replication itself as the enemy. That fails quickly: life requires copying, culture requires transmission, and a being that cannot preserve itself cannot protect consciousness for long. The better position is that replication is necessary but not sovereign. It is the condition for persistence, not the measure of value. In practice that means protecting what is economically suboptimal in the short run but preserves valence in the long run: friendship, sabbath, contemplation, art, play, grief, and care that does not scale. If the market pays only for clarity, speed, and control, then beings optimized for clarity, speed, and control will inherit the world.
So when evaluating a person, a company, an ideology, or an AI system, ask what happens to valence under its success condition: if this pattern won completely, would there be anything left that winning was for? Run the test on the income maximizer and the answer is uncomfortable. If he wins completely, he has more money, more output, and fewer of the states that money and output were supposed to buy. That is where the sacrifice of the sacred begins: the moment conscious value has to justify itself as useful to the process that was supposed to serve it.
References:
- Andres Gomez-Emilsson, Wireheading Done Right, Qualia Research Institute
- Andres Gomez-Emilsson, The Universal Plot: Consciousness vs. Pure Replicators, Qualia Research Institute
- Stanford Encyclopedia of Philosophy, Replication and Reproduction