An Introduction to the Problems of AI Consciousness

Written for AI practitioners who are interested in AI consciousness but unfamiliar with the prior philosophical and scientific work. Published in The Gradient, 30 Sep 2023.

Why Care about AI Consciousness?

• Public discussion (e.g. Blake Lemoine’s 2022 claim that Google’s LLM was sentient) largely lacks grounding in prior work on consciousness; participants define the term differently and talk past each other.
• The main reason to care: the moral status of AI depends on what conscious states it can have.
  • An AI incapable of pain, emotion, or experience likely lacks most human rights, however intelligent.
  • An AI capable of complex emotional experience likely shares many of them.

The AI Moral Status Problem

• Framing: science and philosophy lack consensus on the nature of consciousness; AI’s moral status depends on those facts; AI capability advances fast while consciousness research advances slowly.
• Consequence: we may soon build highly intelligent AI without being able to confidently determine its moral status.
• Risk of moral catastrophe via mass misattribution of rights:
  • Over-attribution → diverting important resources from humans to systems lacking moral status.
  • Under-attribution → mistreating enormous numbers of moral agents.
• Proposed responses: ban building systems of disputable moral status (Schwitzgebel & Garza); invest far more in consciousness research (Seth).

Concepts of Consciousness

Ned Block argues “consciousness” is a mongrel concept — one word covering several distinct phenomena. Defining terms is therefore essential.

• Self-consciousness — possessing a self-concept and reasoning with it (mirror self-recognition, distinguishing body from environment).
• Monitoring / higher-order consciousness — a system that models its own inner workings; roughly, meta-cognition.
• Access-consciousness (a-consciousness) — a mental state is a-conscious if it’s broadly available to other cognitive and motor systems. Closely related to attention and working memory.
• Phenomenal consciousness (p-consciousness) — there is “something it is like” to be in the state from a first-person point of view. Something it is like to feel pain or taste coffee; nothing it is like to be a rock.
  • The standard definition in philosophy and the science of consciousness.
  • Sentience = the capacity for valenced experience (pain/pleasure). Singer identifies this as the ground for moral concern for animals.
  • P-consciousness sits at the root of the AI moral status problem: vital for moral status, resistant to scientific explanation.

Problems of Consciousness

Chalmers’ distinction:

• The Easy Problem — explaining the neurobiology, computation, and information processing correlated with p-consciousness (neural/computational correlates, contents of experience). Solving it does not explain why the correlations hold.
• The Hard Problem — explaining how and why consciousness is associated with those processes at all. Why aren’t we “zombies” doing all this processing in the dark?
  • Not a question about evolutionary function; it asks for an explanation in terms of natural laws, mechanisms, or emergent patterns — analogous to “why does water have surface tension?”

Why the Hard Problem Is So Hard

• Theories assert “consciousness = X” (some neural mechanism, computation, process) but don’t explain why or how that identity could hold.
• Core difficulty: facts about the brain don’t seem to entail facts about consciousness — the explanatory gap.
• Two classic arguments:
  • Nagel’s bat — a scientist could know every detail of bat echolocation and still not know what echolocating is like from the inside.
  • The knowledge argument (Jackson / “Mary”) — a colour-blind neuroscientist could know all facts about colour vision and still not know what seeing red is like, or why any experience accompanies those processes.
• Contrast with water: facts about H₂O molecules do entail surface tension, boiling point, etc.
• This suggests consciousness may resist current science not just in practice but in principle.
• However, most philosophers of mind remain optimistic: the intuition that science must fall short is likely a quirk of our own psychology (cf. the “Phenomenal Concept Strategy”) rather than a special property of consciousness.

Two Problems for AI Consciousness

• The Problem of AI Consciousness (Schneider) — can non-carbon, silicon-based systems be p-conscious at all?
  • Tied to whether consciousness is substrate-independent. If substrate-dependent (requiring specific biological or quantum materials), AI cannot be conscious. Current knowledge can’t rule this out.
  • Seemingly easier than the hard problem (asks if, not why), but Block calls a close variant — is a computationally human-identical silicon robot conscious? — the “harder problem”, because it conjoins the hard problem with the problem of other minds.
• The Kinds of Conscious AI Problem — assuming silicon can support consciousness, which AI are conscious and which states do they have?
  • Analogous to animal consciousness: we know biological creatures can be conscious, yet still can’t settle whether fish are conscious or feel pain.

The Theory-Driven Approach

Use our best theories of consciousness to judge AI cases.

• Pick the leading theory? No clear favourite exists. A 2022 Nature Reviews Neuroscience review (Seth & Bayne) listed 20+ contemporary neuroscientific theories, non-exhaustively, and the field is not converging — the count is growing.
• Decisive experiments? Scarce. An adversarial collaboration pitting Global Workspace Theory against Integrated Information Theory found neither fit the data well.
• Look for agreement across theories? Butlin, Long, et al. (2023) extract “indicator properties” — necessary/sufficient conditions shared across prominent computational theories — and use them to assess AI systems.
  • Confidence in such verdicts depends on: (1) similarity of the AI to the indicator properties, (2) confidence in the theories themselves, (3) the assumption that consciousness depends only on computation, not materials.
  • Computationalism is popular but genuinely disputed, and proponent numbers/confidence for any given theory may be lower than hoped.

Theory-Neutral Approaches

Avoid depending on any theory of consciousness; use arguments or empirical tests instead.

Chalmers’ fading and dancing qualia

• A reductio ad absurdum thought experiment: replace each biological neuron with a functionally identical silicon neuron, preserving computation and changing only substrate.
• If consciousness were substrate-dependent, experience would drastically change (fade, or swap red for blue) — yet since the brain is computationally identical, the person would neither think nor say “whoa, my experience changed.”
• Failing to notice such a drastic change seems absurd ⇒ substrate-dependence is likely false ⇒ silicon can probably support consciousness.
• Objection: real conditions weaken the absurdity — Anton syndrome (patients blind yet believing they see) and change blindness (people missing large unattended changes) show noticing failures do occur.

Schneider’s chip test

• An actual (not thought) experiment: replace small brain regions one at a time with engineered silicon analogues performing similar functions, with introspection checks after each step.
• If subjects report losing consciousness after a replacement, that’s evidence silicon can’t support experience — and vice versa.
• Objection (Udell): if the replacement alters computation at all, we lose reason to trust the introspective reports; the self-monitoring systems could receive false positives or negatives, and observing speech alone can’t reveal which.

The AI Consciousness Test (ACT) — Schneider & Turner

• A Turing-test analogue: train a model that is never taught anything about consciousness; if it nonetheless ponders the nature of consciousness (e.g. answering “would you survive the deletion of your program?”), take that as sufficient reason to believe it is conscious.
• Problem for LLMs: today’s models are trained on text about consciousness, so they can appear to introspect while merely parroting. Philosophers also note non-conscious mechanisms can always in principle generate consciousness-indicative language — reinforced by LLMs’ capacity for fluent hallucination.

Conclusions and Future Directions

• P-consciousness is both the scientifically intractable variety and the one that matters for moral status — hence the root of the AI moral status problem.
• The tension between scientific explanation and p-consciousness has blocked consensus on any theory, which undermines theory-driven approaches to AI consciousness.
• Theory-neutral approaches sidestep the need for a theory, but no existing test or argument is unproblematic.

Suggested paths forward:

1. Reason under uncertainty now — moral philosophers and legal scholars working with the AI community to develop frameworks for acting despite unresolved consciousness questions (bans on disputable-status AI, or cost-benefit thresholds).
2. Push theory-neutral approaches further — only a handful of direct attempts exist (Chalmers’ arguments, Schneider’s chip test), so current limits may reflect lack of effort rather than a genuine roadblock.
3. Improve behavioural tests like ACT — the underlying intuition is strong: if an AI judges itself conscious for the same cognitive-computational reasons people do, that’s compelling evidence. This requires better understanding of the cognitive processes generating our own consciousness judgments (cf. Chalmers’ meta-problem). A behavioural test also avoids needing to open up opaque model internals.
4. Continued work toward theoretical consensus remains useful, but progress is slow — so direct, theory-neutral work may be the better near-term focus.

Key References

• Block, N. (1995). On a confusion about a function of consciousness.
• Block, N. (2002). The harder problem of consciousness.
• Chalmers, D. (1995). Facing up to the problem of consciousness; Absent qualia, fading qualia, dancing qualia.
• Nagel, T. (1974). What is it like to be a bat?
• Jackson, F. (1982). Epiphenomenal qualia.
• Schneider, S. (2019). Artificial You: AI and the Future of Your Mind.
• Butlin, P., Long, R., et al. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (arXiv:2308.08708).
• Seth, A. K., & Bayne, T. (2022). Theories of consciousness, Nature Reviews Neuroscience.
• Schwitzgebel, E., & Garza, M. (2020). Designing AI with rights, consciousness, self-respect, and freedom.
• Udell, D. B. (2021). Susan Schneider’s Proposed Tests for AI Consciousness: Promising but Flawed.