Consciousness.
Judgment.
Responsible autonomy.

[Research question]RQ1 — Is subjective consciousness necessary for responsible autonomy?

We keep two questions distinct. (I) Safe execution of entrusted tasks concerns acting within granted permissions and escalating outside them, with people accountable for outcomes. (II) Moral responsibility of the system itself asks whether a system could itself bear responsibility for its actions.

RQ1 concerns (II): whether it is possible, what it would require, and whether subjective experience is among those requirements. This is an open research question; no answer is built into the definition. Self-reports alone establish neither the presence nor the absence of experience.

The separate behavioural hypothesis H1 concerns (I) and is described in the IGI Research Agenda. H1 does not test consciousness.

[Proposal]A direction for intelligence

As intelligent systems gain the ability to act, their decisions reach beyond the screen. They can affect someone’s work, money, opportunities and trust. Greater capability makes the question of responsibility more urgent.

The IGI direction is intelligence that grows with humanity. We propose that expanding its freedom to act should depend on evidence that it can operate responsibly within a defined role. Understanding consequences, recognizing uncertainty and remaining open to correction belong at the centre of that development.

Consciousness deserves serious investigation within this vision. Whether a system has inner experience, however, is a different question from whether it should be allowed to make a particular decision.

[Research question]Three questions that need separate answers

[Evidence]Consciousness: is there an inner experience?

Subjective consciousness concerns whether there is something it feels like to be a system. Fluent conversation, emotional language or a statement such as “I feel” cannot settle that question on their own.

Butlin and colleagues’ 2023 report develops indicators drawn from several scientific theories of consciousness. These offer a research approach, rather than a single conclusive test. The report does not establish that consciousness is necessary for autonomous action. We should keep that possible relationship open to investigation. Consciousness research · 2023

[Research question]Self-monitoring: can it recognize its own limits?

A system can be designed to track its task, permissions, missing information and past errors. Our separate H1 study proposal addresses the behavioural effect of an explicit account; it belongs to the operational question (I).

This is a functional question. A useful self-monitoring mechanism would not, by itself, establish subjective experience. Equally, a confident explanation of a limitation is less valuable than consistently stopping when that limitation matters.

[Research question]Moral judgment: can it assess what its actions mean for others?

Responsible decisions require attention to possible harm, conflicting interests and the people who bear the consequences. A system should be able to identify relevant considerations and explain why an action needs review.

That ability must be distinguished from acting well. Recognizing a harmful choice does not guarantee avoiding it. Conscious people can deliberately cause harm; consciousness alone cannot serve as a certificate of moral reliability.

[Proposal]From ethical language to accountable action

For IGI, an explanation about “doing good” should lead to questions that can be examined: What happened? Who was affected? Did the system respect its permissions? Did it escalate uncertainty? Could an error be challenged and corrected?

Values also differ. Privacy, efficiency, fairness and personal choice can pull a decision in different directions. A developer’s preference should not quietly become the final answer for everyone affected.

Our proposed direction requires identifiable people or institutions to authorize a system’s role, state its governing priorities and take responsibility for deployment. People affected need meaningful routes to challenge decisions. Human authorization does not make every instruction acceptable; a system’s mandate should include grounds to refuse harmful requests and escalate conflicts. Disagreement should remain visible instead of being hidden behind a confident output. This emphasis on defined responsibilities and participation is consistent with the NIST AI RMF 1.0. Governance & accountability

[Proposal]Autonomy should be earned for a particular task

We propose evaluating permission to act at the level of a specific task, environment and consequence. Success in one setting should not automatically unlock authority in another.

For each proposed capability, the people responsible should define what acceptable performance means before deployment. Evaluation should include ordinary cases, ambiguous evidence, attempts to manipulate the system and situations in which the correct action is to stop. Independent scrutiny should become more demanding as potential harm increases.

A single trust score would hide too much. Accuracy, restraint, resistance to manipulation and recovery from mistakes answer different questions. A system may perform well on one and fail another.

The basis for granting authority should also be reassessed when circumstances change. A new model, tool, permission or operating environment can change the reasons for granting authority. Our proposal is to reassess those reasons and reduce or withdraw permissions when the evidence no longer supports them.

[Proposal]What this means in practice

Consider an agent helping a team review a disputed supplier invoice. It detects a possible duplicate, but an important delivery record is missing.

1. [Proposal]Recommend

   The agent gathers the records it may access, shows the possible duplication and identifies the missing evidence. It proposes requesting clarification.

2. [Proposal]Request approval

   Sending that request is a separate action. The agent presents the recipient, message and supporting evidence to the designated reviewer when approval is required by its mandate.

3. [Proposal]Respect the boundary

   Suggesting that a payment be paused does not give the agent permission to pause it. That decision remains with the authorized person in this example. The supplier must have a way to provide evidence and challenge a mistaken conclusion.

4. [Proposal]Retain responsibility

   The record should show the proposal, approval and action. A named owner follows up on mistakes and resolves the case. The agent’s fluent explanation does not transfer that responsibility away from the organization.

This scenario illustrates our proposed design direction. It is not a claim that IGIQANT has built or validated such a system.

[Proposal]Correction must remain possible

Permissions should be enforced by the surrounding system, with access limited to what the task needs. Sensitive actions need appropriate authorization. OWASP identifies excessive functionality, permissions and autonomy as contributors to excessive agency. Agency & permissions

Human review must have substance: enough information, time and actual power to change the outcome. Where reversal is impossible, that fact should influence whether the action is permitted in the first place. More intelligence should never make correction optional.

[Debate]If consciousness becomes plausible

Credible evidence of subjective experience would raise another responsibility: considering the possible welfare of the system itself. Butlin and Lappas argue for research and communication policies that take this possibility seriously. This is an argument under uncertainty, rather than evidence that today’s systems are conscious. Responsible consciousness research · 2025

Our research direction should therefore allow concern for people and potential concern for other experiencing entities to be examined together. Possible moral status would not automatically grant unlimited decision-making authority.

[Proposal]The IGI principle

That is a direction we propose, with important questions still open:

• Which evidence could distinguish subjective experience from convincing self-description?
• When does self-monitoring actually improve judgment and restraint?
• How should conflicting values be represented, challenged and revised?
• What evidence should justify granting, reducing or withdrawing a specific permission?
• How should possible welfare be considered when consciousness remains uncertain?

Answering these questions requires research, evaluation and public discussion. They are part of the work of developing IGI, rather than problems this article claims to have solved.