Company · News · Article

Autonomy in AI is not about Capability; it is about Responsibility

At the 1st AI & Ethics Conference in Athens, a central question framed the discussion: How much autonomy should an AI system really have, and when must a human step in?

The debate about autonomy in artificial intelligence is often framed in terms of technological capability: “Can it do it on its own?”

But the correct question is not whether it can, but whether it should and under what conditions.

Autonomy is not binary. It is not simply “yes” or “no.” It is a continuum, and it must be designed based on three factors: risk, reversibility of decisions, and the level of uncertainty within the system.

Risk as the First Determining Factor

First, risk.

The greater the potential impact of a decision on human life, fundamental rights, critical infrastructure, or geopolitical stability, the less acceptable full autonomy becomes.

A system that recommends music or categorizes emails can operate almost fully autonomously. A system that influences medical diagnoses, judicial decisions, or military actions cannot operate without meaningful human oversight.

The Role of Reversibility

Second, reversibility.

If a decision can easily be corrected, the risk is reduced. But when a decision is irreversible, for example an action that leads to physical destruction or loss of life, human intervention is not merely desirable, it is necessary.

The less we can “take back” an action, the more essential it becomes for humans to remain within the decision loop.

The Challenge of Uncertainty

Third, uncertainty.

Every model operates with probabilities. When a system functions in environments it was trained for, using clean data and without adversarial interference, its performance may remain stable.

However, when it operates outside its training domain, when data is incomplete, or when there is deliberate manipulation, as often happens in cybersecurity or conflict environments, the probability of error increases dramatically.

In these cases, human judgment becomes critical. Humans are capable of recognizing the paradoxical, the unusual, or the ethically problematic.

Human Oversight Must Be Real

In practice, this means we should not speak merely about “human in the loop,” but about meaningful human oversight.

The human cannot be decorative. They must have the real ability to intervene: to stop a process, revoke a decision, inspect the data, and assume responsibility.

If the human simply presses “approve” on thousands of decisions per minute, oversight becomes an illusion.

Scaled Autonomy

The correct approach is scaled autonomy.

A system should have greater freedom in low-risk environments and strict boundaries in high-risk ones. There should be predefined operational limits, clear rules of engagement, complete logging of actions for auditing, and “safe failure” mechanisms. When the system is uncertain, control should return to the human.

Ultimately, the objective is not to maximize autonomy.

The objective is to maximize reliability and legitimacy in decision-making.

In critical domains, human presence is not an obstacle to innovation. It is a prerequisite for trust.

Artificial intelligence can accelerate analysis, detect patterns, and generate predictions. But the final responsibility, especially when the stakes are high, must remain human.

Can Europe Develop Reliable Defense AI on Its Own?

The short answer is yes, but only if we properly understand what “on its own” and “reliable” truly mean.

This does not imply isolation from allies. It means strategic autonomy in critical areas, including data, computing power, foundational models, and control over systems.

Reliability in Defense AI

In defense, reliability is not measured only in terms of accuracy.

It is measured by how a system behaves under pressure in conditions of misinformation, cyberattacks, and incomplete data. It is measured by whether it can be audited, certified, tested again, and operate predictably.

Therefore, the issue is not simply whether Europe develops large models. The challenge is to develop resilient, controllable, and certifiable systems.

Four Key Pillars

To achieve this, four pillars are required.

First, access to high-quality operational data.

Defense AI requires real data from sensors, cyber environments, and operational contexts. Without common standards for data management and sharing among member states, development will remain fragmented.

Second, computing infrastructure and supply chains.

Without access to reliable computing centers and without reducing dependence on third countries for critical components, true autonomy cannot exist. Europe must invest in secure infrastructure and strategic supply chain planning.

Third, testing and certification mechanisms.

In defense environments, every system must undergo strict stress tests, red teaming exercises, and resilience assessments against adversarial interference. Reliability cannot simply be declared. It must be demonstrated through repeated testing.

Fourth, ecosystem and procurement processes.

Europe possesses strong universities, research institutions, and innovative companies. However, the transition from laboratory to operational deployment is often slowed by bureaucracy and fragmentation.

Unless procedures are accelerated and common European standards are introduced, the gap with other global actors will continue to widen.

Europe’s Strategic Advantage

Europe has an important advantage: a deep tradition in regulatory frameworks and rights protection.

If Europe manages to transform this tradition into a global benchmark for trustworthy defense AI, it could shape international standards. That would represent a strategic advantage.

Resilience Over Scale

Therefore, yes, Europe can develop reliable defense artificial intelligence.

But not in a fragmented way, and not purely through technological ambition.

It requires a coordinated strategy across data, infrastructure, certification, and industrial policy.

In the end, success will not be determined by who has the largest model.

It will be determined by who builds the most resilient, controllable, and operationally reliable system.