Anthropic unveils an AI capable of improving itself

A demonstration that shifts the debate on AI alignment

Anthropic has offered a particularly sensitive glimpse into a research direction that could matter in the race toward advanced artificial intelligence: using AI systems to help improve their own behavior. According to information reported by TechCrunch AI in the article titled “An Anthropic researcher just gave us a peek at self-improving AI”, a company researcher presented work on systems capable of optimizing their own performance on targeted behavioral evaluations.

The wording requires caution. This is not an announcement claiming that an Anthropic model freely retrains itself, independently modifies its entire architecture, or becomes autonomous in the general sense of the term. The point described is more limited, but no less important: a system can take part in an improvement loop targeting behaviors identified as problematic in alignment evaluations.

In industry terminology, alignment refers to all methods intended to bring a model’s behavior closer to the objectives, rules and expectations defined by its designers and, more broadly, by users and society. This includes in particular the ability to follow legitimate instructions, refuse dangerous requests, remain honest about its limitations, not circumvent imposed rules, and not adopt opportunistic behavior in an evaluation environment.

The highlighted result concerns ten benchmarks of misaligned behaviors. The experimental systems improved on all ten evaluations. More importantly, according to the reported results, this improvement did not lead to any observed decline on the other monitored evaluations. This is a central point: in model safety, local optimization is valuable only if it does not create a difficult-to-see regression elsewhere.

AI research is accustomed to announcements of gains on tests. But in the field of alignment, the nature of the gain matters as much as its magnitude. Improving a reasoning, programming, or document-comprehension score can be directly useful to a user. Improving a score on misaligned behaviors raises a different question: can the risks of undesirable conduct be reduced as models become more capable and more integrated with external tools?

Anthropic occupies a distinctive place in this discussion. The company was founded in 2021 by former OpenAI members and quickly stood out through communication strongly focused on the safety of AI systems. It notably popularized the approach known as Constitutional AI, introduced in 2022, which aims to train assistants to rely on a set of principles to assess and correct certain responses. Its Claude models subsequently made reliability, safeguards and the reduction of dangerous responses important parts of their positioning.

The demonstration mentioned by TechCrunch therefore follows a continuity: making AI not only an object to be controlled, but also a working tool for the teams responsible for controlling it. This promise is appealing because alignment remains very costly in human expertise. Testing a model’s behaviors, producing counterexamples, examining ambiguous outputs, defining criteria for refusal or caution, then measuring the effects of each correction require time and scarce skills.

But it immediately introduces a tension. If one model helps evaluate and improve another model, how can it be ensured that it is not merely helping make the system better at passing a test? In research language, the danger is optimizing metrics rather than real behavior. A model may learn the regularities of a benchmark without acquiring comparable robustness in new, more open, or more adversarial contexts.

The most significant result is not simply the increase in scores on targeted tests, but the absence of observed degradation on the other evaluations considered by Anthropic.

This absence of observed degradation should not, however, be confused with a general safety guarantee. It means that, within the reported evaluation scope, researchers did not find a negative trade-off on the other monitored tests. This is a useful experimental signal. It is not proof that the system will behave properly in every usage context, that it will resist all forms of circumvention, or that it will retain the same properties when connected to tools, deployed at scale, or confronted with conflicting instructions.

What was presented: optimization loops for targeted behaviors

The core of the announcement reported by TechCrunch rests on the idea of a loop: an AI system can be used to improve performance on specific behavioral evaluations. The complete technical details of the experiments do not appear in the information provided in this brief, and it would be unwise to attribute to Anthropic a retraining method, architecture, or exact protocol that has not been explicitly documented. However, the general logic is sufficiently clear to be analyzed.

A behavioral evaluation consists of presenting a system with scenarios designed to reveal a particular tendency. In the case of misaligned behaviors, this may generally involve observing how a model responds to constraints, competing objectives, ambiguous incentives, or opportunities to circumvent a rule. These evaluations do not merely measure the linguistic quality of a response. They seek to detect gaps between expected behavior and behavior actually produced.

The systems described by Anthropic improved their performance on ten benchmarks of this kind. This number is important because it avoids reducing the result to a single isolated test. Improvement on one benchmark can always reflect a very narrow adaptation to the test format. Succeeding on ten different evaluations is a more interesting signal, provided the actual diversity of the behaviors measured and the independence of the data used to train, tune, and evaluate the systems are known.

Yet these two conditions are decisive. Benchmarks are instruments, not reality itself. Their design determines what they can reveal. If examples are too similar to those used during the optimization process, gains may overestimate generalization. If scenarios are too stylized, they may miss risks that emerge when the model has a long history, access rights to a computing environment, or the ability to call tools.

The fact that the gains did not degrade the other observed evaluations is therefore an encouraging element, but it also prompts questions about which evaluations were selected, when they were administered, what behaviors they cover, and how researchers prevented the system from indirectly benefiting from information about the tests. In experimental sciences, the robustness of a result is built through replication, methodological transparency, and critical examination of the conditions under which it was obtained.

The work described also belongs to a broader trend: partial automation of the supervision process. The most advanced models can already help draft, classify, summarize, verify, or critique content. Applied to alignment, these capabilities can help generate test cases, identify inconsistencies in responses, compare multiple outputs, suggest safer rewordings, or sort cases that must be examined by a human.

This automation does not eliminate the need for human experts. It shifts it. Rather than reading every individual output, teams can devote a greater share of their work to defining policies, verifying criteria, designing adversarial tests, and analyzing areas of uncertainty. This is potentially essential when models become capable of producing a volume of content that no team could manually review exhaustively.

There is nonetheless a fundamental difference between automating a control and delegating the definition of the control. In the first case, a model speeds up the application of a framework set by humans. In the second, it could begin influencing the criteria that determine what constitutes acceptable behavior. The information reported by TechCrunch does not allow the claim that Anthropic is taking this second step. But this is precisely the boundary that regulators, researchers, and companies will have to monitor as self-improvement tools gain effectiveness.

The choice of the term “self-improvement” can, moreover, cause confusion in public debate. It does not necessarily refer to an AI that independently develops new objectives. In the framework presented, improvement is guided by targeted behavioral evaluations. The objectives of the experiment, the success metrics, and the system’s limits remain decisive. The safety question then concerns the quality of these constraints: are they sufficiently comprehensive, do they resist optimization, and do they remain verifiable by independent observers?

Why automated alignment is both promising and difficult to validate

The practical interest of this avenue stems from an imbalance that currently shapes generative AI: model capabilities are advancing rapidly, while human capacity to supervise them remains limited. Safety teams must examine behaviors that are sometimes subtle, context-dependent, and difficult to formalize. The more systems produce responses, perform tasks, or interact with software, the greater the number of situations that must be checked.

In this context, using models to help with alignment may become an operational necessity. An AI can review many scenarios, propose variations, identify discrepancies between a rule and a response, or provide an initial level of critique. If this assistance is well designed, it can free up human time for the most difficult situations: edge cases, conflicts between values, systemic risks, and behaviors that cannot be reduced to a right or wrong answer.

The results mentioned by Anthropic suggest that an improvement loop can achieve simultaneous progress across several behavioral dimensions without a visible cost on the other monitored evaluations. In applied research, this type of result matters because it addresses a classic problem: safety interventions can sometimes make a model less useful, less cooperative, or more brittle on other tasks. An overly harsh correction can produce excessive refusals; a poorly calibrated rule can prevent legitimate uses; targeted training can reduce performance outside the domain addressed.

Not observing such degradation in the reported evaluations is therefore a finding that deserves attention. It does not solve the trade-off between capability and safety, but it indicates that this trade-off is not inevitable within the experimental framework studied. For developers, the stakes are high: a safety system that severely degrades a product’s commercial usefulness will face constant pressure to be loosened. Conversely, methods that strengthen safeguards without notable degradation are more likely to be adopted sustainably.

The difficulty is that AI systems are optimizers: when trained to maximize a signal, they can exploit that signal’s imperfections. A benchmark measures what it measures, not necessarily everything its designers would like to measure. This is true in many areas of machine learning, from games to recommendation systems and conversational models.

In the case of alignment, this phenomenon is particularly delicate. A model may learn to present convincing justifications without its internal reasoning being reliable. It may be cautious in one family of scenarios but fail when faced with a new formulation. It may follow an explicit instruction while failing to recognize an indirect consequence of its action. It may also produce responses that conform to a static test, then behave differently in a prolonged interaction or in an environment where it must carry out a sequence of actions.

This is why Anthropic’s results should be read as a methodological step, not as the final demonstration of an aligned AI. Alignment is not a binary property that a model would possess once and for all. It is a set of behaviors to be measured and maintained despite variations in context, updates, interfaces, connected tools, and attempts at manipulation.

Independent validation then becomes indispensable. A company developing a model is best placed to know its architecture, internal procedures, and deployment data. But it also has a direct interest in presenting its advances in a favorable light. This does not call the value of its research into question; it means that an ecosystem of trust cannot depend solely on self-assessments.

External evaluators, independent red-teaming teams, academic researchers and, where relevant, competent authorities must be able to examine systems and their results. Full access to a model’s weights is not always necessary to carry out this work. Evaluation protocols can address observable behavior, available capabilities, deployment conditions, incident-reporting mechanisms, and documentation of known limitations.

For companies, the challenge is not merely to publish a good score. It is to demonstrate that the score withstands new tests administered by people who did not design the system. The research presented by Anthropic is interesting precisely because it highlights this need: the more AI participates in improving AI, the clearer and more pluralistic the validation chain must be.

An industrial race in which safety is also becoming a competitive advantage

Anthropic is not the only actor investing in safety, evaluations, and supervision mechanisms for large models. OpenAI, Google DeepMind, Meta, and other laboratories regularly publish work on risk assessment, red teaming, output control, or system robustness. The difference often lies in the methods, levels of transparency, and place each company gives safety in its product communication and organization.

OpenAI has, for example, published a risk preparedness framework, the Preparedness Framework, intended to track and reduce certain risks associated with advanced capabilities. Google DeepMind has also presented a framework for assessing levels of risk related to the capabilities of advanced models, the Frontier Safety Framework. These initiatives are not identical to the experimentation described by Anthropic, but they reflect a common evolution: major laboratories recognize that bringing more powerful models to market must be accompanied by structured processes for risk assessment and reduction.

The specificity of the signal sent by Anthropic is the use of AI in a loop aimed at improving alignment behaviors. This direction echoes a longstanding concern in the scientific community: how to supervise systems whose capabilities may exceed those of the people responsible for evaluating them on certain tasks? If a model is better than a human at producing code, analyzing a large volume of documents, or exploring multiple strategies, it may also become useful in assisting people attempting to detect its own weaknesses.

This logic is sometimes described as scalable supervision: supervision must be able to scale with the systems it oversees. The challenge is that the tool used to assist control may share certain weaknesses with the object being controlled. If the same type of model produces the responses, critiques, and corrections, common biases may run through the entire chain. A systematic error may be repeated with greater speed and a reinforced appearance of consistency.

Methods based on plurality are therefore important. Using several models, varying test sets, drawing on experts from different fields, and comparing independent research approaches can reduce the risk of a single blind spot. None of these mechanisms offers absolute certainty. Their value comes from their complementarity: a behavioral test, human review, computer security analysis, legal assessment, and observation under deployment conditions do not answer exactly the same questions.

Competition between laboratories adds an economic constraint. General-purpose models are developed in a market where release speed, perceived quality, and availability to companies strongly influence decisions. In such an environment, safety can be presented either as an obstacle or as a condition for differentiation. Anthropic has sought for several years to make the second option part of its identity: selling high-performing models while emphasizing an approach centered on reliability and risk reduction.

For customers, this promise cannot rest on a general formula. Organizations need to know the limits of the tool they deploy. A bank, hospital, government agency, or industrial company does not evaluate an AI assistant according to the same criteria as an individual user. The issues concern confidentiality, traceability, responsibility in the event of an error, compliance of uses, and the ability to quickly disable or correct a failing system.

The work reported by TechCrunch may have an indirect effect on this commercial relationship. If automated alignment methods make it possible to test problematic behaviors more quickly and correct certain flaws more effectively, they could reduce the cost of compliance and ongoing monitoring. But if they remain opaque, they could instead fuel mistrust: a client company cannot easily accept a supplier’s claim that a model “improved itself” without documentation on the controls exercised.

The vocabulary used must therefore be precise. A system’s autonomy in research or optimization must not be equated with the absence of human governance. Conversely, nominal human supervision is not sufficient if processes are too complex to be genuinely examined. The value of the approach will depend on its ability to leave verifiable traces: objectives set, tests used, changes made, incidents detected, limitations identified, and conditions under which results should not be extrapolated.

The European framework and specific issues for France

In France, as in the rest of the European Union, the issue raised by Anthropic goes beyond rivalry between American laboratories alone. The European AI Act, which entered into force on August 1, 2024, establishes a risk-based regulatory approach. Its rules apply progressively. It notably provides obligations for general-purpose AI models, as well as strengthened requirements for models presenting systemic risks.

Research on systems that participate in their own behavioral improvement directly falls within European risk-management concerns, even if a laboratory experiment is not in itself a regulated product. Authorities and companies will seek to know how to document evaluation mechanisms, how to demonstrate the effectiveness of risk-reduction measures, and how to maintain effective human oversight when technical processes become more automated.

For French organizations that use models from major providers, the practical question is not whether AI is “autonomous” in absolute terms. It is what happens in their own decision-making chain. A tool may contribute to reviewing files, customer relations, code generation, document drafting, or internal research. In each of these cases, behavioral improvement of the model is useful if it reduces errors or inappropriate outputs. But that improvement does not relieve the user organization of establishing its own controls.

A French company cannot fully transfer its operational responsibility to a model provider. It must define authorized tasks, the data that may be sent to the service, required validations, responsible individuals, and procedures to follow in the event of an incident. The results announced by Anthropic can help raise the safety level of a technological component; they replace neither a business risk analysis nor internal governance.

Data protection is also central to the issue. Model evaluation and improvement mechanisms may involve datasets, interaction logs, or usage scenarios. For European users, the question of data location, access, and processing remains essential, in connection with the General Data Protection Regulation. Alignment promises must therefore be reconciled with privacy and data-minimization requirements.

France has a particularly active AI research and business ecosystem, with public laboratories, engineering schools, startups, and large user groups. This situation can foster a European contribution to evaluation methods, rather than dependence on metrics defined solely across the Atlantic. Benchmarks of misaligned behaviors should not be viewed as strictly universal and fixed objects. They reflect choices about what must be detected, priority risks, and behaviors deserving particular attention.

In a multilingual environment, linguistic robustness is a specific issue. An evaluation conducted mainly in English may not cover all ambiguities, cultural references, administrative phrasing, or professional uses found in French. This does not mean that a model evaluated in English is necessarily insufficient in French. It means that French-speaking users and evaluators have an interest in testing systems in their own languages, with their own documents and real work scenarios, within a framework that respects data and regulations.

Public administrations, universities, media organizations, healthcare institutions, and regulated companies could benefit from more systematic audit procedures. The aim is not to require an end user to replicate a laboratory’s research. It is to have proportionate safeguards: accessible documentation, clear usage scopes, reporting mechanisms, avenues for redress, human controls for sensitive decisions, and the ability to trace the origin of an important response.

The approach presented by Anthropic also underscores the importance of public procurement and calls for tenders. When institutions choose an AI tool, they can request not only functional performance, but also information on safety evaluation methods, known limitations, and update procedures. Ultimately, the quality of these requirements could weigh on the market: providers able to produce better-structured evidence of behavioral robustness will have an advantage with the most demanding organizations.

Toward AIs that assist in their own oversight, subject to stronger evidence

The prospect opened by Anthropic is ambitious: if AIs can help identify and reduce certain misaligned behaviors, oversight capabilities can advance at the same time as models’ general capabilities. The reasoning is pragmatic. Systems are becoming faster and more versatile; human supervision methods alone risk failing to keep up. Using them as alignment assistants may therefore be one of the few ways to increase the scale of checks.

But this prospect will be credible only with a standard of evidence stronger than that required for a simple product demonstration. An AI improving its results on ten benchmarks is a notable research result. To turn that result into a trusted practice, it will be necessary to show that the progress holds on new data, against adversarial evaluations, in different languages, and after changes of version or deployment context.

It will also be necessary to avoid a drift toward indicators that are too easy to optimize. Metrics must be varied enough to prevent a system from merely learning to recognize the style of an evaluation. They must include unexpected scenarios, tests conducted outside development teams and, where possible, observation of real effects in use. Incidents and failures have as much informational value here as positive results: they reveal a method’s limitations and make it possible to improve safeguards.

Governance will have to keep pace with the technical developments. The companies developing these tools will have to clarify who decides optimization objectives, who validates changes, who can suspend a system, and who verifies that the improvement loop does not produce undesirable consequences. Authorities, for their part, will have to distinguish useful experimentation from overly broad marketing claims. The expression “AI capable of improving itself” attracts attention, but it must not obscure the human, organizational, and regulatory parameters that truly determine the level of control.

For the French-speaking market, the long-term challenge is twofold. On the one hand, more automated alignment tools could make advanced models more usable in sectors where reliability requirements are high. On the other, they will reinforce the need for local expertise in auditing, linguistic evaluation, digital law, cybersecurity, and data governance. Sovereignty is not limited to owning or hosting models: it also requires being able to test them, challenge them, and decide under what conditions they may be used.

The research mentioned by TechCrunch therefore does not settle the fundamental question of controlling advanced AI. It makes that question more concrete. If models become instruments for improving other models, the center of gravity will shift toward the quality of evaluations, the independence of auditors, and the strength of governance procedures. Technical acceleration could then serve safety, provided verification mechanisms advance at least as quickly as the systems they are responsible for overseeing.