Faraday, the AI agent that automates scientific replication

An agent designed for replication, rather than scientific synthesis alone

British lab Inherent, founded by former DeepMind members, is introducing Faraday, an artificial intelligence agent designed for a very specific research problem: replicating scientific results. According to TechCrunch's article on the announcement, the tool is not limited to reading, summarizing, or commenting on publications. It is designed to reconstruct the experiments described in research papers in order to reproduce their results.

The distinction is essential. Since the rise of large language models, demonstrations of AI applied to science have often emphasized literature research, code generation, writing assistance, data analysis, or hypothesis formulation. These uses can help a researcher, but they do not directly answer a central question: can a published study be reproduced from the information it contains?

Faraday operates in this more demanding area. Replicating a paper does not mean extracting its conclusion or producing a convincing paraphrase of its protocol. It involves translating a scientific description into a sequence of actions rigorous enough to reconstruct an experiment, carry out the necessary steps, and compare the results obtained with those reported in the publication. This work combines careful reading, methodological interpretation, programming, data handling, parameter control, and evaluation of discrepancies.

Inherent says its agent outperforms Anthropic and OpenAI systems on this specialized task. This claim, reported by TechCrunch, deserves to be taken seriously because it concerns a capability different from general conversational performance. But it also calls for caution: in research, an internal demonstration, however impressive, does not replace independent validation, public evaluation protocols, and a detailed analysis of the conditions under which results are obtained.

The announcement comes at a time when the scientific community is taking an increasing interest in agents capable of chaining operations together rather than simple conversational assistants. A chatbot can explain a chart or suggest a method. An agent, in the sense promoted by many AI companies, must be able to pursue an objective through multiple steps, use tools, and verify its own work. Applied to replication, this ambition becomes particularly sensitive: the agent must not merely produce a plausible answer, but help determine whether a published claim withstands a methodical reconstruction.

The name Faraday also places the project in a scientific rather than consumer-product mindset. Inherent is seeking to make AI a research “teammate,” according to the wording echoed in the title of TechCrunch's article. This expression should nevertheless be read precisely. A teammate does not mean an autonomous substitute for scientific judgment. In this case, the system's potential value will depend on its ability to make its operations inspectable by humans: the sources used, the choices made when a protocol is incomplete, the generated code, the data used, the errors encountered, and the reasons for any replication failure.

Reproducibility, a structural challenge of modern research

Replication has a particular place in the scientific method. A publication generally describes a question, a protocol, measurements, an analysis, and an interpretation. But publishing a result is not, by itself, a guarantee that the result will be obtained again by another team. To be verifiable, work must be sufficiently documented, data must be accessible when possible, methods of analysis must be explicit, and experimental conditions must be capable of being recreated.

In practice, this chain is often incomplete. Some papers do not make all of their code available. Datasets may be restricted, protected, too large, or impossible to share. Essential parameters may not be reported with sufficient precision. In experimental work, material, biological, or environmental factors may influence the result without being easily captured in a paper. In computational disciplines, the exact version of a library, the hardware used, or preprocessing choices can be enough to alter a conclusion.

The debate over the “reproducibility crisis” is not new. A survey published by Nature in 2016 among 1,576 researchers notably showed the scale of this concern: 70% of respondents said they had failed to reproduce another scientist's experiment, while 52% said they had not succeeded in reproducing one of their own experiments. These figures do not mean that the majority of published work is invalid. They do show, however, that reproduction is a difficult, costly process that is too rarely treated as a normal stage of scientific production.

The situation is not limited to errors or fraud, which are distinct cases. A replication can fail for many reasons: missing information, legitimate variations in materials or context, a different sample, statistical uncertainty, difficulty reconstructing a processing chain, a software change, or ambiguous interpretation of a text. Conversely, successfully replicating something does not automatically mean that all the conclusions of a paper are established beyond doubt. Replication is therefore a verification tool, not a binary, universal verdict.

This is precisely where Faraday's positioning may be of interest. Research produces a body of literature that no human team can systematically verify. Redoing an experiment requires time, skills, infrastructure, and sometimes considerable financial resources. Academic incentives still often favor novel findings over replication of existing studies. An agent that accelerated the reconstruction of computational experiments or data analyses could therefore reduce part of this cost.

However, domains must be distinguished. Replicating a paper that includes code, accessible data, and a structured computational procedure is very different from replicating a clinical experiment, a trial involving living organisms, a field study, or a protocol requiring specialized instruments. Automation is likely to be more immediate where experiments are digital, inputs and outputs are defined, and the execution environment can be recreated. It becomes more complex when reproduction depends on a physical laboratory, rare samples, tacit know-how, or regulatory constraints.

The word “replication” also covers several levels. Re-running an analysis exactly from the same data is not the same as reproducing the protocol with new data. Confirming an observation in another context is likewise not identical to reconstructing a paper's software pipeline. The scientific value of automation will therefore depend on how precisely Inherent defines the task assigned to Faraday and on the criteria used to judge a replication successful, partial, or impossible.

What the claimed performance against OpenAI and Anthropic means — and what it does not yet demonstrate

Inherent says Faraday performs better than Anthropic and OpenAI systems at replicating research. At first glance, the comparison is notable. Anthropic and OpenAI are among the companies that have contributed most to popularizing general-purpose language models capable of writing code, handling documents, and using certain tools. Their systems are frequently used by researchers to explore the literature, prepare scripts, or interpret results.

But a comparison between a specialized agent and more general models requires a methodical reading. A system explicitly trained, structured, or equipped to reconstruct experiments may logically have an advantage on a replication benchmark. That is not enough to establish that it performs better in every scientific task, or even in every form of paper verification. The decisive question is not only ranking: it concerns the nature of the tested cases, the tools allowed, the system's degree of autonomy, and the scoring method.

TechCrunch reports Inherent's assertion, but performance claimed by its publisher remains a claim as long as it is not supplemented by an independent evaluation. To be interpretable by laboratories, scientific publishers, and funding bodies, such an evaluation should in particular specify the selected corpus of papers, the fields covered, the resources made available to the agents, the available data and code, as well as the exact definition of success.

A robust benchmark should also distinguish several situations. A paper may be perfectly documented and therefore relatively simple to reconstruct. Another may contain ambiguous areas or not provide access to the necessary resources. In the latter case, the most useful agent is not necessarily the one that produces an answer at all costs; it may be the one that clearly identifies absent information, formulates the necessary assumptions, and indicates that faithful replication cannot be guaranteed.

This requirement is particularly important with language models. Their ability to produce highly plausible text and code can become a risk when the objective is verification. A scientific agent must not silently fill a gap in the protocol with a likely solution and then present the result as an exact reproduction. Instead, it must separate what comes from the paper, what is inferred, what is chosen by convention, and what cannot be established. In a research environment, traceability is not an interface detail: it is a condition of trust.

Inherent's claim must also be placed in the emerging competition around research agents. General-purpose models can offer great flexibility: they read varied texts, write code in different languages, and can be connected to multiple tools. Specialized systems, for their part, can offer a more constrained framework, dedicated procedures, and control mechanisms suited to a scientific task. Neither of these two models is mechanically superior in every context. The choice will depend on the objective: exploring, assisting, automating, verifying, or documenting.

From this perspective, Faraday's promise is less that of a chatbot that can talk about science than that of an agent that turns a publication into an executable process. It is a narrower ambition, but potentially a more structuring one. If confirmed, it would shift attention toward metrics more demanding than the writing quality of an answer: successful reconstruction rate, fidelity to the protocol, detection of missing information, reproducibility of the code created by the agent, clarity of execution logs, and ability to report its limitations.

Relative performance against Anthropic and OpenAI therefore does not exhaust the issue. A demonstration may show that Faraday achieves better results in a given setting without yet answering broader questions: does it work on recent papers unseen during its development? Can its results be reproduced by other teams? Are its errors easy to detect? Can its operational reasoning be audited? And is the tool capable of recognizing that a publication does not contain enough elements to be properly replicated?

Toward an infrastructure of trust, provided doubt is automated too

The potential interest of an agent like Faraday is not limited to time savings. Automated replication could become an infrastructure layer for research, alongside code repositories, data archives, reproducible execution environments, or preprint platforms. Under this hypothesis, a paper would no longer be merely a static document: it would be accompanied by a set of resources and tests enabling systems, and then humans, to verify that its results can be reconstructed.

Such an evolution could change the work of several stakeholders. Authors would have a tool to detect inconsistencies before submission. Reviewers could spend more time on substantive scientific choices, after obtaining an automated overview of code execution or the availability of computational steps. Publishers could encourage more structured publication formats. Teams wishing to build on an earlier study could benefit from a clearer starting point than merely reading a paper.

The most important benefit would not necessarily be to produce a simplistic “replicated” or “not replicated” label. Research rarely operates according to such clear-cut categories. A useful system could map areas of fragility: a given figure is reproducible from the files provided; a given result depends on an undocumented parameter; a given analysis requires a precise version of software; a given conclusion cannot be tested without access to non-public data. This mapping would already be valuable information for scientific debate.

To do so, Faraday would need to be able to embrace uncertainty rather than conceal it. One of the main challenges of autonomous agents is their potential tendency to pursue a task even when the available information does not allow it to be completed rigorously. In scientific replication, this persistence can become counterproductive. An AI that invents a missing detail in order to obtain a chart close to the one in the paper can create the illusion of validation. An AI that stops its work, explains the blockage, and requests clarification provides a service more compatible with scientific standards.

Transparency should cover the entire chain. What text was read? What data was downloaded or excluded? What computing environment was used? What code did the agent write or modify? Which parameters come explicitly from the paper and which result from an assumption? What discrepancies were observed between the reconstruction and published results? Without answers to these questions, a benchmark score or summary sentence would not be enough to create trust.

The issue also raises a question of responsibility. When an agent concludes that a study cannot be reproduced, that conclusion may have consequences for authors, institutions, or fields of research. It should not be interpreted as an automatic accusation. Failure may reflect insufficient documentation, a technical difficulty, or a legitimate restriction on data access. Likewise, when an agent succeeds in recovering a result, the community must be able to examine how it got there. AI-assisted replication does not dispense with peer review; it can change its scale and organization.

This nuance is all the more important because scientific publications fulfill several functions. They communicate results, but also serve as a historical record, an evaluation medium, and a reference for subsequent work. Adding a layer of automation means not confusing the technical execution of a pipeline with the overall validity of a theory. A script that runs correctly does not, by itself, prove that the initial hypotheses are correct, that data was collected without bias, or that the interpretation is justified.

Faraday thus sits at the intersection of two promises: the automation of complex scientific tasks and stronger verifiability. The first is now widely driven by the AI market. The second is more difficult, because it requires designing systems that accept contradiction, incompleteness, and external oversight. It is on this second dimension that Inherent's project will have to prove itself.

Implications for French and European laboratories

For French and European research teams, the question is not only whether Faraday performs better than other systems on a given task. It concerns the conditions under which a replication agent can be integrated into research practices, data infrastructures, and already complex governance frameworks.

France has a scientific ecosystem in which universities, public bodies, hospitals, joint laboratories, and companies coexist. In such an environment, projects produce data of very different kinds: computational datasets, experimental observations, sensitive data, clinical information, heritage archives, or industrial results. A replication tool may be immediately relevant to certain digital work, especially when code and data can be shared. It will be much more constrained in cases subject to confidentiality obligations, contractual restrictions, or rules protecting individuals.

The European framework adds an important dimension. The General Data Protection Regulation imposes strong requirements when a project involves personal data. For an AI agent, the transfer of data to an external service, hosting arrangements, retention of traces, and access granted to teams must be examined carefully. The promise of automation cannot lead to bypassing the security or confidentiality rules governing certain scientific work.

The European regulation on artificial intelligence, the AI Act, is another contextual element. It does not automatically turn every tool used by a researcher into a high-risk system. But it illustrates the growing role of documentation, transparency, and risk-management requirements in the deployment of AI in Europe. These are precisely the characteristics on which the scientific usefulness of a replication agent depends. In this field, regulatory compliance and methodological quality can converge: traceable AI is both easier to audit and more useful for understanding a result.

French-speaking laboratories may also see a linguistic and documentation issue. International scientific literature is overwhelmingly written in English, but internal protocols, laboratory notes, administrative documents, and certain training resources may exist in French. A truly operational replication agent would need to fit into this heterogeneous environment without losing the necessary terminological precision. In science, a poor translation of a parameter, unit, or experimental condition can be enough to invalidate a reconstruction.

Beyond language, access is central. If advanced replication tools remain reserved for a few companies or well-funded universities, they could widen the gap between institutions. Conversely, open standards for describing experiments, depositing data where possible, and publishing execution environments could make verification more widely accessible. Inherent's announcement thus recalls that competition will concern not only AI models, but also the formats, interfaces, and infrastructure that will enable these models to work.

Funding bodies and publishers could play a decisive role. They can encourage code deposition, dataset documentation, the use of reproducible environments, and publication of more detailed protocols. An agent like Faraday would derive much of its effectiveness from these practices. Conversely, no algorithmic sophistication will be able to fully compensate for a paper whose necessary resources are absent, inaccessible, or insufficiently documented.

For European companies developing scientific AI solutions, the Faraday case also constitutes a market signal. General research assistants are already numerous. Differentiating through a concrete verification function, with performance measures and value directly linked to laboratory workflows, may be a more defensible path than simply providing a conversational interface. But this requires a higher standard: scientific users cannot be satisfied with an elegant answer if the protocol does not hold up.

The next test will be open validation

Inherent's presentation of Faraday highlights an important direction for AI applied to science: moving from intellectual assistance to controllable execution of research procedures. If the agent truly succeeds in reconstructing experiments and recovering their results reliably, it could help make verification more frequent, faster, and better documented in certain fields.

But the immediate challenge is not to proclaim that an AI has solved the reproducibility crisis. This crisis does not have a single cause and cannot be addressed by a single product. It involves the quality of methodological descriptions, access to data, publication incentives, replication costs, statistical practices, material constraints, and the ability of independent teams to challenge a result.

The next step for Inherent will therefore be less media-driven than methodological. The startup will have to show how Faraday is evaluated, on what types of work it operates, what its limitations are, and how third parties can verify its own conclusions. A transparent comparison with Anthropic and OpenAI systems would be particularly useful if it made it possible to understand not only which tool achieves the best score, but also why: better reading of methods, better code generation, use of specific tools, error control, or greater ability to recognize missing information.

Over the longer term, replication agents could turn scientific publication into a more dynamic object. A paper could be accompanied by an execution trace, verifiable resources, and reports indicating what has been reproduced in different environments. Researchers would not relinquish their role of judgment; they would have tools to focus that judgment on areas that are genuinely uncertain or controversial.

It is in this ability to organize doubt, rather than erase it behind an automated answer, that Faraday's credibility will be determined. For Inherent as for its competitors, the challenge is not merely to produce an agent that appears to do science. It is to build a system whose results can themselves be replicated, discussed, and challenged by the scientific community.