Quantinuum researchers tackle AI’s ‘interpretability problem’, helping us build safer systems
The Artificial Intelligence (AI) systems that have recently permeated our lives have a serious problem: they are built in a way that makes them very hard - and sometimes impossible - to understand or interpret. Luckily, our team is tackling this problem, and we’ve just published a new paper that covers the issue in detail.
It turns out that the lack of explainability in machine learning (ML) models, such as ChatGPT or Claude, comes from the way that the systems are built. Their underlying architecture (a neural network) lacks coherent structure. While neural networks can be trained to effectively solve certain tasks, the way they do it is largely (or, from a practical standpoint, almost wholly) inaccessible. This absence of interpretability in modern ML is increasingly a major concern in sensitive areas where accountability is required, such as in finance and the healthcare and pharmaceutical sectors. The “interpretability problem in AI” is therefore a topic of grave worry for large swathes of the corporate and enterprise sector, regulators, lawmakers, and the general public.
These concerns have given birth to the field of eXplainable AI, or XAI, which attempts to solve the interpretability problem through so-called ‘post-hoc’ techniques (where one takes a trained AI model and aims to give explanations for either its overall behavior or individual outputs). This approach, while still evolving, has its own issues due to the approximate nature and fundamental limitations of post-hoc techniques.
The second approach to the interpretability problem is to employ new ML models that are, by design, inherently interpretable from the start. Such an interpretable AI model comes with explicit structure which is meaningful to us “from the outside”. Realizing this in the tech we use every day means completely redesigning how machines learn - creating a new paradigm in AI. As Sean Tull, one of the authors of the paper, stated: “In the best case, such intrinsically interpretable models would no longer even require XAI methods, serving instead as their own explanation, and one of a deeper kind.”
At Quantinuum, we’re continuing work to develop new paradigms in AI while also working to sharpen theoretical and foundational tools that allow us all to assess the interpretability of a given model. In our recent paper, we present a new theoretical framework for both defining AI models and analyzing their interpretability. With this framework, we show how advantageous it is for an AI model to have explicit and meaningful compositional structure.
The idea of composition is explored in a rigorous way using a mathematical approach called “category theory”, which is a language that describes processes and their composition. The category theory approach to interpretability can be accomplished via a graphical calculus which was also developed in part by Quantinuum scientists, and which is finding use cases in everything from gravity to quantum computing.
A fundamental problem in the field of XAI has been that many terms have not been rigorously defined, making it difficult to study - let alone discuss - interpretability in AI. Our paper presents the first known theoretical framework for assessing the compositional interpretability of AI models. With our team’s work, we now have a precise and mathematically defined definition of interpretability that allows us to have these critical conversations.
After developing the framework, our team used it to analyze the full spectrum of ML approaches. We started with Transformers (the “T” in ChatGPT), which are not interpretable – pointing to a serious issue in some of the world’s most widely used ML tools. This is in contrast with (sparse) linear models and decision trees, which we found are indeed inherently interpretable, as they are usually described.
Our team was also able to make precise how other ML models were what they call 'compositionally interpretable'. These include models already studied by our own scientists including DisCo NLP models, causal models, and conceptual space models.
Many of the models discussed in this paper are classical, but more broadly the use of category theory and string diagrams makes these tools very well suited to analyzing quantum models for machine learning. In addition to helping the broader field accurately assess the interpretability of various ML models, the seminal work in this paper will help us to develop systems that are interpretable by design.
This work is part of our broader AI strategy, which includes using AI to improve quantum computing, using quantum computers to improve AI, and – in this case - using the tools of category theory and compositionality to help us better understand AI.
About Quantinuum
Quantinuum, the world’s largest integrated quantum company, pioneers powerful quantum computers and advanced software solutions. Quantinuum’s technology drives breakthroughs in materials discovery, cybersecurity, and next-gen quantum AI. With over 500 employees, including 370+ scientists and engineers, Quantinuum leads the quantum computing revolution across continents.
At the heart of quantum computing’s promise lies the ability to solve problems that are fundamentally out of reach for classical computers. One of the most powerful ways to unlock that promise is through a novel approach we call Generative Quantum AI, or GenQAI. A key element of this approach is the Generative Quantum Eigensolver (GQE).
GenQAI is based on a simple but powerful idea: combine the unique capabilities of quantum hardware with the flexibility and intelligence of AI. By using quantum systems to generate data, and then using AI to learn from and guide the generation of more data, we can create a powerful feedback loop that enables breakthroughs in diverse fields.
Unlike classical systems, our quantum processing unit (QPU) produces data that is extremely difficult, if not impossible, to generate classically. That gives us a unique edge: we’re not just feeding an AI more text from the internet; we’re giving it new and valuable data that can’t be obtained anywhere else.
The Search for Ground State Energy
One of the most compelling challenges in quantum chemistry and materials science is computing the properties of a molecule’s ground state. For any given molecule or material, the ground state is its lowest energy configuration. Understanding this state is essential for understanding molecular behavior and designing new drugs or materials.
The problem is that accurately computing this state for anything but the simplest systems is incredibly complicated. You cannot even do it by brute force—testing every possible state and measuring its energy—because the number of quantum states grows as a double-exponential, making this an ineffective solution. This illustrates the need for an intelligent way to search for the ground state energy and other molecular properties.
That’s where GQE comes in. GQE is a methodology that uses data from our quantum computers to train a transformer. The transformer then proposes promising trial quantum circuits; ones likely to prepare states with low energy. You can think of it as an AI-guided search engine for ground states. The novelty is in how our transformer is trained from scratch using data generated on our hardware.
Here's how it works:
- We start with a batch of trial quantum circuits, which are run on our QPU.
- Each circuit prepares a quantum state, and we measure the energy of that state with respect to the Hamiltonian for each one.
- Those measurements are then fed back into a transformer model (the same architecture behind models like GPT-2) to improve its outputs.
- The transformer generates a new distribution of circuits, biased toward ones that are more likely to find lower energy states.
- We sample a new batch from the distribution, run them on the QPU, and repeat.
- The system learns over time, narrowing in on the true ground state.
To test our system, we tackled a benchmark problem: finding the ground state energy of the hydrogen molecule (H₂). This is a problem with a known solution, which allows us to verify that our setup works as intended. As a result, our GQE system successfully found the ground state to within chemical accuracy.
To our knowledge, we’re the first to solve this problem using a combination of a QPU and a transformer, marking the beginning of a new era in computational chemistry.
The Future of Quantum Chemistry
The idea of using a generative model guided by quantum measurements can be extended to a whole class of problems—from combinatorial optimization to materials discovery, and potentially, even drug design.
By combining the power of quantum computing and AI we can unlock their unified full power. Our quantum processors can generate rich data that was previously unobtainable. Then, an AI can learn from that data. Together, they can tackle problems neither could solve alone.
This is just the beginning. We’re already looking at applying GQE to more complex molecules—ones that can’t currently be solved with existing methods, and we’re exploring how this methodology could be extended to real-world use cases. This opens many new doors in chemistry, and we are excited to see what comes next.
Last year, we joined forces with RIKEN, Japan's largest comprehensive research institution, to install our hardware at RIKEN’s campus in Wako, Saitama. This deployment is part of RIKEN’s project to build a quantum-HPC hybrid platform consisting of high-performance computing systems, such as the supercomputer Fugaku and Quantinuum Systems.
Today, a paper published in Physical Review Research marks the first of many breakthroughs coming from this international supercomputing partnership. The team from RIKEN and Quantinuum joined up with researchers from Keio University to show that quantum information can be delocalized (scrambled) using a quantum circuit modeled after periodically driven systems.
"Scrambling" of quantum information happens in many quantum systems, from those found in complex materials to black holes. Understanding information scrambling will help researchers better understand things like thermalization and chaos, both of which have wide reaching implications.
To visualize scrambling, imagine a set of particles (say bits in a memory), where one particle holds specific information that you want to know. As time marches on, the quantum information will spread out across the other bits, making it harder and harder to recover the original information from local (few-bit) measurements.
While many classical techniques exist for studying complex scrambling dynamics, quantum computing has been known as a promising tool for these types of studies, due to its inherently quantum nature and ease with implementing quantum elements like entanglement. The joint team proved that to be true with their latest result, which shows that not only can scrambling states be generated on a quantum computer, but that they behave as expected and are ripe for further study.
Thanks to this new understanding, we now know that the preparation, verification, and application of a scrambling state, a key quantum information state, can be consistently realized using currently available quantum computers. Read the paper here, and read more about our partnership with RIKEN here.
In our increasingly connected, data-driven world, cybersecurity threats are more frequent and sophisticated than ever. To safeguard modern life, government and business leaders are turning to quantum randomness.
What is quantum randomness, and why should you care?
The term to know: quantum random number generators (QRNGs).
QRNGs exploit quantum mechanics to generate truly random numbers, providing the highest level of cryptographic security. This supports, among many things:
- Protection of personal data
- Secure financial transactions
- Safeguarding of sensitive communications
- Prevention of unauthorized access to medical records
Quantum technologies, including QRNGs, could protect up to $1 trillion in digital assets annually, according to a recent report by the World Economic Forum and Accenture.
Which industries will see the most value from quantum randomness?
The World Economic Forum report identifies five industry groups where QRNGs offer high business value and clear commercialization potential within the next few years. Those include:
- Financial services
- Information and communication technology
- Chemicals and advanced materials
- Energy and utilities
- Pharmaceuticals and healthcare
In line with these trends, recent research by The Quantum Insider projects the quantum security market will grow from approximately $0.7 billion today to $10 billion by 2030.
When will quantum randomness reach commercialization?
Quantum randomness is already being deployed commercially:
- Early adopters use our Quantum Origin in data centers and smart devices.
- Amid rising cybersecurity threats, demand is growing in regulated industries and critical infrastructure.
Recognizing the value of QRNGs, the financial services sector is accelerating its path to commercialization.
- Last year, HSBC conducted a pilot combining Quantum Origin and post-quantum cryptography to future-proof gold tokens against “store now, decrypt-later” (SNDL) threats.
- And, just last week, JPMorganChase made headlines by using our quantum computer for the first successful demonstration of certified randomness.
On the basis of the latter achievement, we aim to broaden our cybersecurity portfolio with the addition of a certified randomness product in 2025.
How is quantum randomness being regulated?
The National Institute of Standards and Technology (NIST) defines the cryptographic regulations used in the U.S. and other countries.
- NIST’s SP 800-90B framework assesses the quality of random number generators.
- The framework is part of the FIPS 140 standard, which governs cryptographic systems operations.
- Organizations must comply with FIPS 140 for their cryptographic products to be used in regulated environments.
This week, we announced Quantum Origin received NIST SP 800-90B Entropy Source validation, marking the first software QRNG approved for use in regulated industries.
What does NIST validation mean for our customers?
This means Quantum Origin is now available for high-security cryptographic systems and integrates seamlessly with NIST-approved solutions without requiring recertification.
- Unlike hardware QRNGs, Quantum Origin requires no network connectivity, making it ideal for air-gapped systems.
- For federal agencies, it complements our "U.S. Made" designation, easing deployment in critical infrastructure.
- It adds further value for customers building hardware security modules, firewalls, PKIs, and IoT devices.
The NIST validation, combined with our peer-reviewed papers, further establishes Quantum Origin as the leading QRNG on the market.
--
It is paramount for governments, commercial enterprises, and critical infrastructure to stay ahead of evolving cybersecurity threats to maintain societal and economic security.
Quantinuum delivers the highest quality quantum randomness, enabling our customers to confront the most advanced cybersecurity challenges present today.