Understanding AI Models and the Development of MAIA
As artificial intelligence models become increasingly prevalent and are integrated into diverse sectors like healthcare, finance, education, transportation, and entertainment, understanding how they work under the hood is critical. Interpreting the mechanisms underlying AI models enables us to audit them for safety and biases, with the potential to deepen our understanding of the science behind intelligence itself.
Imagine if we could directly investigate the human brain by manipulating each of its individual neurons to examine their roles in perceiving a particular object. While such an experiment would be prohibitively invasive in the human brain, it is more feasible in another type of neural network: one that is artificial. However, somewhat similar to the human brain, artificial models containing millions of neurons are too large and complex to study by hand, making interpretability at scale a very challenging task.
To address this, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers decided to take an automated approach to interpreting artificial vision models that evaluate different properties of images. They developed MAIA (Multimodal Automated Interpretability Agent), a system that automates a variety of neural network interpretability tasks using a vision-language model backbone equipped with tools for experimenting on other AI systems.

“Our goal is to create an AI researcher that can conduct interpretability experiments autonomously. Existing automated interpretability methods merely label or visualize data in a one-shot process. On the other hand, MAIA can generate hypotheses, design experiments to test them, and refine its understanding through iterative analysis,” says Tamar Rott Shaham, an MIT electrical engineering and computer science (EECS) postdoc at CSAIL and co-author on a new paper about the research.

Key Features of MAIA
The automated agent is demonstrated to tackle three key tasks:

Labeling individual components inside vision models and describing the visual concepts that activate them.
Cleaning up image classifiers by removing irrelevant features to make them more robust to new situations.
Hunting for hidden biases in AI systems to uncover potential fairness issues in their outputs.


“But a key advantage of a system like MAIA is its flexibility,” says Sarah Schwettmann PhD ’21, a research scientist at CSAIL and co-lead of the research.

Neuron by Neuron
In one example task, a human user asks MAIA to describe the concepts that a particular neuron inside a vision model is responsible for detecting. To investigate this question:

MAIA first retrieves dataset exemplars from the ImageNet dataset, which maximally activate the neuron.
For example, an image shows people in formal attire, focusing on their chins and necks.
MAIA generates hypotheses like facial expressions, chins, or neckties and designs experiments to test these hypotheses individually.

Evaluation of MAIA's Explanations

Accuracy Testing: Synthetic systems with known ground-truth behaviors are used.
Predictive Ability: The authors design an automated protocol to measure how well MAIA's descriptions predict neuron behavior on unseen data.


The CSAIL-led method outperformed baseline methods describing individual neurons in various vision models such as ResNet, CLIP, and the vision transformer DINO.

Applications of AI Component Interpretability
Understanding the roles of individual components is vital for:

Auditing AI systems for safety.
Identifying and removing unwanted behaviors in AI models.
Building a resilient AI ecosystem where monitoring tools evolve alongside system scaling.


Note: This work was supported by multiple organizations, including the MIT-IBM Watson AI Lab and the National Science Foundation. Findings will be presented at the International Conference on Machine Learning.