Model Card

What Is Model Card?

A model card is a structured disclosure document published alongside an AI model that describes what the model is, how it was trained, what it is good at, where it fails, and what safety evaluations it passed. It gives engineers the information they need to decide whether a model fits their use case.

Key Takeaways

• Model cards are the datasheet of the AI world: intended use, training data summary, benchmark results, known limitations, and safety evaluation outcomes in one document.
• Every major lab now publishes one with each frontier release, and procurement teams increasingly require them before a model can be approved for production use.
• A model card describes the model as tested by its developer. It says nothing about how the model behaves inside your product, with your prompts, on your data.
• Reading the limitations section is the highest-value ten minutes in any model selection process, because that is where the developer tells you exactly where the model breaks.

How It Works

The format traces back to a 2019 research proposal that argued models should ship with standardized reporting, the way electronic components ship with datasheets. A typical card covers the model's architecture family and size class, the cutoff date and general composition of its training data, evaluation results on public benchmarks, safety testing such as red team exercises and refusal-rate measurements, and explicit statements about intended and out-of-scope uses. How completely those sections get filled in varies widely: a 2024 Stanford analysis of more than 32,000 model cards on Hugging Face found the limitations, evaluation, and environmental-impact sections have the lowest completion rates, while training details are filled out most consistently [1].

For frontier models the card has grown into something closer to a technical report. Developers publish scores on coding, reasoning, and agentic benchmarks, describe how the model behaves under adversarial prompting, and document capability thresholds relevant to safety policies, such as whether the model materially helps with dangerous tasks. Some vendors split this into a short card plus a longer system card that covers the full deployed system, including guardrails and filters layered on top of the raw foundation model. Disclosure has been improving under public pressure: the mean transparency score of major foundation model developers rose from 37 out of 100 in October 2023 to 58 out of 100 in May 2024 on Stanford's Foundation Model Transparency Index [2].

In practice, engineering teams use model cards at two moments. During selection, the card is the primary evidence for comparing candidates before running internal evals. After deployment, it becomes a governance artifact: the record of what was known about the model when it was approved, which auditors and risk teams reference later.

Example

A CTO is choosing a model to power an internal coding agent. Two candidates score within a point of each other on a public LLM benchmark, so the team pulls both model cards. One card discloses that agentic tool-use evaluations were run with a specific scaffold and lists degraded performance on long multi-step tasks. The other card shows stronger sustained-task results but flags a higher rate of fabricated file paths in its limitations section. The team picks the first model, adds a path-verification check to their pipeline anyway, and attaches both cards to the architecture decision record so the reasoning survives the people who made it.

What People Get Wrong

The common mistake is treating a model card as a certification. It is a self-reported disclosure written by the vendor, not an independent audit, and the evaluations it describes were run under conditions you cannot fully reproduce. A strong card is a reason to shortlist a model, never a reason to skip your own evals on your own tasks.

FAQ

Who writes a model card? The organization that trained and released the model. Some are reviewed by external red teams or safety institutes, and those reviews are usually noted in the card, but authorship and final content stay with the developer.

Is a model card legally required? In most jurisdictions, not by that name. Transparency rules such as the EU AI Act require documentation for general-purpose models that overlaps heavily with model card content, so in practice the card is becoming the standard vehicle for meeting those obligations.

What is the difference between a model card and a system card? A model card describes the trained model itself. A system card describes the full deployed product around it: the safety filters, moderation layers, and tool integrations. One model can appear in many systems, each with different behavior.

Sources

1. Liang et al., Stanford. "Analysis of 32,000+ Hugging Face model cards: limitations, evaluation, and environmental-impact sections have the lowest completion rates." https://arxiv.org/abs/2402.05160. Accessed August 2026.
2. Stanford CRFM. "Foundation Model Transparency Index: mean developer transparency score rose from 37/100 (October 2023) to 58/100 (May 2024)." https://crfm.stanford.edu/fmti/May-2024/index.html. Accessed August 2026.