There's a comfortable belief in icon design: people will understand it because it looks like the thing. If that were true, every well drawn pictogram would work, and nobody would need to test anything.

Here's a number that says otherwise. A 2022 study evaluated 108 pharmaceutical pictograms with 52 adults. Only 17 reached the 85% correct comprehension criterion the researchers used. The best one scored 100%. The worst scored under 2%. These weren't sketches. They were published pictograms, drawn by professionals, meant to help people take medicine safely.

Why resemblance isn't enough

Umberto Eco spent a good part of his career on this question, and his position is worth knowing: iconism is conventional, not natural. Resemblance itself is coded. You recognize a drawing of a horse because you learned to read drawings of horses, not because the drawing has some horse inside it. He developed the idea in the late 1960s and 1970s, including in A Theory of Semiotics (1976), and came back to it decades later in Kant and the Platypus (1997).

The working rule that falls out of this is simple:

Your icon isn't understood because it looks like something. It's understood when someone else understands it, and the only way to know is to ask.

Universal icons are rare

Nielsen Norman Group's research on icon usability found that very few icons are close to universal. Home, print and the magnifying glass for search make the list. Most others don't.

The heart is a good example. In one product it means "like" without saving anything. In another it means "save". The star competes with it for favorites, bookmarks and ratings. So NN/g's recommendation is to put a visible text label next to icons, all the time, without requiring any interaction to see it.

But they also warn against the opposite mistake. A label doesn't rescue a bad icon. If the icon points people in the wrong direction, the label has to fight it. Icons aren't free.

It also helps to know which kind of icon you're dealing with. NN/g sorts them into three groups:

• Resemblance: What it does: Depicts the physical object it stands for; Example: An envelope for mail
• Reference: What it does: Shows an object that stands for the concept by analogy; Example: A clamp or vise for file compression
• Arbitrary: What it does: A shape that only means something by convention; Example: The warning triangle, the question mark

Arbitrary icons only work if they're taught. Budget for that, or don't use them.

The comprehension test

The standard method for testing whether people understand a symbol is described in ISO 9186-1. Its core is refreshingly plain:

• The question: "What do you think this symbol means?" And where it applies: "What would you do if you saw it?"
• Open answers. The person writes or says what they think. Never multiple choice. Multiple choice lets people guess by elimination, which tells you about the options, not the symbol.
• Random order. Show the symbols shuffled, at a consistent and generous size.
• The result: the percentage of correct answers. That's it. The method measures; it doesn't interpret for you.

Two practical rules make the difference between a real test and a performance of one:

• Nobody tests their own set. You know what it's supposed to mean, and so do your teammates. Test with people who didn't draw it and haven't seen it discussed.
• Test at the size it'll be used. An icon that reads at 64 pixels on your artboard may be a smudge at 16.

Three kinds of wrong

When you score the answers, use three categories, not two:

• Correct: matches the intended meaning.
• Wrong: doesn't match, but doesn't lead to a dangerous action. Redesign it.
• Wrong and opposite: the person understands the reverse of what the sign means. This is the category that matters most. A single case is reason enough to rebuild the sign from scratch.

Where do the pass percentages come from?

If you read around, you'll find three pass thresholds quoted over and over. They're not equally solid.

• 0%: This is what ISO 9186-1 itself sets as a pass mark. None. The standard describes how to measure and explicitly leaves the pass score to the organization that adopts it
• 85%: Verifiable. It's a criterion in ANSI Z535.3, the US standard for safety symbols, not in ISO. It's the one pharmaceutical pictogram studies usually use
• 66% / 67%: Widely repeated, weakly sourced. It circulates as "the ISO threshold", including in peer reviewed papers that cite ISO 9186-1. The published test method doesn't contain it. The 66% figure is often attributed to a 2005 paper by Foster and Afzalnia, but that paper deals with a different kind of test (judged comprehensibility), and I couldn't find where it proposes that number

One number you can check, one that's attributed but nobody seems to have verified, and one that holds up only because it keeps getting repeated. That's how a "fact" gets manufactured in a field that doesn't check its citations. When you report a result, say which threshold you used and where it comes from.

When the industry doubts itself

Even big, well funded projects fall short. The Universal Symbols in Health Care program, led by SEGD and Hablamos Juntos with funding from the Robert Wood Johnson Foundation, produced a set of hospital wayfinding symbols in the mid 2000s that grew to 50. SEGD later published a piece openly questioning whether the set would pass comprehension testing, and the project's own reporting acknowledged that many of the symbols fell short of the 85% target.

That's not a scandal. That's what honest testing looks like. The scandal would be not publishing it.

Testing in context

A comprehension test shows the icon alone. That tells you whether the drawing communicates. It doesn't tell you whether the icon works in your product. NN/g separates the two:

Out of context:- Recognition: show the icon by itself and ask what it represents.- Appeal: a simple rating scale.

In context:- Information scent: what do people expect to happen if they tap it?- Time to locate: how long it takes to find the icon for a given task.- Standard usability testing and A/B tests.

And a warning from their research: don't assume simple objects are easy to recognize. In one study, shopping bags and ribbons got mistaken for briefcases, locks and clipboards.

Context also includes hiding things behind icons. NN/g's study of hamburger menus found that hidden navigation got used far less than visible navigation on desktop (27% of cases versus 48%), and tasks took noticeably longer. An icon that hides content has a cost even when people understand it.

Accessibility checks that aren't optional

Meaning isn't the only thing to test. A few checks come straight from WCAG:

• Contrast (1.4.11, level AA): standalone icons and interface graphics need at least 3:1 against adjacent colors. A typographic "×" used as a close button counts as a graphic here, so it's judged at 3:1, not by the 4.5:1 text rule. Check light and dark mode separately.
• Target size (2.5.8, level AA, new in WCAG 2.2): at least 24 by 24 CSS pixels, or enough spacing that a 24 pixel circle centered on each target doesn't overlap its neighbors. The enhanced level (2.5.5, AAA) asks for 44 by 44.
• Text alternatives: name the action, not the drawing. "Print this page", not "printer". "Search", not "magnifying glass". If visible text already carries the meaning, the icon can be marked as decorative. The common mistake is marking an icon as decorative when it's actually the only thing that explains a button.

Iterate from the answers

Every change you make after a test should be tied to a specific answer. Something like: "Three out of ten people read the bag as a briefcase, so I added the handle cutout and shortened the body." Not "I felt it needed more personality."

An icon isn't approved because you like it. It's approved because someone who didn't draw it understood it, and you wrote that down. Everything else is opinion with a grid behind it.

Further reading

• Umberto Eco, A Theory of Semiotics (1976).
• Piero Polidoro, "Umberto Eco and the problem of iconism", Semiotica 206 (2015). A short, clear reconstruction of Eco's argument for people who haven't read Eco.
• Nielsen Norman Group, "Icon Usability", "Icon Classification" and "Usability Testing of Icons".
• W3C, Understanding WCAG 2.2: Non-text Contrast (1.4.11) and Target Size Minimum (2.5.8).
• W3C WAI, "Functional Images" tutorial.