Notes on Predictive Interfaces

This past spring I gave a talk at the Big Design conference in Dallas called How to Raise a Predictive Interface Without It Biting Your Users, a talk I think went pretty well. Below are some of the thoughts that ended up in the final presentation. But before we move on, I’d like to revive an experiment I wasn’t able to fully pull off on the stage. Below, you’ll find a space to tell me the tone of how the talk should end. If you choose it, it’ll invoke a small language model in the browser and rewrite the ending accordingly. Will it work? Only you can determine if you want to find out.

What if an interface determined itself?

Designers with any experience under their belt have spent a lot of time learning how to design systems of reusable components. Their behaviors, their rules, their patterns of usage; examples and guidelines. These foundational elements make it possible to construct consistent experiences at scale. And if made well, a design system is essentially a way of encoding decisions so they don’t have to be made again.

But what if the system assembling the experience – or even the components being assembled – could make some of those decisions for itself? Not just what content or data gets displayed, but which components get used, how they’re arranged, and which interactions are actually necessary for the person using the experience. Building an interface and an experience that could be composed dynamically based on customer information, historical preferences, and signals from the current session.

Would an experience that could reorganize itself around the customer’s apparent intentions perform differently from (better than?) a conventional, predefined journey? As the saying goes, if you want to make an agentic interface, you must first invent the ways to give up control.¹I agree that Carl Sagan’s formulation of “If you wish to make an apple pie from scratch, you must first invent the universe,” is far more consequential.

The idea of an interface adapting isn’t novel; I still refer to test and target opportunities as mboxes, codified by Offermatica a long time ago. What’s interesting at this moment is we have a technology that can realistically take on some of the control of the interface options, and ultimately the composition of a flow, in a way that’s not pre-determined but can be coherent.

Deterministic and Probabilistic UX

The conventional experience design develops a sequence of interactions that everyone more or less needs to complete. We draw a map, which the users follow.²This is absolutely an over-simplification of user-centered design, but the analogy has the benefit of acknowledging the final result of a designer’s decisions is a constrained set of options a user must choose from, not a space where a user is in control of the experience in a truly meaningful way. We may create dozens or hundreds of possible paths, and we may allow the experience to change based user attributes or business rules, but those variations are anticipated and specified to answer to our understanding of human circumstances and technology quirks.

I’ve started referring to this as deterministic design. It can be incredibly complex - great ones often are!³ – but ultimately all the behaviors and reactions of the experience have to be predetermined.Mediocre ones can be complex too; the difference I’d say is how visible the contours of that complexity are. Consistency and predictability have been two of our most dependable design virtues. Give people predictable controls, familiar patterns, and interactions that behave as expected. One of the consequences is can have to spend enormous amounts of time mapping edge cases, only to fail to imagine a particular combination of circumstances someone brings to an interaction that can break their flow.

Mostly for the better, human beings don’t arrive as use cases. They arrive as people. A system capable of interpreting context should help us accommodate more of that variability. I’ve been calling that type of experience, one that is crafted to have some level of agency in how it works, a probabilistic design. With a probabilistic design, the system can redraw portions of the map while it’s being used. That means the system plays some role in making decisions about composition and sequence that haven’t been defined yet. Which introduces something really interesting to a user experience: whether the system will make appropriate decisions about paths we haven’t defined.

Design doesn’t just ship, it runs.

This introduces something we’ve generally worked very hard to eliminate from interface design and systems development: uncertainty about what the experience will actually do. If intelligence is making decisions about the interface and the flow, then the choices we make about its objectives, signals, constraints, and permissible behavior will continue expressing themselves long after we’ve finished designing.

Any system will need to have a set of capability boundaries. An airline booking flow needs to know that the underlying reservation system won’t accept a payment type it doesn’t support, or that a particular combination of inputs won’t produce a valid booking.⁴ These constraints establish a scope for what the probabilistic experience can reasonably accomplish.Or that an airline booking flow shouldn’t try to process your cell phone payment.

It also needs a set of rules governing how and when a component, an interaction, an interface, or even an entire flow can be assembled. Contracts that define acceptable inputs, outputs, behaviors, and relationships between components. We’re not asking an intelligence to invent an entirely new reservation system in production just because it thinks it might be helpful.

But technical validity isn’t the same thing as good judgment. A system could assemble an entirely valid interface that unnecessarily complicates a simple task. It could make an accurate inference about someone’s intentions and act on it without permission. It could eliminate a confirmation step that seemed redundant but was, to the customer, rather important.

None of those necessarily violate a technical contract. The interface works. It just might not be an experience we’d have chosen to design. This suggests that the contracts can’t simply describe what the system can do. We also need to establish the conditions under which it should do something. When is an inference sufficiently reliable? When does the system have permission to act? When should it ask, explain itself, or simply leave the decision to the user?

I think it’s these rules, limits, and norms that are likely to become the next interesting design surface. Not simply the arrangement of components on a screen, but the principles governing how those arrangements are decided. It’s a big change to move the value of design from the decisions expressed by an interface to the conditions under which an interface can make decisions without us.

From consistency of interface to consistency of intent

A system like this would present different interactions to different people while remaining quite consistent about the principles governing its behavior. At least, that’s the ambition. The intelligent version has some discretion to elevate information, rearrange controls, infer preferences, and even eliminate steps that appear unnecessary. Importantly, though, the intelligence doesn’t have unlimited freedom. It operates within a collection of component contracts establishing what can be assembled, what data can be collected, and what the underlying systems will accept.

There’s an interesting change in what the designer is actually producing.

We’re ideally spending less of our time specifying the interface a customer will see and more time defining the possibilities from which an interface can be assembled, along with the circumstances that should influence those decisions. We’re moving from designing what an interface will do to designing how an interface decides what to do.

A prediction is not permission

An interface can be consistently right about what someone wants and still be wrong to act on it. As we give systems greater discretion over the experiences they create, we need some rules of engagement that govern both how the intelligence behaves and how users can exercise control over it. Four have emerged as useful guides.

Nudge. Confirm. Act. A prediction should be offered before it’s imposed. Confirmation introduces friction, but friction isn’t necessarily a failure of design. The more consequential an action, the stronger the case for asking permission.

Autonomy should be earned. Start with suggestions, then expand the system’s freedom through demonstrated competence and explicit user permission. Accuracy isn’t consent, and silence isn’t trust. The hundredth decision may carry very different consequences from the first ninety-nine.

Show the because. Conclusions without premises ask users to trust the system on faith. Make the reasoning behind an inference visible enough to evaluate or challenge. And consider how you tune it so it faithfully represents why the system made its decision and not just creates something that sounds convincing.

Design the undo first. The ability to reclaim control isn’t merely a safety net; it’s part of what makes surrendering control acceptable in the first place. When an action can’t be reversed, the threshold for autonomous action should be that much higher.

These have established some consistent expectations for how an intelligence should exercise judgment. They don’t, however, establish whether that judgment ought to be exercised in the first place.

Intelligence is not thoughtfulness

This is where I find myself becoming considerably less interested in what these systems can do and more interested in what they should do. I’ve been thinking about Hannah Arendt’s examination of thoughtlessness in Eichmann in Jerusalem,⁵Her enduring contribution was to suggest that doing monstrous things doesn’t require monstrous people, just ordinary ones who are very good at their jobs and insufficiently inclined to question them. A worthy reminder for all of us who do things that affect people. particularly the distinction between competently executing an established process and exercising judgment about whether that process is appropriate. There’s an uncomfortable parallel in the way we talk about intelligent systems: we’re becoming extraordinarily good at making them capable of doing things, without necessarily becoming more thoughtful about which things they ought to do.

I’ve come to three questions as a kind of ethical check when considering the introduction of probabilistic design into a customer experiences. It’s not a comprehensive framework, but they’ve proven useful in establishing a habit of interrogating the purpose of intelligence, rather than simply honing its application.

There’s an important distinction between creating value for a user and capturing more value from that user. The two aren’t necessarily opposed. A system that helps me find what I want more efficiently may create value for both of us. But the same predictive capabilities could be used to identify what I’m likely to buy impulsively, what friction I’m willing to tolerate, or when I’m most susceptible to persuasion. Both applications might produce terrific business metrics, but one is ethically questionable.

The problem isn’t that the business wants to make money. It’s whether we’re using intelligence to make the exchange more valuable for the customer, or simply making the customer more valuable to the business. So it’s worth asking: when those interests diverge, is the system helping the user make a better decision, or helping the business extract the most from the situation?

What feels thoughtful coming from someone who knows you can feel invasive coming from someone who doesn’t. Imagine talking to someone you’ve just met, who tells you where you went for dinner last Thursday, the medication you’ve recently started taking, and that you’ve been looking at houses in another neighborhood. Even if the information is accurate, you’d probably wonder how the hell this person knows so much about you. And why they think it’s appropriate to bring it up. You might happily share those same details with a close friend.⁶ The difference isn’t the information itself, but the relationship in which it’s being used.I mean I probably wouldn’t share the meds, but for the sake of argument …

Digital systems are capable of creating the same discomfort. We tend to think about this in terms of privacy and consent, but I think there’s a simpler, more human dimension: the appropriateness of knowledge within a relationship. A customer may have consented to the collection of information without expecting the business to draw every possible inference from it, much less demonstrate that it has. Being able to know something about someone doesn’t mean you’ve earned the familiarity that comes with knowing it.

Imagine that we’ve created a wonderful AI-assisted journey. Responsive, conversational, anticipatory. It removes friction and does things on behalf of customers that once required considerable effort.

But not everyone wants an intelligence acting on their behalf. Choosing not to delegate a decision is itself a perfectly reasonable decision, and one that the experience ought to respect.

Some people want to explore their options, understand the choices they’re making, or simply retain control over the process. Imagine that the deterministic experience is clumsy, neglected, and difficult. We’ve effectively communicated that customers willing to surrender some agency deserve a better experience than those who prefer to exercise it themselves.

Sometimes intelligence makes entirely new interactions possible, so there may not be a meaningful conventional equivalent. But when people can reasonably accomplish a task without AI, should a desire to retain control be met with an inferior experience? That isn’t simply a question about interface quality. It’s about where we invest our effort, whose preferences we take seriously, and the rationale behind introducing automation in the first place.

A different kind of design

I keep coming back to the idea that we’re not merely producing a more sophisticated class of interface, but actively seeing a change in what it means to design one. Done well, the designer’s contribution extends beyond the specification of an artifact into the establishment of a system’s judgment.

What signals should the system consider? What should it optimize for? When should it ask permission? Which inferences should it decline to make? How should it respond when it doesn’t know? And how do we determine whether the choices it makes in production are consistent with the intentions we had when designing it?

That last question is particularly uncomfortable. Much of our design process is organized around getting something ready to ship. Research, design, prototype, test, refine, launch. Hopefully we iterate. But a probabilistic experience is going to encounter circumstances we never modeled, and its behavior may change as the system, its context, or its inputs change. Testing the artifact before launch can’t tell us everything about the experience people will have with it afterward.

We’ll need different tools. Ways to examine inferences, test the consequences of uncertainty, detect drift, and evaluate the decisions being made in production. Some of these capabilities exist in engineering, risk management, and model governance.⁷ They just aren’t necessarily part of the designer’s everyday working environment.Honestly, these last two weren’t topics I thought a lot about before starting in my current organization. Perhaps they should be.

A note on the title

At some point, the language of building an interface began to feel inadequate. We build things to specifications. But the systems we’re discussing won’t necessarily behave precisely the same in every circumstance. Their value comes from being able to respond to situations we didn’t anticipate.

That made me think about the difference between building something and raising it – maybe a ridiculous analogy, since an interface isn’t a child, and attributing human qualities to an LLM is a good way to confuse ourselves about what it actually does.⁸My bias to this language may be in part because I’m a parent of teenagers, who at this point don’t always follow my directions, or creatively misinterpret them, but seem to have developed good judgement.

Still, the distinction seems useful. Raising implies establishing boundaries without expecting to prescribe every future decision. It implies teaching, correcting, paying attention to emerging behavior, and accepting some responsibility for what happens after the initial work is done. The metaphor also accommodates something we don’t particularly enjoy acknowledging about intelligent systems: they may occasionally do something unexpected, ridiculous, or harmful despite our best efforts to constrain them.

This is either the most interesting design problem of our careers or a fairly terrifying expansion of our responsibilities. Probably both.

If you set a tone at the beginning, this is where the experiment begins.

And I wonder whether that might ultimately become one of the more consequential design responsibilities of this new medium. Because when the artifact continues making decisions after we’ve walked away, what exactly does it mean to be finished designing it?