AI is getting warmer faster than our human systems

AI for care vs. AI that cares

This July marks my two-year anniversary of developing a condition that no doctor can fully diagnose and everyone wants to treat differently.

Anyone who’s experienced a chronic health issue knows how grueling it can be. Consult after consult exposes you to a system that’s lost its compass for treating patients as whole people rather than inconvenient, expensive puzzles. You become an adversary to both your own body and the system meant to heal it.

You also become deeply accustomed to hanging out in waiting rooms.

Digital tools and human misery abound in these waiting rooms. The moment you arrive, you’re asked to fill out forms on a screen while someone at a desk glares at you.

I get that healthcare providers need more efficiency, and that the iPads are getting us one step closer to electronic health record nirvana, where we finally stop filling out forms.

But when you squint, it’s clear that waiting room technology is just a sustaining innovation on a system that’s fundamentally more mechanical than personal. We’ve just dropped the pretense that it was anything but that.

That efficiency premium drives the experience: instead of technology making the patient experience a few degrees warmer, it’s somehow gotten even colder.

Healthcare isn’t unique. The same pattern is emerging anywhere tech meets myriad bureaucratic systems. That begs the deeper question: as the technology advances by leaps and bounds, how might AI actually make our waiting rooms, classrooms, and exam rooms warmer, not colder?

AI for human systems—not against them

I’ve been thinking about these dynamics in education too, particularly in light of Sal Khan’s recent admission that Khanmigo, his tutor chatbot meant to revolutionize learning, had amounted to a “non-event” for most students. Kids didn’t really use a tool that Khan had promised would change their lives.

As Khan told education reporter Matt Barnum, “AI is going to help, but our biggest lever is really investing in the human system.”

That line elicited glee from Khan’s long-time skeptics. They’re not all wrong: if we invested in human intelligence at the rate we’re pouring gazillions into AI, our education systems might be radically better—particularly if teachers made as much as tech bros.

But Khan’s language, and his adversaries’ response, show that we’re trapped in a false dichotomy between humans and tech. What I wish he’d said? “Self-service AI is going to help, but our biggest lever is really investing in the human systems.”

Because the best case scenario isn’t that human systems “win out” over technology, but that we see a proliferation of AI systems that support the human systems students need: fostering the teacher-student and peer-to-peer relationships that keep kids motivated because someone believes in them, even when they don’t yet believe in themselves.

In short, we should be building AI that makes education warmer, not colder.

That’s hard to picture when even our human systems have a mechanical quality to them, and most of us think of AI as synonymous with chatbots that cosplay humans. But AI could be built to be fundamentally prosocial in ways that bolster human systems.

It’s already happening in little pockets of the market: Josh Nesbit at the Relational Tech Project, building local solutions with—not for— neighborhoods; Phil Komarny at Maryville University, building more seamless, humane systems for credit transfer and re-enrollment; Deepti Doshi at The New Public, building Roundabout, a more civil and joyful version of Nextdoor; and Jean Rhodes of MentorPro, building mentoring software that keeps the human mentor not just in the loop, but at the helm.

The common thread? These builders are unlocking AI as backend infrastructure to a human support system, where people are seen and connected by and to one another.

The sheer existence of those efforts illustrates that our challenge isn’t actually one of supply—we can build tech that makes our systems warmer. Rather, it’s a demand-side story: if systems like education, healthcare, social services, or local government were actually designed to make humans feel more human, they would demand tools that did the same: helping the humans working in those systems show up in ways that make patients, students, and citizens feel seen, understood, and supported.

It’s in the absence of human-centered demand that the market bends the opposite direction: AI that automates bureaucratic processes becomes rational and tantalizing. And if incentives reward efficiency at all costs, it becomes inevitable. Tech systems and elusively “human” systems start working at odds.

Charting a path toward warmer human systems

While I hate when people say “we’re at an inflection point” (saying it as often as we tend to essentially means we’re spinning in place)... we’re at an inflection point.

Because the story AI is writing is not just about how tech infiltrates our traditional systems. It’s also how direct-to-consumer tech is stepping in where our systems and institutions fall short.

In that story, traditional systems that integrate tech get progressively colder, while consumer AI tools get better and better at synthetic warmth. For example, a meta-analysis of research on healthcare chatbots found that ChatGPT has a 73% likelihood of being perceived as more empathic than a human practitioner in a “head-to-head matchup”. In other words, AI tools are doing a better job at making people feel seen than people are.

Stats like that tell us as much about humans as they do about tech. But they also tell us something deeper about system design: the healthcare system was never optimized to build or reward medical professionals who provide the empathy patients crave. (Not to discount the heroic professionals who do provide it—but they’re working at odds with system incentives.)

Play that out at scale, and we’re headed toward a two-tier system that keeps pitting AI against human systems instead of putting it in their service. A system where warm bots continue to outpace human warmth, rather than tools that enable and scale human warmth.

I can attest to this in my own journey. The desire to feel seen and cared for doesn’t go away when you’re years into a health challenge—it just gets buried under liability waivers and low expectations. More than a few times over the last two years, I’ve broken my own policy over this—the one where I don’t outsource human connection—and started describing my symptoms to ChatGPT. Because, relative to most of my doctor visits, it does feel like it cares.

That’s a hard fact and feeling to swallow. But it’s one we need to confront head-on. We didn’t get cold waiting rooms and warm bots by accident. They’re both reflections of what our systems are built for and of the human(e) dimensions where they’re falling short.

But the future shouldn’t be decided by AI becoming more human in an attempt to meet our deepest needs. It should be shaped by human systems that do.