Who's Liable When AI Gets It Wrong? | Robert Kellar KC on AI, Law and Liability in Regulated Professions

[00:00:02] This is the TechDental Podcast, the strategic intelligence hub for leaders shaping the dental industry. We break down how AI, data and operating discipline drive performance and scale. I'm Dr. Randeep. Let's dive in.

[00:00:22] This podcast is only for general discussion and information purposes. Nothing I say should be treated as legal advice or relied on as legal advice by anyone listening. The issues we're going to discuss in this podcast are really very fact specific, particularly in this space. So please do take proper legal advice from a qualified lawyer before acting on anything at all that we discuss and don't rely, I should say, on AI as a substitute for that.

[00:00:49] Welcome back to TechDental. Picture this, a dentist sits down with a patient, an AI tool flags a lesion and recommends a treatment, and the clinician acts on it. The scan was wrong, the patient is harmed. The only question that matters then is, who is liable? Right now, English law cannot give you a clear answer on that. Not for the dentist, not for the group that bought the software, and not for the company that built it.

[00:01:15] Every practice deploying AI today is making that decision inside a legal vacuum, and most of us have no idea where it ends. My guest is one of the few legal professionals in the country who is actively mapping out that vacuum and navigating this emerging and fast-paced field. Today, we are lucky enough to be joined by Robert Keller, KC. Robert, welcome to TechDental. Thanks for having me, Randeep Singh Gill.

[00:01:39] Can you give us some background on your legal career so far and your interest in the world of AI technology? Yeah, I'm a barrister. I qualified in 1999, and I've spent my career doing a range of different things. But in recent years, I spent most of my time litigating high-value clinical negligence claims and dental negligence claims and doing some regulatory work as well, acting for doctors and dentists.

[00:02:07] And I remember a few years ago, I think before ChatGPT was even something we'd heard of, I was asked to speak about the relationship between AI and legal liability for medical practitioners. And it's something I've had an interest in since and I've written about and talked about with increasing frequency as the tech begins to take off. I want to start with the foundation because I think everything else we talk about today is going to be built on this.

[00:02:34] For nearly 70 years, the way English law has judged whether a clinician was negligent comes down to one test. It's built entirely around a human being making a human judgment. Now, put an algorithm in the room. The decision is assisted or shaped or in some cases effectively made by a system that no clinician fully understands yet. So my first question to you is, when a claim arises from an AI-assisted diagnosis, who does English law look to at first in terms of liability?

[00:03:03] Walk us through the liability chain as it actually works today. The reality is that the first person that the law will look to to compensate is the healthcare practitioner, the dentist. And the reason for that is that both as a matter of good practice and these days as a matter of good governance, the healthcare practitioner, so the dentist, will be the one supervising the AI and making the clinical decisions. So ultimately, it is they who will remain responsible. But next in line will be the deployer, so the hospital or the dental practice.

[00:03:32] And they might be liable for a couple of reasons. One is because they will be vicariously liable for negligence of the healthcare practitioner making the decision. But also, they may have their own systemic duties. So they may have their own duties of audit, of maintenance, of validation. And that might give rise to some form of direct liability. And the final legal person in the frame is the product builder, the developer. They may be liable for producing an effective product.

[00:04:01] So those are really the three potential defendants that will usually be in the frame. So the chain still runs finally to the clinician, which takes me straight to the test itself. That 70-year-old standard was designed for human decision making. What happens to it when the decision is assisted or aided by an algorithm? Does the standard stretch to fit it or does it break and do we need a new standard?

[00:04:26] My view is and has been for a while that as AI disrupts medical and dental practices, it's going to disrupt the law as well. And I'll give you three disruptions that I predict at increasing levels of disruptiveness, if you like. So disruption one is that I think we'll see higher standards of care.

[00:04:45] So whereas once upon a time, dentists might be forgiven for incidental or difficult to interpret areas of caries in decaying teeth. When we have increasingly able AI systems that can flag up caries in adults or children, those errors may be less forgivable. So disruption number one is higher standards of care.

[00:05:08] Disruption number two is I think that we may end up in a position where doctors and dentists are required in some circumstances to use AI. So a duty to use AI itself. I think we're not quite at that tipping point. But as the royal colleges and the august bodies start to say that the use of AI is standard practice in things like radiology or disciplines like pathology,

[00:05:34] it may be that the state of the art becomes expected practice. But I think as you alluded to in your question, the third possible disruption is that we have to reconceptualize how we think about negligence and how we apply the test for negligence. Because as you pointed out, it's the Bolum standard we've been applying for years. So what would a responsible body of dentists or doctors do in any particular situation? That's the litmus test for breach of duty.

[00:06:03] But what do we do when an AI system, an advanced AI system makes a recommendation which is at odds with what a responsible body would do? Now, our first blush, our first instinct might be to say, in the end, clinical judgment trumps everything. Clinical algorithms or guidance is not a substitute for patient-specific judgments.

[00:06:23] We might say that the same applies to AI, that in the end, if AI is in conflict with what a clinician thinks they ought to do, then they must press that mental override switch and do what they think is right. And whilst that might feel like the obvious answer, is it the right one?

[00:06:45] I ask rhetorically, because the whole point of AI is to surface insights based on data that humans can't get to themselves. So in the case of radiology, perhaps it will spot concerning patterns in breast tissue that no human radiologist would see or interpret.

[00:07:06] In dentistry, perhaps it will spot very early signs of incipient early decay, which no human dentist would regard as being concerning, but are concerning. And if we are always requiring dentists and doctors to press the mental override switch and exercise their own judgment, then there's an argument that we are depriving patients of those superhuman insights.

[00:07:33] And so my own view is that within the next decade or so, we're going to see cases which push at the limits of the traditional bottom standard and perhaps dilute it or do away with it altogether. You've written about clinicians becoming liability sinks for AI, where the fault naturally flows to.

[00:07:57] Tell me what that actually means for a dentist at a multi-site group using an AI tool every single day who could not tell you how it reaches its conclusions. What I mean by liability sink, and I have to say this is not my expression. It's an expression I think used elsewhere in the literature, so I'm not going to take credit for that one. But what it means is the idea that because doctors or dentists are in practice the people supervising the systems, they end up carrying the can where things go wrong in practice.

[00:08:25] That's the first port of call where AI influence decisions create harm. The reality is that I think there are two dangers for dentists who are using AI. Danger number one is automation bias. So it's this risk that over time dentists uncritically follow what AI is recommending without applying their own independent judgment.

[00:08:49] And so that when AI makes an error, a finding is made that dentists of ordinary care and skill should have known better and are found liable on that basis. So automation bias is the first trap. And danger number two is the opposite, really. It's the risk of liability for disagreeing with AI.

[00:09:13] So if AI gets it right and you get it wrong as a clinician, that gives rise to its own risks. So what would be my advice to avoid those traps? I think one is always apply your own independent judgment. Be conscious of the risk of independent bias, number one. And number two, if you're using a particularly reliable AI system and you're going to depart from its recommendations, make sure you document your reasoning carefully. Make sure you've got a record of why it is you are disagreeing.

[00:09:42] We all know good record keeping is one of the mainstays of staying out of trouble. And this is another area where I think that applies. And I think this is really the interesting thing because the people buying the software, they're not really thinking about this. They're thinking about accuracy and workflow and not necessarily where the liability will land. Dental group operators are making procurement decisions right now. What are the legal questions they should be asking of their vendors? That's a great question, Randy.

[00:10:11] I think the first question is, where does liability sit where this thing that I've just bought goes wrong? One knows from experience that operators can sign contracts which excludes the supplier's liability entirely or perhaps caps it. And therefore, the operator, the dental practice or group is left holding the baby, if you like, for liability purposes.

[00:10:34] So it's important that you read and understand from your supplier where it is liability sits from a contractual perspective. I suppose the second question to ask is, can you explain to me or to us how a decision was reached if things go wrong? So if a patient brings a complaint or a claim, can the operator, the supplier, explain why it is the AI flagged or didn't flag a particular finding or recommended a particular treatment?

[00:11:03] All of the difficulties with AI is that its inner workings can be opaque. And systems vary on how good they are at explaining why or why not they arrived at a particular decision or with what degree of confidence they arrived at a particular decision. And the more opaque system is, the more difficult it is to exercise independent clinical judgment.

[00:11:32] And also the more difficult it is to defend a claim if things go wrong. So, yeah, the explainability piece, the transparency piece is something to drill your supplier on. I think the whole liability gap argument assumes uncertainty is a problem to be solved. But some would say regulatory ambiguity is exactly what lets innovation happen before the law catches up. Is the absence of a settled framework actually a feature for early movers and not a bug?

[00:12:02] And does pinning it all down make dental AI safer or just more expensive? I think this tension between innovation and regulation is a very hot topic at the moment. I think in the end, all that can be said is that it's about getting that balance right between under-regulation and over-regulation. If you over-regulate, of course, you stifle innovation. If you under-regulate, you compromise patient safety and trust and in the long run, the uptake of the technology.

[00:12:31] There are different approaches around the world, radically different approaches. The EU has taken the approach of high regulation and the US very light touch, if any, regulation. My own view for what it's worth is that this sort of balance or tension between regulation and innovation is not a zero-sum game. It's not one or the other. And in the end, regulation is probably a good thing, I think, for innovators in the long run.

[00:12:59] And that's because, from my perspective, it provides certainty. It provides clarity about how to avoid risk and liability exposure. And those are good things in the air for innovators and for entrepreneurs. Robert, the professional accountability system in dentistry rests on a simple idea. A registered professional made a decision and they can be held accountable for it. That's the whole premise. AI breaks that premise.

[00:13:27] If a dentist follows an AI recommendation that turns out to be wrong, were they exercising clinical judgment or were they deferring to a machine? And if it's the second one, has the entire framework of professional accountability quietly become inadequate in the age of AI? I think the answer to that is quite simple, in a sense.

[00:13:46] The registrant, so the dentist, will always, in the regulator's eyes, be responsible for the decisions that they make, whether or not they've used AI to inform their decisions. And if they rely on AI systems without exercising that independent judgment, then the reality is that they will be placing themselves at regulatory risk. I mean, the regulators have made that as clear as they can.

[00:14:11] And so, again, it just comes back to, from a dentist's perspective, exercise your independent judgment and document why you're doing what you're doing. Whether or not you're agreeing with the AI, if you're disagreeing with it, you should document why. If you're agreeing with it, you should demonstrate you haven't simply delegated judgment to the system. You've done that in an informed and independent way.

[00:14:36] And I appreciate some of this is hypothetical because it's in emerging technology, but in a fitness-to-practice investigation that comes out of an AI-assisted treatment, how is automation bias likely to be treated in the future? Would it be a mitigating factor, an aggravating one, or something the framework simply is not equipped to weigh yet? I think it probably is equipped to weigh that issue. I think there's a personal view is that it's at least capable, in principle, of being a mitigating factor.

[00:15:01] If you use, for example, a sophisticated AI tool to diagnose decay radiologically, I can see that it would be a mitigating factor to rely on a false positive or a false negative that an AI system produced to justify your erroneous treatment. But like everything in the law, everything is context-specific. It all depends on the facts. The weight you give to that mitigating feature depends on a number of factors.

[00:15:31] What steps were taken by the practitioner to educate themselves about the risk of error in the system before relying on it? What justification for the decision was apparent on the face of the AI system that was being used? Was it properly explained? Did the system provide a confidence score, and if so, was that accurate?

[00:15:51] Was the output produced by the AI in the end out of kilter with what a responsible body of dentists would do, and if so, to what extent? And is there evidence in the end, coming back to this theme, that the dentist took some steps at least to apply their own independent judgment? And if so, as I say, is there evidence of that, and what was the basis? So these things are not cut and dry. They're open textured, and there are quite a few factors, I think, that are parallel and taken into account.

[00:16:22] So the regulator is holding the line on the individual accountability, while the technology seems to be pulling in the opposite direction, which raises the timing question. The GDC standards predate AI deployment at any real scale. What does the regulator need to do before there's enough AI in enough surgeries to start generating actual case law on this? And from where you sit, is it moving fast enough?

[00:16:45] Well, on the question of whether it's moving fast enough, according to my understanding, the GDC has at least taken some steps to inform itself about how AI is being used in dental practice. So in 2024, it commissioned a rapid evidence assessment and surveyed the field and looked at how AI was being used across the field in things like robotics and deep learning models to detect caries at an early stage.

[00:17:13] But as a personal view, I think more is needed and faster. So far as I'm aware, there is no explicit AI-specific guidance from the GDC at the moment. I'm aware that the GMC has some limited guidance, but the GDC has none, so far as I'm aware. And some clarity will be good for the profession.

[00:17:35] So some basic basis to cover would include things like what due diligence is required from practice managers and groups before they deploy AI tools on their patients? What degree of human oversight is necessary and appropriate? What does that look like in practice? And whether and how consent obligations are engaged when AI is used in a patient-facing setting.

[00:18:04] So in short, in my view, more is needed by way of guidance from the regulator and more quickly. So for a group putting AI across a multi-site estate, what professional governance structures should be in place before a single clinician touches an AI tool in front of a patient? I think in a sense that will, without fudging the question, it will depend on the size of the practice and the nature of the tools being used in the practice or the group.

[00:18:33] But a few general bases to think about. A designated AI clinical and governance lead would be a good starting point. Someone who really understands the regulatory terrain and the sorts of issues that arise and is responsible for that aspect.

[00:18:49] A pre-deployment due diligence framework or policy to make sure that any medical devices being used in the practice are properly regulated with the MHRA, are clinically validated. That there's been a proper review of the contractual provisions and the insurance cover to cover some of the bases that we discussed earlier.

[00:19:12] Perhaps a written policy on the responsibility that dentists in the practice have, emphasizing this point about dentists always needing to exercise their own judgment and the fact that they will remain responsible for decisions. And then finally, although not finally in terms of what, but finally from the obvious points that I can think of, some sort of incident monitoring and audit system.

[00:19:37] So where AI produces an expected, incorrect output or output that needs to be overridden by dentists, making sure that is properly recorded and audited somewhere and reviewed at group level, not simply at site level or by an individual dentist. So there's a lot of questions that is, can you think of what you're doing in the process and went through, not simply at the same level of trust. And if there's a pattern of problems that can be addressed, perhaps fed back to the supplier.

[00:20:04] So there's some starters for 10 to get you going, Randy. Everything we've been discussing so far about the regulator, it's assuming that they should be acting proactively. But by design, the regulatory body is reactive. It responds to the cases that come before it and case law develops on its own. is there a serious argument that it should just wait for the litigation to show where the framework breaks rather than trying to anticipate problems that might look completely different in practice

[00:20:31] than they do in theory? I think in short no would be my answer that's probably one area where I do have a fairly clear and strong view in the end the regulators function and this applies across the healthcare professions not just dentistry it is to regulate dentists but also to provide clear guidance to dentists as to how they meet their own standards it's not fair in my view as a matter

[00:20:57] of policy simply to discipline dentists first and provide the guidance later and on a more practical level in terms of the wait and see approach the reality is that most dental regulatory cases don't reach the high court thankfully and so that's the position as far as I'm aware so if we're waiting for guidance from the higher courts it could be an awfully long wait before we get anything meaningful so I think burden in the end is upon the regulators to provide some early guidance and

[00:21:25] not simply wait and see how things develop. This takes us nicely to the third party in all of this the company that built the software the vendors so far we've talked about about the clinician the regulator let's turn to the vendor now because this is where it gets a little bit strange from my point of view that there is a law from 1987 the consumer protection act that makes the maker of a defective product strictly liable the claimant does not even have to prove negligence but it was written

[00:21:54] long before anyone imagined software making a clinical decision and whether AI software even counts as a product under that act has never really been settled by an English court here's the part that should make every vendor sit up the EU has already answered this question the UK to my mind has not so let me start there how do you read the current state of English law on whether AI software is a product under the 1987 act and what would it actually take for a court to resolve it?

[00:22:22] That's a good question and surprisingly the current law as I understand it is still quite ambiguous on the point I think the clue is in the title to the act it's the dated 1987 as you pointed out and that tells you that it was drafted a long time before the advent of AI and chatbots and even the internet as we know it so

[00:22:45] it's a fairly aged piece of legislation there is a line of sort of authorities which came to the conclusion that in this act and in others which are similar that software cannot be a product because it's not a tangible good and that's because the act was drafted at a time when most products were physical objects

[00:23:08] and not things which live in sort of ethereal world of cyberspace so I think in terms of the answer to your question where AI is integrated into a physical product that is bought by a dental practice I think in a sense there'll be no difficulty about that there would be more difficulty I think hypothetically if the AI system was something which was uploaded from the cloud and sat within an existing product

[00:23:35] that was already owned by the practice I suspect if push came to shove most high court judges would strain to give the act a purpose of interpretation that means trying to make it move with times but it would strain to do that whether they would get so as to say that AI was a product I suspect they would find a way of doing say but not with any certainty as you pointed out Randy in the European context

[00:24:03] the Europeans have now produced an updated product liability directive and within that it is abundantly and explicitly clear that AI and software is a product I think will follow suit here eventually but it may be that will require some form of intervention from parliament to clear up any ambiguity and on the EU law for a dental technology vendor sent selling into both markets and for a UK group

[00:24:29] choosing a platform today what does that gap actually mean in practice between the UK law and the EU directive from a practical perspective I suppose there are two points one is that we are not regulation free at the moment I mean we do have the medical devices regulations so if a UK group is making procurement decisions the first question is this registered in the UK under the

[00:24:56] under the medical devices regulations so that's the first point but you're right to say that there is a gap in terms of AI specific regulation our regulation is in one sense weaker we don't have all the bells and whistles that exist under the under the AI act that's in the EU which is much more rigorous in some ways and has a number of AI specific

[00:25:21] requirements which don't exist at the moment in UK law how should dental practices groups how should they think about this from a perspective one practical move would be to use the EU compliance framework as a proxy for as it were gold standard safety and gold standard governance and if you make procurement decisions by asking is this particular product EU AI act compliant

[00:25:49] and do you have a conformity assessment to prove that then you can say with a very high degree of confidence that even though the UK law doesn't require it you've got gold standard compliance and that even if the UK eventually catches up as it may do in the foreseeable future you've got a product which is going to stand the test of time so I think from a practical perspective

[00:26:11] it's about trying to keep pace with EU compliance. Product liability needs harm from a defective product but in most cases in clinical scenarios the harm doesn't come directly from the AI being defective on its own it comes from the interaction between an imperfect tool and a human making a judgment so is product liability actually the right vehicle for most dental AI claims at all or does it end up back in clinical negligence with all its messiness?

[00:26:37] I think in the end what many claimants will do if defective AI is part of the picture this issue will only be engaged where there's a genuine issue about the AI being part of the problem I think what most claimants will do is they will sue everybody that's something we are used to in clinical negligence so the dentist will be in the frame for not applying their own independent judgment and getting it wrong I suspect that the dental practice or group will be in the frame as the dentist employer

[00:27:06] therefore vicariously liable they'll also be in the frame because it will be said they didn't do their due diligence on the system or didn't audit it or validate it properly and finally for good measure if you've got a competent lawyer who knows their way around product liability they may put the developer or manufacturer in the frame too under product liability so my own view is that in the early innings of litigation around these issues claimants will sue everybody

[00:27:36] and leave it to the defendants to work out between them who's going to cough up which tends to be how things happen in the clinical negligence world now it will just take a different form in the future Robert you responded to the National Commission's call for evidence earlier this year in your view what's it likely to recommend and which of those recommendations has actually changed the commercial calculus for a dental group deploying AI? It's a good question it's the million dollar question but we're as we sit here today

[00:28:04] with some way of getting the commission's recommendations back they are going to be looking at the regulatory piece and the liability piece what they will recommend I simply don't know I suspect if you are looking for clues you might look to the EU liability directive because the EU has already developed a framework and they have some interesting innovations to name a couple of features of that directive one interesting feature is that

[00:28:31] rather than product liability being tied to this open-ended open-textured test of the standard of safety that the general public is entitled to expect liability is tied to regulatory standards so the regulatory standards and the liability framework speak to each other I suppose one part of the calculus is make sure you are if that is going to be a recommendation if we are going to get a regime which maps to the European regime

[00:28:59] make sure that your devices are compliant with UK regulatory standards and make sure of that because if they are not that might be something which exposes you to liability another innovation in the EUA Act and there are quite a number of them is that there are quite onerous disclosure obligations on defendants so if a claimant can make out a plausible case that an AI system has caused harm

[00:29:28] that it's incumbent upon defendants so that could be manufacturers or it could be dental practices to disclose all of the relevant information at their disposal about why a particular decision was reached so that comes back to the discussion that we were having earlier that make sure in your contracts with suppliers that you are in a position to explain as much as you possibly can about how our system operates in general

[00:29:57] and why it produced a particular decision in a particular case so this is me with my crystal ball gazing hat on I don't claim to have any particular insight as to what the commission will do but it wouldn't be a huge surprise if they used the EU framework as a model Robert, this has been an absolutely fascinating conversation but we're coming to the end of our podcast and that means our traditional lightning round

[00:30:26] so these are some quick fire questions so hopefully they don't put you on the spot so the first one what's the single governance action a dental group should take in the next 90 days to cut its AI liability exposure? I think it would be a point an AI, dedicated AI clinical lead who knows something about governance and if they don't know something about governance mug up on it pretty quick because it's going to be an increasingly important issue and if you can't do that employ a good lawyer who knows something about it

[00:30:56] and let them do some auditing for you and make some recommendations And the one question every dental leader should ask their AI vendor before that system goes anywhere near a patient Who's liable when things go wrong? Me or you? And are there any caps on your liability? That would be my, certainly my list And the final one is crystal ball time again a prediction

[00:31:25] when the first high profile AI clinical negligence case in dentistry reaches the courts who's standing in the dock? I think unfortunately it's going to be the good old fashioned dental practitioner who will bear the brunt of the claimant's ire in the end they're going to be the easiest person to sue and it will be on the basis that you didn't apply your own independent judgment you applied automation bias or you disagreed

[00:31:54] with an AI savant when you shouldn't have done I think it will be the unfortunately poor old dental practitioner who is the counter carrier Oh, maybe we won't be sending our AI avatars to court then Or perhaps you should instead of the dentist That may be a future product to make but Robert this is exactly the conversation this industry needs right now Thank you for your time It's been absolutely fascinating Where can people connect with you and follow your work? Probably the easiest place is on LinkedIn

[00:32:22] as everybody is these days so feel free to connect with me on LinkedIn when I write stuff or talk about stuff I generally post there so that would be my recommendation Fantastic and to our listeners if you're building, operating or investing in dental AI and this episode resonated with you or your team the full written analysis is at www.techdental.com forward slash insights share, subscribe, like and comment all the links are in the show notes Until next time

[00:32:51] I'm Dr. Randeep and this is Tech Dental You've been listening to the Tech Dental Podcast strategic intelligence for dental leaders navigating structural change If you're responsible for growth performance or long-term value in this industry make sure you're subscribed I'm your host Dr. Randeep We'll see you next week you