Guardrails for the Future of AI with Viktor Mayer-Schönberger, Professor of Internet Governance and Regulation at the University of Oxford

Richie Cotton: Hi, Viktor. Thank you for joining me on the show. So, since we're talking about guardrails today, to begin with, can you explain what is a guardrail? 

Viktor Mayer-Schönberger: A guardrail is, uh, a guidepost. It helps us in our decision making. I mean, we make, uh, lots of decisions every day.

literally thousands of them, but some of them are really consequential. And, uh, if we make the wrong decision, that could have, uh, huge consequences that could mean difference between life and death. Uh, and so making good decisions is really helpful and having somebody that can provide some guidance on how to make good decisions is even better.

Richie Cotton: So I like the idea of tools to help you make decisions. Uh, so can you give me, uh, some examples of how guardrails might be used in a business context? 

Viktor Mayer-Schönberger: Sure. Uh, but let me give you an example of what, what a good guardrail is first, uh, outside of the business context, just so that you understand and the, and the listeners understand what's the difference between a good guardrail, flexible enough and adaptable enough and, and something that's really not good.

So a really good guardrail is for example, uh, the rule that we should drive on the right side of the road, if you're in North America or on the left side of the road, if you're in the UK. That's a guardrail. Um, it helps us avoid accidents. Uh, it's, uh, hugely efficient and beneficial. Um, but we can break that guardrail if we want to.

For example, when we want to overtake a car, and there's no other car coming our way, we can actually, uh, change to the other side and overtake that car. Uh, the guard, the real good guardrail, uh, helps us, guides us in our decision making. Uh, but we are still in the driver's seat. Uh, we still make the decisions.

Uh, it doesn't nanny us. It doesn't tell us what we do. It empowers us to make good decisions Um, but it uh, but it doesn't go beyond that. That's a good guardrail So in the business context, for example, if we now bring it back to the business context, uh, a good guardrail is one Uh, that, uh, that helps the manager, that helps an organization, uh, to, uh, reach its goals, to reach its aims, um, and, uh, uh, that may come in all kinds of forms and shapes.

Standard operating procedures, uh, in companies are guardrails. Uh, in fact, standard operating procedures in airplanes are guardrails too. And pilots should keep to the standard operating procedures. but there are exceptional circumstances where they can break them. And the same is true in the business context, of course, as well.

Richie Cotton: Okay. So it sounds like, um, there's quite a range then from just sort of simple rules of thumb in order to give you good advice through to, um, processes and procedures up to, I guess, laws, counters, guardrails as well. Absolutely. 

Viktor Mayer-Schönberger: And some laws are quite strict and you can only do a certain thing, and, and other laws are quite flexible as well.

And, and keep in mind, there's a law that we cannot speed on the highway beyond, in the U. S., 55 sometimes 75 miles per hour, but we can still speed. We just have to live with the consequences of a speeding ticket, for example. Uh, in other words, That is a pretty good guardrail because it contains, and it has that flexibility built in.

And similarly, in a commercial context, a good guardrail helps decision makers, but doesn't disempower them, doesn't take decision power away from them. 

Richie Cotton: Okay, so it's like, this is a really good idea, but you have the freedom to break it if you deem fit. You mentioned that sometimes making decisions can be a life or death situation.

Um, do you have any examples where a lack of guardrails has caused some sort of problem? 

Viktor Mayer-Schönberger: Yes. Um, and, uh, if you forgive me, I'll, I'll, I'll offer, uh, uh, an example of the, uh, from the aviation history. Again, uh, we guardrails with this very stark story. Um, It's about, uh, 20 years ago, 20 some years ago, um, at that time there was a new, uh, device or relatively new device that was built into commercial airliners to avoid, uh, head on collisions.

It was, uh, a collision warning system and, uh, these two, um, boxes in two airplanes that got, uh, too close to each other, they would negotiate and one would order the crew to ascend and the other would order the other crew to descend, thereby avoiding the head on crash. And when these boxes came around, initially there was no good guardrail, no good rule that mandated that pilots in all or almost all circumstances should follow the directive of the machine.

Uh, to avoid the collision. Um, and so, two, uh, airplanes, uh, over, uh, the Swiss German border, uh, got very close to each other, uh, both had those boxes in there, but only one airplane had a standard operating, airline had a standard operating procedure and the pilots complied with it. uh, and thereby, uh, descended.

Uh, the other side did not believe the machine, uh, did not have standard operating procedures, not guardrails in place, descended as well, a crash, and 100 people died. 

Richie Cotton: Uh, that's about as bad as it gets in terms of outcomes there, so that's a pretty tragic event. So, uh, yeah, I can certainly see it having Um, a procedure and making sure that other people are following that procedure as well, uh, can be incredibly important Um, all right.

I think we need I think we need a happier story to uh to balance this So do you have any examples of when guardrails have been helpful and there's been a positive outcome? 

Viktor Mayer-Schönberger: Absolutely. Um, and and you see already, um as i'm drawing in examples from the aviation industry from from from uh automobiles and and and and driving Transcript Uh, all the way to, uh, the commercial context, uh, guardrails really are everywhere.

Um, uh, one of the, the happy outcomes in a way, uh, with respect to guardrails is, uh, from Africa, uh, where there is, uh, oftentimes there are oftentimes contested resources just like water. Uh, and in West Africa, a number of, uh, small nations depended on, uh, the, uh, Walter River, uh, water supply. Um, but, uh, uh, some nations.

took more water than the others. Um, and so they came together, uh, and in a very inclusive, uh, process, uh, came up with guardrails, with a framework on how to divvy up that limited resource, um, and, uh, and then ran by it. It was not It was a fixed framework. It was somewhat flexible. It also needed to be adapted to changing uses and changing context and also climate change, of course.

But overall, it was a very positive story as it enabled not just a limited number of people to get access to good water, but a very large group of people that transcended. a particular jurisdiction or a particular nation. 

Richie Cotton: Okay, yeah, so, uh, certainly different nations agreeing on how to share resources feels like, uh, quite a tremendous thing.

Uh, it doesn't happen very often that more likely to argue, uh, or go to war with each other over, uh, over resources. So, and what's 

Viktor Mayer-Schönberger: really interesting is, I think, That, you know, um, you would never think that this was easy or possible, but it actually happens. So we are capable as, as a species to come up with quite flexible, but also quite pragmatically operative, efficient guardrails.

Richie Cotton: Okay, uh, so I think you've convinced me that guardrails can be very useful. Now, I'm curious as to what makes a good guardrail. So you mentioned before that they need to be empowering of individuals. Are there any other principles like that that make a good guardrail? If I 

Viktor Mayer-Schönberger: know exactly what I want to achieve, then I can design the guardrail, I can design the the regulatory mechanism to achieve that goal.

The problem is Uh, what if the goal is somewhat wobbly or what if the context may change? Think about artificial intelligence regulation these days, uh, And and how do we go about this? It seems to me that the european union With its ai act recently passed seem to understand perfectly what the problem was, and then create the perfect solution for the problem.

Except, what if they didn't understand the problem very well? Then the mechanism to achieve the goal may ultimately turn out to be quite inefficient. And so, in other words, uh, if we don't really know what we want to achieve, or we don't completely understand the problem, we need some guardrails that are perhaps a little more flexible, but most importantly, help us to learn, uh, to learn from the decisional mistakes that we make, and also help us to then, uh, adapt the guardrail itself.

Uh, so that it is, uh, uh, uh, better suited, uh, for the context in which we're in, uh, and that's really hard. That's unbelievably difficult to do, uh, especially for, for classical state regulators or multinational regulators who want to lay everything down, uh, in, in great detail, uh, so that there is no ambiguity whatsoever.

Urs and I in our book argue that when we don't really understand the problem completely yet, one of the design principles is to build some flexibility into the guardrail and some ability for the guardrail to be adapted. but also to create guardrails that help people learn, learn from their decisions that they make so that they don't make the stupid mistakes again.

Richie Cotton: Okay, uh, so that sounds sensible. It also sounds a bit like, um, like the agile software approach coming to creation of guardrails as well. Like you need to do something and then, you know, a few weeks later you review it and change your mind about what you're doing next. 

Viktor Mayer-Schönberger: Yeah, yeah. And what is interesting, Richard, that you bring this up, um, when we talk to programmers or people in the software industry, I say, yeah, exactly.

That's what I do a lot with agile, uh, programming and all that, because oftentimes I don't really understand my user perfectly yet and I need to adapt. I need to also adapt to, to, to new user, um, wants and new user preferences, uh, and all that. The problem is really that the regulators haven't understood this.

The regulators have over the last 150 years gone from relatively flexible rules to ever more detail and tight rules, uh, which squeeze all the flexibility out of the system. 

Richie Cotton: Uh, actually, that's interesting that you mentioned flexibility and having very detailed rules. So, my understanding of, like, uh, Very lame understanding of law is like, in the U.

S., you have these very deep, detailed rules, and in, uh, Europe, you tend to have, like, broad rules that sort of covers everything but, um, to a lesser extent. It Is there like, is one better than the other, uh, when it comes to guardrails? Do you want, um, deepness or broadness? You 

Viktor Mayer-Schönberger: know, the truth is much more sobering.

Um, the truth is, yes, in the United States we have case law, and that is quite detailed, but we also have statutory law, uh, stat, uh, statutes, uh, rules that have been enacted, lots and lots of them. And by the way, in Europe we have pretty much the same thing. Um, so when you look at the, the number of laws that have been passed, either, uh, by legislatures in the United States or by legislatures in Europe, whether it's the UK or the European Union, you pick, um, it has grown almost, um, by leaps and bounds over the last, um, two decades or so, uh, legislators, uh, have gotten incredibly productive in enacting laws, uh, which of course limits the ability to be flexible, uh, and to stick to flexible rules, uh, and, uh, it, it, it, it may reduce ambiguity, an interpretative, um, flexibility, but it also creates, uh, the potentiality for statutes that are outdated and no longer are in sync with the reality on the ground.

Richie Cotton: Okay. Uh, that's very interesting. Um, and I had no idea that there were just dramatically more laws than there were a few decades ago. Um, all right. So, uh, The example you gave early on was about, um, speed limits in cars and how there's a sort of law there that says you can't go faster than this, but individual drivers have the ability to speed if they want to.

Um, so it seems like a good guardrail is one that people are actually going to follow at least most of the time. So, um, How do you design your guardrails to make sure that they are actually followed? Mostly 

Viktor Mayer-Schönberger: by keeping your customers in mind. Much like a good, uh, user interface designer, keep your customers in mind, uh, and, and, and their goals.

What do they want to achieve? If a guardrail helps people ultimately to achieve their goals, and if you can communicate that, to it. Um, if, if, if you tell people that, um, at, at the speed limit of 65, uh, rather than 85, um, uh, they, they, they still reach their, um, destination, uh, within just a couple of minutes later than they would otherwise, uh, but with 20 percent less time.

fatal crashes and accidents, uh, then that might be the sensible thing for a lot of people to do. Um, and you may not ever reach 100 percent compliance, but you don't need to. Uh, if 90 percent or 95 percent of the people, uh, follow the rules, that's pretty darn good. Uh, and, uh, that, uh, reduces, for example, in, in, in, in, uh, car traffic reduces, uh, accident rates dramatically.

Um, so what you are trying to do is basically look at what people want to achieve, try to empower them and try to communicate how you empowered them, uh, and hope that you did it 

Richie Cotton: well. Okay. So really it's about like, uh, being sure that people are aware of like what the guardrail is. So I guess in that case, yeah, it's the, it's the signposts that tell you what the speed limit is and also educating people as to why the guardrail exists.

So if they know why it's there, they're gonna want to You know, 

Viktor Mayer-Schönberger: I, I'm a rock climber. A lot of times when I climb, uh, there's fixed ropes, uh, and you clip into the fixed rope and then you climb up and, and, and, and so you're always secure because you're, you're clipped into, uh, that, that, that rope. Um, but there are some times there are people who just come from behind and want to overtake.

They can unclip and they can overtake you and then continue on. Uh, but it's, it's their own decision, uh, and they have to live with the consequences. And I think that's the important point. We don't want to nanny people. We don't want to disempower them. We don't want to take the responsibility, but also their freedom and liberty away from them, um, through the guardrail, through well designed guardrails.

Uh, we want to empower them to make better decisions, uh, without actually caging them. I 

Richie Cotton: was getting mad and terrified about people unclipping themselves from a rock face and and trying to overtake. That does sound incredibly dangerous. Uh, so, but yeah, I can see how, uh, you'd want, um, it to be their own decision to do that rather than, uh, the, uh, standard procedure.

All right. So, um, I'd like to talk a bit about decision making. Uh, so, uh, you mentioned like one of the goals around guardrails is to help people make better decisions. Could you just talk me through how this works? 

Viktor Mayer-Schönberger: For good decision making. We basically need two elements. One, we need to have the right information available, um, in order to make our decisions.

And then, two, we actually have to somehow weigh and calculate and balance all the information that we have, and factoring our preferences in order to come to a decisional option. And that's the hard part. But, uh, you know, if you think getting all the information in place is the hard part, just think about the fact that we have learned over the last 40 years or so that we have numerous cognitive biases, uh, that, uh, that, that, that, that shape our decision making and oftentimes hinder us in, in making the right rational decision, whether it's, um, uh, confirmation bias or availability bias.

There's all kinds of, uh, biases that we are as humans susceptible to, uh, and there is no easy way to untrain those biases and to get rid of them. So cognitive psychologists have said is that rather than oftentimes trying to find the best of, or the better between two mediocre decision options. It is better to broaden your decision space so that you have more options available, which ultimately may lead to, to better outcomes.

We are not particularly good at choosing with our biases between two options. But we are, as humans, pretty good in coming up with lots and lots of decision options if we try. Uh, we, we can kind of, um, dream up new decision options relatively well. Uh, and so a good, uh, decision making should therefore provide us with, uh, good information, but then also help us to broaden our decision space and then to navigate that decision space appropriately.

Richie Cotton: Okay, yeah, um, that certainly seems to be a good idea to help, um, compensate for all these sort of human biases we have. Like, uh, yeah, the tricky decision is do you want the salad or the french fries? It's like, uh, you're going to be biased towards one. Um, okay, uh, so, uh, it does seem like there's often a lot of uncertainty in decision making.

And you mentioned that sometimes you don't know quite what to do. what your goal is or what your users want. Uh, can guardrails be used to help, uh, with this level of uncertainty? 

Viktor Mayer-Schönberger: Uh, yes, they can. Now, we need to understand that, uh, there is no certainty there. Uncertainty means that, uh, even if you have guardrails that work 80 percent of the time, it means 20 percent of the time they're off.

But, but they were sort of a good starting point, if you want. Atul Gawande, an author and medical doctor in the United States, had a wonderful book out a couple of years ago called The List, and it was a a a list of standard procedures in the emergency rooms, uh, uh, for in U. S. hospitals. And what he showed was that if people stick, if doctors stick to the list of, you know, the, the seven or 10 things that need to be done in this order, when a new patient comes in, um, it significantly improves the chances for a positive outcome for most patients.

Not all patients because you have those odd cases that unfortunately with the the list doesn't cover And it may make the situation actually worse. That's why you need the empowered Physician who can then kind of discard the list and say in this case. I need to do something else But but but overall Um, Guardrails help us, these standard operating procedures, these lists, help us to, uh, to cover, uh, the, the most obvious cases.

And that's why 

Richie Cotton: they're useful. Okay, absolutely. I'm a big fan of checklists. I'm quite often, uh, forgetting things, particularly admin tasks, where I can't hold them in my brain. So having those checklists has helped me, uh, go through and make sure that I'm doing absolutely everything I should be doing. Um, all right.

So, um, You mentioned one of the big principles of this is about making sure that you have enough flexibility around the guardrail so that people can be empowered. So what's your strategy for deciding like, is this guardrail too strict or too lenient? How do you get that balance right? 

Viktor Mayer-Schönberger: These rules of thumb that I was alluding to, whether it's the rule of thumb that a guardrail should enable learning or whether it's the rule of thumb that the guardrail should facilitate the empowerment of individual decision makers, those are more like design principles.

They are not Rules themselves, but they're, they're, they're kind of principles that should guide our designing of, uh, of, of the guardrails. And so, uh, the designing, uh, process is not a scientific one. It, it, it is artistic in a way. Uh, and, uh, it requires a lot of trial and error, uh, which by the way is. very difficult for regulators, as I mentioned before, because regulators usually want to regulate and then they want to forget about the regulation for the next 10 years, at least.

Uh, and that's not what, um, flexible good guardrails are. Uh, they need to be constantly checked and, uh, fine tuned, uh, and adapted to changing environments. Um, and, and so, um, what we require is not just a good, uh, guardrail, a well designed guardrail. But what we require is an institutional structure around it that can kind of revisit it constantly or frequently and adjust it.

And to set that up is even harder than to come up with a flexible guardrail. A flexible guardrail doesn't help you if you have no, um, institutional structure around it that can then step in and increase flexibility or sort of rejig the guardrail, uh, one way or the other. Um, so you always need to think about the sort of, uh, institutional environment, the context in which a guardrail, uh, exists, um, and, and, and, and how to make sure that the, that, that you stay on, on top of, um, keeping it, uh, uh, flexible and adaptable.

Richie Cotton: Okay. So, um, it's not just the guardrail itself. It's, it's the all sort of the institutional framework to say, is this actually going to work or not? All right. Um, this is getting slightly abstract, so maybe we need a concrete example. Can, can you just talk me through some examples of guardrails in business that are used for helping you make decisions?

Viktor Mayer-Schönberger: Absolutely. Uh, think about environmental laws, for example. You can try and limit the amount of carbon dioxide that is emitted by factories. And you just have a lot. It says, uh, uh, no factory can, uh, emit more than X, uh, number of tons of carbon dioxide a year. Uh, that's a very inflexible, very fixed rule, but it's pretty Very clear, doesn't have any room for interpretation, or very little room for interpretation.

Uh, think, uh, about a very alternative concept where you say, Look, um, every, uh, uh, every company, uh, every factory that's emitting, uh, carbon dioxide, uh, gets a, uh, emission certificate. And if you don't, uh, uh, emit as much carbon dioxide as you have the certificate for, you can trade that certificate on an emissions market, an emissions certificate market.

And every year we cut 10 percent off the total amount of certificates that are available. Um, so, uh, what we do is we sort of set the goal, but we do not define, predefine the pathways that individual companies set to reach that goal. Um, and that's a pretty flexible guardrail in a way, um, because it leaves open, uh, a lot of different ways to achieve the, uh, the envisioned goal.

Uh, and it creates incentives for innovation. It creates incentives for companies to even go beyond the, uh, What the, the, the current limit is, because that gives them an opportunity to trade their emission, uh, certificates and make extra money of it. Uh, all these kinds of things. This is a, uh, a pretty flexible framework and a pretty flexible setup.

It uses the market, uh, as an, uh, incentive structure. Uh, it uses, uh, innovation and human ingenuity, uh, that's built into it. Um, uh, uh, and, uh, and it is. Uh, capable of sort of lowering, uh, the limit of carbon dioxide emissions over time, while the hard rule just sets a limit and sticks to that limit. 

Richie Cotton: Okay, that's interesting because it seems like the focus there is less about having, um, rules in place to enforce stuff, uh, in the segment laws are, and this is more about the focus is on this is the goal, and these are the incentives to make you adhere to this goal.

Viktor Mayer-Schönberger: Yeah, and it helps individual decision makers in the companies to make the right choices. because it creates incentives for them to make certain choices rather than others. 

Richie Cotton: Okay. Um, now you mentioned that, um, you don't always get these, uh, guardrails right the first time. So you need some process of, uh, well, having feedback to make sure you can improve the guardrail later.

Can you talk me through, First of all, how do you test whether or not your guardrail is actually working? 

Viktor Mayer-Schönberger: Absolutely. And that's one of the hardest parts, right? You have a guardrail, um, and the, uh, a guardrail is working if it helps people, uh, achieve their goals more effectively than before. Um, whatever their goals are, we are not gonna judge the goals.

Uh, here. This is just, uh, uh, uh, the, the, the, the, the guard rail is a, uh, a mechanism to, uh, achieve, uh, uh, an exogenous goal. Um, now, measuring that is actually not easy, uh, and, uh, and when you start measuring it just in terms of economic impact, uh, a sort of cost benefit analysis of some sort, then you are starting to measure something.

Um, but you're not capturing, of course, the full, uh, the comprehensive picture. You're just capturing what can be economically captured in data. Um, so it's a good first step, but it's not complete and comprehensive. Um, and what we need to do is to therefore, uh, develop a better measuring tools, uh, to, to measure the effectiveness of guardrails.

Um, we are. At the beginning of that, but we have made quite some headway going beyond, uh, simple cost benefit analysis, uh, capturing longer term externalities and these type of things, um, uh, but, but there is a long way to go. However, having said all of this, the truth of the matter is I, I've advised politicians and policymakers over the last 25 years.

And the truth is. The ugly truth is that they don't even do most of the time a simple cause benefit analysis. Even that would be better, uh, than what they are doing. It's, um, oftentimes shoot from the hip. Even small steps towards, um, getting, um, some assessment of effectiveness, uh, would go a long way to improve, uh, guardrail design.

Richie Cotton: Okay. Yeah. It does seem like if you, um, proposing a law, you should do some kind of thinking about one of the costs and one of the benefits. Absolutely. 

Viktor Mayer-Schönberger: I advised a German government and that was one of our recommendations and it was the one recommendation that immediately got binned. That's 

Richie Cotton: a, that's a little bit worrying.

Um, okay. So, uh, it seems like a lot of the key then to being able to test whether a guardrail is good or not, is, is the goal that the guardrail is for well defined, so I don't know, let's talk about that. Smart goals where they're like testable and all that kind of stuff. So, uh, if you've got a well defined goal, then hopefully, uh, you should be able to test what's, what's happening.

All right. Uh, so. I think we've mostly been talking about guardrails so far in the context of like helping humans, but now AI is sort of reaching the point where it can automate some human decisions. So can you talk me through like, um, first of all, like when should artificial intelligence replace human decision making and when should it complement it?

Viktor Mayer-Schönberger: That's the 100 million question at the end, isn't it? At least.

Or, or even more than that. I think, um, the, the answer is relatively straightforward. It may be surprising, but it's just relatively straightforward. If the decision making is very routine, um, and, and, and doesn't require a change context or anything like that, then there is no reason why the machine shouldn't make the decision.

If you enter an elevator and you press the button for the third floor, you let the machine make the decisions of closing the door and getting you to the third floor. Uh, that is a very efficient process and you don't need to question that. Nevermind artificial intelligence here. Um, so when, whenever there is, um, uh, a very straightforward, uh, efficient decision to be made, then that can be, uh, uh, automatized.

Um, but whenever it gets more complicated, whenever the context is changing, whenever goals are more muddy, uh, sometimes contested, or, um, battling with each other, uh, then I think it is perhaps better to keep the human involved, not just in the loop, but sort of in the loop. at the end pressing the button, but to be really involved in the decision making process, not because the human makes the better choice between two mediocre options, as I imagine, as I said, but perhaps because the human might be able to come up with a third or fourth or fifth options we haven't yet considered that is actually better than the options that are that are already on the table.

Uh, and so in that sense, I think there is also a good division of labor, if you want, between AI and, and the humans. Uh, AI is very efficient, uh, for alerting us and, and, and, and, uh, uh, focusing us, uh, on a solution that is already available. Uh, humans are, perhaps better positioned to come up with novel solutions.

Richie Cotton: Okay, um, I like that. So if you need some kind of creativity or there's some novelty, uh, in the, uh, situation, then probably humans are going to perform better than AI. 

Viktor Mayer-Schönberger: And it's, it's, it's oftentimes it's important to then keep humans in the loop, even for relatively routine decisions. Um, I'll, I'll give you another example from the aviation industry.

Um, some years ago, uh, Asian commercial airliners, uh, were always using autopilot to auto land their, their, uh, uh, airplanes, uh, when they came to the U S, um, because it was very smooth and everything. Uh, but then, uh, One day, the Autoland function at San Francisco airport was down, and a very large Asian airliner had to land.

Uh, uh, manually, beautiful weather, everything, uh, but they crashed the aircraft and people died. Um, and, uh, that led, uh, airline companies all around the world to mandate that their pilots have to continue to land by hand so that they continue to train, experience, learn, uh, what they're doing and how to land the aircraft, uh, so that they know what they need to do when they have to do 

Richie Cotton: it.

That's again, quite a horrific tragedy there. Um, and I can certainly see how having that continuous training is going to keep people, you know, keep their, keep their focus. Brain sharpen, uh, make sure that the, the quality of the landing when they have to do it, uh, is getting better. I sort of, um, like that, the similar thing in the data world is like generative AI can write your code for you, but, and it's fine till you don't have access to it and then you need to write your own code.

Um, okay. Uh, so, um, if you are designing guardrails, uh, for AI, then what would those look like? Are they going to be different to guardrails for humans or is it? just the same sort of thing? 

Viktor Mayer-Schönberger: No, um, if you design guardrails, uh, in the context of AI or, uh, how AI should be used, um, it's important to keep in mind that same design principles that we mentioned before, for example, to empower individual decision making.

So rather than delegate away a, uh, a complex decision from the human because it's too complex for the human. Uh, that's not a good idea. Uh, we should keep it with the human and we should provide some, uh, uh, guidance, uh, on, on how options could be generated, uh, how additional, how the option space could be, uh, broadened.

Um, uh, we should also, uh, uh, understand that Uh, guardrails should be designed to enable human learning rather than AI learning. Um, it turns out that cultural learning is actually really powerful and has, uh, propelled our species from a relatively middling, uh, mammal species, uh, about 100, 000 years ago.

even 25, 000 years ago, uh, to, to something that can actually land on the moon. Uh, and, uh, there is no other mammal that even aspires to do that, at least to my knowledge. Um, and so, uh, in that sense, um, learning and cultural learning is, uh, hugely important and we can design AI systems to enhance and enable and facilitate, uh, human learning.

Uh, rather than take that chore away from the human and embed it in the AI. 

Richie Cotton: Okay, uh, that's kind of interesting and I like the idea of cultural knowledge and you can sort of build on what, uh, other people have done before you. Yeah, uh, there aren't many of the apes going to the moon, I suppose. Um, okay, uh, so, uh, I think this leads to the idea of like transparency of how decisions are made.

So certainly once you've got AI in there, uh, sometimes you end up this black box. So, um, when do you need to worry about how decisions are made rather than just the answer? It's a big question. 

Viktor Mayer-Schönberger: It's a big question because behind it lurks, uh, an even larger question about causality and the role of causality in, uh, Our understanding of the world.

Um, and of course we are, um, we are. living beings that make sense of the world through cause and effect, um, and, and, and that means, uh, we always want to peek in the black box, into the black box, um, even when it is not possible. Um, now what is really interesting is, um, The truth is a lot of times in our own lives, we cannot, we, um, out of people's minds or our own sometimes is a black box.

Uh, we may arrive at a decision, uh, not completely knowing why we came to that decision in the first place. And then we take that decision and then we post rationalize. And, uh, the goal of the post rationalization, of course, is to tell us, A, there is cause and effect, and B, uh, we knew it all and calculated it accordingly.

Um, and that gives us that warm and fuzzy feeling that we are in charge, uh, as, uh, post rationalization. I am not sure that actually we are, uh, so, um, clear eyed. Uh, in, in fact, I think we are quite often, um, uh, uh, deciding based on black boxes. So maybe, therefore, dare I say, but that's tentative, dare I say, it's less important to peek into the black box of AI than to understand it.

and make sure what role the AI box fulfills in the decision making process. So what is, what is the, what, what, what, what is the tool for rather than how does the tool work? 

Richie Cotton: That's interesting. Uh, now I can certainly see how you got that sort of, um, confirmation bias way. Something goes right. I always think, yeah, definitely, definitely I did that on purpose.

But, um, if it goes wrong, they'll be just sort of quietly forget about it. Um, Or blame others. Oh, blame others. Yes. That's another good trick. Um, okay. So you think, uh, you just need to define what the role of AI is within the decision making process. Okay. Uh, can you make that concrete? Have you got like a real example of how, uh, that might, how you might go about doing that?

Viktor Mayer-Schönberger: I think I, I, um, I mentioned that before, uh, already, uh, and that has to do, for example, with, um, uh, positioning the AI as something that does not take the decision power away from a human being, uh, away from a decision maker, uh, but rather facilitate decision Uh, the, uh, the ability of the decision maker to, for example, broaden the option space.

So, concrete, in concrete terms, um, if, uh, if, if I'm, uh, facing a decision, I shouldn't ask ChatGPT, uh, which of the two choices should I go for, but ask ChatGPT, tell me all possible options. That gives me a sort of a, a lay of the land, a, a a, a sense of the option space available, and then use that as a, uh, a, as a jumping board to come up with even further additional decisional options, uh, that perhaps others haven't considered.

That is, uh, taking the AI tool. Uh, and utilizing it, giving it the role of, um, facilitating and enabling my coming up with additional decision options, uh, rather than delegating my decision making to the AI box. 

Richie Cotton: I like the idea of an option space. It reminds me of, uh, when you have small children, you say, do you want to eat the carrot or the broccoli?

And they never think about the third option that they could ask for ice cream. Uh, so the, it ends up, uh, encouraging them in the right direction to eat vegetables. 

Viktor Mayer-Schönberger: And, and by the way, Richie, if, if, if I may add, um, one of the interesting things is that, uh, that children, uh, uh, uh, hone, uh, over the first couple of years from about year one to year four or five, hone those what if skills, uh, their, their skills of sort of thinking in larger option spaces, what if I don't choose broccoli or carrot and go for something else, eventually by the, uh, by the age of three or so they will be there.

Richie Cotton: Okay, uh, they, uh, they, uh, they learned the scam very quickly. All right. So, um, just going back to the relationship between AI and humans and decision making. Um, if, when you have these systems that are sort of partially automated and you have, um, some, um, model or AI saying, okay, the answer is no. It's very easy for a computer to, uh, for a human to say, oh, well, computer says no.

And they don't do any thinking. So how does using AI change the accountability in decision making? 

Viktor Mayer-Schönberger: Yeah, exactly. That's a, um, great question. In a way, I alluded to it when I gave that example of the, uh, of the, uh, pilots that, uh, did not know how to land the aircraft anymore because they were so reliant on their technology.

It wasn't A. I., but it was, uh, a similar technology that kind of took their decision power away from them, um, that they forgot how to do it in the first place, that they had unlearned, uh, that particular skill in a way. Uh, and decision making, making decision is a, uh, powerful, but also a hard skill to learn, and it requires constant practice.

We are lazy people. very lazy, uh, beings. Um, and so we don't like to make decisions. Uh, we, it's easiest if you can delegate decisions, uh, and don't have to make them particularly hard ones. And that's why we have to practice them. Uh, and so, uh, what we need to design, um, uh, AI systems for, uh, and in general, uh, decision assistance systems for, um, is to, um, force us to continue to practice our decision making, um, and to help us get better at it, um, to, to, to learn, to experiment, to help us experiment to an extent, um, but, but, but not to nanny us or to take the decision making power away.

Uh, and, uh, and, and that's a very tall order, uh, because of course, There, there is a huge temptation to say, we're going to use a machine because the machine is going to reduce fatality rates, accidents, rates, whatever, um, bad decision rates, um, significantly. Um, but the problem is that, uh, if we use the machine to make decisions, uh, um, uh, and we use generative AI that's trained on training data of actual decisions that generates new decision data.

That then the AI can be trained with again. If we do that over and over and over again, the option space is narrowing because the AI will always make the same decisions. So the training data will always confirm an ever narrowing band of decisional options. Um, that, only works if the reality doesn't change.

If none of our goals, none of our preferences, none of the context ever changes. If it does, we require diversity in our decision space, which the machine cannot provide us easily. That's why humans are better have a leg up there. We kill each other once in a while making bad decisions, but overall experimentation has furthered the species.

Richie Cotton: Ah, this is like very close to the idea of, um, model drift in terms of, uh, machine learning where it's like the, the reality of, uh, what you need to make decision about is changed from what the model is predicting and the quality of the model decreases over time. Okay. Interesting. All right. So in business, suppose you're a manager, you're now convinced that guardrails are a really good idea.

What do you do in order to start, uh, increasing the number of guardrails or implementing them in your own team or company? Cool. 

Viktor Mayer-Schönberger: Uh, start an inclusive process, um, and, and bring the, the, the various people and stakeholders in because you're at. Like you're designing a user interface, you are trying to bring the stakeholders in and trying to get them to accept some guardrails.

And so what they need to have is a say in how they're designed and what their aim to do. To the extent that you do that, you have a higher chance of compliance. Um, you have a higher chance of acceptance. Um, and, uh, you may also end up with higher quality guardrails, uh, at the end of the day. Um, so that's very important.

The process of. generating the guardrails, uh, is, is, is an important one. Uh, it takes time, it takes efforts, it's costly, um, but, uh, at the end of the day, you come up with something that's quite flexible in the book. Uh, we, uh, conclude the book in the final chapter, uh, with, uh, the rule of Saint Benedict, uh, uh, uh, uh, uh, uh, Monks, uh, and the Order of Saint Benedict, uh, these rules are a thousand years old, uh, and they were designed to be so flexible, um, to start with that they're still in place today, um, because they have had flexibility built in and they had a lot of buy in as well, uh, over the years and generations of monks that were involved in it.

Fine tuning it. Uh, that's a, um, uh, an example for a, a good buy-in, um, uh, if, if you want the design of the creative common licenses, uh, or some of the sort of Wikipedia set up, uh, or also good buy-in, um, designs, uh, the. Volta River resource example that I gave, uh, earlier, uh, as another one, but keep in mind in all of these cases, uh, this wasn't the cheap, uh, overnight process.

It took real effort, uh, to get all the people involved at the table, uh, to get them to, uh, discuss and clarify the goals, the preferences, to make sure that the mechanisms, the means, uh, built into the guardrails are robust enough and are, are effective enough to, uh, to further the goal in mind. Uh, all of that, uh, doesn't come easy, uh, or doesn't come cheap.

Richie Cotton: I like this idea of just, um, speaking to all the stakeholders, like figuring out who's going to care about this and getting their opinion on what they actually want. I think that's going to help, like, define the goal more clearly, uh, and sort of probably, um, yeah, you'll find out if they give me any problems up front before you put this card right in place.

Um, since that sounds like it might be a fairly sort of time consuming process and, like, quite difficult to, um, set initially. Are there any examples of, like, really simple guardrails that make, like, a good first guardrail project? 

Viktor Mayer-Schönberger: Um, ha ha. Yeah, uh, I wish, uh, I, I think there is actually a very pragmatic way to, to, to, to start.

And that is to come up with a guardrail, but give it a sunset clause. Um, and say, okay, we'll, we'll, we'll use that guardrail for the next X amount of time. And then we'll, we'll take stock. Um, we experiment. Uh, I, I think that in itself is incredibly helpful, uh, because it also tells everyone around the table, uh, that this is now in place.

but it's not edged in stone. Um, it's something that we'll try out, uh, and we'll then, um, huddle and reconsider if it was the wrong choice. And it is perfectly okay if it was the wrong choice and we get rid of it. Um, that's just how we learn. Um, so, um, also have, uh, a bit of an arrow culture. in your organization is useful.

Uh, Spotify is famous for its error wall. Um, uh, I, I think that's a, a, a, a fabulous way where, you know, if, if a sub team, uh, at Spotify makes a mistake, especially if it was a costly one, they have to write it on a little piece of paper. piece of paper and put it on the mistake wall, uh, so that others can read it.

Uh, the idea is that you shouldn't make the, you should always make an error, a mistake. Uh, it's, uh, it just happens, but you shouldn't make it, not make it twice. Uh, and that's a way to learn from other people's mistakes. Uh, and, uh, yeah. That's not a way or a strategy to blame others, but a strategy to learn from others.

There we are again, cultural learning standing on the shoulders of giants. And if you have sunset clauses for guardrails, then what you're signaling is, this is not permanent, this is just temporary, and we're just starting off with this, but we understand that we may need something very different. 

Richie Cotton: Okay. Um, I like that idea of the, uh, the error wall.

I think like I probably need to, uh, need to get a bigger office just to feel all my mistakes on it. Uh, but yeah, uh, I can, I can see how you want to make sure that, um, it's very clear that this isn't a wall of shame. It's a, it's a learning tool. Cause yeah, there's a possibility that could get very misinterpreted.

Um, okay. So, uh, we're talking about like, uh, trying to get an easy first project on the flip side, are there any, um, Are there any guardrail projects that like to have a high impact on your business? Like where's, where's the, the big money at guardrails? 

Viktor Mayer-Schönberger: In Oregon, in most organizations, the big money in guardrails is, uh, when you look at the role of people, uh, and what the, the role of people is.

If the role of people is to make standardized decisions or relatively standard decisions, uh, then, um, that is not a particularly good utilization of human ingenuity, uh, of human fantasy, uh, of human dreaming. Um, and then you need to rejig the role that, that, or for, for, for, for this particular post or position.

Um, it seems to me, um, because, uh, otherwise, uh, you are, uh, utilizing humans for something that they are not particularly effective at. Um, and similarly, um, if you are using AI and AI tools. for things that are, uh, that, that require creativity or unbelievable dreaming, uh, of things that don't exist, uh, then you are, um, misusing, if not abusing, uh, uh, AI.

Uh, AI is a phenomenal tool to tell us what's already out there, what's known, uh, what kind of, uh, pattern are implicit that we may not have recognized as, uh, uh, important yet. Um, and, and so in a way, uh, sharpening and shaping the, the roles in organization that people have, uh, can unleash enormous amounts of productivity, uh, that I think, uh, uh, has not been tapped into that latent productivity.

Richie Cotton: Okay, I like that idea. If you've got, um, routine decisions, then that's something you want to automate away. If you've got something that requires creativity, you want to get all your best humans onto that and get them being creative and having interesting thoughts. Alright, um, and in terms of who gets involved in creating guardrails, you mentioned that, um, Um, you probably want to get all their stakeholders together to talk about stuff.

What does that mean in practice? Like who's going to be in charge of setting up a guardrail? And can you give me like a, have you got a concrete example of like, um, who might be the stakeholders for a particular guardrail? 

Viktor Mayer-Schönberger: Obviously that the, uh, depends on the project, but let me take the example of Creative Commons, for example.

Um, uh, and with Creative Commons, uh, you had, um, content providers as stakeholders, but then you also had, uh, those that, uh, were involved in the design of large platforms, uh, online platforms like Wikipedia, uh, that utilized a lot of content, uh, as, as stakeholders. Uh, then you had some of the large search engines like Google, for example, uh, being part of it because you want to be able to find Creative Commons content easily and straightforwardly.

That's. findability, uh, it's important. Uh, then you want to have some of the regulators and lawyers at the table so that you're not constructing something that is against the law, uh, or, uh, has other regulatory problems. Um, and, uh, and, and at the end you create a user community. Um, that is quite heterogeneous.

Um, uh, and, uh, when you talk about Creative Commons, uh, a lot of people say, oh yeah, that's that, that licensing scheme, uh, or they say, oh yeah, Creative Commons, uh, that's the, the kind of huge repository of content that can be easily used and reused, but the truth is that that's just the hardware. uh, the wetware around it, or the conferences, the online discussion groups, the forums, uh, the hundreds of thousands of hours of volunteers that people put in, uh, in order to, uh, evolve, uh, the Creative Commons licenses over 20 years, for example, uh, and And, uh, what, what was quite interesting in the Creative Commons process is that they were quite inclusive, welcoming NGO groups, but also welcoming industry stakeholders, uh, as long as the goal was clear, uh, and remained clear, uh, to create, uh, an easy and straightforward, uh, process.

Copyright licensing scheme that was machine readable, standardized, and enabled. The reuse, uh, uh, of the, the, the, the, the cost-free reuse of intellectual property. 

Richie Cotton: Okay. Um, I like that. Um, a. Not just, um, people getting together as being the sort of method of feedback, but you also got, you've got like discussion forums and reports and all that kind of stuff.

And there's a lot of sort of, uh, communication infrastructure going around that then as well. 

Viktor Mayer-Schönberger: Exactly. And, and, and we have a lot of the wonderful digital tools that enable us to do that. You know, you have. Uh, vikis and you have forums and then you have zoom and then there's lots of ways by which we can Engage with each other and then of course we can also engage with each other Uh face to face, uh in in actual conferences and workshops and symposia.

Richie Cotton: Okay, wonderful All right. So, uh, i'm hoping people have got some good ideas on uh, how to get going with uh, Creating guardrails then, uh, do you have any final advice for anyone who's interested in this just get started? You Get 

Viktor Mayer-Schönberger: going. Uh, the, the, the, the one, uh, mistake that you can make is not start, because then you will never have a good guardrail.

Um, only when you start, yeah, you'll fail, you'll fail half a dozen times or a dozen times, doesn't matter. Uh, get up, dust off, uh, rejig your guardrail, uh, and continue doing it. Uh, we all have started like that. Uh, every good guardrail framework started like that. Um, uh, but, uh, I think we don't have any other chance than to have good guardrails to improve our decision making so that we can better face the challenging and perhaps even existential, uh, uh, questions that we, we, we, we face as a species.

Richie Cotton: All right, super. Uh, I like that. Just get started. That's a good motivation. Excellent. All right. Uh, thank you for your time, Victor. Thank you very much, Richie. Wonderful to 

Viktor Mayer-Schönberger: talk to you.