The Real Reason Your Product Team Needs a Feedback Loop with Todd Olson, CEO at Pendo

Richie Cotton: Hi, Todd. Welcome to the show. 

Todd Olson: Great to be here. 

Richie Cotton: Yeah, great to chat with you. To begin with, I feel like I spend a lot of my day complaining about software, and I think it's, the same is true for a lot of people. So why is there still so much bad software around? 

Todd Olson: So I think there isn't enough attention to detail and design around how to build it, and very often people think that you can just build something and have people use it.

They don't take the same rigor in measuring it and improving it and trying to make it better. So to me, it's the discipline and lack of focus on iteration and design that to me is often why there's such bad software. 

Richie Cotton: Okay. Yeah, I love the idea that you've gotta focus on the details, make sure that you are being disciplined right throughout design, tracking stuff.

I think we're gonna get into a lot of this in more depth. But before we get to that, I'm curious, so because AI has made developing software a lot easier, you can do things a lot faster, is that gonna improve software quality or make it worse? 

Todd Olson: A nuanced answer to that. Do I think that in the right system it could improve software quality?

Sure. I've talked to companies over the last few weeks that are like, "We have not zero bugs, but almost no bugs. The second a bug is found, then, we have some agent pull it, and then we actually fix it." I think in certain mature areas and that, that could be correct. But if you're just, like, creating and generating pure AI slop without ri- lot of focus and rigor around how good is it, how is it being used, what's the architecture of it, no I think you, you've, you run the risk of actually having more bad software.

I think the other risk you have is you have a number of people who are not professional software creators creating software. And imagine if you had a large group of professional musicians or professional anything just creating something without some, experience, formal training, any training on what good looks like.

If you don't know what good looks like, it is hard to inspect. And that's the thing that a lot of folks worry about, is how do we help all these people, which are new builders using these amazing solutions to make sure that what they're building is of high quality, that it scales. Yeah, that users can actually use it successfully.

What are the patterns that we use? Like this is a discipline, this is a practice that's existed for decades now. There's a lot of learnings from those decades. How do we embody it all? And that-- Look, the coding agents have some of those learnings built in, but it's not perfect, and there's still a lot of honing to get a great software product out.

Richie Cotton: Absolutely, yeah. So you mentioned quite a few things there. So it's like the idea of having a good system, like a good framework for building stuff, having the right training, 'cause I guess, yeah if you're Miles Davis, you can improvise a great tune. Not everyone is the Miles Davis of of software, so yeah maybe we'll start with the systems. Do you wanna talk with your what does this good system for software development involve? What are the components? 

Todd Olson: You're providing a lot of context to the coding agent that you're starting to believe. So what does context mean? Context means good information about the problem domain, good information about the users.

Who are the users? Almost like a persona. Like what do they care about? What are their challenges? I think you need to think about what sort of data you supply to the coding agent so that they're intelligent. What are people doing today in your application? Where are they getting stuck today?

What are people using today versus not using? Investing a lot... Let's say you have one of the, your highest used pages and workflows in your product, and the coding agent decides just to change the entire thing because it gets some requirement that it has to make, and that's where it decides to put it.

That's pretty high risk, right? Versus if you told the coding agent, "Hey, this is our most heavily used thing. This is our most important workflow It probably on its own wouldn't decide to put things there, right? So that's to me, a good example of what context does in this problem domain, is that by giving it more information, you may make a different decision.

Just if you're a human being, if you had more data, you're gonna make a different decision, right? And I think, so the, the quality of the data being provided, that to me is like what you need to start thinking about first, and what data do you need to make good decisions on what to build next. 

Richie Cotton: Okay.

Yeah, I love that. Just think about the popularity of pages. If you have got an agent like that's gonna update your corporate homepage, that's a very different impact to always just updating some kind of minor page like a, a single product page or something. Okay. Since you mentioned data, one of, one of my favorite things and one of the favorite things of our audience, talk me through what kind of data is important then for providing context.

You gave the example of like popularity of things. Is there anything else? 

Todd Olson: Yeah, look I think we look at journey level data. Where are people flowing? Where are they going? What are they, like where's drop off in your application? So if you have an area where you're trying to get someone to, from, point A to point B, where do they get stuck?

Where do they drop off? I think journey level data is super interesting. Look, I think with the rise of all this agentic software, there's a whole new set of data that has emerged around how are people interacting with agents. What are they asking for? How is the agent responding? Is the user getting frustrated?

When they go to have agentic conversations, is it invoking certain MCT tooling or not? I think starting to unpack how human and agents communicate and interact is a whole new growing data set, which I think become really important for shaping the future. Because if you think about if you believe that a lot of our future is going to be interacting with agents- Getting a really rich data set on what that interaction means, how good it is, where there's opportunities for improvement.

Yeah, I think that's gonna be s- some-- a very interesting area. And I think about that versus there's really n- no other way to get that information, to know what a user wants an agent to do, right? Other by, other than instrumenting and, capturing full telemetry on all the conversations and how things are working.

So that, that's definitely a, another interesting area for me as well. 

Richie Cotton: Absolutely, yeah. So I love the idea of tracking like just what users are doing. So there's gonna be a lot of click data, things like that. I guess more broadly, for people who aren't familiar with with this product, how do you collect these data sets then?

How do you go about finding out what your customer wants from your product? 

Todd Olson: Yeah, look, we're a part of every application, so you embed Pendo within your application. Once we're embedded, we're actually capturing all the behavior. And that's everything, all the, traditional clicks, the journey-level information, so what, what order people are op- operating in.

But also we can use the same install to capture all the agentic conversations, so the human-to-agent coordination, and even the agent-to-agent coordination. So all of this is basically think of us as having something within your application that we're constantly monitoring everything that's happening inside that application.

We also complement that with the ability to capture data like qualitative data. When I think qualitative data, everything from basic surveys and polls to full on what features do you want, potentially even capturing things like voting on features. So I think what we wanna do is we wanna provide a more holistic view, es- essentially how users are interacting and what they want out of software, both quantitative and qualitative.

So yeah, that, that's essentially how we work. 

Richie Cotton: Okay, yeah. So it's a mixture of at least automatically collected data sets, and then maybe it's you go through like the whole user interviews and things like that. You can get more in-depth, you get these more qualitative things.

How do you get... If you've got agents as users now, 'cause I think there's a lot of agents using software, is that a different way of collecting data? Do you collect different things? Like, how do you know what an agent wants? 

Todd Olson: Yeah. Look, there are some ways to capture when agents are using software, and there's, there's different interactions.

Some of it's, agents are calling APIs, or they're calling command line tools or they're calling applications via, again, MCP tooling. So I think there's ways in which that we can instrument those calls to capture an- analytics too on, yeah, when are agents or how are agents interacting with software versus how are humans.

And I think at some, some level it's the system-level communication. People are calling it headless. If you think about headless systems, those are the ones where I think capturing how agents are interacting via APIs is gonna be a good part of the future. 'Cause at the end of the day, as, as a software provider I think as long as you're using our data, using our systems, I don't know if we care so much if it's headless or non-headless, as long as we are like, essentially the system of record for whatever task it, it is that we're solving.

Richie Cotton: Okay. Yeah, so this idea of headless software access, it seems to be increasingly popular where you don't need to bother actually using the software yourself. You just use your favorite LLM and get that to ask the software f- for you. Actually, do you wanna talk me through how that changes software, like, development and design then?

If you're designing it to be used in a headless way? 

Todd Olson: Yeah. I guess if you step back, the first thing you have to think about is what makes your software valuable, or what makes your application valuable? For many folks, it's probably not the UI. It's probably not how you interact with the software.

It's what your software's accomplishing, right? I'll give you us as this example. We're an analytics provider. Our job is to give you data on how to make your software smarter. And whether you log into our product and look at a dashboard to do it, or whether you use Claude to get the answer to your question or use it I don't care.

I care that my data's accurate. I care that you use it. I care that you're ultimately able to take it and take action upon it, 'cause that's actually where you're gonna get the most value out of our products, right? So if I think about that, a lot of it is how do we surface enough data that people can take action on it?

And for there I think we wanna make sure that in a headless environment, we're able to translate what the user's asking for, essentially the question, to good, accurate answers, to the point where, the LLM knows which tools to call for what purposes so there isn't confusion. Frankly, we, what we, last thing we want is the LLM to call the wrong tool, present the data incorrectly so that it leads to essentially a false conclusion or something to that, that degree.

So I think we're trying to focus on accuracy relative to the questions being asked. That's a, a big part of the design is- and then what do people wanna do? And what other actions do people wanna take via a headless environment? So a lot of the way we're thinking is anything you can do in our UI, you should be able to do headless.

Anything. But then some th- some activities are visual, where a visual interaction's actually a better one. I wanna see something, in which case, you know, what does that look like? That could be providing a screenshot. That could be something else that's, again, a different way of delivering the same value.

But yeah it's consumed in a different manner. 

Richie Cotton: Okay. Yeah. I love that, that you think about the important question is always what do users actually want, and are we actually providing them with the right answer? Is it accurate? So we talked a bit before about collecting bits of data to make sure that getting answers from to make sure that y- you are finding out what your user really want.

The second part is maybe harder, is about making sure that you're providing the right answer. This is a notorious problem with AI, is that sometimes it makes stuff up. What techniques have you got to make sure that you do give the right answer when you're developing software that way? 

Todd Olson: There's a lot of techniques, and, some of the basic techniques that people talk about are things like evals.

Evals essentially, think of it as like an automated, I'll call it testing suite, but think of it more as a grading suite, where AI is grading the answers of AI to see the success. So you can get scores on that and make a ton of improvements with evals. But evals alone do not make products better, because there are situations where the eval may score something well, but the actual user has a different experience.

So I think there's no replacement for really measuring how users interact with software. So if they're operating in a headless environment, like how many back and forth, are they happy with the answer? Understanding, again, when people are complaining around the collaboration with the agent. So we have this concept within within our products of rage prompts.

So when are people getting pissed off of interacting with an agent? Because it's c- it to me, do the users know it's wrong, but it keeps giving back the wrong answers? To me, that's, those are good areas to go deeper on. 

Richie Cotton: I love this idea of a rage prompt. Can you give me an example of this and what do you do about it?

Todd Olson: Yeah. Rage prompts are everything from, exclamation points to capital letters, to profane language to sharp language, repeatedly asking the same question without changing it. These are all areas where the users are clearly pissed off, and we try to measure that on a per use case basis to understand in what scenarios are people being driven to rage.

And that becomes a sort of signal rich area where you can then foc- you know, hyperfocus almost on improving those types of interactions. We know that when someone, let's- I'll use a fictitious example of a travel site. If we know that rebooking something, so changing existing booking leads to a lot of rage, maybe we invest a lot of R&D on smoothing over that process, really going deep in the problem so that we address it, 'cause that's what we're seeing, and that's causing people frustration, and we don't want people to be frustrated. 

Richie Cotton: Absolutely. I think that sounds like almost the worst possible user interaction, where they're getting so mad at your your agent and your chatbot that, yeah, they start swearing at it, they start cursing at it, or s- or something like that.

So that's absolutely what you don't want. In terms of figuring out what's going wrong, then do you have to do this manually? Is it a human sort of just going through curses, m- chats full of curses? Or is it can you do this automatically? 

Todd Olson: Yeah. We have the way to, we have a way to do it automatically, where we're capturing it all and labeling the conversations as such.

We can surface the, we can surface those insights right to you so that you can actually fix them. 

Richie Cotton: Okay. That sounds useful if you're not having to troll through all these horrendous messages. 

Todd Olson: No but actually, let's go back to the headless environment. What if you are interacting with us headless, and you ask us, "Give me the top three areas that's causing people massive amounts of rage," and then we return those things, and then you ask, "Let's diagnose the application to understand how we can actually address and fix these r- fix these issues."

That's actually a real-world scenario where someone leveraging this data in this context. Goes back one of the earlier questions of how do we make sure that whatever we build is of high quality? If you're saying use this data to then improve this application based on it... you're gonna build a better software application.

But again, it's all about the context you're provided. It's all about the data you have. It's all about using it to make sure you're actually iterating and improving things. 

Richie Cotton: Okay. So is there then a feedback loop that you have where all this kind of existing behavioral data from the application is then fed back into the agent that's then building or updating the application?

Can you build that kind of loop? 

Todd Olson: That's ultimately the vision, yeah. Yeah, absolutely is... and every org- Now, look, every organization runs a little bit differently, but yeah, I think that's the ultimate vision is how do you complete this feedback loop where you're getting actual usage data of, actual interactions in production- You're learning things from that, areas for improvement, and you're feeding it back into an agent to continuously improve.

Isn't that ultimately what we want? And then you then take it a step further. Okay that's just like addressing issues. Imagine we wanna run experiments on things that we think could make the product even better, but we're not quite sure. We don't wanna hurt the product, but we, if we can run experiments, we're again collecting real world data and then taking action upon that data that's pretty amazing.

Richie Cotton: Absolutely. Yeah, when you've got the positive feedback loop from bot to bot and things start magically improving themselves that, that's the dream, right? Okay, anyway we talked a bit about understanding user rage. Are there other kind of interesting insights you get from all this tracking?

Like what else can you spot that is exciting from a user point of view? 

Todd Olson: I think look, again, I, if I value step back, first you understand what the user wants to do with your software. What's it even asking? I think invariably when you build something, you normally in your mind have a set of use cases that you're building towards.

But I think one of the new Let's call it new opportunities with agentic software is that people could type anything into the box, anything at all, even if it's not a use case you built for. So right there, it's actually a pretty good proxy for, hey, this is important enough to users. Now you can measure how often people ask it.

You can measure what type of customers are asking it. That to me is a treasure trove of information. What people wanna do is gonna help you steer the future. 

Richie Cotton: All this feels like a very big change for what product managers do, 'cause traditionally, a lot of the job of being a product manager is trying to figure out what do users want.

Talk me through what's the new role for a product manager then? How does the job work? 

Todd Olson: In this instance, yeah I just said that users are telling you what they want. So in that sense your job's in some way it's a little bit easier. I think the opportunity for bus- the product managers to go up higher in the stack.

Think about the market opportunity. Think about the competitive landscape. Think about things like pricing and packaging. I can tell you the developer who's writing in cloud code's not thinking about that. So I think someone on the team needs to think about what's the marketability, excuse me, of this solution?

How unique is it? I think in a world where you could build anything, we all know the answer isn't to build everything. What are the things that we're going to be building that we think steer us in the right direction, that we can charge more money for potentially, that drive stickiness? I think there is a, a business level rigor that, that needs to be applied.

And then the other thing is invariably there's gonna be areas where you have new users new ideal customers that you're seeking to drive growth for your business who aren't represented in the dataset you currently have because they're net new. And for that, yes, you probably need a product manager to go out and get to know that persona really so that you can steer how you're actually building things.

So I think the role of product is super interesting, and a lot of the constraints we've had in product has been because we couldn't develop or we had You know, where things were really slow. I think back to a lot of our product decisions we make is like I would build this thing, but it's gonna take me six months versus this other thing."

That's a hard trade-off. Now when that six months becomes couple weeks, I think we start having different discussions. What is what can we charge for this thing? What markets does this open up? I th- I think it, again you have this opportunity to go higher in the stack.

Richie Cotton: Okay. That's fascinating that then product becomes slightly more business-focused. And so you're not worrying about the nitty-gritty of individual features. It's more about how can we sell this? How can we do that? And so I guess, are there knock-on effects then for what engineers and what design do?

'Cause normally product engineering and design, they're very interrelated. So if product managers do more business stuff, what do, what are the other parts of this department do? 

Todd Olson: Look, I think the designers are still figure- are still really focused on how the us-users interact and having an overall design system.

Honestly, designers are also responsibility for making sure the product represents the company's

Richie Cotton: brand identity. 

Todd Olson: Like in a world where Claude's generating a lot of stuff, do we want all of our software to look exactly the same? No. No one does. We pick things intentionally based on the brand and the identity of the companies, and we usually associate something with that brand.

And I think design will have a really key role in making sure whatever's built fits the ethos of the business, whatever that ethos is. So I think now the big difference is instead of just creating a prototype of it or they actually can build it or fix it themselves. So I think the exciting thing about designers, I think, I do think designers now I'm seeing a lot more get much closer to code.

A lot of times, developers didn't prioritize making design fixes because they were just so expensive to make. So I do think this is a huge opportunity for design and UX. We're actually making UX fixes internally that we would've never made before due to resource constraints. So now I think we're gonna see a golden age of design, I think, where people can make things look the way they want it 'cause they're gonna be less constrained around resources.

And the developers I think their roles are slightly changing. Obviously, the role's a lot less around writing code, and the role should be a lot more around architecture, scalability, brittleness like sure, quality can be measured certain ways, but you don't want some system that breaks a thousand ways to Sunday, right?

You want something that can, supports edge cases and things like that. You want, again, things that scale. There's gonna be thought thinking more about costs. They always thought about costs, but now they can really obsess on costs. And in a world where you can pick an open source model for certain use cases and pick a more robust model, I think this, this optimization of cost is gonna be a real issue as well.

Richie Cotton: All right. Okay. So we got product going more business focused. We've got design getting more technical and being closer to the code, and then the coders are thinking, more high level about system architecture and robustness and these kind of architectural decisions. Okay. All right.

And then I guess related to the, w- we started from talking about having more data. So do you think all three roles are gonna become more data-driven then in some sense? 

Todd Olson: Oh, 100%. 100%. And part of the beauty is developers are often relying on product teams or other folks to see data, specifically about user behavior.

Now they can just access themself. I think a lot of systems, ours included, is democratized data via MCP. If you're in coding agent and connected to our MCP server, you can just ask, "What are the top used features?" Or how often is this page being used?" Or, "Where do people drop off in this particular area?"

And it'll just tell you which Again, makes you, allows you to build a and design better new features for the future. 

Richie Cotton: Okay. Yeah, I love that. Just having a bit more input. A- again, they're just providing context in order to help me figure out what to do next. Okay. So suppose you're sold on this idea, how do you get to this new way of working?

Like, how do you go about transforming your your product development capabilities in order to take advantage of all this new tooling? 

Todd Olson: I think you just, this is a time where I do think people just need to be really curious and try a lot of things. Or maybe even said differently, organizations need to give their teams time, free them up to try a bunch of new things.

That's the area where I know it may seem counterintuitive because everyone's pushing everyone to move fast fast, which we create this culture where you don't have time to experiment with new things. But if you're not taking advantage of these tools and you're sitting here saying, "How do I get there?"

Step one is give your team some time to play around with things. Give them space. Trust me, you will see the gains later, but you need to do it first. Classic kind of slow down to speed up, right? And slowing, part of slowing down is just running experiments, trying things out. Here's an idea. Have multiple people at the company try different solutions so you figure out which one works best for your environment.

And, we're doing that, but I could rattle off 20 tools that I've found useful for us. Fine. Not gonna be the same 20 tools that work at every company, and I think each organization has a little bit of its own style of doing things and but you, yeah you need to encourage people to, to push the boundaries and try things.

And also the big thing that I've noticed here too on, on changing it is very often those of us who've been working in the industry for the past five, 10, maybe 15 years, we've got enough experience that there's certain go-tos that we always do. Like w- when we have this problem, we always go to that. We have another problem, we always go to this.

The landscape has changed. It's worth reevaluating your traditional go-tos to make sure there isn't some new system that is like a step function better or different than what you had before. And you have to be comfortable taking some risks. And that's kind of the era we're in right now, a much more risk-taking, call it entrepreneurial environment, but that's what we're seeing.

Richie Cotton: Yeah, and I do love the making sure that people are curious and they actually have time to try new things. And yeah often like the, the tools that you used a few years ago, it's like actually, maybe that's not the best tool anymore. But you mentioned risk-taking then. So there's a trade-off here between taking a risk and making sure that you don't mess up things too badly, so y- you're not having your bot break your company homepage or whatever.

Talk me through, like, how do you manage this risk control and making su- and innovating while not destroying the existing business? 

Todd Olson: Like any r- any risk you take, you have to score it on what's the criticality to the business, right? So risking something that could leak like sensitive customer data, yeah, no, don't do that risk.

That, that could be bad for everyone. You don't wanna, end up on the front page of The Wall Street Journal for leaking people's data, right? So I... Look I think that's when we should not probably take risks or be very careful. I, basic questions. Is it a one-way door or a two-way door?

And I'd be very careful with one-way doors. Oh, 

Richie Cotton: This is the Jeff Bezos- 

Todd Olson: Yeah ... 

Richie Cotton: concept, right? Do you wanna explain that one? 

Todd Olson: Yeah. I ask about it all the time. We all know the one-way doors. One-way door is you go out, locks behind you and you can't get back out. A two-way door is you walk in and you don't like what you can walk back out the other way, right?

So I think some decisions are one-way doors. Once you go there, can't put the genie back in the bottle, right? But some decisions are ones where you can undo it. So like I, I think a lot about the, can something be undone? And look, a lot of things can be undone, but what's the pain to undo it?

It's a slightly different way of saying it, and some things are really painful, and it's tough to take things away from people. Like really tough. 

Richie Cotton: It sounds like you've got a, a story you wanted to tell around this. 

Todd Olson: No, I think about... Look, there's a g- we all saw in the news, but it's public, so I don't want...

but, HubSpot, made changes to, their terms of service around some of their new products, and they got a lot of hell out of it. Now, after getting hell they undid it, so you could argue it was a two-way door. But the reputational pain and hit they took for doing that, yeah, not good for the business, right?

So that's like an area where they took risks, and that was a policy change. But yeah, I think there's a lot of like technology changes we have to be really thoughtful about, what the risk, real risks actually are. 

Richie Cotton: Yeah, absolutely. It's very easy when you're innovating lots that some of the bets aren't gonna pay off a- and you make mistakes and yeah, you can annoy customers in that way.

So it does sometimes require a bit of care and think about exactly how you are changing things before you make those big decisions. Okay. All right we were talking about AI transformation and how you go about changing all your approaches for the better way of working. So talk me through, do you have any tips for overcoming hesitancy around change?

Like quite often when you're changing processes, like people are like, "Oh, I don't wanna do things differently." How do you do change management that way? 

Todd Olson: I think, look, I think the key thing in change management is really explaining why they're changing. Just saying you have to change, "Hey, change," is different than saying, "This is why we need to change.

This is the benefits of changing afterwards." I think the other thing that I like to always remind people is that a lot of times these change, changes, excuse me, represent professional growth. I don't know about you, but I'm not the kind of person that like I get bored. I don't want to do one thing all the time.

I wanna grow and learn new things and try new things. Like that's why I went into a technology-oriented field, that it is changing. Talking to people around the professional growth they're gonna see is, and reminding them that's a, that's part of the why, that's part of learning things keeps your brain fresh and is ultimately good.

Ultimately good for anyone. And then I think again, giving people space to fail. When they first use a new tool for the very first time, they're probably not gonna be as effective as if they used their old tools. That's something that we've seen. Yeah, giving people space is the thing that I think management needs to do the most of.

It's like we're gonna ask you to do something brand spanking new But we're gonna give you space to fail with it a little bit. Try it, see what works and see what doesn't work. 

Richie Cotton: Absolutely. I've definitely experienced that. I l- so I love the idea of having space to fail, 'cause there's been tons of times where we went, "Okay, yeah, I'm gonna try and automate this thing I do," and it's taken me a day to do something that would normally take me an hour, and then it's it's not quite as good as if I'd done it myself.

And yeah you get things wrong to begin with. I like the idea of just giving your employees a little grace when they're changing what they're doing. Okay. Actually maybe that's a measurable thing. Have you got any success stories from companies who've had success with it?

Who do you think is w- doing product better now than before? 

Todd Olson: Products better than before. I kinda have a great story of, like how people are doing product necessarily better, but I think people are doing product definitely differently, and they're moving a lot faster.

Now look faster isn't always, always translate to outcomes, but I definitely think that's one of the big changes we're seeing. Look, y- you go on the internet, you'll see success stories of companies like, the companies that are well-regarded companies. I think Ramp's a great example where what, they seem to be moving very quickly and delivering great product in this era.

Stripe continues to do really well in building, new products for this era. I think one of the great case studies that we've seen in the past, this new a- agentic area is Fin, who's recently announced w- will be acquired by Salesforce. Great transformation story, but also a company who's really in tune with the market wants and needs and move really fast to capitalize it.

I think they're a really interesting story. So there's stories that are out there that people are starting to tell. So it's definitely possible. That's the thing that's exciting. I can... You can start to see what's possible in the future, and then it's just about how do we all get there.

Richie Cotton: Yeah, some great stories there. I would say yeah, we use Ramp at Data Camp. I hate doing my expenses slightly less now, so yeah, it's a g- good story. Yeah that's a real win for a lot of people. So yeah I love that. Beyond that is there a secret sauce then you think to what makes these things successful?

Todd Olson: I think the secret success is companies that are applying an, an engineering mentality to leveraging this technology to really advance and sol- solve their business. So let me explain and unpack what that means. Very often people go to use a new technology, they'll try it, it'll do some things well, do other things poorly, and it's very easy to sit there and say, "Pfft, this technology's not ready for it yet, man.

Like it was good at this, but it wasn't so good at that. We shouldn't use it here. Maybe we could use it a little bit here." The companies that are doing great, when they see something it's not good at, they attack that problem and figure out how to make it do it well. There's a in- pre-built assumption inside that it's possible to be good at that.

It's just out of the box not good, and it's our job to make it better. And I think that's, to me, an engineering mindset. Engineering mindset is like, "Hey, my goal is to use this amazing new technology to solve these problems in this way. When it doesn't do those things automatically, I'm gonna iterate on it.

I'm gonna improve it. I can maybe I provide more data to it. Maybe I do other things." But it's that that mentality that drives results. Now I've s- also seen in other companies with other e- even people, just a resistance to making these changes or a reluctance or like Honestly I've heard some people try to find really good examples of what AI's bad at and focus all their energy on what it's bad at.

It's terrible at this, how many bl- you know, posts have you seen it, the how many Rs in strawberry and it gets it wrong and yes, these models aren't great at every single general purpose problem, and you can definitely find things they're bad at.

Definitely, if you search. Doesn't mean you shouldn't use it, and doesn't mean you can't create systems that accommodate for these variances and then make sure that ul- ultimately still create a winning system. So I don't know, I... that to me is the attitude that the winning teams have around this, is "We're gonna find a way to make this thing work."

Richie Cotton: Yeah, I love that, 'cause there's definitely a gap sometimes between the level of hype around what AI can do out of the box and then what's possible with it, 'cause and yeah, sometimes you try a model and it gives you a garbage result. But the, the goal of the product teams is to make sure that you get from that stage of not working immediately to something that's actually adding value.

So yeah I love that difference and a lot of the differences around attitude. So are there any particular it sounds like are there some soft skills that you think are particularly impo- important around this then? 

Todd Olson: The soft skills around open-mindedness, experimentation, being optimistic.

Th- those are the big ones to me that I'm seeing, one, one of the funny things I've observed, and like this is my own opinion, so But I think this is a great time to be ADHD, even better time than usual, because like folks that are like, they can just try a bunch of things.

It's, and it's honestly easy to run multiple agents at the same time. So if you struggle from like extreme focus, like these systems are really good for you, right? 'Cause they can do a lot of work for you. And I do think that... Now, I don't know that like long term like I'm saying that like extreme, being extremely defocused is a positive thing that can give great results.

But I'm seeing like folks that like try a bunch of different things, this is like a heyday for them because they get to experiment a ton, and I think that's what we need. We still need a lot of experimentation to find what works and what doesn't work. 

Richie Cotton: That's really interesting. Oh, I saw a post recently, I think it might've been from Alex Karp.

It was like one of these tech CEOs who's saying one of the biggest things you can do to be successful in the world at the moment is being neurodivergent. And so yeah, maybe yeah, maybe ADHD is the way forward. Just, Yep. All right. I'll have to stop concentrating on things so much.

Just to wrap up, I always want more people to learn from. So whose work are you most excited about at the moment? 

Todd Olson: Oh, geez. Obviously the w- a lot of the work that's coming out of Anthropic and, Boris from Claude Code, I read a lot from that team. I like the Fin people.

I like what they're saying a lot. I think I'm learning a lot from a lot of the posts. I've already mentioned some of the other companies whose blogs I read and, I think now where there's no shortage of good content out there, obviously all the VCs, Sequoia's writing good papers.

Actually, on LinkedIn the other day I posted an article by someone from Foundation Capital, Jaya. I think she's doing great work. Love her posts. So there's just a bunch of great articles I'm reading. Everyone's like trying to figure this stuff out and yeah I think that's all very, very interesting.

I actually just saw a podcast recently from the founder of Clay. Clay's a super interesting company, right? So I think those folks are all the ones that, yeah, I'm learning a lot from right now. 

Richie Cotton: Yeah. A ton of great people and companies. There's a lot of exciting stuff going on.

Yeah, I think Jaya, is it, Gupta, is it? She does stuff on context graphs and management. Yeah very nice stuff. All right. Super. Thank you so much for your time, Todd. 

Todd Olson: My pleasure. Have a good one