The Next Generation of Business Intelligence with Colin Zima, CEO at Omni

Richie Cotton: Hey Colin, welcome to the show. Now I'll read, make you justify your existence a bit. So there are a lot of very mature BI tools at the moment. Why do we need another one?

Colin Zima: It's interesting because BI is one of these spaces where every company is also trying to build their own thing, and sort of everyone thinks they can do a little bit better. I mean, I think the funny story for me is I actually started a previous company and I really hated it the first time. I hated doing the startup thing because it's extremely difficult to sort of wake up and motivate yourself every day if you don't strongly believe in what you are doing.

Colin Zima: And almost the only reason that Omni exists is because I sort of experienced this frustration as a data person over, like the last 20 years, where I had touched a bunch of different tools and sort of felt the good and the bad of them, and I wanted them all to exist in one place. So the very simple story is I started in finance, grew up in Excel, sort of learned everything data from an Excel context, and ended up later in BI.

Colin Zima: Got exposed to looker data modeling like very heavy SQL concepts. And I was always frustrated that tools couldn't do both of those things at the same time. Like there's a time and a place to cut corners and go really fast and be a little messy. And there's also a time and a place to operate like a software engineer and have governance and centralization, and we just kind of never felt like it was getting executed.

Colin Zima: And, after looker went through an acquisition and sort of started slowing down, we said, let's go, let's go try to build it and that's kind of what on me has been.

Richie Cotton: Okay. It does very interesting. There's this sort of divergence of tools. I certainly yeah, there's a ton of very technical tools around. Like if you are hardcore data scientist or hardcore sort of data model, a lots of great tools that then spreadsheets are ubiquitous as a model for more casual analysis. Yeah, you're right that the two things haven't really come together in some way.

Richie Cotton: So yet maybe, two people just told me through, like, when do you want to do these sort of fast casual allowances? And when do you want something that's a bit more rigorous?

Colin Zima: I think that is actually sort of the core of the problem for a lot of this stuff, which is you don't always know. I mean, obviously there's foundational concepts that you want modeled as sort of the baseline so that everyone can do better. The example that I always cite is Salesforce is the API puts out deleted records. Maybe someone needs those apps.

Colin Zima: Very obscure moment, but effectively no one needs the deleted records. And so that is stuff that you want controlled in the ETL cycle, cleaned up for everyone. For all intents like you never want an end user to see those things. There's other stuff where you're starting an analysis and you realize you're at the corner of what you are doing, and to make a structural change would take you two hours and sort of sidetrack you, and you just want to sort of cut a corner and do a thing and sort of share the question, or share the answer to the question that the person ask you or that you have really quickly.

Colin Zima: And I think the, the, the challenge is often finding the place where you want to be governed and you want to create a sandbox is actually really difficult because the lines blur all over the place. Like even for stuff that I would consider sort of messier for analytics or product analytics is sort of a canonical example. It doesn't matter how many people are on the website or how many people did a thing.

Colin Zima: Not really like you're looking for directional analysis. Even there, you probably want a template, things that you come back to all the time. But you know, if you're if your data collection is Json, you want to be able to like reach into the Json blob, grab out a field and go do a thing. And so I think the really hard part is that when when the paradigm shift is a completely different set of tools or infrastructure, it gets really hard to actually have that subtlety you get locked into, you know, the structure of a looker and really hard core data modeling, analytics, or you kind of throw your hands up and you say, give me

Colin Zima: like a CSV and I'll go make it happen in Excel. And like same with like data science use case. I think my strong view is all of these things tend to blur together in different ways at different times. And you don't always know. And thus you need tooling that can actually be flexible and adapt and kind of even when you start in one area, you might end up in the other area.

Colin Zima: So like the example I always sort of thought was sometimes I'm writing one off SQL because I have a one off question, and then you realize that thing you did was important and needs to be sort of extended or shared by everyone. And as soon as you have to kick it out of your ad hoc tool into your governance tool, you sort of lost the thread.

Colin Zima: And so I, I think it's very hard to sort of say like this data is good, this data is messy, and it tends to be sort of like everyone and everything, but sort of blurry lines between all of them.

Richie Cotton: That's fascinating. Yeah. I think, you're right that a lot of data projects, you're not quite sure where it's going to end up and what consulting is like. Oh, this is just going to take me half an hour. Turns out this big project involves lots of different teams, and it was a lot more complicated than you think or imagine.

Richie Cotton: You go the opposite direction. Something like this can be very complex, is actually can build a simpler. You end up with the wrong tools. So. Okay, cool. So, let me go through what's your approach then. Like how do you have tooling that suits both casual analysis and something more rigorous?

Colin Zima: I mean, I think the most important sort of takeaway that we had at looker. So for people that weren't familiar with looker or I mean, we probably maybe are, there's a data model, and that data model creates little building blocks of SQL. So everything that you do on top of a data set, when you're writing SQL, you can think about the little bricks that you fit together to write a query, like everything in the Select statement, the relationships, the Where clause.

Colin Zima: These are all just little snippets of SQL that have used in different places. So if I ask for users by region and I've divided regions into North America and Europe, it doesn't matter whether I'm grouping by it or filtering by it. Region is a concept I want to use in many places, and those core concepts drive all of the centralized BI tools.

Colin Zima: There's a data model. You template out these things. You create this sort of rigid structure and effectively put pivot tables on top of them. The sort of insight that we had sort of leaving looker was that all of the things that users are doing in UI's, in even spreadsheets, or writing raw SQL can actually be templated into these pieces.

Colin Zima: If I write a giant blob of SQL, we might not be perfect at it, but we can break it into its component pieces and actually understand what it's made of. And in many use cases, you don't want to go do that. You don't want to take your 200 lines of SQL and break it into thoughtful building blocks that can compose together in a variety of different ways.

Colin Zima: But essentially what our tool does is, as you are writing SQL or as you are doing Excel things, we understand the components that sit under them. So we are silently building data models. And thus when you decide that you want to take it from being sort of quick and messy to being more structured, we pre-prepared those pieces for you so that we can absorb them into a data model and sort of work with them like a mature BI tool.

Colin Zima: And that was sort of the concept is like, I think every major tool runs into these problems over time, like Tableau put prep underneath in looker, you can write SQL like ultimately every tool is trying to bend it to all of these different use cases. But the challenge is, if you're not thinking about that as you build out the tool, moving between these layers can become very challenging because your post-processing calculation layer that's like Excel doesn't talk the same way as your data modeling layer.

Colin Zima: But if you actually do the work up front to sort of think about how these things weave together, that progression can become simpler. And that was sort of the really core insight that we had about building the tool that enables you to write a calculation that is row one minus row two. And we understand that that's like a lag function in SQL and can kind of pull it back and do things with it.

Colin Zima: If you want to use it in another context, that is essentially the building block of Ami.

Richie Cotton: Okay. So I do like the idea of automatically doing the data modeling in the background. Now I'm probably going to offend some analytics engineers here, but I would say in general data modeling is not like the fun part of working with data. So yeah, having that, automated does seem like, some or even it's like partially automated decision lights can be useful for a lot of cases.

Richie Cotton: I'd love to know a bit more about how, or where Omni fits into a larger sort of data stack. Now, we had Tom Tongass, the venture capitalist on the show last year. He was talking about a postmodern data stack included on me, another as, telling modern tools like Mother Duck. So talk me through, well, guess why do we need a postmodern data stack?

Richie Cotton: And how does it relate to that? The earlier modern data stack?

Colin Zima: I would say the postmodern data stack is not an official term by any means. But I think what's interesting is that, so I was actually one of the first looker customers. That's actually how I landed at looker when I spent eight years there. And in 2010, the idea of running analytics in a data warehouse that is disconnected from the BI tools are very foreign concepts.

Colin Zima: There were columnar warehouses, but it was essentially Vertica and green plumbed, like on premise stuff. And over the last 15 years, and really, this was about ten years ago, I would say that the worlds were coalesced around a way of doing data analytics. And again, there's there's obviously a mix of all this stuff, but you need to collect your data together, and that is a combination of sort of sucking things out of SAS tools and grabbing it from databases.

Colin Zima: So ETL has become a concept that is important to everyone, but it's really cool at this point. It's you take it from a place and you dump it into a cloud data warehouse or by train. Probably the most popular version, but there's a ton of players there. You've got the warehouse now, and pretty much everyone has settled on some version of a cloud data warehouse.

Colin Zima: So redshift, snowflake, data bricks, mother duck, BigQuery, things that process data quickly and cheaply. I would say that the kind of newest player in the space is the transformation layer. And, DBT is probably the most popular version of this, but you've also got SQL Mesh and Coalesce and a few other sort of transformation as a service products that effectively turntables in your warehouse to other tables in your warehouse.

Colin Zima: To kind of grossly oversimplify them. And then you have a B ice will sit in on top of them. I would say those four layers have sort of become the core of what a B ice stack looks like right now. That is the modern data stack, the postmodern data stack, sort of. Whatever it is. Then you have a bunch of other tools on the side, like catalogs and observability and what I call it, the more niche, tools that sit in a place.

Colin Zima: But those four are the core. The interesting thing about being a BI tool, and I see this sort of coming from a looker background, is what's interesting when you're the last mile, is you sort of have to take responsibility for all the other tools in the stack, because if the dashboard is wrong, you are the one that is wrong.

Colin Zima: And so it's it's always interesting to sort of think about our place in sort of how we interact with those other tools, because again, going back 15 years, if if Tableau is extracting and ingesting the data in memory and presenting it, it actually owns the stack end to end. But now in some ways that has sort of mutated into this four part best of breed stack, where we need to understand how we interact with those layers better.

Colin Zima: So we have spent an enormous amount of time, for example, trying to be the best tool in the world alongside a DBT or SQL mesh, or call us, for example, because data schemas have become much more fluid in 2025 than they were in 2010, the idea that the table would change its shape over time, was, was sort of a problem for data people, you know, like that was called a migration.

Colin Zima: If you're if you're like a database engineer now, it is just sort of like a day to day operation. And that was a that was, for example, like a very large change for the BI tool that I would argue that even looker that is a relatively modern tool like 2012 was not fully prepared for. And I think when you look around the ecosystem as sort of data players now, especially in the BI layer, very few tools are built with sort of the expectation that schema is fluid.

Colin Zima: For example. And that is, I would argue like that is core now to the postmodern data stack, because this stuff is still frequently changing.

Richie Cotton: Okay. So the idea of fluidity is very interesting. So I think maybe one of the most common complaints of business intelligence is oh, the dashboards broken. Something upstream change. So talk me through like what's a new workflow for, making sure all your dashboards continually work? Like how does maintenance work now? Yeah, I.

Colin Zima: Mean, some of those niche players I think are attempting to play a part in it. So I'm sure all the observability tools would talk about, you know, data quality and making sure that the tracking is correct and things like that. I mean, honestly, I think the biggest part here is actually philosophically, how all these tools fit together.

Colin Zima: So now you need to make sure that Fivetran got the data there. You need to make sure that your data flake is up and running. You need to make sure that DBT did what it expected, and then the BI tools to attach to all those things from like the BI tools perspective. Really what it is, is just understanding the lineage of where things come from.

Colin Zima: Like we need to know that if we're dependent on column A and column A changed its name or moved its schema or something like that, that we can actually sort of touch and feel those things. So like going back to kind of what I was saying, the sort of ability for the data tool to adapt to change, I would argue, has become more important than almost sort of anything in the tool.

Colin Zima: Like very frequently we'll just have someone whose data moves from one schema to another, and it might have been planned in some sense, but, I think historically that might have been like one week to go fix everything. And in a kind of very mature embedded analytics deployment, you need to do those things in lockstep because your data is in production to your customers.

Colin Zima: And so I would say, like there's there's now these ideas of sort of software engineering or software lifecycle for data in a lot of places. So my dashboard and the data model and the data warehouse or DBT or something like that, you actually plan a change across all of those layers together, and you can go into a branch like we have literally get branching, or I can go make a change across all three of those layers at the same time and push them to production almost.

Colin Zima: That is the way that those types of things need to get handled. But I again, like on the flip side, there's also the really easy version of this answer, which is just like we have great tools that can do alerting across all these layers now, and some of it is just doing the legwork to actually understand who is using what and pay attention to it.

Colin Zima: So, a lot of it is really just the basics of, you know, setting up some monitoring on the content that people use and watching it.

Richie Cotton: Okay. So actually, that, idea of like, branching does seem, very interesting. So I presume that means you've got access to, like, a whole suite of, like, standard software development sort of testing tools. Then how do you go about testing like the, the pointing part of things, or is that the easy bit?

Colin Zima: I generally think that the developer, like the, the data team sort of gets to punt on that responsibility, because the sort of weird thing about a data team is in lots of contexts now, data analysts have sort of become product managers. We're producing things that look and feel like production assets, like we have reporting in our product. RPM lead and VP of product built it all, deploys it all and sort of manages it all.

Colin Zima: And it's literally a product that's in production. But at the same time, they're partnered with someone like us to essentially build out the software that powers all of that reporting. And so in some ways, you sort of yields the, the, the actual software development piece of it, of, you know, building charts, interaction, and things like that.

Colin Zima: And you can focus on the reliability of actually getting data into those objects and making sure that it looks nice and feels good. So I would say, like our customer is certainly more focused on making sure the data is right and good, and the UI development is sort of left to us as the partner to make sure that the buttons work and the sort of, you know, the application stays up.

Richie Cotton: Okay. So, really it's like it's kind of separate things like make sure the data is in the suitable form for the database. You can test it all programmatically. And then, yeah, you probably want someone on the, the bit beside to just check that the interface works. All right. Oh. So, it's gone back to the start of our conversation.

Richie Cotton: It talked about how there are some sort of very casual analyzes where it's like, you know, you're basically it's like sort of spreadsheets plus, and then there are some, much more sort of rigorous, analysis from a dashboard level. The different requirements on the BI software for, for casual analysis versus for more, rigorous things like imagine like a dashboard sort of powering like company cash flow, very different things, like, analyzing podcast listeners.

Colin Zima: I think that's exactly right. And I think, it's sort of obvious, but the builder is different and the consumer is different. I'd say the canonical internal example that most data teams are starting with is they're building a dashboard for their CEO. And the dashboard that you build for your CEO is very different than the dashboard that the data person has for themself to go monitor, you know, data quality.

Colin Zima: And so I think the challenge is that and this is, I would say, a broad challenge across BI in general. The dashboard just being sort of like the main entry point. There are many different types of builders and many different types of consumers, and those different builders and different consumers have very different expectations for the product.

Colin Zima: So to put it very tangibly, we need end users to be able to build dashboards quickly and simply. I need someone on the finance team or the support team to be able to build a four tile dashboard for themself to monitor their job. That doesn't need to be the prettiest thing in the whole world, but that they can do in ten minutes.

Colin Zima: And I also need to be able to build a dashboard for the CEO that can combine data from all over the business. That looks exactly the way that they expect. And that maybe could even get delivered out in external use cases that might be pixel perfect that the design team intervened on and sort of label layouts are particular or color palettes are particular.

Colin Zima: And I think you can sort of see that there's these trade offs now, because I need the simplest tool in the whole world, and I also need all of the complexity that you could possibly imagine. And that visual side, I would say, is even just one note of it. There's also sort of like data complexity, which is like, I might want to do it through a spreadsheet or really casual sort of querying interface, or I might want to do really complex SQL and, you know, even data science use cases, which frankly, we don't support right now.

Colin Zima: And I think the challenges that we need to keep all of these in balance. So I'll give you an example of sort of like a way that we have tried to tackle this. We have the most common type of query on a dashboard is what we call a KPI. It's a number, maybe with a change, maybe with a spark line.

Colin Zima: Again, no rocket science to it. There's a lot of numbers on the dashboard, and we've got a little configurator UI so that you can, you know, select a number and put a spark line on it. And those are just like click two buttons and I get a spark line in that number. So no config, very little control or less control, but like very easy to build.

Colin Zima: At the same time, we will let you lock unlock the HTML and CSS of that object and literally write web front end code to design it completely differently. And that's where a designer can come in and say, like, I need this three pixels to the left. And you know, I want to bring my own font pack. And I don't know, like, I want it to be mobile responsive in a different way.

Colin Zima: And I think that's the type of example that is really difficult to build in software, because you're trying to make the simple, simple and the complex sort of like unbelievably open. And the dashboard is like the centerpiece of that for the organization because you have these users that want to configure everything and, you know, build a web page, you know, actually act like a web developer.

Colin Zima: And then you have people that are just trying to pin for charts and get the tool out of the way. And so I think that is I would almost call that singular problem, like the core challenge of building BI is like, how do you reduce the number of buttons while also increasing the number of buttons? But that is sort of the the torture of building a tool for everyone.

Richie Cotton: Absolutely. So, yeah. Certainly. There are so many company dashboards where it is just it's just numbers and then time saves is like, how does this metric change like quarter on quarter? But then occasionally you want to do something really complicated is like, okay, I want a heat map that would kind of do that. Not sure.

Richie Cotton: Yeah. So I'm curious then, since we're talking about user interface, but there's been a lot of push for, adding chat like interfaces. So, generally to almost every product. Is that changing? Business intelligence interface as well?

Colin Zima: I think it is, because, it's interesting again, because I, I think this actually kind of oddly fell sort of into our vision accidentally in that what we are trying to build is sort of a product that can bend into many different use cases. So Excel and SQL and pivot tables, and it turns out that AI has just become sort of our natural language has become just another interface for how people want to do things.

Colin Zima: So, the examples I love to give our there's places where AI is a really friendly way of doing things like lookups are a great example. Like when you ask ChatGPT like make me a grid of tariffs by country, it can do that very quickly. And a tabular form is really, really nice if you ask it to book a plane flight.

Colin Zima: It's not probably the best interface because like you want a two and a from and you want filter dropdowns, you want to see the sort of neighborhood of controls that you have available. And so in some ways, yes, it is changing BI because people do want to be able to come ask natural language questions and sort of be on their way.

Colin Zima: But I think the idea that it could or should replace everything is also pretty tenuous, because there's a lot of places where UI actually makes it easier to do what you want. And I think, like if you think of the BI experience with 10,000 fields, all of which are named things that you don't understand, that's probably the example where I can compliment it really nicely.

Colin Zima: I think on the other side, again, like the plane flight example or like a really well curated document, it helps to see the five fields that you can choose from and the way they're structured, and like to have a calendar picker on them and have UI attached. And so I think that the way that most folks will interact with these things is actually very complementary in a lot of different ways.

Colin Zima: Like I'll give you another example, we've got this Excel layer, one of our engineers, literally of his own accord, attached our Excel layer to an owl. And we copied Excel's dialect. Exactly. We reinvented nothing. We literally, like, copied their syntax. And it's actually magical what an LLM can do to write Excel functions, because, like, I, I pride myself on my Excel skills, and I can go in and write like really elegant functions that can do things.

Colin Zima: But you can you can literally just say like writing if statement that that says whether someone's over six feet tall and it'll just go right, you like the the 30 characters you need in in one second. And so I think what is going to happen increasingly is just AI now becomes a component for the web in the way that dropdown filters or calendar pickers have been in the past.

Colin Zima: It's just UI that you can attach, but it needs to be thoughtfully attached. It's it's not like no one is just looking for text boxes everywhere to go to everything via natural language. It needs to be thoughtfully sort of dropped into use cases that it actually solves problems well with.

Richie Cotton: Okay. So, I enjoy a disaster story. So do you have any examples of why you think I have been forced into these things in a bad way?

Colin Zima: I mean, I think the canonical example tends to be like the customer support phone call where you're talking to the rep and you're trying to get through the phone tree because you know that you have a special use case, and it just keeps offering you documentation over and over again. I think those tend to actually be the hardest use cases, like I think in by there, there tend to be a lot of different use cases that require sort of train of thought.

Colin Zima: And over time I might solve these, like it might be able to sort of do number one and then do number two and then do number three. But if you try to do them all in one shot, like, you know, the find me my best performing users and do X, y, z with them and then compare them to the population.

Colin Zima: I think one challenge of natural language, at least as it exists today, is it'll often just try to do it all in one shot. And if it misses any of the component pieces, then you're sort of back from square one and you need to start from scratch. And so almost like decomposing question and answering. And again, like I think a lot of these problems will get solved better and better over time.

Colin Zima: But I think it tends to revolve around being able to introspect the responses that you're getting, and also places where UI is just better to do things. So like again, like, I think filtering is, an example where I can be amazing, like filter for users in the US. I don't need to go find, you know, country and then go country equals us.

Colin Zima: I think that a calendar picker is probably a little bit easier, you know, in, in many circumstances than, you know, typing in, I, I don't know, I think we're going to see but, I think it's essentially provided sort of synonym based UI for kind of everything that anyone needs to do.

Richie Cotton: Okay. So yeah, that's interesting. The idea that if you need things to be right, then you want to have small problems. So like the equivalent, like a single formula or a single, like calendar picker widget, rather than saying, go build me this whole dashboard in one step, because then things are not going to be quite as you want them.

Colin Zima: Even as you say that that might accelerate you in a really significant way. I do think the key, though, is if if the end of the road is sort of the outcome of the AI and you're stuck if it's all or nothing, I think those tend to be the bad use cases. I think the ones where here's a dashboard, do you want to tune it up?

Colin Zima: I think those are the use cases that are most effective, where you can sort of stay in the loop and adjust the result sets. So like our CTO has been using cursor for the last like two weeks and sort of slowly started evangelizing it. But one of the things that he's sort of been paying attention to is that often you need to say like, no, you did this wrong.

Colin Zima: Try again. Like, no, you did this wrong. Try again. Or like, this is the style with which we do things like redo everything that you did, but do it in this style. And so it's like it's these building blocks and sort of human in the loop pieces that I think are most important for AI. And I think this is actually, one of the interesting divergences that we're seeing in analytics is some people are trying to have I go write SQL and some people are having AI work through a semantic layer.

Colin Zima: And I think what we're going to see is that the only way that it can be successful in data is through a semantic layer, because SQL at its sort of core is not decomposed enough that the user can go back and forth with it. Where if we look at it from a semantic perspective and we can actually understand, you know, anyone can understand the pieces, it gives us the sort of release valve to go to those things.

Colin Zima: So I'll give you an example. Today we've been working on sort of like multi pass querying. So one of our engineers was demoing this morning. It was like filter the last year for people that did something on Saturday and then, you know, show me this other query if that is 200 lines of SQL. And it got Saturday wrong.

Colin Zima: And you have to sort of inspect the three places in the SQL block, it says Saturday to go adjust it because like, I don't know, time zone was wrong or something like that. That's frankly going to be impossible for any user to do. Like it's going to be possible for technical people to if it builds something in the UI and the Saturday Sunday dropdown says Sunday instead of Saturday, and I can go click it and hit Saturday.

Colin Zima: And the whole chain of law works. That is sort of successful. I to me, and I think that sort of summarizes, I think the sort of push pull that we're going to see for AI with data because in many ways writing SQL is easier. The internet is full of SQL. And, you know, all these lines are very good at writing SQL.

Colin Zima: The problem is that SQL is sort of monolithic. And if you're writing these monolithic objects, it gets very, very difficult to go back and forth with them without sort of rebuilding the whole thing from scratch. And so it's going to play out. But I think the biggest thing that I believe is that this stuff will have to go through semantics and sort of tie to UI for end users to be successful with them.

Colin Zima: But we're going to see how it plays out.

Richie Cotton: Okay. Yeah. So certainly, that ties in quite closely with my, experience of writing SQL is like writing the first time is kind of fine. But then when you try to debug stuff, especially once you get like a really long query, that becomes incredibly difficult and then you're like, oh, I should probably write this is smaller common table expressions to begin with, but I didn't.

Richie Cotton: And then yeah, imagine.

Colin Zima: Where someone else's SQL now, you know, and now it's someone else's SQL every single time. But I, I think that's exactly the summary of it is like, you can touch the one thing and understand how that decomposes, but as soon as that gets really big, it gets really, really challenging.

Richie Cotton: Okay. So you mentioned the idea of, semantic layer. So just talk me through like just explain, what's the semantic layer like, when should you build this? What should go into it? Why do you need it?

Colin Zima: At its most trivial, a semantic layer is how tables join together and what columns mean, and what sort of, aggregations of columns mean. So it is really just here's my raw data in a database, and the semantic layer is when I ask a question about an aggregation or something about that raw data, how to explain it in business terms.

Colin Zima: So I got a users table. It's full of users. Maybe some of them are employees, maybe some of them are customers, and maybe some of them are vendors. If I search for user accounts, when we talk about that internally, if it means customers, the semantic layer is saying, take that user's table, trim off the people that are not customers, and the count of them is users.

Colin Zima: That is really all that a semantic layer is. Obviously there's like nuance on how you write it and what and it's sort of the workflow around all of those things, but it is just taking normal day to day business conversation and translating it into database speak. And, I think that sort of obviates why it's so important for a natural language to be able to have that translation layer, because as you start applying creativity there, you might lose some of the nuance on, you know, what a count of customers is.

Colin Zima: And you really need those building blocks to sort of effectively decompose business language into database language.

Richie Cotton: Okay. Yeah, I like that. Particularly in creativity involves that seems like an important because there's so many business metrics where there's a lot of creativity in how you define sort of like customer lifetime value, like how you attribute like where a customer came from. It's like the definitions change a lot because, yeah, every team wants to calculate them differently.

Richie Cotton: So having like a standard definition use modulus is a great idea. All right. Wonderful. So I'd like to talk a bit about, getting, the casual analysis crowd into business intelligence tools because it does give me like how many businesses are powered entirely by spreadsheets. So there's always been this, sort of, I guess, hub, like, step up in difficulty or perceived difficulty from going from, do something in the spreadsheet to I'm doing something in business intelligence tool.

Richie Cotton: How can you make it easy for everyone if they're not technical to access BI?

Colin Zima: It's a very good question. And honestly, I think we are going to struggle with this problem forever. Because in some sense there's a combination of problems. There's the semantics that we talked about, which is, you know, if you come and ask me for customer data, I can hand you a very, very clean version of that table that you understand really well and can go work with down the line.

Colin Zima: So in some sense, like that's not an Excel thing. That's not a bi thing. That is a human made you a really small data set that represents a very curated piece of data. And so I think a lot of what tools can do to improve that is sort of help people with that first step to trim down the degrees of freedom that enable them to understand something and work with it.

Colin Zima: The challenge becomes that the universe of things that people want to do is very broad, and a bi tool is highly generalized. And so the way I sort of think about that is, again, like there are divergences here where we could heavily curate the things that are available to people at its most pure. We could get everyone one table with ten columns, and that is all that they can work with.

Colin Zima: That is perfectly well understood. And so the good thing that we get there is a really simple piece of data, that probably most people could work with and a BI tool. The problem there is it can answer very few questions, and the challenges as you sort of make these trade offs slowly to the other side of the universe being like, here's the whole data warehouse, maybe with a semantic layer, but you can do whatever you want with it.

Colin Zima: Now, suddenly every question is available, but each one requires more context. It gets very difficult to sort of make the trade offs between those. And I think sort of like, again, I'm a big guy, but I think BI gets beat up for this concept and Excel is sort of, you know, viewed as the solve for this, but it's not really sort of a BI versus Excel thing.

Colin Zima: It's, small data set versus all the data in the whole world thing. And so I think the first thing that you can do is just offer a lot of curation. I think the second thing is that BI tools have repeatedly sort of forced users to learn the things that they don't want to learn. And I can give you an example, which is like in looker, we built a calculation language from scratch and it looked a lot like Excel.

Colin Zima: You know, the functions were similar. They did similar things, but ultimately it was a completely different language. And I think lots of companies, sort of re approach these problems and try to solve them a little bit better instead of absorbing the knowledge that users have and bringing it into the tool. So, paralleling to us, we literally copied Excel.

Colin Zima: Exactly. If you want to make a calculation you type in equals. You click on B1, you click on B2 divided by B2, and you press enter. And there are trade offs there. It, it, we have to bend to the will of Excel and everything that we do. But the user can now bring up familiarity, just sort of write that function and work with the tool.

Colin Zima: But the way that it works with and so like I do think natural language will open up more things for people, especially over like a well curated semantic layer. I do think, though, at the same time there is nuance in the world that is very hard for any user in any context. The example that every BI tool always gives, that I always kind of chuckle at is sort of like, how many customers do we have?

Colin Zima: You know, the semantic layer solves. How many customers do we have? Speaking to someone at looker, you know, who has worked with semantic layers, that's a more nuanced question than the customer realized, because the engineering team and the finance team and the sales team all have different definitions of what a customer is. And that's not a data problem.

Colin Zima: That is a human, you know, business problem, because finance only recognizes them when they have revenue, and sales recognizes them once they're using the products and a product recognizes it once they open up infrastructure for them. And, you know, like maybe a semantic layer can get clean enough to explain all that nuance to people. But the reality is like, if you and I are having a conversation about how many customers we have or something like that, I'm probably going to say customers over and over again, and they completely casual about the nuance between those things.

Colin Zima: And those tend to be the problem that business users get stuck on is not bi problems and they're not tool problems. They're life is hard and nuanced, and if you want to be perfectly right, you need to understand some of that nuance to do your job well. And so that is, you know, in some ways, me throwing my hands up and saying, I don't have to solve that problem.

Colin Zima: But like, I often don't think it's actually a tools problem. I think a sufficiently motivated user gets out what they need. And we can always make our tools simpler. But a lot of stuff does require smart people to collaborate. And, you know, nuance.

Richie Cotton: Yeah, definitely. I can certainly say you have, one person in a, in a meeting is talking about like the total number of customers, whether they've registered on your site and someone else is talking about, well, these people obviously given us some money. So therefore their customer. Yeah. There's a scope for some company misunderstandings that one.

Colin Zima: Thing that we've learned that's actually been very amusing about this is once you start attaching LMS to your environment, the language becomes even more important. So I'll give you an example. We have a bunch of different versions of where a user came from, and one of them is called channel, and one of them is called source. And as I speak to you, I actually can't tell you which one is which.

Colin Zima: I know one of them is like partnerships versus inbound, and one of them is like the type of campaign they came from. When you use an LLM and you use natural language, it is extremely pedantic about which one you wrote. If you write a channel, it's going to bring you channel. If you write source, it's going to bring you source.

Colin Zima: And in some ways it actually requires the user to have even more nuance because it's just going to kind of one shot the answer for you. And, you know, maybe in the future it'll say like, did you mean source or channel? Like the people confuse these things. But the reality is the LMS are even more pedantic about language and almost magnified the problem of sort of this nuance in some sense.

Colin Zima: And it's one of those things that we sort of just learn by trying to do this more and more, that we're actually trying to sort of figure out how to solve.

Richie Cotton: Interesting. Like I'm guessing the solution is just document your terms properly, like have some kind of glossary. But I don't know, is it is there something better than that?

Colin Zima: I think it probably is something like maybe it's it's sort of like understanding when things are very similar or sort of used similarly and saying, like, I did this, but this is also an option. It's sort of like the type of thing probably a user would do, which is like if you were sitting next to me, I would pull up one and you'd be like, no, that's not right.

Colin Zima: I'm looking for the one with this. And then it would go find the other one. And I think that almost is how it is going to have to work. But it's going to be interesting to sort of see how you have to reverse engineer, you know, loose semantics because human languages is imprecise in some sense. And now these LMS are sort of much more tuned to the specific differences in language that you use between different contexts.

Richie Cotton: Okay. So I think, something for everyone. Audience. Be careful. Okay. Sure. Yeah. Using the just the precise language, when you're when you're prompting these things. So I guess let's talk about the even harder case then. So when you're in a heavily regulated environment or there are consequences to your analytic senses being wrong, how can be AI tools help that?

Colin Zima: I mean, I think the answer there, it actually comes back to the trade off, which is that you need to reduce the degrees of freedom for your users. So, you know, in that example of like, we can go Wild West and let people do their thing or we can lock things up a little bit more. I think when you start talking about more regulated things, be that like HIPAA data and people's names or your financial metrics, you need to reduce the surface area of access and the degrees of freedom for your users.

Colin Zima: So I think the really simple answer is the tools need to be more rigid in those circumstances. And, you know, you need to apply row level access filters and column level masking and these much heavier weight BI concepts so that you can make sure that your revenue is stable over time. The other thing I will say in sort of the the nod to there's always a human problem underneath, I will note that our revenue has changed several times, and that has been the result of hygiene in our sales ops process.

Colin Zima: And so there's also an aspect of the data is as good as the data that you have. And often times that is actually source system problems. And so I think that I underappreciated as a data person how important the pipeline of data creation was to the quality of my data, because as a data person, I was just a pure consumer of that.

Colin Zima: And as a founder and leader of Omni, changing the way that our sales process work to make sure that, you know, sales force entries were better was actually the main determinant of the quality of the data in our reporting. And so I think, again, it's like the tools need to have the things that you need to be able to lock things up and make them curated and versioned and all that kind of stuff.

Colin Zima: And then you need business process to be able to make sure that there aren't 12 transformations in DBT in your data warehouse that introduce complexity into that reporting and that your data collection is good.

Richie Cotton: Okay. That certainly seems to make sense that you want data quality to be taken care of, like, right from the start, the process, like not just once. It has to be layer. Do you have a sense of who needs to be responsible for all this? Is this like a head of data governance or who needs to care about, making sure the data quality is good and people have the right access?

Colin Zima: I think I think it depends on the size of the business. You know, like, we're about 100 people. There's, we don't even have a data team. I, I'm probably the most core version of the data team right now. Which gets me on the product a lot. So, I mean, I think at, at smaller companies, it has to be the, the, the line of business that sort of owns the data set, you know, like sales needs to be responsible for sales numbers and then it's going to flow through a pipeline.

Colin Zima: And but ultimately, you know, that is theirs to own. I think as you get larger you get specialization. I would argue that I don't think that you want specialists for governance or data management. Like I think that the closer that you get to the practitioners in the tools owning their own things, the sort of more successful you are.

Colin Zima: Obviously like, I have not managed data at JPMorgan, and they probably have a different set of trade offs. And like maybe standards that they want to drive through the organization. But I think ultimately you want to get it as close to the practitioner as possible. Because they're the one living day to day in these objects. And, you know, we'll drive the most quality naturally.

Richie Cotton: Okay. So that makes sense is like, basically the bigger you go, probably the bigger the problems are with, with, scaling data quality. And so, yeah, maybe you need a dedicated team for that. But, at a small level in smaller companies, you probably want it to be close to the practitioner. All right. Okay. So I guess, a related idea is that there's been a bit of a trend from just doing sort of individual analysis to creating data products.

Richie Cotton: Do you want to talk me through, like, how do you go from analysis to, data products and where buy fits into this?

Colin Zima: The first thing I would say is like, I think in some ways data products are like an over marketed term. You know, it's I think it was probably created by vendors trying to do something new or sort of carve out a new niche in the market. I think going back to sort of our previous conversation, there are more polished objects where the recipient of them, and I'm talking data objects at this point where the recipients of them are much sort of stricter in their needs and their consumption, like data, data team as product manager.

Colin Zima: And there are other types of analyzes that are very ad hoc and very small. And I think ultimately what a data product is, is just a highly curated, very well planned data asset in comparison to more of these ad hoc things. And so the types of things that you care about are how does the user interact with it, you know, is there read right on it, what is the visual quality of that object?

Colin Zima: But I really think about these things as mostly products, not data products in any special sense. Like you're either delivering a product out to your end user or you're sort of doing data analysis. And so the kind of data product, sort of term is even a little bit strange to me. The example I love to cite here is if you've ever used Zillow or any sort of home buying website, those are essentially data products.

Colin Zima: You know, it is a map with a bunch of listings on it that has drill through control and effectively presents data. Now it's doing it in a way that no one would consider it a BI tool. The experience is highly defined for the use case that the user has. I am navigating a map. Maybe I need to lasso.

Colin Zima: Maybe I want to search by a bunch of different facets and then when I click on things, I get a very specific experience that is built around what I'm doing on that page. So I've got a giant picture section, I've got a little table of data, I've got another little table of like previous sales. And each of those objects is delivered in a visual way that is more aligned to what the user is doing with them, then the constraint of the tool.

Colin Zima: And I think that is essentially the description of the data product is an experience for the end user that is meant more to the end users needs than a dashboard. So I think there's always blurry lines here because like, what is the difference between Zillow and a dashboard? I think like an engineer might say nothing and like a bi person might say everything, but I really think that it's it is much more of a gradient of experience, of just how do you make your BI tool feel a little bit less like a dashboard and a little bit more like something that was built to solve a problem for the user?

Colin Zima: And I think we'll see more of that over time as tools get better, but also just more, more business tools have become consumer ized and consumers have higher expectations than business users. And so kind of everyone is pulled towards things that feel better. And so I think it doesn't sound good to say like dashboards that actually feel good, but I think that's most of the definition of what a data product is.

Richie Cotton: Okay. So I like that. I'm not sure whether people at Zillow go like, I especially yeah, your entire business is a fancy dashboard, but but, so it was interesting you said that consumers have higher expectations. Some business users. That's kind of a sad statement for business users. So how do you create these data, experiences that are great in business.

Colin Zima: So I think this is actually the funny thing because I think, speaking is like a builder of business products. I think the interesting problem is that often the consumer product is much more finely directed. So again, going back to the Zillow example, Zillow does it need to do open a data analysis on any type of data set across any database, in any dialect, across many different types of users?

Colin Zima: It needs to deliver a very curated, very specific experience for a single use case that is highly well-defined. So there's no corner cases. The null handling doesn't matter. It doesn't have to deal with time zones like it owns the data set, and it owns the presentation layer. And so I think when you look at a consumer app like a Netflix, for example, is doing one specific thing, it needs to help you pick movies, and it means that the UI can be highly tuned to a specific experience.

Colin Zima: If I look at Omni or a BI tool, I need to deliver that. Maybe, I maybe I do need to deliver a movie picking dashboard, but I also need to deliver something that looks at the database logs for exceptions. And I need to get Salesforce data, and I need to tell you about your marketing pipeline. And because our tool is so flexible, we have to make all sorts of different trade offs where we have to generalize well, but maybe sacrifice specific use cases.

Colin Zima: And I think again, this come back to this is the hard thing about building data products, is we need to build simple things for end users that are really fast to build, where the trade off is time to value. And we also need to be able to build Zillow in the product and have something that is pixel perfect and lays out, you know, in a very consumer ized way.

Colin Zima: And I think this is ultimately the trade off between B2B products and B2C products. There are very few B2C products that have the same surface area as an ERP or a CRM or a BI tool, and I think this is often why it's so challenging, because the trade offs are really different. I do think at the same time, that doesn't mean you have to deliver crappy, you know, dashboards.

Colin Zima: And so I think what we're trying to figure out is like, how do we reduce the surface area so that you can build something really polished and control the CSS of a dashboard and lay it out like a web app? But we also need to do that for when you shop with, you know, your database, you know, nulled out a column and we don't want to blow up the whole BI tool or something like that.

Colin Zima: So it's difficult to find the trade offs. But I think ultimately that is how we get better is like the we are able to care about these problems down. I think I do think these things are improving over time, but there are trade offs that you have to make.

Richie Cotton: Okay, so it sounds like, the ways that around these trades, at least for people who are building dashboards, the creators, that they should try and aim for something very targeted towards solving specific problems for their users. Is that your advice then, or what's the best way to create a great.

Colin Zima: I think that is exactly. I think sometimes people build dashboards, and that just means I'm going to throw eight queries on this web page that do a thing. And I think if you think about that from a product builder's point of view, they're thinking, what is my user trying to do and how do I get them to that outcome?

Colin Zima: And as we start building dashboards better, they will feel more like products. And again, like I think if you reframe it from I'm building a dashboard about our business to I'm making everyone in the business aware of what's going on. Very small reframing, but I think probably significantly changes how you build that object for your end user. Like what is the drill through path?

Colin Zima: How do they understand the objects that exist there, you know, as people use it? Are you evaluating whether this tile matters and whether it should get deleted? And another thing should get added. And again, like it's not rocket science is just thinking about it from the user's perspective and what they actually need.

Richie Cotton: Okay. Yeah. So it seems, very similar to the idea of things like data storytelling where you've really focused on what the people want, what they care about, what do they want to hear, and I want to target my language, in that way.

Colin Zima: Though, though, I would say, like, I think storytelling the challenge there is often, again, like this trade off thing, I think you can tell a really effective story on stack piece of data. It's really hard to be a good, dynamic storyteller. You know, like it's easy to make a PowerPoint presentation with four slides that are static and say, like this happened here, this happened here, this happened here, this happened here.

Colin Zima: As soon as those charts are alive, making that a great story becomes a little bit trickier. And I think this is where you need to think about. Like if you're trying to tell a story, you know, maybe you should take star pictures and tell the story. And that's a snapshot in time that is a little bit different than, you know, a dashboard that is how did the business do today and the types of things that it needs to be successful.

Colin Zima: So again, like, I think it all comes down to like, what are you actually trying to do? Make sure that you're doing that.

Richie Cotton: Okay. Now you've enticed me. Like, is there a way to, do this? Like, have you got a tip for how you can tell stories that are dynamic?

Colin Zima: I hate to just be like, I might solve this, but I. I think there are some interesting things that it's like, if I get a few slices of data, I can actually pass that into an algorithm that can say, like, here's what's been happening. I think the risk there is like, if you've ever done one of these for fantasy football or something like that, it often just sort of reads the values on the page and says the most trivial things in the whole world.

Colin Zima: So, I think it's, I think it's tricky, but like, again, I would really go back to you need to think about what you're doing. Like, are you trying to have something that every single day has a new insight for the business? Or is that even realistic? Like great example internally we used to meet every single Monday to do corporate metrics.

Colin Zima: We realized that, you know, every every three week period sounded pretty much the same because our business doesn't change that quickly. And so now we do a monthly meeting and the numbers actually move. And so if you're trying to build a data story on a dashboard that needs to that someone's going to open every single day, you probably need a very different set of values and sort of actions that a user can take versus a sort of monthly review.

Colin Zima: And so like, I think the time matters. I think the intent matters. I'm not sure that you can sort of have a genius insight on a dashboard dynamically, but I don't know, like maybe LMS will figure this out over time too.

Richie Cotton: Okay. Yeah. So, I like that idea that in theory at least, lm very good explaining things. So you can probably get an LMS to explain the output from a dashboard. Other updates as often as you like. But also, yeah, certainly if your business metrics are changing wildly from week to week, then you have a problem. Yeah.

Richie Cotton: So, I like that idea of, maybe, maybe monthly is fine in most cases. Or at least you hope it is all right. So just related to this, are there any, skills that you think are important for business intelligence analysts to learn?

Colin Zima: I mean, it's going to sound pretty obvious from the rest of my stuff, but, I mean, you have to understand how the business works. Like, you obviously need the foundational understanding of sort of data movement and all of those sorts of things. I think those are well trained and pretty obvious to people. I think the really big thing is you want to understand what is driving the business and how and why, and that's going to help all of your analysis be better.

Colin Zima: Like the way I would explain it is, build the sales dashboard like you're responsible for sales. Don't build it like someone asked you to build a sales dashboard, because when you're responsible for sales and you build that dashboard, you're going to ask the next question about every single thing that you have there, and you're going to find the thing that is missing there.

Colin Zima: And I think that is always the most important version of how people do things. Like, I always think of this example where I asked, the data team at looker, we were doing a big repricing and sort of the whole question was what should pricing be? And I got a dashboard to have 25 tiles on it. And my first question was like, great, what do you think pricing should be?

Colin Zima: And the person was like, I'm not really sure. And in some sense then that means that you haven't actually sort of taken the last step to figure out why we're doing the things. And so it's like it sounds really, really basic, but sort of just like, be the kid that asks why over and over again of their parents, as you do everything that you're doing and all of your stuff will get like 100% better.

Colin Zima: So, I really think that that is the most important thing, for data analysts.

Richie Cotton: That actually sounds very cool, because you get to role play as like head of sales, which is cool. I get to pretend to be an executive little power trip, but also you get to pretend to be a perpetual child who's asking stupid questions, which is also quite fun. So, it sounds, great both ways.

Colin Zima: That that's why it was my dream job. Like, those are the things that I like doing, pretending I'm in charge and, asking why.

Richie Cotton: Nice. Cool. All right, so, what are you most excited about in the world of bi?

Colin Zima: I think I'm most excited. Just sort of see how AI impacts all of the existing tools, like LMS, are just moving so quickly and have created so much uncertainty. I think it's interesting because even ten years ago, like the ability to parse and use natural language was the hard part. Now the natural language parsing is the easy part, and the hard part is back to being data.

Colin Zima: And so it's it's kind of amazing to see the new things that become available to us as builders because of the advancement of just intelligence and all this kind of stuff. And, it just changes every single week. So it's going to be really interesting to see how that keeps going.

Richie Cotton: Nice. Yeah. I mean, certainly, a lot of exciting stuff. Just continuing. So, brilliant. All right. So, I always need, follow recommendations. Whose work are you most excited about at the moment?

Colin Zima: I don't, like, I feel like I'm a nerd. I like reading other competitors docs to understand what they're building. So that's that's, that's my light reading. I really like the Logan Bartlett podcast. It's it's a business podcast. So he pretty much talks to business leaders, but I think he does a really, really nice job of sort of drawing out just interesting insights from its mostly CEO guests.

Colin Zima: So I would say that's probably the main, that's the main kind of business thing that I look at. I mean, the DBT round up is pretty good. The analytics engineering round up is a good one to follow. Ben Stansell on Substack writes good stuff as well. I need to think a little bit harder about how to answer that one next time.

Richie Cotton: All right. Okay. So, Logan Bartlett and, Ben Santos earlier, Ben Santos is of, data and regular sort of webinar I. So, yeah, he's, he's, got a lot of great content. All right. That was going to be my last question, but I need to follow up on something. So you said that you spent a lot of time reading competitors documents.

Richie Cotton: Now, are you actually reading documentation or are you just getting into them? To summarize it for you.

Colin Zima: I actually read the documentation. I still feel like I am trying to get better at thinking about ways to use LMS, but I think I've just been in by for so long. I like to really try to understand the nuance of how people are building stuff and the trade offs that they're making. So, you know, whenever anyone announces something new, instead of reading their announcement, I just go straight to the docs and try to understand what the feature is actually doing.

Colin Zima: So I can sort of back out what they were trying to do, why they were trying to do it, what kind of trade offs they made. So I guess, like, I don't know if you're building SAAS, go read your competitors docs. It's like, it's a goldmine for understanding how thing works.

Richie Cotton: I think you might be the first person who's been enthusiastic about reading documentation, but yeah, this is a great tip. I was like, yeah, if you compare this doc, see what they're doing, see how they're building stuff, probably can have some useful insight there. All right. Wonderful. Thank you so much for your time, Colin. 

Colin Zima: Thanks for having me