Not Only Vector Databases: Putting Databases at the Heart of AI, with Andi Gutmans, VP and GM of Databases at Google

ely. Like, just think about a chatbot, like, Hey, when's my flight today? Right? The flight is actually recorded somewhere in the database. So we've got to make sure that that foundation model ability to interpret your question ultimately gets translated a database lookup. It actually looks up your flight.

So that's where the connection comes in.

Richie Cotton: Yeah, that makes lot of sense. So, because there are just maybe hundreds of different kinds of database out there, what sort of databases do you need for your AI applications?

Anti Gutmans: I would say what we've really been seeing over the past year or plus yeah. is that pretty much all databases have started to add the capabilities needed to be able to be integrated into generative AI applications. So, we also have the purpose built vector databases, right, that kind of emerged with Gen AI.

But what you're actually seeing is a trend of, relational databases, whether that are, you know, databases like like Postgres with pgVector. they're all adding vector capabilities. They're all adding support for some tooling ecosystem like Lanchain. And, what I tell customers is, yes, there are going to be some databases are going to be somewhat better than others on things like vector lookups and so on.

But really, if you already have your data in a specific database, the first thing I would think about is, hey, does that database actually have the base capabilities to help me exploit that data with Gen AI?

Richie Cotton: Okay, that's interesting that it's not just about vector databases, which is still, I guess, quite a lot of the limelight with respect to generative AI, but you actually need relational databases as well. Are there any particular properties of databases that you think are important here?

Anti Gutmans: Yeah, absolutely. So one is, you know, vector support. So being able to do efficient vector indexing vector search. So for those who don't know, vector embeddings is a way to represent semantic data. good old days of full text search, we will basically do lookups with a piece of text.

And usually use an inverted index to go and look at documents that actually match that. In the era of Gen AI and vector embeddings, we can actually, you have a semantic representation. It's usually a vector of numbers. And we can index and then do a similarity search and actually find things that are very similar.

So this way you can actually find things like, you know, King being very similar to Queen as an example, you can find similarities in data. And so that's very powerful, both on its own and then also in the context of, retrieval augmented generation with JetAI. workload. So one is basic vector indexing and search, super critical for Gen AI.

But then also capabilities like natural language interfaces to the database. having the ecosystem integration with, tools like LandChain and Llama Index, being able to transparently call vector embeddings, APIs and services like Vertex AI. So we're seeing, you know, just a bunch of capabilities starting to be built into the database to just make it so much easier and more efficient for customers to build these gen AI apps.

Richie Cotton: Maybe let's talk about this in the context of Google and Google Cloud Platform in particular. So there are hundreds and hundreds of services on there now. I have to say, I tend to get lost and not sure which ones I might need. So can you just give me an overview of like, what the different database services are and which ones apply to AI?

Anti Gutmans: Yeah, absolutely. And, you know, I run the operational databases for Google, so I'll talk mostly about operational databases, but we also have analytical engines like BigQuery. But on the operational database side, kind of think about it as three categories. The first one is the category of managing third party databases.

So these are databases as is, like MySQL, Postgres, Oracle, SQL Server, Redis. those are managed services that just enable our customers to focus their time on innovating as opposed to integrating and managing the systems. Then the second category is what I call the modernization category, where we primarily position AlloyDB as a great migration target for customers coming off legacy databases and really wanted to modernize on open source and open APIs.

And by the way, dad, dad. Engine also plays a very important role in Gen AI because Postgres has become very popular to Gen AI developers with PG Vector and in AlloyDB we've actually created a lot of additional differentiation around building Gen AI apps. And then the third category is what I call the cloud first databases.

These are databases we first of all start to build. to run Google's billion user businesses. Those are engines like Spanner, Bigtable. We also built Firestore, a document database on top of Spanner. And those are the very transformative databases. Now, the way I look at this is all databases need to be Gen AI enabled.

So we've been building in vector capabilities across all these databases. We've been building them into the ecosystem like Lang chain and Vertex AI. And really our belief is everywhere where customers have data needs to basically be easily exploitable for Gen AI applications. So

Richie Cotton: whatever the standard database, you're probably already using your Postgres or MySQL or whatever. And you've got some new things like AlloyDB and then you've got stuff for, I presume, very big data that's, well, things like Firestore or Spanner, is that right? Okay. So you mentioned AlloyDB is like the sort of the new thing that is like a modernization target.

What does that mean? Like, how is it different from some of the existing databases? 

Anti Gutmans: Think about AlloyDB as being 100 percent Postgres compatible, but really Postgres on steroids. We've taken the Postgres engine and we solved some very fundamental challenges that customers had when they were coming from very high end databases and wanting to modernize on open source. So we solved issues like performance, scalability, manageability, So, for example, it is four times faster than stock Postgres.

We have an analytics accelerator that allows it to go 100 times faster. We've solved the linear scaling issue, so you can actually literally scale up on cores, which you can't really do with open source Postgres. And by the way, we've made this available everywhere, so it's not only on GCP. We have a version of it called AlloyDB Omni that customers can actually download.

They can run it on premises to modernize. They can even run it in other clouds like AWS and Azure. So that, that I would say is Allo Tob. And we're just seeing tremendous success both in customers coming from legacy databases or just customers who have very high end Postgres workloads and just need that additional scalability and performance.

Now the other piece that is related up to, to Gen AI is we've also been innovating really quickly on Gen AI within Alloy db. So for example. The vector capabilities are, it's actually built on 12 years of research we've done inside of Google that runs already internally. Some of our Google applications, we've brought some of that innovation into AlloyDB on vector processing.

We've also been working on natural language interfaces to AlloyDB. So a lot of innovation also around GenAI happening in this engine.

Richie Cotton: Okay. So, the idea of something that's a bit like Postgres, but faster and more scalable, that does sound quite appealing.

Anti Gutmans: Exactly. And our, you know, our commitment is 100 percent compatibility. So no vendor lock in, you know, use AlloDB and, you know, if at any point in time you want to just use open source Postgres, you can. Of course, you'll then forego some of those benefits I mentioned.

Richie Cotton: So, you mentioned that your focus is mostly operational databases rather than analytical databases. Do you want to talk through like what's the difference and whether you would care about one or the other for using with AI applications? Okay.

Anti Gutmans: Yeah, it's a great question. And you know, the lines are blurring. And I would say that definitely uh, as a vendor, because of how we've built our infrastructure, we're probably best positioned to actually make this a much easier experience for customers across both transactional and analytical engines.

But just to give folks a bit of a, an idea, transactional engines, you know, tend to be very much focused on short latencies, short latencies. transactional workloads very much on the serving side of the house. Analytical engines tend to be in a columnar format as opposed to a row format. Of course, there's always a, it depends answer because, you know, some systems are a bit hybrid and because they're in the columnar format there, they can be much more efficient at processing very large amounts of data and are usually built to be more of a data warehouse data lake capability.

Now, is a market leading. Analytics engine AlloyDB and Spanner are market leading transactional engines. Well, what's really interesting about some of the work that we're doing at Google is because we have built our infrastructure with compute and storage completely desegregated.

Our ability to start blending these worlds is actually quite high. So for example, BigQuery can actually query spanner or bigtable in production with full workload isolation at any scale. So customers can now benefit from having both the transactional analytical capabilities without having to move data.

Another thing we did in Alloy db, we added a culinary engine. which automatically will translate from the key columns of your data to a columnar format and then give you the 100 times faster analytical on the transactional engine. So a lot of work happening, not just on making sure both our analytical and operational databases are market leading, but also the fact that our customers have an easier and easier time to efficiently use whatever they need.

on the same piece of data.

Richie Cotton: So in theory, if you've got storage data storage in one place and then separate databases, you can have different people within the organization. querying the data in different ways. So, I'm trying to think of a a good use case for this. So you can have your data teams querying, using an analytical database, and then maybe what commercial teams making use of the transactional capabilities.

Anti Gutmans: Let me give you an example. Take Spanner as an example. Spanner runs the most mission critical workloads for our customers. is underlying systems like Uber and others that people are very familiar with. But you often have folks in the back office like data analysts who want to do kind of real time operational analytics to make sure they can optimize, the business or alert on issues on the business in the traditional way you have to actually move data from the transactional system to analytical system to go and analyze it.

But given that we built this data boost capability that allows customers to do real time querying from BigQuery to Spanner with full workload isolation, meaning it's not going to have any negative impact on the transactional workload, like that is very unique in the market. And it's because of how we built our infrastructure where any piece of compute on our site can access data.

data in microsecond latencies, in traditional architectures that we see with other cloud providers, they use blob storage. It's usually tens of milliseconds of latencies. And so these systems become very complex. They use caching, they become very brittle. We don't have that issue of Google because of how we built our system.

System. 

Richie Cotton: for organizations who are like, okay, we, we want to perform a database transformation program, you want to update things like, where do you get started?

Anti Gutmans: I think it, there's always, and it depends, it really depends on your use case. But I would say what we've definitely seen become probably the most common, the most popular engine in enterprise is Postgres. You know, we're seeing about 50 percent professional developers use it.

we've actually made sure that three of our engines have a Postgres interface. That is Cloud SQL, that basically has managed Postgres, AlloDB, which we talked about, which is Postgres on steroids. And then even with Spanner, we added a Postgres interface, so developers could use a familiar interface. So I would say, I would probably use that as a starting point.

But then, depending on the use case, you probably want to broaden your aperture. So for example, if you want caching, You may want to look at Valkey, which is the new Redis fork. If you have a data fabric you're building, you need very high ingestion rates on key value. Bigtable can be a really good solution.

So we do help our customers, make that determination of the right engine based on their workload.

Richie Cotton: And beyond the tooling I suppose the skills you need to work, these things are probably going to change. So I guess maybe first of all, who is. Typically involved in working with these cloud databases.

Anti Gutmans: first of all, the benefit of these fully managed services is the customers can really focus their time on innovating and not integrating. And so we're seeing as developers have a lot more flexibility now to choose the database they want because we're doing the management for them.

And so they don't necessarily have to have as much in house skill set. of really operating these databases as they used to have. So the developer side of the house is definitely critical. The second team we tend to work with is the platform engineering team. That are the teams that really think about the resiliency of the application architecture as a whole.

They think about security, governance, data protection. And so they tend to support the development teams on the operational side. again, they don't have to be as deep on the database because we manage it for them. But they do have to make sure that the application architecture as a whole is sound and that they're managing that application in a secure and available manner.

Richie Cotton: Okay, so, in this case it's mostly software engineering teams, data engineering teams, infrastructure teams. How about the, the rest of the organization? Like, does it change the way, for example, analyst teams need to work or other people who are working with data? Like, once you move to a cloud database, is that going to change people's workflows or what they're doing?

Anti Gutmans: think what we're seeing is that a lot of the, data and lists persona are still using, kind of our traditional data warehouses. But, but these data warehouses becoming way more powerful because now they can also reach into the data lake.

They can reach into the operational databases that AI capabilities. So what we've basically done is we've built big query as kind of that single entry point for the more of the analyst practitioner. And then from there, they can both use their existing SQL skills, but some of the new skills they're going to learn is, how to maybe do more federated queries, how to use embedded AI, in their queries.

So there are some new skills they're picking up. But we're definitely giving them an ecosystem, an interface that they're quite familiar with.

Richie Cotton: Okay. So that's kind of cool that like most of your standard SQL skills is the same, but then you've got a few extra things. So you mentioned federated queries and embedded AI queries. Do you want to talk me through what those things are in a bit more depth?

Anti Gutmans: Yeah, absolutely. And actually, you know, even before I talk about that, just one more interesting anecdote is, I still think SQL is still the language that most folks feel comfortable with. So even in a service like Bigtable, which was really the service that launched a NoSQL movement. We just announced SQL support.

NoSQL databases, most of them actually have added SQL support over the years. So, that is still a good interface. So to your question, though, on both the federation and also the embedding. So federation means you can actually quite easily query across different data sources. So even if you're in BigQuery and a lot of your data is ingested in this part of BigQuery native story, if you have some logs that have been indexed in your data lake, on, Parquet or using a file for an ingestion format like, Delta Lake or Iceberg, you will actually be able to join both data in your, in BigQuery and data in the data lake.

It also integrates with services like AlloyDB, like Cloud SQL, like Spanner. So basically in a single query user could actually query across these systems and join them. So they can always make sure that they're actually querying the freshest data. It doesn't change very much in their skill set, because it's still the same SQL they love.

It's external table, so it's very natural to use. So that is one element. I would say on the AI side, we have AI functions that make it very easy both to start, both to train and fine tune models based on the data customers have, or to do inferencing, meaning, let's say you have a column in BigQuery that has text.

and you want to calculate a vector embedding, we can make it very easy for them from within SQL to basically generate another column with the vector embedding based on the text column. So, that is just one example. There are obviously a lot of different inferencing APIs. But the nice thing is the practitioner doesn't have to leave their tool.

They're still using SQL. So we've really made it super easy for them.

Richie Cotton: Okay. That does sound interesting. So, I think. Perhaps a lot of times when you have text data, you really just want to turn that into like something a bit more tangible, like kind of almost feature engineering, and so being able to do that in SQL rather than have to like go and break out Python or something, and then you mess it about with going backwards and forwards between the database and maybe local analytics to seem like it makes a lot of sense.

And are there any other skills that you think are becoming more important with the move to cloud databases?

Anti Gutmans: I do think understanding how to use Gen AI is becoming more and more important. So the good news is you don't have to be an AI expert anymore. You don't know how to do training, model training, and so on. So the pendulum has really swung from the data scientists and model trainers to the developers and what I call kind of the inferencers.

And so some of the new skills that folks need to learn is how to do prompt engineering, how to do retrieval augmented generation really understanding the notion of vector search, vector indexing, vector search, how it works. There's always these tradeoffs between performance and recall.

So I definitely think that, skill that folks need to pick up is how to leverage these foundation models. also a lot of the folks who are actually learning those skills are the developers because they're the ones who are actually building these applications. So I think that's been really exciting.

I also think that because this data. Now has to be low latency has to be live. What you're seeing is, you know, it's not just about analyzing data in the data warehouse. We're seeing a lot of reverse ETL happening, meaning once we have insights, we actually move it back into the database into the serving side so that when a request comes in, we can make that data very quickly accessible to the foundation model.

So I think the other piece where we're seeing is that folks need to have a broader understanding of the data landscape they're in. And actually learn how to use more than one system.

Richie Cotton: Yeah. It sounds like generative AI on the one hand, it's making things easier. You need to know slightly less machine learning for the other hand, having big impacts on data engineering, so you need new tools or new workflows, like reverse ETL, and then also you need to use more tools than you did before.

Okay. So, actually, do you want to talk me through what reverse ETL is?

Anti Gutmans: Yeah, yeah, absolutely. So, you know, in many cases you will bring a lot of disparate system data into your data warehouse like BigQuery or into your data lake, right? we support both with BigQuery and then you're going to run, more batches jobs to really derive like the more deeper insights into your data.

And that could be things like product recommendations. it could be a bunch of other capabilities. Once you get those insights, very often you want to actually move those insights back into the serving side to like a spanner or a big table. All right. all. for fast, low latency serving.

And so, as you think about ETL was how do you bring your operational data to the data warehouse? Reverse ETL is how do you bring those insights from the data warehouse back into the serving side so you can actually connect it to the application. we're seeing a lot of that happening and so we're also making investments to make that reverse ETL workflow much, much easier for customers.

Richie Cotton: Okay. That's interesting that you've been generating a feedback loop. So you brought everything into the data warehouse, you then crunch it and you get some insights and then you're like, okay, those insights are then going to feedback to do something else. All right. I'd like to go back to some of what we talked about at the start, actually.

So, you mentioned how vector databases are probably essential for a lot of generative AI applications. Can you just talk me through some like specific use cases, like where vector databases come in handy?

Anti Gutmans: Yeah, absolutely. And you know, a lot of folks talk about retrieval augmented generation, which is, how do we assemble a prompt for a foundation model that kind of has the data that that foundation model needs? Because we can't just feed it the whole database. It's going to be too expensive, too slow.

And most models that don't actually support that. one method, and it's not the only one. Is to actually use vector indexing and then vector search to retrieve the documents or the roles of the pieces of data. that are most relevant to the question. You'll often see that in kind of the text like interfaces or, even in images, because doing that similarity search on vector embeddings is very efficient in that kind of model.

So instead of bringing in 1000 data points into the model prompt, you can say, Hey, what are the 10 closest ones, the 10 most relevant ones and actually feed it. That is one I would say most popular way, but the other thing that we're seeing is just, if you just think about search, just regular application search, you're going to an online store, you're looking for a cup, right?

maybe you're looking for a red cup and how do you find the most relevant in the past? This was mostly done with full text search. Now you could also use vector embeddings and vector search to also find, you know, similarities that are more semantic based versus just full text search based.

What we're seeing is a lot of customers are actually blending those two. They're doing their traditional search query, often with full text search. They're doing a vector embedding space search. And then they're merging those two data sets and then doing the traditional ranking on what to show to customers.

So this is another really interesting use case. Folks often don't talk about it too much because we're all focused on rag. But the reality is we're seeing a lot of customers in retail and other industries actually just using vector embeddings for much, much better and more accurate search results.

Richie Cotton: I suppose at Google you have a bit of an interest in good search. So, yeah, that's interesting. And certainly there are other shopping websites where I think I'm sure I searched for exactly what I want and it's retrieved like useless results.

Anti Gutmans: And by the way,

Richie Cotton: search is going to be helpful.

Anti Gutmans: even more fun, not just what you want, like take a picture of it and search, right? Like, I mean, this is what vector embeddings do is like, you can actually now just upload a picture and we can show you stuff that is similar. That is really what's so powerful with these vector embeddings.

It's not only text based, it can also be image based or video based.

Richie Cotton: Okay. So I take a photo of like myself wearing a shirt and I'm like, okay, find me other shares that you sell that are a bit like this.

Anti Gutmans: And by the way, we've actually had that capability in Google search for quite a few years. And, I've known about it for a while. I've used it once in a while, not that often. I actually use it all the time now, like I've gotten so used to searching with images as opposed to text.

And by the way, you can also combine, you can like snap the image and add some text. like it's worth trying out because those are the kind of experiences you can now deliver to your customers through your application. 

Richie Cotton: seems like it could affect like a lot of at least retail websites. Does it have applications beyond this? Like, I think about any other sort of corporate website, how can the search be improved there?

Anti Gutmans: you know, I think you know, the sky's the limit on what you could do, but it could also be used in like inventory management and, you know, I mean, there's just a bunch of interesting use cases. Where using images as opposed to just text or other unstructured data can really help you find the information that you need.

Think about just, you know, service, right? Like, actually, I'll give you another example I got from a retailer, which is inventory management on the floor, right? Actually being able to go and snap a picture off the shelf, and it immediately tells you if you have any more in the inventory or not. Or you can actually place an order right there to actually replenish the inventory.

So, a lot of, I'd say service level, um, roles that are in the field that are not like in an office, right, can really benefit from being able to take images and video. And very quickly translate that to action.

Richie Cotton: Okay. Yeah. There's certainly going to be faster than like typing in some like product Key or something and trying to look, do a lookup, write a SQL query while you're in the field.

Anti Gutmans: I mean, the most amazing, the most amazing thing with this is you don't have to be, like, really a sophisticated developer to do this. Not like anyone can do it, right? That's what's so amazing with this technology is any developer pretty much can use a foundation model, give it an image in the prompt get an embedding, do a vector search on their Postgres database, and they get some matches.

Richie Cotton: Okay. That's pretty cool that the workflow is like fairly simple. I mean, I guess it's not something you're going to learn in five minutes, but like that, that's like an hour or two to build, I guess. So we got changes to websites. We've got changes to like operation stuff, like, on the floor stuff.

Are there any other areas where you think this is taking over? Yeah.

Anti Gutmans: think pretty much every industry right now when I think about our customers, whether it's in financial services or manufacturing or retail, you name it, transportation. I think we're seeing all of our customers really rethink. what the future of their business looks like and how they can really leverage AI to be a lot more efficient to deliver better customer experiences.

So the sky's the limit. And I think really consumers are really going to see the benefit from that. Businesses are going to see the benefit, and frankly, employees are going to see the benefit, too. So far, what we've seen is, it's really enabling employees also reduces some of the, undifferentiated heavy lifting employees have often had to do, and really enables folks to focus on the higher value activities.

So, I'm quite optimistic. on how this has been supporting both employees and customers.

Richie Cotton: Okay. So this is pretty cool that like, just because the cost of development has gone down, it's become a lot more accessible. These features are just easier to build, I guess, than they were before. And they've become more economically viable. All right. So there's one of the type of database which I'd like to talk about which we've not mentioned so far.

This is the graph database. Now, I know Spanner has a graph database built in. So, I guess to begin with, just talk us through what is a graph database.

Anti Gutmans: Yeah, and, you know, I'm super excited about what we've done in Spanner and really excited about Graph. So think about Graph databases as a database that really has a focus on not just capturing the data itself, but also how the data is interconnected. You can almost think about it from a relational perspective, unlike, you know, you have foreign keys and so on between data.

But think about this makes it really easy for you to basically make the connection between friends, between things people have purchased, connecting images or interactions, that people have. So it's all about capturing that kind of interconnected in context.

That becomes more and more important as you think about Gen AI because Gen AI, very often you want to exploit like the connectivity between information, you know, and things like knowledge graphs. So you use, graph and use cases like knowledge graphs. recommendation engines so I can easily say like, Hey, what products did my friends buy that I haven't bought yet?

That's a very easy query in a graph database and so you can kind of very immediately, do a recommendation use cases like anti money laundering, where you're looking for, rings connectivity within a graph to kind of look for bad actors I. T. topology management when you're trying to kind of see like, Hey, what are the what are all the instances and networks to have in my topology?

And if one goes down, what's going to be the negative impact of that security use cases? So graph is being used a lot of use cases. I think with Jenny, it's become even more important, both because of things like knowledge graphs. Alright, GraphRag has proven, you know, that additional context you can get from graphs actually improves the accuracy significantly in these rag based applications.

Richie Cotton: Okay, actually, tell me more about this, because it sounds like combining two very exciting ideas at once. Talk me through GraphRag.

Anti Gutmans: Yeah, so you know, as we talked about before, a rag is really a way to feed foundation model, right, with the right prompt and the right data. What we're seeing with GraphRag is because we now also have a better understanding of the connectivity of data, the kind of decisions we can make in what data points are really interesting to customers goes up a notch.

So it's really those relationships that help. Also, some customers build graph embeddings, meaning they actually kind of build more of these semantic relationships between their data points. And so that also helps them rank. the data better. So just think about it as it's another tool in the toolbox to get to a higher level of context and a higher level of connectedness.

Now, one thing we did really special in Spanner is it's not just about graph. We brought graph together with relational, with vector search and very advanced full text search. And so now customers have in a single database, a multi model database, all the ingredients to truly create A lot of context.

For example, you can do a vector search or a full text search, find the nodes in the graph that are most relevant. For example, what are the people that match a certain name or, you know, and then from there you can start traversing the graph to get more of that interconnected insight. And get more of the relevant data.

So it becomes a very interesting connection between kind of search and graph to really get to the highest level of insights and most relevant data points.

Richie Cotton: Okay, so this answer is going to be very useful if you have maybe complex questions for us. So, I presume it's going to be fairly useful in, like, support chatbots and things, because it's got a lot more context around, like, what the actual problem means. 

Anti Gutmans: Yeah, yeah, exactly. So, the problem is, very basic question and answering it, the query doesn't completely understand what you're asking, you can go wrong pretty quickly. I think the graph, you have a higher chance of getting to the right entry point in the graph and then traversing it and being able to infer what the user was actually trying to ask.

those are some of the second approximation benefits you can have. And what we've done in Spanner Graph is we've made it really easy to overlay this on your existing relational data. And we've integrated this with an industry standard called GQL. into sequel. So you can really use your existing data sets that are relational, your existing skill set, which is sequel to start getting the benefit of graph without having to kind of adopt a whole new database, a whole new model.

So we really made it easier for customers to start getting this benefit.

Richie Cotton: Okay, actually, I must confess, I've been sort of interested in thinking I should probably learn how to do, like, all these sort of graph languages, like there's a GraphQL and all these things, but the syntax is just slightly weird enough that I've never got around to doing it. So, what's the easiest way to go about learning how to work with graphs?

Anti Gutmans: You know, without this sounding too self serving, on in the Spanner graph documentation, we have a quick start that kind of gives you copy and paste, like here's how you create the schema, here are queries you can run. I actually did that myself, because I always like, before we launch a product, I really like to play around with it.

that was super easy. And then I was also able to change the queries a bit, and just get a feeling for how, Traversals work and so on. I mean, I had that skill from different graph databases I've used in the past, but not from spanner graph specifically. So that's probably an easy way to do it is just use it.

Look at our quick start and it really gives it a copy and paste so you can just play around with it. In the console, super easy and you'll get a feel for how these SQL embedded graph traversals work.

Richie Cotton: learning by doing, that's that's quite dear to our hearts at DataGap, so, uh, I like that approach. Okay, and it does seem like, maybe AI ought to be helpful with this, because you can write queries in natural language and hopefully it's gonna be able to create. Graphs for you. Absolutely. So, you know, there's a lot of investment we've, we've made on natural language to SQL. that's not an easy problem to solve out to even the state of the art today is probably in the 55, 60 percent accuracy. But we're, you know, we're raising the bar consistently, and, we think we can probably be the best at that.

Anti Gutmans: then while we haven't offered that natural language to SQL yet in the Spanner graph context, that's also something that's obviously top of mind for us to make it even easier for users. That is, I would say, one area of innovation. The other one is natural language to query, which is I do imagine that over time, More users are just going to want to use natural language as a way to create a database, not as a way to generate SQL that they can then review and edit.

That has its own problems because you need a much higher degree of accuracy, but that's also a really interesting space to watch in an area that we're innovating in. So, I guess rather than just generating SQL, it's then it's just executing the query and giving you results. And maybe you inspect this, the SQL, maybe you don't. Okay.

ambiguous. So it's not just about being able to do it. It's also about, well, what if someone asks an ambiguous question? Like, how can we then. disambiguate that, maybe ask a follow up question, right? And so on and so forth. So it's a pretty hard problem, but it's a really exciting problem to be working on.

Richie Cotton: So yeah, I guess what's the solution here. You just make sure that there's some kind of conversation going until both the user and the AI agree, or how do you approach this?

Anti Gutmans: there's, there's lots of, I mean, it's a combination of many approaches. Definitely one of them is, you know, also feedback loop and conversation. But it's also around like, how do you organize the data? How constrained do you have it? You know, there's a lot of elements to go into this. But it is definitely a difficult problem.

But one that we believe is worth solving. So, continue to kind of work in that direction.

Richie Cotton: So have you seen any success stories around like the great use of data in AI?

Anti Gutmans: Absolutely. I mean, all the time we're working with customers. We have a few thousand customers right now using things like PG Vector on Postgres, building AI applications. Actually, some of the use cases I mentioned, we're seeing chatbots. So, for example, Rignology in the regulatory compliance space.

for joining us. I built a chat bot both for their customers and their employees to be able to ask regulatory questions and that then queries their data. We have retailers who have been using it for things like search, I mentioned, like storefront search. So we're seeing a lot of really interesting, exciting use cases on how our customers are using GenAI with operational databases.

To deliver better outcomes, both for their customers or internally for employee productivity.

Richie Cotton: Are there any specific customer stories you could, you're able to talk about?

Anti Gutmans: But Rignology is one of them in the regular regulatory compliance space. You know, we have other customers like Bear, who are in the, you know, in the pharmaceutical space. We're also using AlloyDB. You know, in their context for AI applications I'd have to think about the public references we have.

I can't talk about customers that I'm, I don't 100 percent remember if they're public, but we definitely have customers across a variety of industries. who are using kind of vector capabilities, whether that's in LODB, in MySQL, in lots of our engines,

Richie Cotton: And I guess the flip side is, are there any common mistakes that you think organizations make when they're trying to make better use of data with their AI applications?

Anti Gutmans: one thing that I've seen is in the past, because a lot of the I was kind of driven from the training side and data science side, you saw a lot of that accumulate, like in the data science side of the house. And you'd even have folks who are kind of the data and AI leaders in these organizations, who look at analytics and AI as a way to deliver AI to the organization.

I think what we're seeing, though, is because the pendulum has swung so strongly from, you know, having these off the shelf foundation models, where developers are really in the driver's seat to exploit those. Some organizations haven't quite figured out that shift yet. where that's happening. And so do see kind of this interesting situation where you have these two different personas and roles in the enterprise and they're not necessarily talking that much, even though they both have a shared interest in making AI successful.

So I do think there's an organizational element here that customers are discovering quite quickly, which is how do they really think more strategically about both their data state and And then however, all the personas in the enterprise can, can take advantage of that, not just the traditional analysts, but actually the developers.

That's where we have a quite a strong advantage because of how we've integrated both operational analytical systems into a single data cloud that is very easy to exploit with AI. That's really an area that we've focused on, on help break down those silos, within our customer base.

Richie Cotton: All right. So the big problem is then that normally it's just like, it's one or two teams that really sort of, get to take advantage of these things. And actually you want everyone in the organization to be able to take advantage.

Anti Gutmans: Exactly. And, you know, the other thing that, of course everyone struggles with, which is what's the balance between accuracy and cost and latency? Accuracy. people often ask me like, Hey, what's the best model? I'm like, well, it depends. that it depends is hard. It really depends on your use case.

It depends on kind of the latency tolerance your use case has, the cost elements, the accuracy needs. So I think that's another difficult problem for organizations. We're, of course, focused on really helping organizations figure that out. But that is a non trivial problem for everyone.

Richie Cotton: Do you have any examples of when you might care about accuracy or when you might care about cost or when you might care about latency the most?

Anti Gutmans: Yeah. So for example you know, if you're looking to really process an important document and generate insights from a document, you may actually want to use a more sophisticated, bigger model that may be a bit higher costs and a bit higher latency because it's not in the serving path, So that's an example where probably that additional cost would help. But if you're, for example, doing a chatbot. And inferring what someone needs doesn't have to be as sophisticated, then probably a smaller model that can respond much faster and be cheaper is more effective. Right.

You don't want the customer to have to wait for eight seconds to get a response. You really want them to get like a, response within a second or two. so those are maybe two examples like document understanding, you would use a much more. bigger model, probably more expensive, higher latency model chatbot interface you want to optimize for latency and cost.

Richie Cotton: Yeah, can imagine a chatbot where he's waiting a very long time for every response. That's gonna be a terrible user experience. I want the AI to be faster than a human can type, I suppose.

Anti Gutmans: And then, of course, also vector embeddings, which we mentioned before, you know, generating vector embeddings, let's say you have a terabyte of data, it could take an awful long time. So even there, you want to make sure that, you know, your embeddings model is relatively low cost and low latency as long as it gets the job done, So that's always the but. It's got to be good enough.

Richie Cotton: Okay, yeah, make stuff that's good enough and I guess focus on the speed after that. Yeah, Okay, so before we wrap up, what are you most excited about in the world of databases?

Anti Gutmans: You know, this is one area that I'm really excited about. I think just operational data is so key to really getting the benefits of AI and foundation models that I think that, how we can innovate to make this super easy for customers. I think it's really, really exciting. And so I think that is one element that I'm just super excited about.

I think the other one is how, we start bringing these modalities together. So we, for example, talk about Spanner and like, how do we start being kind of relational and graph and vector search together to make it really easy for customers to build those kind of very high value contexts. I do think graph is at a bit of a tipping point, you know, it's been one of those technologies that like never really took off.

I think the need of this additional context has become so profound that we're really going to see this explode. And so I think that area, even though you, people will say, well, we've had graph for many years and we've had some multimodal databases for many years. I think now is the time.

And I think we've taken a very unique approach to really try and Make a big impact on behalf of customers.

Richie Cotton: That's interesting that maybe graphs going to be the next big thing. And that's kind of a cool thought. So finally, do you have any advice for organizations wanting to up their database game?

Anti Gutmans: thing I tell folks is the technology landscape is moving so fast. The business is moving so fast. You've got to transform your business. Don't waste your time integrating the different pieces and managing your own, like, infrastructure. Right. I think one thing I'm hearing loud and clear from customers is we need more bandwidth to actually make business impact.

So this is where I really advise like where it makes sense, which I think is most of the time, like use managed services enable us to do the undifferentiated heavy lifting for you so you can really focus your time on innovating and delivering business outcomes versus dealing with the plumbing.

I think that is really one piece. The second one is really thinking about. Gen AI and the possibilities it opens up, it shouldn't be one team's problem. It should be in the DNA across the organization. And by the way, it's not always the right fit, right? Some applications don't need Gen AI.

But I think every application team should be empowered. To take advantage of AI in the cases where it can make a difference. so kind of really making sure that that access is the market ties, the costs are clear, developers know how they can engage and use it is important to make sure transformation happens across the organization.

And not just in, a single central organization that doesn't necessarily really have the influence on all the different application areas within the organization.

Richie Cotton: All right. I do like the idea of just making sure that every single team gets to take advantage of these things, like gets to take advantage of the data. All right. Super. Thank you so much for your time, Andy.

Anti Gutmans: Yeah, well, thank you so much for having me. This was really fun and I really appreciate it.