[Radar Recap] Charting the Path: What the Future Holds for Generative AI

Adel Nehme

Hello. Hello everyone and Welcome to our pan ultimate session of the day at data Camp radar. I am so excited for this 1 on what the future holds for generative AI everyone do give us a lot of love in the Emojis. We want to see it. Let us know where you're joining from in the chat everyone give Thomas Edo Nick a lot of love today because they're going to be sharing a lot of excellent insights. I've already been enjoying the behind the scenes conversation quite a lot. So we're in for a treat today. Uh, you know, there are various Ways by which we can look at the generative AI ecosystem today and its future, um, you know from how the technical landscape will evolve to how llms themselves will improve to the infrastructure to the adoption the future of work and you know, how parent of AI in general will really change society and the tech space

00:00:53

Adel Nehme

As a whole in the future so luckily today's guests all come at the generative AI, you know come from the generative AI system, but from different angles, and it will be awesome to pick their brains and see how if you view the future of AI so first, let me introduce EO Liberty EO Liberty is the founder and CEO of pine cone. The managed Vector data database helps to build knowledgeable AI systems. He is also the former head of Amazon aai labs, and the research director at Yahoo. EO is key in building Amazon sagemaker. If we have any fans of Amazon Sage maker. Let us know in the chat and have a computer to over 100 academic research forward his company Pine Khan was founded in 2019 and is now valued at 750 million dollars. That's not bad at all. Let's see you

00:01:43

Edo Liberty

It's great to see you.

00:01:44

Edo Liberty

I wouldn't uh know if that valuation is right, but other than that it was

00:01:44

Adel Nehme

awesome, and next

00:01:50

Adel Nehme

The general partner at Theory Venture Thomas has been a venture capitalist for more than 15 years before founding theory. He has worked with companies including looker Monte Carlo mother duck Omni hex arbitrum and miston Labs quite a few well-known names in the data and AI space and he writes that tomtom.com, which I highly recommend that you check out his blog 1 of my favorite reads of the week Tom great to see you.

00:02:16

Tomasz Tunguz

Pleasure to be here. Thanks for having me on.

00:02:18

Adel Nehme

Awesome. Awesome. Awesome and last but not least is Nick. Elrin CEO of domino data Labs as co-founder and CEO seems like we lost nick. Uh, so I think we're having some people we'll be joining again. Hey, we're getting is again. No worries. Yeah.

00:02:31

Tomasz Tunguz

He's back.

00:02:31

Nick Elprin

I hear it. I'm here. Sorry about that. Yeah, I think there's a power outage.

00:02:37

Adel Nehme

Yeah. Yeah. So Nick is the CEO of nominal data lab as the co-founder and CEO of domino. He had a front row seat to the transformative impact that Dominus platform has Unleashed for its customers next start a dominant 24 after spending nearly a decade build on platforms for data scientists at Bridgewater the world's largest hedge fund since then Nick has built phenomenal into an industry leader whose ongoing Innovation is continuously reshaped the standard for data science and AI platforms. He holds a ba and a master's in computer science from Harvard Nick. It's great to have you on

00:03:10

Adel Nehme

So so just a few housekeeping notes, there will be time for at the end. So make sure to use the Q&A feature before asking for asking any questions if you have any other messages or want to engage in the conversation use the chat feature if you want to network, add your attendees on LinkedIn, we highly recommend joining our LinkedIn group. Any LinkedIn posts will be Auto removed immediately just to make sure that the chat is this a signal to the noise ratio. So let's talk about the state of a generative AI that you know, we've been talking about this behind the scenes, but I'm very excited to pick your thoughts here, you know, all of you approach generative AI from different angles, you know Thomas you're investing in the space either your work at pine pine or you're literally building the infrastructure that is powering a lot of generative AI use cases today and Nick You're Building tooling that that makes building and managing AI models much easier. So maybe let's set the stage for this conversation.

00:04:01

Adel Nehme

How do you view the current state of generate AI today Tom? I'll start with you.

00:04:07

Tomasz Tunguz

Well, I think we're really exciting time. You look at the model performance is rapidly improving where?

00:04:13

Tomasz Tunguz

uh, small medium and large language models are you know, there's a there's a score called the MML which is high school equivalency and

00:04:20

Tomasz Tunguz

Uh, all those models are performing pretty well the Llama model you have open source and closed source that are both thriving with openai and anthropic being closed and then Mistral and the Le llama family of models being open source. There's a lot of innovation at the infrastructure layer, you know, even the chips right? We're seeing pretty meaningful and improvements video with h200 you have a AMD launching chips that are 30% more powerful than the current h100s.

And then even startups that have architectures that profess to be 10 times faster. So there's a lot of innovation at the infrastructure layer.

00:04:51

Tomasz Tunguz

and I think

00:04:53

Tomasz Tunguz

as that evolves and improves

00:04:56

Tomasz Tunguz

there's a parallel desire from businesses to really take advantage of this. There's a dominant feeling that this is a big wave for good reason and so

00:05:01

Adel Nehme

Mhm.

00:05:06

Tomasz Tunguz

We see a lot of innovation a lot of Desire like Enterprise search is a major use case of asking questions about a knowledge base customer support is a major use case applications in legal accounting code completion. And those are probably the 5 most active areas of investment today.

00:05:24

Tomasz Tunguz

and then in terms of the future where we're going is

00:05:26

Tomasz Tunguz

making sure that these models are accurate over multi-step processes.

00:05:31

Tomasz Tunguz

and once we're able to unlock that then I think you'll actually see some massive productivity improvements because you we will we will be able to delegate tasks that

00:05:43

Tomasz Tunguz

that take a lot of time and have a computer do it instead of us.

00:05:46

Adel Nehme

Can either you know, you're building infrastructure to power a lot of generative AI tools let us know how you see the state of turn to V today.

00:05:54

Edo Liberty

Um, yeah, I I shared Thomas's uh, uh tamas. I think I don't know. I'm not sure how I'm pronouncing you your name in his uh in the right way too much but

00:06:06

Edo Liberty

am I is this the

00:06:07

Tomasz Tunguz

You are it's an sh at the end. It's kind of you to ask. Thanks.

00:06:10

Edo Liberty

All right.

00:06:11

Edo Liberty

uh

00:06:13

Edo Liberty

the uh

00:06:16

Edo Liberty

Yeah, I mean the space of course is rapidly involving and so on. I think we see a few.

00:06:21

Edo Liberty

Different Trends. Uh, uh, I think the biggest 1 is Enterprise adoption.

00:06:28

Edo Liberty

which sort of like

00:06:30

Edo Liberty

started in you know fits and hiccups last year and I think this year people really start to

00:06:38

Edo Liberty

Uh invest deeply in that and for that they need here.

00:06:41

Adel Nehme

Mhm.

00:06:42

Edo Liberty

models and their AI in general to be knowledgeable and trustworthy and dependable and recovered and like they sort of like the excitement of the new technology kind of heads the

00:06:54

Edo Liberty

the rocks of reality of what it means to be able to actually work in in an Enterprise.

00:07:02

Edo Liberty

Uh, I think we see a lot of that.

00:07:03

Edo Liberty

Uh, and at the same time, I mean I share the the excitement. I mean, there's there's a ton of improvements I think agents and assistants are going to

00:07:12

Edo Liberty

Be very big.

00:07:14

Edo Liberty

um

00:07:17

Edo Liberty

We're going to improve a lot of them with.

00:07:19

Edo Liberty

With more knowledge from Vector basis and other uh, you know, improved models and better mechanisms and so on. Um, so yeah, I there was a ton coming and there's a lot of energy and uh,

00:07:33

Edo Liberty

desire to build on those tools, so

00:07:36

Adel Nehme

Mhm. So we're definitely going to unpack a lot of that and maybe Nick from your perspective walk us through how you view the state of Journey to AI.

00:07:37

Edo Liberty

Yeah.

00:07:45

Nick Elprin

Yeah, um, well, maybe I'll be a little more contrarian or or a little less sanguine than taymouth. And um and Ido, you know, I um, look the the Technology Innovation is very exciting, from my perspective working with customers and most of our customers are are large large Global Enterprises. You know, I'd say we're my take is we're still kind of in the inflated expectations period of the hype cycle and and what I mean by that is there's a lot of prototyping there's a lot of experimentation. Um, but most of the real production use case not all of them, but most of them are seeing are still sort of internally facing like tamas said, um, you know Enterprise search question and answer chat Bots and look like that's that's fine. Um, but I don't you know, that stuff is not um world changing and where I've seen companies really get excited about initial ideas for um, Products based on genuine idea that would change a business model that would really be customer facing they have exciting prototypes, but they can't get across the Finish Line because there's still too many issues around accuracy around around around around safety or around Roi frankly. And so, um, you know, I think look I I think that

00:09:00

Adel Nehme

Uh-huh.

00:09:06

Nick Elprin

Maybe we're jumping jumping a little ahead. But but I I think there's a huge opportunity for for geni to impact.

00:09:12

Nick Elprin

Um Services heavy businesses, uh, you know, Consulting businesses, um, um, graphic design agencies. Um, but but I think that for a lot of other businesses, I I worry actually that the the infatuation with Genai has distracted Business Leaders from the benefits and the potential of traditional machine predictive machine learning traditional AI. Um, and I I think that I think that in some ways to sort of set us back on the potential for all AI to really transform businesses and have positive impact on the world so that yeah, like I said, maybe maybe a bit more uh, provocative take but that's how I see things right now.

00:09:15

Adel Nehme

Mhm.

00:09:53

Adel Nehme

Got to let maybe Tom and either react to that like so how do you view kind of the adoption landscape today? So I'll let maybe either first I'll let you take that.

00:10:04

Edo Liberty

Uh sure I agree 100% that there is a ton of experimentation and most of the energy in this field is spent with understanding how to run these things.

00:10:13

Edo Liberty

uh how to make your applications

00:10:16

Edo Liberty

first of all perform what you want them to perform be governed be Dependable be not not make silly and embarrassing mistakes and and so on.

00:10:26

Edo Liberty

uh, and that's hard and that's hard and I agree with Nick 100% that some

00:10:31

Edo Liberty

people might have unrealistic expectations from AI.

00:10:35

Edo Liberty

Uh, that's it. Um,

00:10:38

Edo Liberty

I do think that a very large set of new kinds of workloads is now possible that was not possible with traditional machine learning.

00:10:45

Adel Nehme

Mhm.

00:10:48

Edo Liberty

And oh, what if I would call them like predictive models whose job is mainly to score and predict and rank and and and do stuff that we do very well today already and and I agree with Nick 100% are incredibly valuable and people should invest in them.

00:11:06

Edo Liberty

But the ability to consume language and generate language in a in a meaningful way.

00:11:14

Edo Liberty

Is incredibly powerful. I think yes, of course a lot of people don't know how to

00:11:18

Edo Liberty

commercialize it yet within their business and their experimenting but I think a lot of them already know what they want to do and already know that this is the answer and now the question is how to build it and and how to get there and you know, when can we launched this?

00:11:33

Adel Nehme

Okay, Tom. I'll let you react as well.

00:11:36

Tomasz Tunguz

Yeah, I agree. I think we're still early really people are still trying to understand exactly what's going on with the Technologies are good at what they're not good at. I think there's security concerns around data loss, uh, which

00:11:48

Tomasz Tunguz

uh, and there's this Dynamic where

00:11:51

Tomasz Tunguz

historically the Chief Information Security Officer. The ciso has been the 1 that's been responsible for securing these uh,

00:11:58

Tomasz Tunguz

Companies and now the heads of data heads of engineering are a bit thrust into this role of figuring out.

00:12:04

Tomasz Tunguz

How does that work? I mean even like the procurement processes for some of these Technologies are really tough legal teams are needing to create new rules about well.

00:12:15

Tomasz Tunguz

you know, how do we think about software and there's been 20 or 30 years you have like sock 2 compliance with the procurement process now all of a sudden there's this thing where

00:12:18

Adel Nehme

Mhm.

00:12:25

Tomasz Tunguz

Kind of vendor use my company's data to train a model. Is that valuable or what? Do we want the data that's on inside of VPC or on Prem. Is that something that's really important to us?

00:12:34

Tomasz Tunguz

And so I think

00:12:37

Tomasz Tunguz

even there's like the technology diffusion curve. Like what is this technology do and that's slowing adoption. And there's the who's responsible for securing it and then there's the how do we get this through procurement in a way where nobody's fired? And so those 3 things those 3 Dynamic I think have um, people are still companies organizations are still figuring it out.

00:12:47

Adel Nehme

Mhm.

00:13:02

Adel Nehme

Mhm. Okay, that's really interesting. And you know, how do you see, you know, you mentioned here these challenges, uh, Tom, um, either maybe from your perspective you see technical challenges as well that are hindering adoption today and the Enterprises.

00:13:18

Edo Liberty

yeah, um, I I would

00:13:22

Edo Liberty

say there are 3.

00:13:26

Edo Liberty

Main challenges and uh, not surprisingly. There's there's a 3 big uh Focus areas for us as well.

00:13:32

Adel Nehme

Mhm.

00:13:35

Edo Liberty

uh

00:13:36

Edo Liberty

1 is the adoption of the technology itself. You have a lot of Engineers and and systems, uh systems and and application builders that uh don't have an had the the experience of data scientists or data engineering or ml Engineers, uh, and they are now interested with building those things. So they need to catch up and and learn very quickly and we're trying to do a very good job at

00:14:02

Edo Liberty

at both educating and giving really good tooling and making everything work out of the game.

00:14:07

Edo Liberty

The second thing is people need to actually make the applications more knowledgeable and perform better. So it's not just you know, if if you could build something by just

00:14:17

Edo Liberty

You know hooking up to a an llm great you you're done but most applications don't work. That way. They need to do support they need to know about your documents. They need to know about what a processes they need to connect to your own data somehow.

00:14:21

Adel Nehme

Mhm.

00:14:35

Edo Liberty

Uh, and so that means you have to sum up provide them with knowledge and that is very difficult today.

00:14:38

Edo Liberty

Victor databases in Pine Cone specifically make it a lot easier, but it's not enough. We have launched a system today. Uh today I think or this week, uh that uh allows people just load document and run basically, uh text queries and and so on and answer questions and complete tasks with their own data.

00:15:00

Edo Liberty

uh

00:15:01

Edo Liberty

But that's still a challenge for a lot of people who don't use pine cone. And the last thing really, you know underscores what tar said. It's not enough. You can build an application that will blow the C CEOs like socks off.

00:15:04

Adel Nehme

Yep.

00:15:18

Edo Liberty

Right, but you would still need to be socks compliant and HEPA and and gdpr and the you know have jump through all the fiery hoops. And a lot of them are actually complicated questions, you know, and so it's it's still going to be hard. Uh, so, you know, this Enterprise Readiness is I think a big uh,

00:15:32

Adel Nehme

Yeah.

00:15:42

Edo Liberty

Area of of uh friction for buyers and investment for companies who work in this space.

00:15:49

Adel Nehme

Mhm.

00:16:39

Nick Elprin

Well, you know actually, you know, I was gonna I was gonna yield or or see my time to answer to to maybe you know, I'm I'm not as I'm not I don't have the most expertise on sort of the LM architecture details and so he might have a better answer. Um, but, you know, even before he jumps into that I'll just say

00:16:58

Nick Elprin

My my prediction would be that regardless of how some of those next wave of leaps play out. I think we're going to see a lot more. Um,

00:17:06

Nick Elprin

Juice being squeezed from different ways of combining multiple models together. So more agentic workflows. I know that's that's very a lot of people are talking about that right now. And I think that's also going to enable, uh, kind of harness better harnessing or extracting more value from um, smaller more specialized models. And so I think we'll get more um more system AI systems that are using a lot of more Specialized or small models, um, potentially likely in in sort of a genetic patterns, um to to improve overall performance of these systems, but um anyway to answer your question directly. I don't I don't feel like I have the best expertise.

00:17:10

Adel Nehme

Mhm.

00:17:12

Adel Nehme

Mhm.

00:17:44

Adel Nehme

So let's switch gears then either. I'll let you take that on.

00:17:49

Edo Liberty

um

00:17:51

Edo Liberty

I'll offer a contrarian view that I actually don't models are going to the current way. We think about models are not going to improve dramatically in the way that we'll qualitatively change. What can be done with them.

00:17:54

Adel Nehme

Mhm.

00:18:02

Adel Nehme

Okay.

00:18:07

Edo Liberty

Okay, uh for multiple reasons, they're like physics kicks in uh at some point on how these things are trained. There is the economics of doing that, you know, when model training up top-notch model was quote unquote only 10 million dollars, which was Way Out Of Reach For any tiny company

00:18:24

Adel Nehme

Mhm.

00:18:28

Edo Liberty

But a lot of medium and large companies have that capacity when you need to invest a billion dollars to train a model suddenly.

00:18:36

Edo Liberty

You this is this is really kept to a very very small set of players. Um, and so

00:18:44

Edo Liberty

and for reasons that you know, I you know, it's going to be take a long time to go through. I I I personally think we're reaching a level of saturation that uh, it's going to be hard to qualitatively change and yes, they'll improvements but to me

00:18:55

Adel Nehme

Mhm.

00:19:02

Edo Liberty

Product wise if some accuracy measures for a goes from 80 to 85% It might be a tremendous engineering fit fee and and it could be a big difference but it's not a qualitative change. You didn't go from you know, 15 to 80 and you didn't go from 80 to 100.

00:19:20

Edo Liberty

Okay, and so yes, it's a better model. But is it qualitatively different? I I don't think not in terms of the applications and use cases. What is going to change uh is that we are going to make these models.

00:19:21

Adel Nehme

Yeah.

00:19:27

Adel Nehme

Yeah.

00:19:36

Edo Liberty

A lot more Dynamic knowledgeable plugged into the right places and find ways to make them.

00:19:45

Edo Liberty

Uh, basically improve the ways they train and make them tend to 100 times bigger.

00:19:50

Edo Liberty

Uh, and that's happening but it's not the models, you know, and the the the the way you think about a language model it's going to be completely different like so a quality a qualitative change has to happen.

00:19:60

Adel Nehme

Okay.

00:20:04

Edo Liberty

For that curve to a break basically in my opinion.

00:20:10

Adel Nehme

Okay, so models are going to become more useful but not necessarily raw intelligence. Why is there going to have a simple a jump in?

00:20:16

Edo Liberty

no, no, they they will but uh

00:20:20

Edo Liberty

Well, it's it's hard to explain. I might I might have to stop here. But I will say that I don't think the the models the way they are trained today.

00:20:29

Adel Nehme

Mhm.

00:20:30

Edo Liberty

Uh

00:20:32

Edo Liberty

are I think hitting an an asimtot an asymptote, uh that we're not going to improve much on.

00:20:39

Edo Liberty

in my

00:20:40

Adel Nehme

Okay, and and Tom I'll let you also react to here. How how do you view the next wave of LMS?

00:20:45

Tomasz Tunguz

yeah, I think there's a bifurcation you'll have large language models that will cost like a billion or 10 billion and you'll have a handful of companies that are training them and I suspect they'll probably be government subsidies for training some of these massive massive models because they're

00:20:58

Adel Nehme

Mhm.

00:20:59

Tomasz Tunguz

of national importance. Um, and then you'll have smaller models that are purpose built that are much more accurate that are running on your mobile phone that are much more and so you have this basically bifurcation and it'll be used for different use cases. You know query will come in will be a classifier that says have I seen this kind of query before and if I have it probably goes to a small language model to optimize for a particular task and if it hasn't goes through a large language model, it's basically a big consumer search index effectively. And so I think that's where we're going we call these things constellations of models. We're starting to see Enterprises actually develop some pretty sophisticated.

00:21:36

Tomasz Tunguz

Layers on top there's a caching layers in order to reduce costs because if let's say you're working on a translation.

00:21:42

Tomasz Tunguz

Software company saying you know the word for hot dog in Russian doesn't change and so you don't need to go back to the index every time you can just cash it.

00:21:43

Adel Nehme

Mhm.

00:21:50

Adel Nehme

Mhm.

00:21:51

Tomasz Tunguz

Uh, and so I you know, that's that's the way that I see it. I think you know multimodal is kind of the I think the 2 big waves of the day is multimodal meaning

00:21:57

Adel Nehme

Mhm.

00:22:02

Tomasz Tunguz

Use text to produce video or vice versa or learn from video and use that to improve text models and then agentic systems which Nick and Ido have talked about already which is basically chaining these things together to act have a few minutes the multimodal Technologies are new I think you can see it with clouds artifacts starting to work people creating 3-dimensional models of uh, particle physics with just a handful of sentences. And then it's rendering it's pretty cool. Uh, and then on the agentic stuff, I think the compounding error rates through multi-step processes are just really hard to manage today and there are a lot of different architectures there. There are suggest or critiques. There's adversarial networks that people are starting to use there's the use of the suggestion with a classic classifier in order to minimize the error and that's all research today. It's it's very very early.

00:22:18

Adel Nehme

Mhm.

00:22:27

Adel Nehme

Yeah.

00:22:53

Adel Nehme

Yeah, and 1 thing. I think that is kind of we discussed this behind the scenes right which is you know, um.

00:22:60

Adel Nehme

Are we reaching saturation to training data for example is like 1 to kind of training the next generation of models. So and you hinted at this EO like saturation of training and like saturation of models today, maybe walk us through your your your kind of thinking here a bit as well in more depth. I'd love to learn more about how you see the next wave training will happen what you what what needs to be true to get incremental improvements on models.

00:23:24

Edo Liberty

well, I can talk about it because we're working on it, but

00:23:31

Adel Nehme

So Tom, maybe I'll let you in here as well.

00:23:35

Tomasz Tunguz

Yeah, so I missed a question. What was it?

00:23:37

Adel Nehme

Yeah, so do you think kind of we're going to reach saturation on training data when it comes for uh models today? And so what is the way out for training models?

00:23:45

Tomasz Tunguz

Yeah, I think well, okay. So llama 3 8 billion parameter was really interesting. So let me take a step back the way to think about training 1 way of thinking about training is there's a cost optimization function here, which is how much data do I need and how many compute hours do I need to train a model and there was a paper produced called chinchilla which created a humoristic an algorithm to determine exactly what that was and meta decided on the Llama 3 8 billion parameter to go Way Beyond it where they spent they trained it on 15th and the performance there was actually this very expensive to do but the performance was really good especially compared to larger larger models. And so I think we're we're sort of figuring out. How do we squeeze as much juice from the lemons that we have and exploring that surface area. We're probably in a local maximum the

00:23:50

Adel Nehme

Mhm.

00:24:09

Adel Nehme

Mhm.

00:24:30

Adel Nehme

Mhm.

00:24:35

Tomasz Tunguz

the other Dynamic and you know, EO and I were talking about it before but

00:24:38

Tomasz Tunguz

That llama model was trained on about 15 trillion tokens. Our estimate is just someone with like 20 roughly 20 trillion tokens on the internet and the question is. Okay. Well we want to train a model on like a 100 trillion tokens or 500 trillion tokens.

00:24:46

Adel Nehme

Mhm.

00:24:54

Tomasz Tunguz

Where does that come from? And I think that's

00:24:57

Tomasz Tunguz

that's TBD. Nobody knows right? It could be video. It could be content that's yet to be created. It could be synthetic data. Uh where computers are actually, you know to identifying where models are weak and then producing data that supplement it. So, um,

00:24:59

Adel Nehme

No.

00:25:04

Adel Nehme

Mhm.

00:25:11

Tomasz Tunguz

I don't have a great answer for you.

00:25:13

Adel Nehme

Okay, so I think we'll find that and then maybe if we kind of look at you know, Nick you mentioned this earlier today earlier in the conversation that you know, we're still an experimentation phase for a lot of tools and you know, but all 3 of you hinted at kind of the uh, deployment challenges with generative AI Solutions.

00:25:31

Adel Nehme

Today, you know a lot of infrastructure technology to uh, enable fast deployment of LMS think monitoring the debugging retraining systems, all of that, uh, still needs to be built. Uh, so have you if you maybe the maturity of the llm middleware ecosystem today, uh, or like the generative AI middleware ecosystem today so Nick, maybe I'll start with you here as well.

00:25:51

Nick Elprin

Yeah, I think it's similarly early. I mean and the other thing I would say is I don't I think um.

00:25:58

Nick Elprin

I think a mistake people are making is thinking it's um conceptually a completely different thing from like like let's say the traditional mlops challenge is we've all we've had over the last 5 ten years. So I think you know, I heard I hear some people think well LMS and Genai make it so we don't we don't have mlops problems anymore. It's a different set of problems and rather. I think it's more like the

00:26:08

Adel Nehme

Mhm.

00:26:22

Nick Elprin

the Gen AI use cases take all the same problems. We had with traditional mlops, but they just they just exacerbate them or now. Now it's like if we're playing a video game we've activated the hard mode. Um, so you know you have you have a lot more

00:26:27

Adel Nehme

Mhm.

00:26:38

Nick Elprin

Um a lot more intensive compute resource requirements infrastructure requirements, uh, you know, it's gpus it's not just CPUs you need to scale these things that comes with a whole bunch more costs you need to deal with. Um, I mean, we have customers that

00:26:52

Nick Elprin

Have have gone from deploying again for Prototype use cases traditional predictive ml to now Genai and all of a sudden their compute bills are going out of control. And now they they before they didn't need to worry about things like elastic autoscaling for inference. Now they do because the gpus cost a lot more. Um, you you know, you talk about talk about model governance model monitoring. Um, uh, that's all based on the the fundamental characteristic of any sort of predictive AI system that um that their probabilistic and so their behavior can they can have unexpected failure modes or behaviors can change, um, you know, or their their their performance can change as sort of the world around them changes. Well Genai has a much wider surface area of of unexpected failure modes than say a traditional a predictive ml model. And so now the set of things you need to do to check and guardrail in the set of um, human or human processes that you need to put these things through as we move them.

00:27:24

Adel Nehme

Mhm.

00:27:54

Nick Elprin

Sort of a quality assurance process. Um, that's all much more complicated. So yeah, I mean like the the sum it all up. It's like all the same problems with that what they have we had with mlops are now, um incremented or exponentially harder for for Genai. Um,

00:27:56

Adel Nehme

Mhm.

00:28:11

Nick Elprin

and and I would just add I think I think a lot of these challenges are

00:28:14

Nick Elprin

Not neces you asked about middleware. I'd say a lot of these challenges are not um strictly technical they are they are business process challenges. There are people in process orchestration challenges, uh, if if we're going to put a Genai model or system into production in an in an Enterprise that might touch customers especially in a regulated environment of saying and insurance company, um, a life science is company, uh, what what checks have been what checks have been executed to ensure that is compliant. That is safe. These are um,

00:28:21

Adel Nehme

Yep.

00:28:49

Nick Elprin

Everyone that we work with is still trying to figure that out.

00:28:52

Adel Nehme

Great and then Thomas from your prospective you're investing in this space. Right? Like you have a very elevated, you know bird's eye view over the space. I'd love to kind of see how you see the kind of infrastructure space for LMS evolving today.

00:29:07

Tomasz Tunguz

Well, there's I mean lots of exciting things happening right you just building a great business like a database that's a really important. We're investors in a company called um superlink, which is a called a vector computer which allows you to take structured and unstructured data and put it together, uh in a unique way.

00:29:16

Adel Nehme

Mhm.

00:29:21

Tomasz Tunguz

Uh, and so I think where you know, but it's really early in this ecosystem. Where as we talked about buyers are still trying to figure out exactly what it is that they want to build and until they know what that is. It's hard to have a view on exactly what the ideal architecture is. Then you have, you know, open Ai and the other big companies also deciding strategically what areas of infrastructure do they want to play with right? So openai released an open source evaluations framework, um to help

00:29:32

Adel Nehme

Mhm.

00:29:50

Adel Nehme

Mhm.

00:29:51

Tomasz Tunguz

the software Engineers understand exactly how well a model is performing before they release a little bit like a testing harness and that was a move for them into a category with there are a lot of startups already.

00:30:02

Tomasz Tunguz

So it's you know, I guess for startups, I guess the way I'd put it is you're dancing with elephants. There. Are these big companies who really care about these markets Microsoft as a 5 billion dollar run rate business and 18 months open AI is north of 3 billion and you have others that are growing pretty fast and they're still deciding what parts of the stack they want to play with you look at um,

00:30:12

Adel Nehme

Mhm.

00:30:26

Tomasz Tunguz

uh, you know open ai's acquisition of Rocket last week, I think and you know open questions about what did they do with that technology and that team

00:30:28

Adel Nehme

Yeah.

00:30:34

Tomasz Tunguz

So the chess the chessboard is not set. There's uh, there's still a lot of moves to be played and uh and you have to navigate I think startups have to do what they do best, which is being Nimble and navigate the ecosystem as its rapidly forming.

00:30:49

Adel Nehme

Yeah, and either you're building a company that's at the center of the infrastructure space and llms, uh, you know walks through how you see this space of evolving over the next year and I don't want to I don't want to hear like product road map. That's not the question, but I would love to see kind of how you think the the ecosystem will evolve and then the outcome

00:31:09

Edo Liberty

Um, I agree with how much 100% I mean, this is the it's a very dynamic.

00:31:17

Edo Liberty

space uh

00:31:18

Edo Liberty

and um

00:31:22

Edo Liberty

I think that the um

00:31:25

Edo Liberty

there's going to be 1 trim that's going to still uh Prevail here, which is

00:31:32

Edo Liberty

uh

00:31:34

Edo Liberty

Uh companies and Enterprises specifically, uh going to find AI more.

00:31:41

Edo Liberty

Production ready and useful, uh, very soon. Uh, we are working on it others are working on it the the elephants in the room are working on it.

00:31:43

Adel Nehme

Mhm.

00:31:52

Edo Liberty

It's it's obvious that this needs to happen.

00:31:56

Edo Liberty

Uh, and so exactly who does what part of the stack and who gets to own what piece of the pie I think is is anybody's guess. Uh, we of course, you know want to have the biggest piece that we can know and out of that. It's obvious. It's like any other SEO. Uh, but um, I, you know, I'll be intellectually honest and say that nobody nobody can really tell

00:32:19

Adel Nehme

Yeah, and I think the segue is to my next question really. Well is everyone's working on this problem of deployment and making sure that you know models are very productive, you know, uh, you know, I think Nick mentioned earlier, uh that we're at the peak of inflated expectations. Uh, but we will reach the slope of Enlightenment at 1 point in time here, right? And once we reach a cortical mass of adoption, uh, if generative AI, you know, you know reaches critical mass in terms of adoption. How do you think it will shape the nature of work or

00:32:52

Adel Nehme

The but what does that feel look like? So maybe Thomas I'll start with you.

00:33:00

Adel Nehme

Yeah, yeah.

00:33:02

Tomasz Tunguz

All right, just going off mute. I think um, you know the way that we think about it is there's lots of toil.

00:33:05

Tomasz Tunguz

within work, there are lots of repetitive actions that we do, uh and each

00:33:12

Tomasz Tunguz

Each discipline has its own right within sales. There's the role in sales development of understanding leads and qualifying leads and deciding which ones to pursue within the world of legal. There's paralle work, which is the administration of documents within the world of accounting, uh, data entry across different kinds of tax forms that have different formats and uh within the world of software engineering it's remembering which arguments go and which order and a function call.

00:33:23

Adel Nehme

Mhm.

00:33:38

Tomasz Tunguz

And I think that the future of work with AI is having computers solve a lot of those wrote tasks and work alongside of that's these co-pilot systems that we have and today the productivity gains at least from some of the early companies that suggests. It's about a 50 to a 75% productivity gain. Maybe it's a little bit of aggressive. Um, computers will not love each other. That's true Luis, uh,

00:34:01

Tomasz Tunguz

But I think if we look to the Future where work is actually automated and we can delegate tasks to computers. The way that we delegate tasks to humans. The best analogies is looking at automobile manufacturing lines where robots have replaced a lot of the the labor and there you see about a 275% Improvement in productivity where 1 robot takes about uh can produce about the output of 2.7 humans and I think I think we should be able to get close.

00:34:28

Adel Nehme

Mhm.

00:34:31

Tomasz Tunguz

That's not based on anything.

00:34:33

Tomasz Tunguz

Except a rough guess.

00:34:35

Tomasz Tunguz

But I think you know order of magnitude we should be able to get to a place where a lot of this toil this rope work. That is unappealing is uh is automated right like

00:34:46

Tomasz Tunguz

A you know, 100 years ago in America there were 4 million human dishwashers people who watch dishes for restaurants.

00:34:51

Adel Nehme

Wow, okay.

00:34:51

Tomasz Tunguz

And then the the first robot introduced in most people's houses was a was a dishwasher. And now we don't, you know, the the chores that we give to our children is loading the dishwasher instead of washing the dishes. I think that's that's a very good analogy for future work in White Collar work.

00:35:07

Adel Nehme

Yeah, that's that's very very fast. And I'll let you hear also. React how do you see, you know, 1 spine contains the biggest part of the pie as much as possible. How do you see the view? Uh the future of work here?

00:35:20

Edo Liberty

Look, I mean.

00:35:22

Edo Liberty

I think we are producing great tools. Um, there are um,

00:35:31

Edo Liberty

I think we're very far from any model or set of models or agents completely replacing somebody's job. I think that's that's

00:35:42

Edo Liberty

That's very unlikely unless your job is like so many all and and just whatever like but except for extreme cases. Okay, all the lawyers and doctors and and accountants and analysts and this and that the they're not going away. What they are getting is fantastic new tools that make them, uh, work more efficiently more correctly, you know, uh offload the grunt work that maybe they they don't want to do.

00:35:44

Adel Nehme

Mhm.

00:35:51

Adel Nehme

Mhm.

00:36:12

Edo Liberty

uh

00:36:12

Edo Liberty

And that that's going to happen. I mean, I'm I'm I'm 100% sure of that and and uh history shows that when you have something like this that actually improves overall product productivity for society which ends up being good for everyone.

00:36:27

Adel Nehme

Mhm. Okay, great. And then with productivity, you know, when the Paradigm changes for productivity generating Technologies, you mentioned this Thomas, uh, the the shores 1 used to give to their kids was washing the dishes. So it's slowing the dishwasher that skill is sh has changed right the scale from washing dishes to loading dishwasher. What do you think the skills needed? Like, what would your advice be for individuals today to build the skills needed to kind of uh adapt to an Era where generative AI is widely adopted so Nick I'll start with you on that question.

00:37:01

Nick Elprin

Yeah, well, I think I mean building on what tamas and either both said.

00:37:08

Nick Elprin

um

00:37:10

Nick Elprin

If if G plays out the way everyone's expectations. Hope it will it's going to be leveraged for people. It's not going to replace people and so in any given industry or any given job someone has today.

00:37:25

Nick Elprin

There are a set of things that can be delegated away. But the there's going to be a Core Essence of that job. That is the

00:37:33

Nick Elprin

Insight part or the creative part, um or the architecture part that is going to be that is going to remain um on delegable to to 1 of these AI systems. And so, you know, I think yes, if you're a software engineer, then you don't need to remember the order of the arguments to a function call, but you do need to be able to

00:37:57

Nick Elprin

understand a customer requirement and and based on that understanding design a software architecture that appropriately reflects the abstractions you need to to model the customer's problem domain and you know that so so the the the conceptual or more in in Insight based parts of any job are going to be increasingly important, you know, if you're a if you're a a graphic artist, um, you'll still need to be able to have a picture of something in your head that

00:38:03

Adel Nehme

Mhm.

00:38:31

Nick Elprin

1 and then you can you can interact with you can coach you can instruct um an image generation system to to create that but it'll be it'll be getting a computer to create what you are picturing in your mind's eye. Um, so maybe the the technique of how do I use Photoshop to you know to to um to compose this thing? I'm picturing maybe that will will go away.

00:38:46

Adel Nehme

Mhm.

00:38:54

Nick Elprin

Um and and you know and just combining that thought with you know, your your question a minute ago about what this all means for the future of work.

00:39:03

Nick Elprin

the interesting opport the interesting potential Dynamic I am thinking about is

00:39:10

Nick Elprin

um, what will this mean for a whole wave of disruption that flows across every Services industry. So if you're a consulting firm, if you're a graphic design firm, if you're if you're a business fundamentally a Services business, I think that you are at risk over the next

00:39:25

Nick Elprin

You know 5 years of being disrupted by a much more efficient much more scalable much more High leverage model of executing that business.

00:39:30

Adel Nehme

Mhm.

00:39:36

Adel Nehme

Okay, that's and Tom. I'll let you also react what do you think the future skills to to adapt our

00:39:46

Tomasz Tunguz

Well, I remember when all the search engines came out. Alright Google meta web Yahoo, Alta Vista and you needed to figure out exactly how to use them. Well and the people who could use them. Well were a lot more productive at work.

00:39:54

Tomasz Tunguz

Um, and so that's where we are. So I think prompt engineering is probably the skill of the day today.

00:39:59

Tomasz Tunguz

Uh, because if you can do it really well, you can produce a blog post. You can produce marketing copy. You can produce images you can produce code.

00:39:59

Adel Nehme

Mhm.

00:40:06

Tomasz Tunguz

And those marginal benefits if you're 50% more productive than a co-worker or a competitor you will be promoted and uh, that's sort of the name of the game. Right? I think all software is sold because it's a promotion and in a different form somebody buys a software and champions it because they can be promoted. So I think prompt engineering is the thing that or the skill that is the most broadly applicable. There was this awesome study of I think BCG or McKenzie 1 of the big Consulting companies studied the way that

00:40:33

Adel Nehme

Yeah, and yep.

00:40:35

Tomasz Tunguz

yeah, it was called Centos and cyborgs and 1 of the most interesting use cases there was

00:40:40

Tomasz Tunguz

asking the AI to simulate a potential software buyer and

00:40:44

Tomasz Tunguz

be the um

00:40:46

Tomasz Tunguz

be the foil in a conversation and we're seeing this in education too where uh students are creating their own personalities to talk about how to solve a calculus problem or chemistry problems that

00:40:58

Tomasz Tunguz

so I think that's that's

00:40:59

Tomasz Tunguz

yeah, that's probably the broadest and most applicable use case today.

00:41:02

Adel Nehme

Okay, great. And then either I'll let you finish us off with you know, how do you view the the state of skills in an era where adoption is has matured.

00:41:14

Edo Liberty

um

00:41:15

Edo Liberty

It's very hard to it's very hard to to to know to be honest. And I I might not have the the best, uh, like a very, uh,

00:41:23

Edo Liberty

Strongly held opinion on exactly. What's that's going to look like I remember also when search engines came out, like people imagine those sorts of professions that ended up not existing and

00:41:36

Edo Liberty

uh and so on so

00:41:38

Edo Liberty

I think prophecy was is on on this topic is is a bit hard. I think it's it's obvious though that

00:41:45

Edo Liberty

uh

00:41:46

Edo Liberty

this is a like a set of very powerful tools are going to be produced and

00:41:52

Edo Liberty

professionals who don't Master those set of tools are going to be left behind, uh, you know, if you're you know,

00:42:00

Edo Liberty

Fewer that lawyer that rejected the use of a PC and kept using your you know, you know type and whatnot. Yeah, you you didn't you didn't uh stick around for a long time.

00:42:13

Adel Nehme

I couldn't agree more and I think we have time for 1 question from the audience. I'm going to take this 1 because everyone everyone here hinted at the uh, you know.

00:42:26

Adel Nehme

Potential Improvement of agentic systems in the future. So maybe I'll ask was looking for it.

00:42:37

Adel Nehme

Do you see there? Is this do you think that we might see agentic approaches increasing in effective in Effectiveness to the point that it reaches the level of predictive? So for example, can you reach a point which genetic systems are so good that you can predict with accurately that they will work. Well most of the time are we headed to that type of future because you know Tom Tom as you mentioned the compounding error rate across the different steps of the agentic workflow. I'd love to see where you think agent systems are headed in the near future. So let's start with you David Tom.

00:43:09

Tomasz Tunguz

yeah, I think uh, we okay, so I think let's

00:43:14

Tomasz Tunguz

We'll put it a different way.

00:43:16

Tomasz Tunguz

I think we'll get to a place where we will Brute Force innovation in other words like uh, we'll kick off massively parallel jobs that are trying to figure out what is the next Generation chemical compound and it's not that the systems will predict its that we will just throw so many computers out it that it's some point in time. They will solve they will find a solution and so um and we'll just, you know, it'll discover huge Solutions space and 1 of the answers will be there. I I don't believe computers can predict very effectively. Um, and if you look at classical machine learning, they've had a I mean they've had a really hard time they can predict things that are very stable and um and repetitive and cyclical and Google made the Google really beautiful business model based on very predictable patterns, but for things like core Innovation or creation that is not just recombination.

00:43:37

Adel Nehme

Okay.

00:43:56

Adel Nehme

Mhm.

00:44:08

Adel Nehme

Mhm.

00:44:10

Tomasz Tunguz

Is just not the right technology to solve that.

00:44:13

Adel Nehme

Okay, Nick. I see you're nodding in agreement. I'd love to see your thoughts here as well.

00:44:18

Nick Elprin

Oh, yeah, I was just gonna agree with tamash about the um, you know brute forcing Innovation. There's a um a hedge fund we work with that's doing that exact thing using Genai to come up with candidate investment algorithms. And then of course it's easy to or not, you know, you can you can test and check each 1 um, and their their point of view is look if we generate 10,000 and 900 or you know, 99999 or bad ideas, but we get 1 good 1 that's worth it. Um, but I think I think the that Bruce Force Innovation you only get the ROI for these areas where the innovation has a big payoff and you can kind of justify the investment to to search a big space because both yeah the vast majority of the ideas would be bad.

00:44:36

Adel Nehme

hmm

00:45:00

Adel Nehme

Yeah.

00:45:04

Adel Nehme

Either I'll let you finish this off. How do you how do you see a genetic systems evolving?

00:45:11

Edo Liberty

um

00:45:13

Edo Liberty

Well, I'm I'm both in support and violent disagreement, uh with the large. I mean, I think brute forcing doesn't work, uh exponents tend to be uh, exponential. Uh, and anybody who wrote like 5 nested for Loop knows that it gets pretty terrible pretty quickly.

00:45:33

Edo Liberty

um

00:45:35

Edo Liberty

and so

00:45:38

Edo Liberty

um, yeah, I I don't think this is solved by just throwing more Hardware at it for the love of God. We've been throwing literally billions of dollars billions of dollars worth of energy on this thing and it's it's you know, uh, we didn't solve everything so I don't think that's the way to go. I I do think that some Innovation comes from those systems that then agentic systems can actually discover interesting things because humans also can produce exponential amount of compute in in our own heads and the people who do research are also, you know energy bound

00:46:14

Edo Liberty

and so

00:46:14

Edo Liberty

You know whether we do this in Brute Force, I don't think so, whether it actually produces interesting insights and Innovation, I think 100%

00:46:24

Adel Nehme

Okay. I think this is a great place to End chat chat. I want to make sure everyone send as much love as possible to our speakers today. Thank you so much so much. Thank you so much Edo, Nick, especially Edo and Nick are both on vacation in Greece and Italy respectively. So I really appreciate them making time out of their uh, you know, uh pressure vacation time as Leaders of their organization. I really really appreciate your time and thank you so much for everyone who attended and again you a round of applause you speak speakers and see you in the last session on our our closing session. Thank you all so much.