No-Code LLMs In Practice with Birago Jones & Karthik Dinakar, CEO & CTO at Pienso

Richie Cotton: Hi, Birago and Karthik. Welcome to the show.

Birago: Great to meet you. Thank you so much for having us.

Karthik: Nice to meet you, Rishi.

Richie Cotton: Excellent. So, to begin with, I was thinking that most generative AI applications that are developed by either machine learning teams or engineering teams, and these are people who think in code. So, why do you need no code AI?

Birago: Subject matter experts are the people that understand the data best. They're the ones that should be training the model. So you shouldn't have, a machine learning person toiling over a screen of the limited list of words and then trying to explain that back to the person who actually understands the data best.

So you should just give the agency and the empowerment to the person who understands the data. And then they can, at their leisure, create the best models possible for their own use case and situations.

Richie Cotton: Okay, so I like that if you know the database, you can be a domain expert and you should be the person who's trying to reason about what data the AI needs. Thanks. Okay I'd like to know some examples of these domain experts who might want to work with no code AI. Karthik, do you want to take this?

Karthik: Imagine you are in the newsroom, maybe in the BBC or any other media company, and you're an editor trying to understand news feeds from other organizations. You're working on one particular story. Somebody else's story may not be the kind of story that you would want to write. So one example would be If there is some expert within, say, a news broadcaster that specializes in one area, then the dashboard for this person should be highly customized and personalized by their own subject matter expertise, which is pretty nuanced.

That would be one example. Cardiologists who interpret clinical trials and can dig through the content of a research article and can see what is good or not good. They should be in charge of training their own models to analyze cardiology work. And another example, which I think is interesting is let's say you're an analyst somewhere in the government.

and you're trying to see how you can keep track of propaganda and narratives. And I think the old joke is one man's propaganda is another man's point of view. And so, you would want a group of machine learning models that analyze coming in from various sources according to how you see the world.

And those would be areas where it's really very difficult to get a machine learning engineer. to somehow get a dump of knowledge or a transfer of nuance from a subject matter expert. But the value is actually in the nuance. And if you cut the middleman out and amplify how nuance can be embedded effortlessly and seamlessly by the subject matter expert, then that's a win win.

And that's what we are really passionate about. Interactive deep learning. by embedding subject matter expertise who may or may not know how to write code.

Richie Cotton: Okay, that's very cool and lots of great examples there. I think particularly with the newsroom example and the propaganda example there's so much subtlety in the language used that you really do need that domain expertise just to make sure that language generated by the AI correct.

And actually, the newsroom example is particularly interesting because I hadn't really thought about this before, but if you're a politics editor versus a sports editor versus a finance editor, there's lots of very different language being used there from just one team to the next.

Birago: When Karthik and I were at MIT a long time ago, we had a class project. We were trying to support using machine learning to mitigate some of the problems of social networks with teams. And one of the things that we found is that teen vernacular that's a nuanced language. It's a nuanced nomenclature.

And all of the language models at the time were based on the Wall Street Journal. And as you can imagine, teens don't speak in the language of the Wall Street Journal. And so, While we're cool, we still needed to go find some high schoolers from Cambridge High School to be able to help us with generating the most appropriate model for that use case.

And what we found is from our research was that that subject matter expertise That nuance that Karthik was talking about doesn't translate so easily. It's pretty lossy when you start to shift between domains. And so bad means good, good means interesting, interesting means not interested at all. That kind of disambiguation is best sorted out by human because algorithms will always find patterns and not every pattern is germane.

to the use case.

Richie Cotton: Absolutely. I have to say, every time I hear teenagers speak, I feel like I'm getting older because like Like, so much nonsense from Jen Alsler. Not to uh, I mean, I guess every generation has their own uh, slang as teenagers, but yeah it's getting increasingly difficult to understand as I get older.

So, some great examples there. I'd love to hear more of these industry use cases. Borago, do you have any more examples of different industries that are making use of no code AI?

Birago: Yeah, so we've got some great partners with an incline customers with Sky UK. their customer insights group is running all the customer calls through the models that they create. These analysts are experts in Customer service. And as you know, sky has great customer service. We're really happy to be able to support sky with that those efforts.

But those models are being created by them. They're not created by us. The nuance of customers success within the sky. It's slightly different than, say, Comcast or, Vodafone or any company. And so you really need to have that expertise there to be able to sort out what kinds of questions are you looking for?

What, are the parameters for the use case. And then once you have that, then the frequency of how you update your models, these are all things that only an expert wouldn't really have understanding about. Now it's just some generalized model.

Richie Cotton: Okay, so yeah, I like this idea that it's about the frequency that you update stuff as well, like there is a lot of subtlety in just how you go about implementing this sort of thing. Okay, so maybe let's get into the implementation stuff. So just at a high level, what are the steps involved in creating a custom LLM?

Karthik, sounds like a technical question. Do you want to take this one?

Karthik: Yeah, and the first problem most people and organizations have is they don't know where their data lies. And when they do, it's pretty difficult to fetch the data, massage it, and prepare it in a format that's feedable into an architecture that can, fine tune a large language model, So the first step usually is just find out where your data is if you do know where your data is get it into a format or a state where it's ingestible by a large language model building algorithm. And when you do, the end user interacts iteratively with the Pianzo interface. and provides it input by clicking, by giving it feedback in various different ways, not by writing code.

And the design, which is very important, the design of the interaction is very important. The user provides nuances and all those nuances are captured. Those Nuance that are captured are used to fine tune, train, and then what we say, sculpt the large language model. And this happens in an iterative cycle.

So with more iterations and more cycles, the model starts to behave more and more like how the end user would like it to be, and then they take it through a series of evaluations and then make a decision. I think this is good enough for me right now for it to be deployed to analyze something in production for live data.

And what's really important is it's, it's, It should be very obvious to people that there's a lot of machine learning glitter in organizations because it's difficult to pick things up from some point. But if you structure it in a way that makes it easy to reuse, I can just go to your exercise, Richie, and then say in iteration three or step three, Richie sculpted the model in this way, but I would like to take it in a different way.

And I can just start with that as the starting point. So you encourage reuse and When you do that, you allow people to get their hands dirty, build a large number of models, evaluate them, do A B testing on their own, and then deploy it. And giving this power to people really dramatically reduces the time it takes for them to quickly deploy models, which is very important whenever there's high volume data, right?

For instance, if there's a war happening, for example, the Ukraine Russia war. You are an analyst somewhere in some government intelligence agency trying to keep tabs of propaganda coming from that part of the world. And all of a sudden the narrative that the Wagner group is patriotic has shifted to the Wagner group is now treacherous.

And traits you would want a model to be fine tuned really quickly for you to respond to this. changing event. There's no time to delegate that to a software engineer who can go work on their scrums and then come back with, is this good or bad? And so, it's almost like an Adobe Photoshop, if you will, where users can do a variety of things on the central artifact, which is images.

Maybe you just want to convert it to a black and white image. Maybe you need to apply other filters. We want to do the same thing for language datasets and not have people be too bothered or burdened by some of these technical jargon concepts.

Richie Cotton: Okay that's very interesting, the idea of applying this to sort of a real world situation. at news events because you do need changes to the model then very quickly. Okay, so you mentioned this idea of, providing nuance and sculpting the model. Can you just tell me a bit more about how that works?

Because I think a lot, a lot of these, Germ2BI models, the large language models, you have like the opportunity to say thumbs up or thumbs down on did it generate something well or not. Is there more to it than that then? What does the user have to do to do this sculpting process?

Karthik: Well, the process, the way it unfolds on our platform is first, in a completely unsupervised way, the untrained, raw, Foundation model is going to give you an analysis of what it sees in your data, and you're going to go in there and give it some feedback, and that feedback would be to tell the model what is important, and you would do this in plain language, in natural language, you would write this out.

You would also do the upvoting and downvoting on certain kinds of insights that the model is surfacing. Yes, but not a whole lot. And then you would also give it some feedback to make it more narrow, more specialized, or more broader. For instance, if the Wagner group for, you know, just to pick off from the previous example, is a core area of focus for you, you would You're interested in not just the Wagner group just being mentioned, but the narrative.

Is it being portrayed as an anti national entity, or is it being portrayed as a patriotic entity, And that kind of subtlety and nuance. can be provided by either selecting those narratives directly from the data set, right, which the model or algorithm can surface, or you can tell it in natural language that that's what you want it to do.

And it'll take that feedback and re estimate the model behind the scenes. And in the next iteration, you will see the result of the feedback that you've given in the previous iteration. So if you want to make any correction, you want to refine it further, you want to add new kinds of topics.

Maybe there are different narratives for the Wagner group, not just one, and you would see a need to have separate ones. You would do all of these in the next one. And it's a bit like taking Play Doh and sculpting it repeatedly, but a highly structured and well defined way. And it's usually good to take breaks of five or ten minutes in between, just to sit back and reflect a bit, you know?

 Reflection is always good. Is this what I really want my model to do? That's the kind of thing that you would be able to do.

Richie Cotton: All right, nice. Yeah, certainly taking time for reflection is often a good thing and sometimes, you're in a rush, you forget to do that sort of thing, so that's good advice. But I like that. It's really about making very subtle, small changes to things. It's not about, hey, have a big project to go out and like completely retrain a model.

It's like, let's just make a small tweak to make sure that the language is right for this particular use case.

Karthik: Can I say one thing on this point, Ritchie, that when it comes to nuance, Ritchie's nuance is very different than Birago's nuance, might be very different than my nuance, And so if you have a dashboard, Ritchie has a dashboard, that's analyzing new incoming data based on models you have trained.

Your dashboard may look very different from what Virago's would be. It would look very different from what mine would be. And we want to encourage that. In a way, what we are seeing is when you have domain expertise within organizations, a lot of it is very tacit. Right? It cannot be articulated or captured in structured format, So it's very difficult to transfer that nuance from one person to the other. And usually when it's done, it's probably done in a very lossy way too. When you have Richie's own library of models, but I know the cookbook of how each of those were created by you, I can actually go and trace every single step that you took, and you can do the same thing for me. if it's a bit reductive. at least to some degree, captures nuance in a structured way. And nuance in this context is organizational knowledge. And I don't think that's valued enough in organizations. I think there's so much conversation around AGI and that large language models can do everything. The real prize, according to us and our belief, is How do you amplify the nuance that's in people's head?

That's very difficult to articulate, and it's tacit.

Richie Cotton: Okay, so really, it's less about or any kind of artificial intelligence replacing humans. It's more about trying to get some sort of AI representation of like what people are actually thinking.

Birago: That's correct.

Richie Cotton: it seems like the no code aspect of this is going to have some pretty big implications for changes to workflow and how you go about working with this sort of stuff.

Borago, can you talk a bit about like, how does the no code aspect change things?

Birago: Well, as Karthik just outlined, it's the structure that work structure that you start thinking about. Number one pretty sure that for some listeners, you're thinking, hey, is this going to eliminate my data science job? The answer is no. There's plenty of things that data science teams need to work on.

And you can think of Pienso as a platform that allows a data science team to create the best models by utilizing the subject matter experts within their enterprise who usually don't get. seen or shown any love or any admiration towards a data science project until after it's been deployed. And then when it's been deployed, you have subject matter experts coming around saying, okay can you tweak the output so that it best matches my intuition?

That's actually not taking action based on data. And so for Pienso, what it means is that the subject matter expert is at the forefront of working on the model through the process that Karthik just talked about. It is all no code in that sense, because it's a structured way of codifying the subject matter expert's nuance into something that can be turned into a variable and then refitted onto model.

So in other words, user's intent, their expertise, it gets imbued into the model, but for the data scientist, since you're not the expert, then this is probably not, there's no thing for you to do at this moment. So our platform allows that agency and empowerment to happen. And it's a user interface that allows, it's very, It's not simple because it's a complex kind of thought and reflection that comes goes into it, but the interface itself, it's pretty simple, and it makes sure that the user can walk through the steps of generating a good model and then have a space on our platform to be able to test it out with new data, test their assumptions, and if there's doesn't work, they can go back and continue working from that point on.

In the end, what everyone wants is they want their expertise to be able to inform a model on how the output is going to generate insights. And then those insights can then be acted on and become actionable. There's that old adage of garbage in, garbage out. Well, a bad model is going to produce bad insights or a model that just gives you 60 percent of the answer that you're looking for with no nuance.

Is that really worth time and energy you put into deploying it? Probably not. With Pienso, you're able to do that to, one, your heart's content, then deploy. Then enjoy the insights, then take some measured action on those insights. And it's all enabled through a no code environment for a subject matter expert.

Richie Cotton: that's very interesting, that last point about, how if your model is only right some of the time, then it's maybe not worth deploying. I think particularly like the generative AI, it's just, it's a lot more expensive than traditional software. So it has to be noticeably better than something else in order to be worth deploying.

Karthik: Absolutely. I think in 2018, Richie, there was a stat, I think, from one of the organizations actually a lot of organizations, that only around 20 percent of all machine learning projects in enterprises, large enterprises, were converting from POC to production. We were at an event recently where we met some of the big players in the Gen AI space.

I'm not going to take names. The common lamentation there was it's less than 10 percent of all Gen AI projects are shifting from POC to production. And it's pretty instructive, I think, because One points, everybody is now talking about how the GPU, which is the mainstay for a driver for a lot of this AI revolution that we're seeing now.

It costs 2.3 times the cost of A GPU to 2.6 to maintain it inside a data center for a year. And so I think Andreessen Horowitz came out with an analysis recently that showed a hundred billion has been spent, or probably a little more than that, on GPUs in 2023 as capital expenditure. Even if you amortize that cost over five to ten years to maintain it inside data centers, there is no way that there'll be 230 to 260 billion dollars in revenue.

So there's going to be a huge gap. And it kind of begs the question, why is so much GPU power being spent? Do we really need massive models for enterprises? Do we really think these ginormous large language models are the workhorse for the enterprise? And our answer for that is no.

If the nature of your problem is to Do you need a hammer or would you like a crane to be parked outside your home? We just think the Hammer, which is tailor made, targeted, and focused, is probably a lot better, costs a lot less is environmentally friendly, and has all the other things, hopefully, that, you know, we were talking about earlier, things like Nuance, is a much better, realistic, and practical thing to do.

Then at the cost of being maybe a little provocative, put one's money in. in the AGI bucket. It's very hard to define and for the enterprise, do you really need such massive models?

Birago: Yeah, you know, Pienso is about providing business value to our customers. And so if they don't see business value, then what's the point? And that's why the agency, the empowerment, it goes back to the subject matter expert, the person who understands the data best, meaning who is the person who has the question, nuanced question that needs to be answered?

Put that person front and center at the modeling process, and then we have a much better chance of having a, more likely, valuable outcome at the end. And if you don't do that, you just faith based may not get you the results that you're looking for.

Richie Cotton: Absolutely. agree there has to be some kind of business relevance, business value. Otherwise, yeah, everyone's wasting their time. Karthik, to go back to your analogy, you were talking about like a crane versus a hammer. And I guess, like, in this case, the crane is going to be the sort of cutting edge large language models, you know, the ones with hundreds of billions or trillions of parameters.

How about the smaller models? I know there are a lot of like, well, yeah, just much smaller and cheaper to run models which might be suitable for specific domains. Is that the sort of model that you consider customizing, or is the plan to customize, like, just a larger model?

Karthik: I know you're absolutely right. There are plenty of smaller models right now. A lot of open sourced one as well. Google released their Gemma 2 models 9 billion and 27 billion. They are tiny ants compared to the much larger ones out there. I won't take names, but I think everybody knows what these ginormous models are.

And there are models from Meta as well. The Lama 3 ones, both the 8 and the 70 billion. You can take one of these open source models, compress it, and fine tune it for very specific kinds of tasks, right? And this is the other important point, which is very interesting, I think. Because Virago and I can mild mend and read each other's minds.

I'm going to say what he expects me to say now anyway. We are old enough to remember the debates in the 90s when the conversation was around agents versus direct manipulation in machine learning and artificial intelligence in general, And how this relates to the size of the models, Richie, is. There's one group of people that for a long time in computer science are very driven by the concept of AGI, That you're going to have a human like conversation with an agent, and the agent is going to tell you what the right answer is. There's another group of people that I think Biago and I kind of identify with because of our training and because of the way we were mentored.

We were mentored from that school of thought. This school of thought is called direct manipulation. The idea here is, An assistive AI, together with an interactive interface that involves some kind of direct manipulation by the end user, eventually gets the end user to a desired outcome and not through an automatic suggestion, And if you are fond of direct manipulation, if you have that philosophy then You understand the nature of the beast in enterprises are essentially batch processing. People are trying to classify, summarize, extract. These are very specific natural language processing tasks.

And if you couple that with an interface you don't need an AGI like huge model.

Richie Cotton: So, in the second school of thought, it's going to be a mix of your, I guess, your generative AI, you've got other pieces of software, and then there's a human evolver as well, so it's mixing all these sort of different types there together.

Birago: I would just add, Karthik let's talk about our favorite metaphor. Because you, we use the hammer, but the hammer is really not our best metaphor. I think the cuisine art is our best metaphor.

Karthik: Yeah, we have a lot of metaphors so Richie, feel free to stop us if we are talking about too many metaphors. But at this point, it generally feels like there are so many people with these larger language models, There are at least five to six players, all with APIs that Depending on what you want to do are fairly good and comparable to each other, even if some of them are much better for certain kinds of things than others.

It's a bit like them as commodities. Large language model essentially as a commodity right now. So if Biago and I wanted to put together a Greek salad, we would go to, and this is an American example a Trader Joe's or a Whole Foods Market or your local farmer's store. You can get produce from anywhere.

So the large language model through an API is a bit like raw produce. have choices in where you can go and pick it up. But just because I have lettuce olives, and all of these raw materials with me doesn't mean that that's immediately consumable. in order to make it consumable, you have to bring it into the kitchen.

There's a certain art and science of, maybe chopping that. Maybe there's a cuisine art device that helps you very quickly chop the salad. You know, just prepare it. And you need a recipe book with somebody who's done this maybe a couple of times before. And then finally, hopefully it's edible.

At the minimum, delicious and ultimately like consumable, And so it's a good metaphor to think about large language models. Just because you have an API and you have access to any large language model doesn't mean that you can consume it directly. You need to prepare your data. You need to do flow engineering, prompt engineering.

These are very new technologies. The libraries are very new. They're not yet mature. And at the end of it, my Greek salad would taste maybe a little different than Richie's. Might taste a little different than Virago's, but that's quite okay, So, That's the metaphor that we like and so we think of ourselves as not replacing the cook, but more of a cuisine art device.

Richie Cotton: Alright, this is a very fancy Greek salad, you're buying your ingredients from Whole Foods, you're using your cuisine, not just chopping stuff with a knife.

Birago: Well, I mean, you can think, you think about this and that's like it's chopping with a machine versus chopping by hand, that's at scale. So, you know, if you're trying to analyze 700, 000 calls per day, that's scale, you need a machine. If you're just reading through one or two, just hand chop, or you just read by yourself, that's easy to do manually.

promise of AI is to be able to do human things at scale, not replace humans. that's kind of the path that we're on. We're just really trying to support enterprises and creating value from that, that exercise.

Richie Cotton: Okay, got you. So, once you start doing things at scale, like if you're trying to have the same conversation with a customer several thousand times, you can employ lots of people, but they can have the conversation in slightly different ways, and having AI is going to help standardize that in some way.

All right. So, Karthik, you mentioned data, and since this is DataFrame, we do like to talk about data. So, it does seem like, in general, LMs need a large amount of data to train on, and okay, maybe you're starting with something that's already trained, just fine tuning it. How much data do you need for this fine tuning process, and where does this typically come from?

Karthik: The beauty of large language models is how quickly they can be fine tuned on very little data, right? A lot of starting problem, Ritchie, that most organizations have. is even if they have massive volume of data in their data center, they don't know where to start, And so starting with a few instances, maybe a handful to say a thousand instances or documents, Just to begin with, just to start it as a seed process is good.

And then once you start using those instances to fine tune, obviously depending on are you trying to do it for classification, are you trying to do it for summarization, are you trying to extract something from a piece of text, you can take that model that's fine tuned on fewer instances And then kind of point it at a larger volume of data, which you haven't really looked at.

and take the output of that and refine it and iteratively use a larger chunk if you need to, And so the scale of data really depends on what are you trying to use it for, what do you want the end model to do, what end NLP tasks are you focused on, and With every organization, it's usually a different story, Hrithik.

Some people have a lot of data, and some people don't have that much data. But regardless, there are challenges with either. If you have too much, you don't know where to start, and you don't know how to select data. And if you have too little, you think that maybe you can't do anything. Right? And so, the platform is flexible and powerful enough to help a person navigate this. And if the system feels like it needs more data, it'll tell you, but it'll tell you very gently what it needs. For example, there's nothing in this data about the Wagner group. Can you tell me a little more about the Wagner group?

Richie Cotton: Okay that's quite useful to know what data you're missing. I think often it's a case of, well, I've got this data. Is this good data? I don't know. And so having some sort of feedback as to what data you might need to collect in the future, that sounds incredibly useful. Alright, so just on that note, like, I've noticed a lot of the announcements about new models, like from OpenAI, Anthropic Meta, Google, all these.

They've had these little footnotes in the announcements, like, oh, we've improved our data processing workflows, and this is like, giving us an increase in performance. So it does seem like how you go about processing the data does have some effect on the quality of the model. What do you think people who are creating their own custom LLMs need to know about data processing?

Karthik: it's more of an art and not exactly science. You know, you can read some of the research articles that are out there. But let's imagine somebody in an enterprise who's not technical, who hasn't written a line of Python code, who's asking the question that you are asking, H E, right?

Definitely the way the data is prepared prior to it being fed into a large language model, either for the purposes of pre training or fine tuning is important. And it usually depends on what the end task or purpose is. We make that simplified and easy for the end user. It's not always true for all downstream NLP tasks that you have the nuance to a very precise degree, in some downstream tasks like text extraction, for instance, you do need to be a lot more careful and refined and precise, but for things like summarization, for things like text classification, et cetera, you can do a lot with language just specifying what you want in plain natural language.

So that's an option as well. And usually most people are. They don't find it easy to articulate concept, but they are more able to point to something specific inside their data already and say, this is exactly what I meant, When I want a narrative for the Wagner group, this is an example of the narrative that I want.

And so in such instances, you can actually derive a lot You derive a lot of data processing insights from the way the instances have been selected by the end user as well. So maybe a more refined look at the Wagner group means extra pre processing that you may not have known before.

Richie Cotton: Okay, so, that's something that you don't really need to think about things in terms of, like, the traditional data processing steps, like, you're not, like, worrying about, like, Fivetran or Airflow or something like that, or messing with that, it's just, like, what does this data mean in a domain specific context?

Okay I think this brings us to a sort of Related idea of like, how do you design an application that a very technical subject, it's about like messing about with AI, but it's for non technical users. But Raghav, can you talk us through some of the sort of design challenges you've had there about making this accessible?

Birago: I think the The biggest challenge that people have is that, as Karthik was saying, is that you don't know, as an individual, how to articulate what it is that you're actually looking for. You know it when you see it, but you can't Describe it. And so that's one of the reasons why don't believe in chat based interfaces.

Number one people don't want to learn a new language or a new grammatical structure of how to explain something they didn't know how to explain anyway and prompt engineering. And when you think about what it is that people are actually looking for, it's usually at scale for enterprise. It's it's I'm looking for something, and if you find it, can you summarize it?

And if there's a lot of that mass, can you break it into smaller categories? Can you then give me a bullet point summary of the most important things from that category? How many of the things that I asked to look for, are there present? What are the intersecting ideas or concepts that are related to the thing that I'm asking for.

And if you break it down like that, well, then it's pretty structured. And you can think the way we've created our platform is to be able to allow the user to plug in the variables that they're looking for. There's a large data feed of thousands of articles coming in every single day. And I'm just interested in the articles that are related to electric vehicle batteries.

I don't need everything, just the ones about electric field like EV batteries. And then that same example, you can say, from that, I just really interested in any articles that are associated with batteries that are being developed in some country overseas. Or the minerals or the, that are being used for it.

That's it. But at scale, that's pretty difficult to do. And so our platform is, is creates structure around those ideas, around the things that people are trying to elicit. Not specifically the nuance, but just the general category of what needs to be organized, what can be categorized, and then can I get summarization?

Can I get some other downstream task? And then put that into. My favorite dashboard or back into some section of my data driven application that I use on a daily basis,

Karthik: This is a little provocative, but I'll say it anyway. I think the biggest challenge we've encountered, and this is my personal perspective Ritchie, remember we talked about agents versus direct manipulation and agents in this way would indicate a chat interface, just having a dialogue based mode of interacting with large language models. Our trauma to success journeys that we see in the Pienzo customer, like anyone with Gen AI trauma, meaning people who've gotten their hands dirty and have gotten burnt with the so called hype or promise or whatever is welcome in Pienzo. The challenge that we see is getting people to unlearn, maybe, the conditioning that they've been put through with a chat mode based way of interacting with a large language model.

That's not the only way to interact and derive value from a large language model. It's batch processing, it's at scale, it's to do with nuance, it's to do with repeatability, it needs to do with optimized cost optimized resources, both hardware as well as, engineering. And once people unlearn, oh wait, there's another way that I can interact with large language models and derive value.

It's usually very liberating, but it takes a while for people to get there. And another metaphor, it's a bit like being very young and wanting to live in, pick your favorite large city. Either in Asia, Europe, or North America, or anywhere in the world. It's great, and for a lot of people that really works, that's awesome.

But, you know, for a lot of people, after a while, they just realize, Maybe this is too much for my nervous system, but it's good that I got this out of my system when I was younger. So it it takes a bit of wrangling to get people to unlearn, but that's the primary challenge. I feel like people are just way too conditioned to chatbots when don't think that that's the enterprise workhorse large language models.

Birago: everybody wants the easy button. That's the, that's the challenge, but it's just that every enterprise engagement of AI is custom, all of them are custom. And so to think that there could be an open ended interface that just accepts everything and anything some things will, you know, we love to use it personally, but in terms of an organization trying to use it, It's not so easy 'cause it just doesn't scale the same way.

Richie Cotton: This is absolutely fascinating, because I think there's been a rush over the last few years to try and add a chat interface to absolutely every bit of software, like every company making software is like, oh, we should put a germ2vi chat in there. And I think of all the people working in the germ2vi space, I think you're probably the most negative about chat interfaces that I've come across, which is very interesting.

So, do you have a sense of like, When you might want to chat interface versus, I guess, the alternative is like a point and click interface, like your traditional software, is that what we're talking about? So when would you want one or the other? Do you have like a heuristic for when one is best or not?

Karthik: So, like Preago said On a personal basis, it's maybe useful. So for instance, I'm not a native English speaker, so if I have something very important to write and I want somebody to just proofread it, I can just ask a large language model. If in London or Hawaii or somewhere and I have an evening to spend but don't really know where to go or what to do, I can Just to, get some ideas going for creativity's sake, just talk to a large language model, right?

And so there are, users, maybe in a personal sense, for analyzing large volume of data at the enterprise to leverage the large language model's ability to do classification, summarization, data extraction, and to do it very reliably and in a nuanced way. That's not it doesn't afford itself naturally to a chatbot, And so, maybe from a B2C perspective chatbots are great just for personal use, and everybody's personal use is a little different. Maybe for poetry or creative use or, something that you just need to be done very quickly. workhorse for the enterprise, probably not the best.

Richie Cotton: Okay, so if you're doing the same kind of task over and over again, then having to type the same chat conversation over and over again is probably going to be an inefficient use of your time and better off just have a, button to click to say, okay, do this thing.

Birago: True. And one thing to note is that sometimes when you write the same prompt you get different answers. And so, you know, as an individual I can be frustrated or delighted that there's, there's other things to think about or consider, but at the enterprise level, do you really want consistency because the volume of data that you're going to analyze is so high.

1 percent error at 1 million data records is pretty high. me doing it, it's one.

Richie Cotton: Absolutely. So, yeah, having that inconsistency of results is one of the sort of, I guess, common fears around making use of generative AI. Another big one, perhaps the biggest one, is around data privacy, and particularly creating your own custom LLM and using a platform like yours. That's going to be a big fear.

So, how do you deal with data privacy issues for your customers? Thanks.

Karthik: So, Pienzo is an enterprise platform. We deploy it on whichever environment the enterprise customer wants it. It can be on the cloud or on bare metal. We don't do anything like multi tenancy, Ritchie. So, when an organization gets to the cloud, their own installation of Pienzo, it's in their environment.

We don't have access to their data, we don't want to see their data, we don't have access to their models, we don't want to see their models, and nobody else can touch it anyway. And all of the models that we use, we use optimized open sourced versions. And then, you know, the model libraries for our customers they sometimes, fine tune and refine models every single day.

They have their own library, but we don't want to see it. What we do want to do is incentivize all our users. to experiment, experiment, experiment. To get their hands dirty, train a lot of models, really think deeply and reflect about their nuance, The clarifying act of representing nuance becomes obvious over a period of time and a library of models then becomes an institutional asset, And because no one else has their data, and it's the proverbial hammer from our previous conversation. These are very small focused models, but there are so many of them in their library now. Believe that's a good way to go. And the other reason why privacy in an interesting way also ties with cost and throughput is also very interesting, Because these are smaller models, they're much faster, They can run on less expensive hardware, less expensive GPUs. And because they are small not tapping into any of the massive cloud API, large language models. And they don't need to. Unless you are trying to write a thesis and, you know, just engage in some creative, individual things where the tolerability for.

Variances, you know, high.

Richie Cotton: so in this case, not having access to customer data, that seems like very useful thing. I'm also making use of these open source, or at least, I guess, open weights models. So you've got a bit more transparency in what's going on there. All right, super. Okay, before we wrap up, what are you most excited about in the world of AI at the moment?

I think you both need to answer this. do you want to go first?

Birago: The more experimentation that happens, the better will all be. think it's probably every week there's a new device that comes out that has been embedded with AI that the next two weeks or three weeks it gets review bombed and then it's gone and then the new device shows up that's very similar.

I think that level of iteration is good. Because we'll get to a place where we're creators and the constituents for whom you're building these things for, there'll be a parody and people will be able to articulate what they want and don't want. As Karthik said, people really don't know. What they want because of the they've never had these levels of affordance of scale or affordance of efficiency or speed or so that's that's what we're really looking for.

I'm excited about how I could be used to support education. I think that's a great place for where there's gaps. AI can help funnel those resources towards those things. And then also, of course, just, you know, discovery of, medicines to be able to mitigate some of the health issues that we have, kind of circulate the world.

Richie Cotton: Lots of cool use cases there, yeah, I like the idea of empowering creators and helping out with, well, education is very dear to my heart, certainly working at Datagram, and then also, yeah, the healthcare issues are very very promising there. Karthik, what are you excited about? Uh,

Karthik: I'm really excited about where we are in history. If you look at the 1920s all the way up till the Second World War it was a, elongated period of great turmoil with two wars and, multiple depressions and so on, right? A lot of human suffering. There were very breathtaking advancements that were made in physics, fundamental physics, and also very curiously, at the same time, multiple revolutions in design.

For instance, the Bauhaus, other schools of thought came into being. And I think they really tried to simplify and make accessible to every The, disparate fields of art, design, manufacturing, so on and so forth, right? We seem to be in another era where sadly there is a lot of war and conflict happening.

It's pretty depressing. And just like we had physics, now it's gen AI. There's something very remarkable that's happening. What really excites me is there's so much opportunity to embed. design, maybe a revival of something like the Warhouse principle, Let's focus a lot on designing interactive interfaces making these language model capabilities more accessible, highlighting nuance.

And ultimately, though it's an AI revolution, I think it can shine a really nice spotlight on what we truly hold valuable as humans. which cannot be taken over by anybody, So, I'm very, very excited by what the design movement for Gen AI to make them more accessible is going to bring. And hopefully the conflicts will all end soon.

Richie Cotton: so, I'd say, yeah, it's only intermittently that Art and science take notice of each other and bring the two together. That's maybe one of the big promises of AI. So that's a very cool thing. And yes right there with you. I'm hoping that a lot of these big world conflicts end.

Quicker rather than later. Okay. Excellent. Finally do you have any advice just for businesses wanting to get started with making their own custom LLMs?

Karthik: Usually, giving any advice, feeling is All the advice that we received usually something different happened, Richie, where the advice was non applicable. So it's just hard to give advice. I guess the only thing I'll say is maybe it's important to do your own mistakes but not repeat somebody else's.

there's a lot of Gen AI trauma going around. And so if you get your hands dirty and you commit your own mistakes, but don't repeat somebody else's, that's a good starting point. 

Richie Cotton: All right. Super. So no advice, just try some stuff, make some mistakes. It's always good to get going and actually try doing things. That's the best way to learn. All right. Excellent. Well, yeah. Thank you so much for your time, Barogo. Thank you so much for your time, Karthik. That was a great conversation.

Karthik: Thank

Birago: you as well. Really appreciate it.