Building Diverse Data Teams with Tracy Daniels, Head of Insights and Analytics at Truist

Richie Cotton: Welcome to DataFramed. This is Richie. Better data culture is something that every organization aspires to, because of course you want all your colleagues to be happy and productive when working with data. However, it can be hard to even define what a good data culture looks like, never mind setting up all the people and process initiatives to get there.

That means we spend a lot of time on this show and another day to count content. Trying to provide practical advice on how to deal with those devils in the details to improve your data culture. There's one area of data culture, namely diversity and inclusion, where I realize I've sometimes previously been a bit glib and said, of course you want to get better at diversity and inclusion without providing the advice to help you get there. Today we're going to rectify that and get into the nitty gritty of how to improve that important aspect of data culture. Our guest is Tracy Daniels, the Head of Insights and Analytics at Truist.

Tracy has two decades of experience managing teams at banks, having previously worked at Merrill Lynch and Bank of America. In addition to her management experience, Tracy is also the executive sponsor of Truist's Enterprise Technology and Operations Diversity Council. I'm looking forward to learning from Tracy. Hi, Tracy. Thanks for coming on the show.

Tracy Daniels: Hi, Ritchie. Thank you for having me.

Richie Cotton: So I'd like to start off just by seeing if you have any success stories of companies where they've implemented diversity well, and it's had a big impact.

Tracy Daniels: Yeah, I mean, I think I would be remiss if I didn't start by talking a little bit about how we approach DEI here at Truist. I like to start with the fact that our company, like a lot of other companies, but maybe a little different and a little special. We start with our purpose, in terms of how we think about building better lives and communities.

And that means for the people that work for us, as well as the people that we serve in the community. And part of our values are actually about caring. And so when you start there, one of the goals that we just intrinsically have as an organization is we want to make sure that we are creating really uh, an inclusive environment and we're making the right.

Moves and changes in the way that we do our work in a way that we think really helps, again, our teammates in the community. And so we focus on, goals, just like everything in our business, right? We want to make sure that we have a really great return for our shareholders. Well, we also want to make sure that we have strong representation by women or people of color.

And so we've set goals to that end around increasing our representation both in terms of women and ethnically diverse populations by 15 to 20 percent respectively. And to do that, we have done certain things like we have a chief diversity officer. We have again, diversity goals and a framework that we implement.

And we have pushed to the business units as well the opportunity for them to sort of craft and mold this in a way that can play out appropriately for the line of business, and I'll talk a little bit about some of the things we do in enterprise tech and ops. The last thing that I would also say that we've been very sort of goal oriented around is also making sure that we're improving our supplier diversity.

So again, teammates, community, those that work with us, and there we've also set some goals about increasing representation and sourcing for our RFPs, et cetera. So all of that really speaks to kind of a framework, and I would say intentionality. About creating diverse and equitable and inclusive environments with outcomes that we think are important to our lines of business.

Richie Cotton: That's very cool. I do like the idea of it being a goal driven approach rather than just a sort of general, this would be nice to have and not really having a direction in terms of how you get there. So let's talk about that more before we get to that. Can you tell me about the flip side to this?

What happens if you don't do diversity and inclusion right? Do you have any examples of problems this has caused perhaps in other businesses or organizations?

Tracy Daniels: Yeah, I mean, and so I think on the positive, right, when you do it well, McKinsey has studies about increasing and outperforming your peers. And there's studies that show that, during COVID where there were higher representation of gender diversity there was more stability or a better response, let's say, like in the COVID pandemic.

So then, okay, well, that's good. But what happens if I don't do it? Well, I think you start to see things like products and services that maybe aren't as inclusive as they could or should be, or missing features, quite frankly, that would best serve your customers and clients. So, think, again you miss out as a company.

Again, in terms of being able to connect with your customers, I think that your organization misses out on the potential for really fantastic problem solvers and product developers, et cetera. And I think in these, this day and age, where there's truly a war on talent, and particularly data talent, right?

You don't want to overlook that talent, and you don't want to be overlooked by that talent because they don't see what they would expect to see in terms of a DEI program and or a representation in the ranks of your company.

Richie Cotton: That's interesting that it's really about making sure that you're not missing opportunities, but... It's happening on both the external side like, missing opportunities with your customers, but also in terms of talent as well internally. So, getting back to your point about setting goals around this I'm curious as to how companies can, align the diversity programs with other business metrics and make sure that the diversity program is aligned with their business goals. How do you think about that?

Tracy Daniels: Yeah, so as we mentioned it's, good business, lots of studies show that it's really something intrinsic with how you do business. And I like the fact that we have empowered our lines of businesses to also contribute to the type of programming that we. Create around diversity, equity and inclusion.

And so I think it can be multifactored as we've tried to approach it, right? It's who you hire. It's the products and services that you create. It's the way you interact with like your suppliers et cetera. And I think it can start with something very, basic around, we wanna increase the representation.

I think you wanna look at you know, not just within the general pool, but are you having a pull through effect? Are you promoting people as well into roles? Are you giving them really great, I think, stretch assignments or assignments on newer technology that allows them grow into the technology that's being leveraged.

And then I think it's also about tying it specifically to your goals and outcomes, right? So again, as you mentioned, it's not just a good thing to do. It's because you actually want to see something different show up in terms of how your business produces a product, service or experience.

Richie Cotton: before we get in any deeper into this because we want to talk about both diversity and inclusion. I think it's something where people often get a bit confused about what's the difference between the two. Do you just want to talk about what each of those things means just so everyone's on the same page?

Tracy Daniels: To me, diversity is really about presence and representation, right? From different groups. and perspectives and background. We talk about equity because we also want to think about fairness and justice. And then inclusion is really that environment that you create where are people feeling valued, heard, productive seen, and I think all three of those really work together.

One of the sayings that I really like, there's a woman by the name of Vernée Mays who was formerly the Chief Data Officer of Netflix. And she said diversity was like being invited to the party, whereas inclusion was big ass to dance. So that's the easy way I think about it in my mind.

And again, I think all of these work together for a really comprehensive approach that supports the outcomes that you're looking to drive.

Richie Cotton: Okay. I really do that as an alternative yeah showing up to the party and are you actually uh, I didn't

Tracy Daniels: I said that you never got necessarily asked to dance, you

Richie Cotton: I can do that. Yeah. I'm not sure many people want to see me dance, but like, it'd be nice to be asked. Yeah.

Tracy Daniels: There you go

Richie Cotton: Cool. All right. So, it feels at least in terms of diversity, one of the most important sort of elements from a business point of view is how we go about hiring. I'd like to talk A bit more about hiring strategy, and I guess this starts with the adverts, like how are you showing off, like, how are you attracting candidates or potential candidates? So what changes can you make to your job adverts to make them more appealing to a diverse audience?

Tracy Daniels: Yeah, so one, I think, where you go and where you place yourself, whether it's adverts or your, the people that you want to hire and so let's walk it through a little bit like early career, right, what schools and colleges are you going to? Are those schools and colleges diverse?

If not, do you have additional roles? Places that you would go, historically black colleges and universities, or universities where their body is fairly diverse, so that you can, cast as wide a net as possible. I love this idea of also how do you write your descriptions, your job descriptions.

There's been some work done around what are you asking for, how are you asking for a day in the life, versus, this very rigid, you must have this, and we know. Some populations, women versus men, might respond differently to certain ways that things are written, so I think that's also important.

Who you leverage to interview the panel is also important, right? And there's been studies that show the more diverse your panel can be, the more likely you are to convert folks into the hiring process as well. And then one that I also like to employ and I like to encourage folks to think about is your network, And if I have a very narrow network in terms of who I know or who I go to to ask questions or, hey, do you know a really great person? I can end up with a very sort of homogenous group of candidates that comes through. And my husband likes to, remind me that while I'm an introvert, not everybody likes to go make just general conversation with people and expand their network and, gather new friends.

But I do think it's important to be intentional about thinking about how do you put yourself out there, even if it's uncomfortable or unfamiliar is maybe a better word for it. So that the spaces and places that you you occupy are a little bit more diverse. And you can source from, that population when you're looking for.

New talent

Richie Cotton: that's really interesting. The idea that maybe you need to have diversity in your people's team or your HR team in order to increase diversity of the actual candidates who are going through the interview process perhaps even being sourced by recruiters.

Tracy Daniels: Or even the team, right? Like when we recruit them in, sometimes our HR department does a really great job sourcing diverse talent. And then we have a not so diverse panel that interviews them and is judging and deciding who actually comes in.

Richie Cotton: I want to pick up on something you said about changing the writing within your job adverts and how different people might respond differently. Do you have any examples of this? Like from maybe a... Data analyst or data scientist type job advert,

Tracy Daniels: Yeah. So rather than writing, something like 10 years of LLM models, right? I would challenge myself, do I really need that? Or do I need somebody that's maybe had more contemporary and maybe less time, with those types of models? If I'm going to list out all of these again, five years of this, three years of that, all these languages, can I talk more about what the job entails.

I want you to use these skills, and some of them will be technical, some of them might be interpersonal. You will work with these types of people or these types of problems. that may resonate with a variety of people. And I think about data as being a very multidisciplinary type of field, right?

And so I want to appeal to as many different types of people as possible to help solve these really tough problems that we have.

Richie Cotton: I have to say, if a job advert is just like a list of skills, then that sounds super boring. It's well, that's what you want from me. Do you not tell me what I'm getting out of this? What I'm going to be doing? And yeah, there are also terrible job adverts

Tracy Daniels: Yeah, and I'll tell you, even like my 27 year old male son would not respond to that these days, right? He's looking for like, tell me the adventure I'm going to go on and this horrible problem I'm going to solve and the amazing people I'm going to work with. He would actually respond to a very different kind of writing.

Richie Cotton: That's really interesting stuff. So, maybe one of the challenges with hiring when you think about diversity is you have to think about diversity of skills across your team. You've got to think about like diversity of personality and then. It's fine if you got lots of people applying for the job, but if the candidate pipeline is a bit thin, then you might start to think, well, okay, maybe we don't worry about demographic diversity here, and that's when you can run into problems.

So I'm just wondering, how do you prioritize all the different types of diversity when you're hiring?

Tracy Daniels: Yeah, that's a great question. I think it's, I think it's a hard one, right? The nerd in me wants to, oh, I'm sure there's a formula we can create to make this work, right? But it's not. I mean, I think you cast a wide net. You create a really you spend time creating an inclusive environment. So the people you bring in, regardless of their backgrounds, understand the values of the group, the company.

Understand that they are valued as part of the group and the company and the expectations of how you work and perform, how you I don't want to say rate each other, but how you learn to, work through. issues and solve problems and celebrate and recognize one another are well understood.

And I think that inclusive environment, right, helps to reinforce that the choice that they made to join you was the right one and a good one. So I I like to think that it's not one or the other. I'm not pitting a group against each other. I'm not it's better if you're black. It's better if you're disabled.

I want to create an environment where whoever you are and whatever your intersectionality is, you feel. Included in that this is a place you can really thrive and grow.

Richie Cotton: Okay, and I guess there's some sort of feedback mechanism there where if your existing employees are feeling included, then it's going to encourage them to try and hire more people, and that's going to sort of benefit your pipeline. Does that? Sound like it makes sense.

Tracy Daniels: Yeah, I think so. And I think as the environment becomes more diverse, right, then again, that network and the pools that you have access to also start to diversify. Again, it all starts with intentionality. It doesn't happen because of rainbows and unicorns. It happens because you've you've decided to be intentional.

You decided that this was an important element of how you build teams.

Richie Cotton: Okay. I'd love to talk a bit more about team building in a bit, but another question about hiring before we get to that. So, I think bias in hiring is maybe one of the sort of hidden problems that you need to overcome if you want a diverse team. So what sort of steps can a company take to increase awareness of the types of bias you might find in hiring?

Tracy Daniels: Yeah, I mean, again, I think you start with some of the things we've talked about before, which is looking at your practices looking at who's interviewing, looking at where you're going to get folks from. I love this idea as well as, making people aware. of where there could be bias and giving them tips and frameworks around how they overcome or mitigate biases in the practice.

Because I think if you ignore it and you want to pretend that it doesn't exist, you'll just repeat some of the same failings and miss out on the opportunity. It's more of a, Framework we've used with our data teams in terms of how we encourage them to really think about removing bias, not only in the hiring practice, but as part of our framework, we had Dr.

Brandy Ice Marshall come in and she talked about this peer principle a framework that she used for helping to mitigate bias, really, in data, but our, part of the premise that I love is, it starts with the P, who's actually participating, and it was interesting, we had teammates, data scientists and data analysts, et cetera, across the organization really appreciate like a framework and a concept for them to think about.

Because they, nobody was doing it on purpose or didn't assume that it was all perfect and. And good but helping with a framework for them to think about, okay, how do I get more participation and access on my team? And that being again, an intentional goal and thought that they were having.

Again, I think that awareness and literacy is really important to overcoming what the biases could be in the process itself.

Richie Cotton: I like the idea that there's a framework, sort of definite steps you can take in order to try and identify and reduce these biases. Maybe moving on to inclusion type stuff. So what do you think the most common problem areas are for inclusion? Like, when do you have this like diversity, but then people aren't being asked to dance as it were?

Tracy Daniels: Yeah, I think Again, I tend to be a little bit of a Pollyanna, right? So I don't think people come in and with bad intentions. I think if you're not... being intentional about it, there are just things that could happen, right? And so, I think sometimes it's just slowing ourselves down a little bit so that we can pay attention to what might be, our natural inclination.

So I think about things like is everybody being heard? in the room, because I'm an extrovert and I love to talk talk, are there moments where I need to be quiet or specifically call on somebody I'm not hearing from? and ask not really general questions, but poking on, did I hear from or hear the experience?

Is there a perspective we are not considering? To draw in more voices. and invite them, right, to actually participate in the conversation, in the debate, in the feature that might be being discussed. And again, I think you need all voices in the room to be able to do that in a really proactive way.

Richie Cotton: That's interesting, you mentioned the idea that if you're an extrovert, you need to learn to be quiet sometimes. I guess the flip side is true. If you're an introvert, you need to sort of force yourself to speak up sometimes.

Tracy Daniels: Yeah, that idea of being uncomfortable a little bit, right?

Richie Cotton: Yeah, I guess make yourself uncomfortable to be successful. Um, Are there any sort of less painful ways of doing that? Like, Do you have any tips for like how we can balance out the difference between extroverts and introverts being heard in a sort of maybe systematic way?

Tracy Daniels: Yeah, you know what we also try to make sure that there's a way to get feedback, right? One of the things I love about... technology these days in our society to some degree is there's lots of ways to give your opinion, right? Whether it's through technology, your star rating, or even a simple thing is just drop an email.

And I, and again, I try to make sure I'm very intentional about inviting people to do that. And I'll say things like, I know everybody doesn't just want to speak up necessarily in this meeting, or people may need time to process their thoughts, right? Feel free to drop me an email or have a way for folks to provide that email.

Whether it's anonymous or with their name or whatever, what have you. The other thing that I think is important in this context is to show people that you're listening to invite them to continue to do that, right? Because there's nothing worse than, I give you my opinion, it goes off into the ethers and I don't see anything that happens based on this feedback.

I've told you that I don't feel like we do enough to bring different people into some of the, more technical experiences, or I've told you I don't feel like I can talk about my same gender spouse, right? Okay, well, you need to act on it and be very transparent. that you're doing this because of feedback so that people know that you heard it and you're doing something.

You're not gonna be able to fix everything and react to everything, but I think it is important to show progress against what you're hearing from your teammates as well.

Richie Cotton: I really like that idea of having feedback mechanisms also having different forms of communication so if some people are more comfortable writing things and it's great if they're more comfortable having to chat with people you know you can voice to have a conversation so. related to this one thing that's like the bane of everyone's life at work is meetings and doing meetings.

Well, it's like the secret art that like everyone needs to get better at. So I'm just wondering, do you have any tips for making meetings more inclusive?

Tracy Daniels: Yeah, okay, so I'm gonna, it's gonna be what I say, not necessarily what I'm awesome at doing. Right, so Doug's again, I will sometimes get called out myself, right, on what is the meeting about. So I do try to make sure that I can give people the heads up about the nature of the meeting, maybe a couple of things that we're going to cover.

Again, so people can prepare, and then you're transparent about your intentions, but if it takes me a moment to process. I can maybe do some research or think about what my response is going to be. clear on what the expectations are coming out of it. Are we just going to talk and we're going to, brainstorm?

Or do we need to come out with a decision and how do we get there and drive to it? Again, I like this idea of inviting the quieter voices in the meeting to also participate, whether it's before, during, or after with some of the, feedback mechanisms we talked about. But given that, yes, our life is spent in endless meetings I think those are one of the things.

And then, again, I don't know that this has to do with inclusion, but maybe about effectiveness coming out of meetings. I like when meetings aren't done on the half hour or hour, meaning that, it's done at 20 after, if it's a 30 minute meeting or, 45 minutes after so that I've got time to process action, do the thing I said I would do before I move on to the next meeting.

I think that gives everybody an opportunity to really action and process what they've been through.

Richie Cotton: Absolutely. I have to say, I agree. In theory, all my meetings would have an agenda beforehand that everyone's seen, and yeah, it doesn't quite often look like that. So yeah, aspirational goals. All right. So, maybe let's talk about how you actually go about implementing a sort of diversity and inclusion or even DEI program.

So, It seems like getting some sort of executive buy in is going to be quite important because this needs to be an organization wide program. So how should you go about this?

Tracy Daniels: I think one of the things that's key is Like you said that by, you know, whether it comes top down or what, I do think it's important that somebody at the senior level is the champion in this driving this, right? We're very fortunate here at Truist. It is something that's truly embedded in our executive leadership team and we have a chief diversity officer, whose remit is to help us really grow and expand our goals and our frameworks, et cetera, and the programs that we have around um, DEAD. But it's not just one person's role and expectation. I think the other strength that we have and some of the best practices I've seen at other organizations is when you have a really strong and involved business resource group or whatever you want to call them, whether they're, created by whatever flavor of diversity, whether it's women or, people of color, LGBT differently abled individuals where they get to set the agenda, of what's important.

And how they see these items playing out. And then as I mentioned earlier, getting the lines of business involved. So we have, finance, enterprise technology, our wealth group, they have their own DEI organizations where they're really tailoring how they work through the objectives that we have as an organization and help make a difference in their lines of business.

So I think, again, the focus of the organization, the intent of the organization, leadership top down, but also I think really participation by as many of the teammates as possible makes a difference. And then the last thing I think is I think can also be secret sauce, let's call it, is when allies are really active in this framework, Meaning, I am not personally, physically differently able, but I involve myself in those organizations and learn hear the conversation and participate in things and solutions that we can do. I think that allyship is also really important to help further the goals.

Richie Cotton: I like this idea of having some different organizations representing different communities within wider business organization. I'm curious as to how like the human resources or people teams going to interact with these. Is this something that is driven by them or is this something they're involved in separately?

I'm

Tracy Daniels: That's collaboration. I would say the framework, the overall framework that we have here at Truist is really set by the DEI organization, so Dominica Grimm and her team, and then we take it within the lines of business and we come up with a roadmap for how we're going to implement it.

So within enterprise technology, what does that mean? I may be going to, I may have a different way of improving my hiring, right? I may be looking at different, Conferences in different schools. I may be looking at mentorship programs within my organization that are specifically tailored to technologists.

We actually created last year a women in technology group, That was in addition to the DEI work that we were doing more broadly across enterprise technology. I've seen groups out there for, black women in data as there tends to be a dearth of women of color, let's say in the field.

So, I don't think it has to be one size fits all or just one versus the other. But I do think that there's, what's important for us here at Truist is that there's collaboration. So that I'm pulling through the goals of the enterprise and really just tailoring it for how technology is going to do it.

And then asking, questions about what specifically do I need to do as part of the data organization to continue to support those goals as well.

Richie Cotton: So if you're still running a DEI program, then at some point someone's going to want to say, well, is this a success or not? So how do you go about measuring the success? I know you talked about having some goals beforehand, so maybe you can just elaborate on what counts as success, yeah.

Tracy Daniels: Yeah, so for overall, from a truest perspective, I talked about, increases in our representation. So, looking for increase in our female representation or ethnically diverse representation. I also look at some I don't want to say softer metrics, but looking at the type of programming that we do.

So one of the things I think about in my role for data is I'm always wanting to build a data culture. Well, part of that data culture includes conversations around EI and or, removing bias from, the field or at least mitigating for it and thinking through it. And so we look at programming and sometimes I'll point to I did a three day program, and I trained educated, a thousand people on these topics.

And, I might have gotten feedback, good, bad, or indifferent on that. I might also look for And if I was super sophisticated and everything was inventoried and libraries galore, right? Do we have code or ways that we are cataloging as part of how we do our data work specifically? So taking it from maybe something that is just business specific to something technically I'd actually want to see where I'm 50, 60 percent of our X models are going through some sort of models for detection of bias, et cetera.

So I think you can start small, and as you get more mature, and maybe even more, a little bit more technologically advanced, you can also throw in some additional ways to measure your improvement or even maturity, right, on the topic.

Richie Cotton: That's actually a fast thing. I've not really thought of it before, but it's not just about like diversity of like employees. You just don't, figure out, I don't know if a genie coefficient or something of like your employees by team, but it's also in terms of the models you're creating, particularly like in data and machine learning, it's that's part of like responsible machine learning is figuring out, have you got bias there and just implementing that systematically.

Interesting. So. How do you ensure like the accountability of any managers or sort of team players who are involved in this sort of program

Tracy Daniels: So what gets measured, get managed, right? So there are metrics that we look at. Are we getting better? Are we holding flat? And if we are where we expect to be, okay, what else do you need to look at and start thinking through? The training and education and the conversations we talked about.

So there's training that the organization gets to avoid the things we want to avoid. And then again, maybe slightly softer is this idea of What are the conversations that we're also having, And do people feel like their voice is heard? So we have an employee engagement survey and in there we'll ask questions that are also focused on DEI inclusion and about how people feel about how they're treated within the organization.

So I think there's a variety of metrics that you can look at to really show that you're making the improvement or. If you are not, or you're moving backwards, right, something that lets you know that there's probably work to be done and an opportunity for focus.

Richie Cotton: while we're on the subject of measuring things? I remember there was an experiment with Twitter a few years ago. I can't remember which CEO was on there. I had quite a few, but they were talking about having diversity, measuring it at a company level in terms of Diversity of employees, but then they would have individual teams are saying, okay, well, I'd like a whole team of just women or something.

And the team level diversity didn't matter as much as the whole company level. I'm just wondering how you think about that.

Tracy Daniels: Yeah, I mean, I think there's much slicing and dicing, right, that you can do. And again, I don't want to say it this way, but I'll say it, you're not reacting to everything, but I think you're looking for what's important for your organization. I do things that are important to me, I'll just say it this way, right, as a woman and a person of color.

Levels, So if all of your diversity is in the most junior places of your organization and not represented in senior management on your board, et cetera, I think there's an opportunity, If all of your diverse talent is in support functions, and not in revenue or code producing functions.

I think that's another area to look at. So I do think you want to slice and dice the data for things like equity, right? Because how that could play out is, again, your products and services aren't as fulsome as they could be. People making the most money. it doesn't represent a huge diverse representation of your population, be it, the function or the city or wherever you are.

So I think you do want to look at various dimensions of the data.

Richie Cotton: I suppose related to this it seems like this can be a tricky area for data departments because if you've got a customer facing role, it's a pretty easy to pitch to say, okay, if we increase the diversity of our account executives, you've got People who look just like the customers you can probably see that pretty quickly in terms of the metrics with data diversity and data is harder to measure an impact on the business or at least it's hard to tell a story about that if you're trying to improve.

Diversity in your data teams. What's the sort of pitch to management for this?

Tracy Daniels: Oh it's products and services, right? I mean, and experiences. So I think about some of the studies that have been done. And again not just one reason, but Did the data that I collected, are the teams that are looking at it, are the people that are challenging if the solution is fit for purpose do they really represent a diverse perspective, backgrounds, experience to be able to guide me where I go, right?

And so I think about some of the studies out there where for example the image products or products and technologies that we're using image, Don't do as good a job identifying or differentiating lighter skin from darker skin. A product can have a very adverse impact to somebody with darker skin if you're not training your model to be able to detect that, right?

And we've heard all kinds of stories with horrific examples where, the error rate might be small, but it's still impactful. particularly on certain populations? Or did I train my model or create a model with whatever training data that disproportionately favors one population or another?

We have a joke in my family. we applied for a certain financial product, and I'll leave it at that. And I am the primary breadwinner. And I got declined and my husband, whose uh, stay at home dad got approved. He's got a credit history, had a job, all that kind of good stuff. And later, you wonder and you hear about, models that maybe weren't...

As equitable as they could have or should have, right? And then, again, you learn, because again, I'm slightly Pollyanna. I don't think people go in with this to be malicious or to harm certain populations. But as you learn that the outcomes aren't what you want them to be, you need to go back to the lab and come up with another model, right?

has a different outcome or a more equitable outcome. And when you think about the explosion of AI software and in the decision making process, even in something like using AI to do hiring, I think it's incredibly important that you have diversity in the process of who's making these products, the data that's used to train the products, and the people that are contemplating how these products go to market,

Richie Cotton: And I guess in the credit example, that's straightforward. They've just lost your business. So that's bad outcome for you. Bad outcome for the organization as well. So yeah, getting this right is going to help companies revenues help increase customers satisfaction.

And that's a fairly benign example. I think do you have any examples of where diversity has been built into products successfully? And the company's done it properly. They've made things fair and it has worked.

Tracy Daniels: I don't know that I have great examples, but let me tell you two things. One, I love the way that you ask that question, because I think that's important is that you're building in this concept or consideration, And the reason I say I don't know that I have a great example is because if you did it well, I shouldn't know.

I should just know. that I'm attracted to the product or the service and I have an opinion and it worked really great. So I think about things like legally now there's things you need to do when you're creating products and services for people that may be disabled under the ADA, And if you do it up front, right, I may not need technology that reads to me the description of the picture.

But you've built it in and you could build it in a very elegant way that benefits folks that are just differently abled that may have for example, a sight impairment. And in a way that is perhaps even a side benefit of being instructive or something for me who is not sight impaired.

I think about just consideration in the product. So, again, do you consider things like people's various pronouns, right? And just for data entry, how you're capturing and maybe tagging and identifying data. And then there's the really hard stuff, around training data. And the effects, like we know our data has biases built into it.

from decision structures, whatever, that were in the past. Is there a way that you can augment the data? And are you thinking about that as you're building your products up front? So that you've got... as much data to make the best decision possible.

Richie Cotton: I love that, there are some sort of real practical steps you can do to improve diversity. Are there any sort of process changes you need to make just in order to ensure that you are making all these steps in terms of building diversity in?

Tracy Daniels: Yeah, I mean, I do think you should be building this into frameworks, right? And whether it's your model development process, where at least you're asking the question. I think like AI now has an impact framework that they've put out there. NIST, even in their frameworks, talks about, considerations for, they don't tell you exactly how, but that you're building in consideration for these types of things.

I talked a little bit earlier about Dr. Brandeis Marshall's process, and it was about thinking through PIR, is the acronym that she uses, and it's participation. Who's participating in building these things. But also who's got access, not only to the data that's selected. What gets promoted?

What gets limited? But who's got access to the technology? I love the way OpenAI actually came out with their product, right? Because you've got a lot of people banging on it and asking it questions that will become part of its purpose, right? And part of the lexicon of how The language is leveraged inclusion, and her pair principle talks about very intentionally engaging diverse or marginalized communities and can you evidence it, right, so that you know that you, what, I did, I asked and I got information and I used it or I didn't, and then representation being the last bit around do you have representation from all demographics that you need within the data or the system or the framework?

And then this last bit I also love around, and are you crediting people for their contributions, right? Again, this idea of who's story is being told or who's part of the conversation about what gets created, the value that gets created as well.

Richie Cotton: and in that last case where you're talking about showing off the representation, are you talking about like the people who are creating the product, like within your team, or are you talking about like the users and all the different groups who are actually using your product?

Tracy Daniels: Yeah, a little bit of both, right? But the idea that they're represented in the data, this idea that dark skin and light skin is represented in the data. That the frameworks that you're using are going after making sure that there's representation there as well.

Richie Cotton: And you mentioned OpenAI as being an example. It just seemed this sort of explosion in the use of AI, this is going to be a bigger issue, just making sure that these AI products Are fair for all the groups that are using them. Do you have any advice specific to AI in terms of reducing bias or otherwise making them fairer?

Tracy Daniels: Yeah, I mean, I think we've touched on a couple of them, right? So, we could start with PAIR. I love just having this framework that you can use. How you collect the data and what kind of data that you have. Check it for both, Are there biases built into it, or is there unrepresented data that I actually need to go and get?

one of the things that I kind of love about the data field, it, as well, is this movement into more inclusive design principles, like almost a little bit becoming more like software design, right, where you're actually thinking about how it gets used, how it gets consumed and building that into the design process, right?

So in as much as it's a design bias awareness techniques that we talked about earlier and making sure that it's there. I also think you're gonna monitor right and measure which we all know we should be doing with our models and and data. And so you're looking for ways to look for impacts, And equity on various populations. You're gonna ask yourself, again, this concept should being uncomfortable. You might need to ask yourself on some uncomfortable questions and actually test for that in the outcomes of your your product. And then maybe last, much like we talked about earlier around literacy and culture, I think increasing the literacy of your team around these issues is important.

So, as I mentioned here at Truist, a couple years ago, we actually did it was a three day half session Dr. Brandeis uh, Marshall, around how do we incorporate these concepts in teaching people to think about it. you know, Brookins talks about companies need to think about and actually debate the trade offs.

So as you're going to market, I think having the lexicon and a place where people come together to talk about the trade offs of Okay, really effective, maybe very surgical, but I'm seeing adverse impact here. Are we all good with that? Or are there things we should also be doing and including maybe humans in the loop, To help deliver a message or provide options for, whoever's consuming this product or service.

Richie Cotton: How do you envisage that humans interact with these sort of AI ideas and who gets the decisions? how does that affect diversity?

Tracy Daniels: Well, I think looking at in some sort of framework where you decide where you are applying the technology, right? Where You might not need a human in the loop. For example, maybe on a back office, system, I'm looking at uptime and I'm doing some self healing using like AI, right? I may not need as much of a human in the loop in that.

But I think you do want to consider where there are times and places and spaces where it's best to have human decision makers helping you to really think about the solution, the output, et cetera. maybe I'm an outlier. I feel like I, AI is most powerful when paired with the human, Not to override, okay, I always tell my husband, if Waze told you go left, just go left. You don't need to override her all the time. But I do think it's a powerful opportunity and maybe even a little bit more surgical way than we have in the past to really help humans be more effective and efficient at doing certain tasks, And so I think we're going to end up with different types of jobs that still require humans, right? Because in terms of how you deliver this and how you use that expertise to hone. The output is going to be important as well.

Richie Cotton: think there's gonna be a lot of people want to learn a bit more about this topic. So do you have any advice on communities or resources people can use to find out more about diversity and inclusion?

Tracy Daniels: So, I love to follow certain thinkers. You've heard me mention Dr. Marshall a couple of times. Dr. Rumin Chowdhury, as well as I think she was part of the AI ethics team at Twitter at one point. Dr. Joy Bulamini also and I just love the name of this group. I think she was part of the founding group for Pioneers for the Algorithmic Justice League.

I think following people who think about and talk about these topics is really important and making sure that group is diverse, as well. I think there's some really great studies out there. You heard me mention the McKinsey study. Brookins actually has a really nice independent piece as well about helping to mitigate bias in data.

Not as, sexy and maybe as interesting. Some of the legislation that's coming out of Europe where they're actually tackling this, I think, a little faster ahead of the U. S. I think it's also interesting to read and keep up with. And then, like I mentioned with some of the business resource groups, I think joining support organizations that focus on varying populations in these topics are also important.

So I think about groups here in Georgia, like Women in Technology, or Goals through Code, or, Women in Data and AI are great groups to also follow and participate in.

Richie Cotton: Excellent. And if someone spots a problem with diversity inclusion at their own organization, do you have any advice on how to start tackling it?

Tracy Daniels: I think people should speak up. And not be afraid to do that. I remember being part of a presentation just recently where the collateral was talking about heroes and data. And it was a wonderful lineup of diverse men, but it was all men. And I said, aren't there no women in this universe ecosystem that, you know, and, they just hadn't considered it.

And so the second revision had more genders. Present it, right? So just speak up. I mean, there's very polite ways to say things. You don't make me, you don't need to make people feel horrible about themselves, but I think speaking up is important. I think advocate as a senior leader, I try to make sure that I am talking about these issues and that I'm visible about these issues so that people understand it's important to me.

I think, Again, if you're in the out group, you're not in the group, being an advocate and an ally. So if you're a male for women, if you're white for people of color, if you're cis you should advocate for our LGBT teammates as well. And I personally have been, I think, a recipient of people advocating for me that may not have looked like me or had the same background or whatever, what have you.

And then maybe the last one is to be curious. Where if those of us in the data field, that's, that comes very naturally to us to ask questions and ask more questions and poke and look for the data and then ask some more questions. And I think if we apply that also to one another I think that also helps create moments of understanding and helps to really build and foster that inclusive environment we talked about.

Richie Cotton: That's great advice. Although maybe don't poke your colleagues literally.

Tracy Daniels: Thank you.

Richie Cotton: All right. So, uh, do you have any final advice for people who are thinking about diversity, equity, and inclusion?

Tracy Daniels: Yeah, I think just be intentional about it, again, it doesn't happen just because of a, I hope this all gets better. And be courageous. This is hard, difficult messy, sometimes deflating things to go after, but, continue to be courageous and push through and then keep learning.

Don't poke your colleagues, but keep learning.

Richie Cotton: All right. Perfect. Thank you for your time, Tracy.

Tracy Daniels: No, thank you. It's been a pleasure. And thank you for your interest in the topic. This was wonderful.