Succeeding in data projects comes from alignment: When working on large scale data projects, it's absolutely paramount to get all your stakeholders aligned. Whether it's engineering, infrastructure, product—ensure you have common goals and roadmaps are aligned.
Try to break data silos: One of the biggest blockers when procuring the data you need at the beginning of projects are data silos. For example, marketing data tends to be centralized in a marketing stack owned by functional teams—to avoid this in the future, think about how you can centralize your data.
Prototyping is key: It's always important to start with an MVP to determine the viability of a project. Avoid trying to create the perfect project, or the perfect thing—and focus on iteration.
The most important thing I've learned from agile in getting out to data science is to start with an MVP and to build on the MVP quickly. I find a lot of data science projects fall into the waterfall trap of... We're going to create the perfect project, the perfect thing, and get that out in front of customers. When really you can start with something simple and naive and start learning about how your customers are going to interact with it.
Success comes a lot from alignment. The more that stakeholders, the people that want the problem solved, the data science team and the team that's implementing the project are aligned, the better the outcomes will be. So make sure you're all on the same page about requirements and timelines.
About Brian Campbell
Brian Campbell is currently an Engineering Manager at Lucid Software, where he leads the companies Data Science and Engineering efforts. From helping a 3-person marketing department configure Google Analytics to being the tech lead on custom streaming-analytics solutions, he’s built up Lucid’s data ecosystem through many stages of growth. He’s grateful for the many friends he’s made in every department along the way. When not behind a keyboard, he can be found digging in his garden or at the library with his daughter.
Adel is a Data Science educator, speaker, and Evangelist at DataCamp where he has released various courses and live training on data analysis, machine learning, and data engineering. He is passionate about spreading data skills and data literacy throughout organizations and the intersection of technology and society. He has an MSc in Data Science and Business Analytics. In his free time, you can find him hanging out with his cat Louis.