How To Get Hired As A Data Or AI Engineer with Deepak Goyal, CEO & Founder at Azurelib Academy

Richie Cotton: Hi Deepak, welcome to the show.

Deepak Goyal: Hello, thanks for calling me in.

Richie Cotton: Brilliant. So tell me, what's the difference between AI engineering, and data engineering.

Deepak Goyal: Data engineer is like someone who is trying to create the data infrastructure. I would say, for example, like, you have lots of data, your frontend systems, your mobile app, your. Retail system. Every system is generating lots of data. Now you need someone to, pull in all this data, do some kind of an ingestion cleaning, processing, and make it available in such a form that, that we can derive the insights from it.

That could be very useful for the stakeholders. That will be very useful for taking it to the downstream system. Most importantly, like AI systems. So. Data engineering is going to play a very vital role for the upcoming ai.

Richie Cotton: So how does a data engineer relate to, say, an AI engineer? What's the difference between the two roles?

Deepak Goyal: A great question because a lot of people comes with this question, and this is a kind of a very, thin line sometimes you have between the data engineering and ai. So you think like that. Let's say like you have an amazing infrastructure right across the US you have roads buildup, So that you can think about like a data engineering, and the Tesla moves on that like, the driverless car is something like an AI part of it. So data engineers build the data platform. they bring in all the data, clean, transform, and make it available, like, go layer data, which can be utilized for getting the business insights.

And that's where the AI people will comes in. they'll take the advantage of the playground, the data playground, which has been developed by the data engineers to take up the data from there and then, use it for forward. So. I would say that like data engineers comes first and then on top of that, the AI works start.

So that's how you can differentiate the data engineers and the AI engineers.

Richie Cotton: Okay, so the AI engineering, really, it's sort of built on top of the data

Deepak Goyal: Absolutely. So once you have this data, then the AI engineers will utilize this data to do the prediction forecasting or maybe generating the new data using some gene AI technique like content generation and all. So that's where they'll

start from. 

Richie Cotton: Fundamental for a lot of organizations know the most do.

Deepak Goyal: The most two important skills which any data engineer need to have is. They're going to write a lot of logics using the Python. So that will be the fundamental skill. Second is the sql. So sql, is the heart of the data engineering because you're going to play with data. And fortunately, almost all the tools which you find across the data engineering, you find that in one way or other way, they support the SQL and SQ is easy to learn as well.

So these two skills is something which is going to be the foundational one, which going to be. The route. Once you have built that skills, then on top of that there are further skills which someone needed to learn to become a data engineer, for example. After that, they need to learn a kind of an ETL tool that helps to pull in the data.

Then if they are playing with a huge amount of our data, they need to bring in the big data technologies like Hadoop, spark, pi, spark. Maybe they need to learn the new data warehousing tools like the cloud ones like snowflake in including the Databricks in, that's the main skills that someone need to learn.

Besides that, then you need some. I would say secondary skills, like how this code deployment will happen, how to do the CICD, how do this performance optimization, doing the data governance and all that. Also then add up as you move forward or upward into this data engineering journey.

Richie Cotton: Okay, so actually uh, lots of things to learn there. So it's interesting that you mentioned that so SQL and Python, obviously like two main data programming languages. These are the important places to start. so after that you said it's about data processing, big data processing technology, so, spark and things like that.

And then the sort of cloud data warehouses as.

Deepak Goyal: This SQL Python is basic because without that, probably you cannot learn most of the A TL tools and most of the big data analytics tool. So once you've done that, probably you will not be ready as a data engineer. Then you definitely need to learn at least one a TL tool. Now we have a variety of ETL tools in the market.

It could be like an airflow, which is an open source if you go towards the cloud. Then in the Azure world you will find the A DF Azure Data Factory. If you are more focusing towards the Amazon, or the AWS, then the tool would be there is a AWS glue. So once you have the SQL Python ready, then you need to.

Pick a stream around that. You want to go towards a one specific cloud, or you want to go towards an open source system. So once you pick that branch in that, then you have to learn one ETL tool. So that's one ETL tool is needed. Okay. It could be either on the AWS track, a Azure track, or it could be an open source track.

Then after an ETL tool. The next thing which you need to learn is the big data analytics tool. So in the big data analytics world, the core skill is the Spark Pi Spark, Now the same thing happens is like Pi Spark, you can do on your own, but these days people uses the Databricks. So maybe, again, if you want to go into the Databricks world, you can choose Databricks if you want Microsoft.

Also 90% of the work is like the PI Spark only, but they have recently launched a very popular service that is a Microsoft fabric. So then in that case, you need to learn Microsoft Fabric. Otherwise, if you go ahead with the AWS, then a WS also has the A Ws Databricks, so you can learn that big data analytics tool.

So in short, SQL, Python, then one E. Based on your track and then one big data, an tool. And then later on, you can slowly, once you are ready with this, I think I would say that like 90% you are ready for the data engineering job. Yeah, we just add the remaining skills like ci, CD based on the kind of a tool which you have select.

and add little bit of the more skills around, some behavioral skills and all. And then you are good to go.

Richie Cotton: That actually sounds very straight. Forward, learn some five sql, learn an ET l tool, learn a cloud data warehouse. and you're kind of there. Alright, so, beyond these sort of technical skills, are there any soft skills that you think are important as a data engineer?

Deepak Goyal: I would say that any job in today's time must need to have a soft skills. So data engineering is no exception to that. First and foremost is that you should be able to communicate, Because I mean, maybe compared to the software engineer, data engineer has to. Probably they have to talk to a lot of stakeholders to understand what kind of the output or the results they are looking for.

They have to coordinate with the data analysts. They have to coordinate with the AI engineers. So they might have to communicate a lot of times with the administrative platform admins and all. So they definitely need the first impounding. That is a communication skills that they would be able to, put their point forward first.

Second the behavioral skill, which I feel is that they should have a problem solving skills. a lot of time you get into the stuck that you build some data pipelines and the pipeline is taking huge time to get finished. Or maybe sometimes you develop everything right, and suddenly it happens That.

the retails team or the stakeholder comes to you, or business user comes to you and say that, okay, we are not getting this specific values into our spreadsheets or into our dashboard, which was there. Then you need to be smart enough to understand and try to put your critical thinkings into the play and see that where is the problem.

Then later on you might find that, okay, the incoming data has itself a problem, which is not populating this specific information. I. So you need to have this critical problem solving skills. that is something I would say must have it. And third, the I feel that one of the most important in general in today's time is the adaptability and learning capability.

you need to be someone who is continuous learner, so you cannot stop because. We are into the, era where everyday new technology is coming up. So you need to keep on learning new skills and you, you need to be open for that. So this is what something the soft skill people look for towards, or the kind of a behavioral skill people look for it. 

Richie Cotton: Get on.

Deepak Goyal: Yeah.

Richie Cotton: As much as an attitude thing, as a skill there, I think.

Deepak Goyal: Absolutely. I mean, see, when I personally hire, I have a long history of hiring people. So I look for this very important skill that, as you said about the attitude, because I mean, technology can be taught okay, but the attitude is difficult to develop. So that's, create a difference. So we need to look for that and every recruiter looks for that.

Richie Cotton: Just going back to the thing you mentioned before that you need strong python SQL skills. Are there any particular things you need to know around programming? I mean, these, these a learning, are there any.

Deepak Goyal: There's a lot of AI involvement comes in nowadays. Auto for you. So like the intel sense has been developed a so strong that we are slowly moving to the low or no. So, although still.

People get confused when we talk about the python from the data engine perspective and the AI perspective because in case of the Python for the data engine perspective, you need to know I would say the basic concept of the python, like how to declare variables to do the control lobes, fls, so those kind of a stuff.

Good to know functions, little bit libraries, good enough. But if you go to the AI side, then you need to know a lot of things into the python. I mean, you have a psychic learn, you have a pythons and all, so don't get confused. If you wanna build a career in the data engineering world, then the Python, those data science library is not something needed.

You need to be covering the basics Python, where you are good to write the code and then you would be smart enough to. Little bit of an um, that's it. If you cover the python up to that level, you're good because rest of the work, remember, is done by the PI spark because your panda is not a distributed computing environment.

So they're not smart enough to handle a huge amount of data. But the PI spark would be so majorly you're gonna write the pi. So that's why the fundamental is if come to the. I think normal SQL people understand the only exception to that normal SQL would be here is the windows function. So you need to be a little good around the windows function, like the ramp and lead lag and all.

So those are the extra thing. If you are not aware of that, someone has to cover that.

Richie Cotton: Okay, so, on the Python, it's incredibly interesting that you don't need all these data science, cyber. So I think you're a data scientist, something. And it is pretty much core functionality there. Something that's very essential. But as a data engineer, it's like, keep it simple. Just stick to like the, the core sort of syntax.

A lot of.

Deepak Goyal: This is one of the misconception.

At least 10,000 people to become a data engineer. And every time people come to me have this common misconception that they need to be very expert in the python, but reality is not like that. So because majority of the work you're going to write, all the complex logic you're going to write using the SQ itself.

So if Python, you have the practitioner level knowledge, it's good enough. You don't need to be expert of the Python.

Richie Cotton: So basic python skills on the SQL side. You said you also need window functions. Now this is interesting because this is like one of the first sort of hard concept that. That learning sql. Talk me through why do you need window functions?

Deepak Goyal: See, many times you come across a situation where let's say somebody's trying to identify the running total, or maybe somebody is trying to find out, let me give the top five product, which is get sale on a monthly basis or something like that. So you get into a situation where this kind of functionality's something, it's important to learn also everyone who is learning this data engineering, the, the simplest goal they have in front of them is going for the job interviews.

And these questions are the most favorite for the interviews?

Richie Cotton: Interesting. Okay. Yeah. So, uh, since you mentioned interviews, we talked a lot about skills. So let's talk about getting hired. In terms what hiring managers want, I guess, do they care about qualifications? If.

Deepak Goyal: Most of the time not, and honestly, it's literally based on the, region. I would say in India, If you talk about the India, few people, you find it like very stereotyped. They might look for that. But if you talk about the majority across the globe, like in the us, uk, Australia, all the different parts of the world, generally people are are open.

So I feel that most of the times, which I have seen is like the hiring managers are not very much behind the specific degree. But again, if you are belong to the computer science, it's always, always a plus in cases when you are not from the the engineering background in those cases, or not from the computer's background or maybe like a non-IT background, I would say in those cases, somehow you need to showcase your credibility because the computer science engineer somehow has a credibility that they have their IT degree, So in that case. I mean, you have to, play a little bit different with that. You do some kind of a certification and you have some projects to showcase and all, and that's how you know, you can project yourself and you can justify that. Yes, although I don't belong to a specific category of, folks, but still I am, I'm someone who knows all this and I can handle this job. 

Richie Cotton: So you've gotta demonstrate your computing skills, your programing skills. So if you got some sort of, I guess.

Try hard to showcase that you do have those programming skills. Okay, so, you mentioned projects. Talk me through that a bit. Like I guess, portfolio what might.

Deepak Goyal: In the portfolio case, this is two, two basic things. We need to understand that basic challenge of the portfolios. If you are someone who is working professional, then probably portfolio CRE creation is not that easy. The reason being would be is that. You might be working for your client and you cannot share the data, So you cannot make a GitHub repository and saying that, okay, this is my GitHub rep, and like, I put all the code, which I have done, or all the SQ logics, which I have written in available there. So you don't have that leverage to do it. So in that case, probably in your resume you might be putting it, and that could be considered because the hiring managers also aware of that.

So for them, probably, at least as far as the data engineering is concerned, it's tough to make that the portfolio. Now, if someone who is newly entering it into this data engineering field, what they can do is they can pick some of the, we call like a POCs or some kind of a capstone projects that they can build in and they can, put it onto the GitHub showing.

All the codes and all the logics they have written for it. So that's how they can think of building the portfolio. And if we talk about a specific about the projects, so the projects should be somewhere like, creating like a data lakehouse system where you have some Medallia or a three layer architecture.

That's a very popular framework at the moment, and most of the organization is implementing that. So if you have one of a project around that. Put a lot of value to your portfolio. Then second would be like if you can bring in something into the real realtime streaming and all that is something recruiter. 

Richie Cotton: So just having real world examples of, hey, this is like something that gonna be useful. Off that you do have these skills to be able to create something once you get the job. Okay. Are there any other ways you can make your profile stand out if you're applying for data engineering jobs?

Deepak Goyal: There could be multiple things. So the first is your resume, so you cannot skip that. So make a.

The resume when you are going to make and resume, try to put in something which is quantifiable. So rather than just saying that, I work for some X, y, Z project and I work exceptionally there, maybe that's not going to work. So maybe try to quantify that work. Maybe you start seeing that, okay, I work for this x, y, Z project and I built the pipeline that has improved the, Turnaround time by 30% or something like that. So you need to quantify it. Maybe you need to start quantifying there, like how much data use size you have handled. So rather just. I worked on a huge data and handling that. Start saying numbers. Say that, okay, I was handling the a hundred terabytes of a data on a monthly basis.

So that will go into, put something that people can quantify. So that the first thing would be to make yourself stand out. Having a killer resume and killer resume will have something like this, that quantifiable thing. Second probably is, as we we're talking about it, you can always add us certifications.

So you can think of doing some certification based on whether you are in the Azure Track or AWS track or maybe Databricks or maybe Snowflake. So whatever you are picking the technology based on your liking. So in your space, at least do some kind of a certification so that something is, making you stand out against your, all your peers.

The third thing would be is being more active over the, social media like LinkedIn, which is specifically for the professionals. So maybe, that catches more eyeball compared to others. So you can always say that, okay, I'm quite active when I'm sharing what knowledge I have, and it's kind of, you know, building your, online portfolio every day.

So maybe you can do that. I would say like if you are good at, or you know, speaking.

Making some good information on YouTube videos or something. So in short, you just share your knowledge. So somehow that will make your portfolio saying that yes, this is the guy who knows something. That's why he's sharing with the word. So overall, this whole bouquet of things, if you do it, probably you going to, you know. 

Richie Cotton: yeah, so resume, and that's. Hiring process.

Deepak Goyal: So, hiring process in general, always like, very based on company to company. But it start with you apply for the job. The moment you get shortlisted, then you probably get a first, the screening call from the hr. He might ask you some basic questions like. What do you do?

What two years of experience you have how's early or how soon you can join if we give you an offer and all? So those kind of an initial screening, they do it and then we get into the technical discussion. So, so the second round would be the technical round. So it, it'll be, generally it's a intense technical round where they're going to, try to ask almost everywhere from the start to the beginning.

They might ask you all the tough questions. Around the technology based on your experience. Maybe sometimes we have a second round of the technical as well. So overall, like third round I would say. So that will be an another technical discussion round. Sometimes some organization call it like a system design round where probably they.

Although the system reason is more or more popular in terms of the software development engineers, like SDE roles and all, but in some cases in the data engineer world, some of the organization still do this system design or maybe the second round of technical questions discussion where they are trying to ask more of a real world scenario.

so they probably give you the scenarios like. Okay, we are having a hundred terabyte and you have to design the pipeline. So how you would design this pipeline and you need to ensure that if something is going to break in between then how we can recover that situation or if that you are getting a lead data, how you doing the back filling in some situation.

So they try, try to twist the question around these different use case scenario or most of the failure scenario. So something which you need to prepare yourself for these kind of a scenario based question. Not mostly that is not going to be straightforward. There is something is more of like, you might have experience, you might also get some questions like, okay, what are the different challenges which you faced in technically?

And maybe, tell me the most challenging problem you have solved technically. And then you have to explain that scenario like that. So that is this second technical round. So overall third, then fourth would be the behavioral or the managerial real, where like they asking the question.

To just check like whether you are a right fit for the organization or not, or not, maybe from these they try to test your soft skills as well. They try to check the, the compatibility within the team that okay, whether you're going to be compatible for this respective team or not whether you have a right attitude or not.

So all those the soft skills that okay, they give you some conflicting situation and say that, okay, how you resolve the conflict in this situation, or if you were there in place, how would you have handled it, et cetera. So those kind of a behavioral questions or maybe altogether a some problem solving skills they're probably going to ask in that.

So more of like a little bit of non-tech interview in the behavioral round. And then mostly then would be the next, last round would be the fifth round would be the, again, HR discussion where. The compensation and all. So I think if someone passed through, probably it

Richie Cotton: so it sounds fairly comprehensive. I think this is a fairly standard for a lot of tech roles is you have some sort of screen and then it gradually gets more technical and then you have a, an interview with the manager. So in general, do you have any tips for how to pass interviews? Like what's gonna put things in your favor?

Deepak Goyal: first, foremost. Be genuine. So it's tough sometime to, to twist your personality. So be genuine the way you, you are. And I feel that most of us are in general are good people. So probably that will show up there as well. Coming to the points you should always have a one killer resume, so that is always gonna help.

. Then next important tip, I would say that if you have the attitude of a learning, so you need to keep yourself open because the technology in which we are right now is changing at a very fast pace. So to keep up with that pace, maybe you are working every day, day in, day out. That's maybe if you want to.

To show your skills compared to others, probably you need to open for learning. So keep on learning new technologies. So keep your eyes open. that is another thing. The third thing I would say that, have more networking done, so maybe like through the LinkedIn or somewhere. go out and talk to the people, build your network, because lot of jobs get filled.

I don't have the exact percentage. In front of me, but if I'm not wrong somewhere, I've read it. Around 30, 40% of the job has been filled through the networking itself, right through the references and all. So there's a huge job get filled through that references as well. So build your strong network that also, give you an added advantage outside your technical knowledge and maybe if you can have the credibility in as in terms of the certifications and all.

So that will help you get to the job easily.

Richie Cotton: I quite like that idea of, being authentic. It just seemed like sensible, that you don't wanna twist your personality and pretend to be someone else. Because once you.

Deepak Goyal: Yeah, one question. You can do it, but another question probably where you show up your real face. So maybe even they enlight you by the first question, they start hitting you because they feel like that you may be pretending a lot of things, not just this thing. So I personally feel sometimes that being you is always easiest for you.

Richie Cotton: I guess it's a good, unless you have a really awful personality, which case.

Deepak Goyal: And I feel that most of us don't have, trust me, most.

Richie Cotton: Okay. Yeah. Bring the best version of yourself, not maybe not just show what's an all in the interview. Uh, okay. 

Deepak Goyal: For the tech interviews the straightforward question was fine. I feel that somebody should be very well with the performance because in the retention world, the performance is the key questions, which you can expect. Someone would be very interested that how you handling a huge amount of data. So think about how you're going to handle the performances.

Second probably is the failure situations. So happy scenario, everyone is okay that you'll be able to write the happy scenarios. You probably, they are more interested in understanding that how you handle the failures. So think about that, okay, your normal pipelines, your normal logic works pretty well.

Now if it starts failing, how are you going to tackle that? So that will be something which you can expect. So there is something which you should prepare beforehand so that you don't feel like a surprise into the interview. Performance failure handling the valuers youth size of data handling that is again, kind of a performance itself.

Maybe this is a one very very important question, which I seen the most of the time is. Tell me the most challenging situation you faced in the . Technical interview. And people get caught at this level, and people will let, they really have to talk about the exact challenges they have faced it.

So they need to play smart there, probably they need to talk about something which they are most comfortable with it. so they bring in a one scenario, which they are most comfortable in talking and most comfortable in experience. So bring on that and showcase it like the challenge which we have solid, but maybe it's a challenge for them, so being, you are strong in that area, it'll be an easy task for you.

So bring it something example from that. So beforehand, prepare for such kind of an example that is something is going to ask in general. Maybe on the SQL side, window function, somebody should need to prepare because window function is something. People find it a little hard. So some question might, you can expect on the Windows side, windows function side, and most of the people get stuck around this. And lastly, like any new, new feature got added up. For example, if you're going for a Databricks interview, then just go and read it. What are the, in last six months, what are the new features got add up So people get, people are sometimes are interested that, do you know that new feature in.

So they just want to check like whether you are updated or you are someone who is keep loading new things or not. So just do that and then I think probably that will be your ticket to the another job.

Richie Cotton: I do love the idea of preparing things beforehand. Because certainly I spent a lot of my time fumbling over trying to explain technical concepts. So I think yeah, preparing what you want to talk about beforehand, especially if it's like talking about a project that you did a year ago. You've done a few projects since then.

You probably can't remember like what the hardest part was off the top of your head. You're gonna want to write that down and, just practice saying it before you go into interview.

Deepak Goyal: My ninja tip always for my students, and I think for all our listeners here as well, would be is that even when question comes to you and interview, even, if you know the answer, pretend like that you don't know the answer. So the question throw to you should not jump an answer. Take a pause.

Pretend it like that this is something new question is offering to you. And then answer slowly so it, it feels like the interviewer that Okay, he has asked us some really a tough question. that works pretty well many times.

Richie Cotton: Yeah, just taking a moment to before you, uh.

Deepak Goyal: yeah. I mean, because it's a human psychology. I feel that interview is almost like a fight between the two minds, right? Everyone wanted to win. Like the interviewer wanted to win and feel that okay, he has asked one of the toughest question to the interviewee. so just let them, let him pass and let him, win.

Feeling that that, okay, he has asked really a tough question. And then you take a pause, you feel like, okay, okay, I never have allowed this question. Let me try to fi try to answer like that. So that brings kind of a perception in the interviewer's mind that okay. He knows something because this is not something, the question which he is already aware of it.

Because that will decrease your value in general in their mind sometimes.

Richie Cotton: Actually, so, just related to that, so you mentioned sometimes you get an interview who just wants to ask the hardest question just to kind of show um, what do you do when you have a difficult Yeah. What do you do when you have a difficult interviewer?

Deepak Goyal: If you have a difficult interviewer, the best thing is you need to be polite. Be polite as far as possible, because there is no point in arguing with this person because he's eventually a difficult person. So, and he's always having an upper end because he's the interviewer and you are the interviewee.

So anyways, because he has the final call right to take, so there is no, no point in fighting. So, I would say put your guards down and saying that. Okay. I mean, be polite and answering. Maybe he will try to counter you sometimes that, okay, why this, why not? And he might, have a very rigid personality in that case, polite and put your point forwards with the facts and figure.

So if you, with the facts and figure, then probably he'll left with no room, to argue with you on some of the stuff. You know Very correct. For example. You say that, okay, how you improve the performance, then you will say that, okay, I'm going to improve the performance by doing the cache. And then you say that not just, you just let him complete.

I mean, put the figures as well that because if you cache the data, because your data gets saved into the main memory, and to get the data to read the data for the processor from the main memory is always going take a less time. It has a low latency compared to the data which is gonna be read from the, so performance

something you. You put your answer and add little more extra, description around it with which will, put your points forward with some facts.

Richie Cotton: Join their interview process, even though they're, they're creative of Claude, do you have a sense of when you might be allowed to use generative ai and is it a good idea?

Deepak Goyal: To be honest, at this point, most of the organization is not ready to accept that point. Still at this point, a lot of the organization, you find it, which even does not allow within the organization itself, not allow them to write the code using the, some charge or any AI based tool. 

So I think the world needs some time before we make it little, comfortable. I mean, forget about the.

Still people not allowed to do the Google in the interviews. most of the interviews you see, right? I mean, although like we, when we work as an engineer, I'm, I'm working from last 17 years in IT industry, since day one of my engineering, I'm using the Google every day almost while writing my code and all so there is no shame in doing a Google, but still in today till this state, most of the organization is not ready, to allow you to do the Google. So I think expecting that they'll allow some AI tool to to use it into the interview. I doubt that they will be open so fast.

Maybe some of the startup could think of doing it. Maybe because they might be a little open-minded in thinking that okay, once a person comes in and probably. we need someone that who is going to use this all the availability tool in the world and build some software for us, and we are okay with that.

So I think those startup mindset folks probably can start up with that. Ideally, I think for the bigger organization, I think it's still it's too tough. It's too tough.

Richie Cotton: So gonna depend a, the organizational culture, and it's probably something you gonna ask maybe in that first interview, what's appropriate.

Deepak Goyal: Right at the moment. I think it might take some time because I mean, in day to day, I mean, think about day to day, like when we talk to the people, or let's say you go to your manager and say that, okay, I have Right using the charge. Probably he, he doesn't feel comfortable. People feel little bit, negative side in saying that, okay, maybe you, you don't need it.

That's why you do it. Although, I feel that I, I was not a part of that journey, but maybe I feel that like. 20 years ago, or 25 years ago when whenever the Google came in, like and when people were using the Google help and writing it, the same feeling might have it. So I think that acceptability is yet to come.

So hopefully after that they might be open for the interview to just start using the ai.

Richie Cotton: So one more question on hiring is, suppose you get an offer? How do you go about negotiating your salary or pay package?

Deepak Goyal: The negotiation is a. everyone should have it because it is not something which is needed in your, job roles. It is like a day your negotiations skills is gonna pay you big time. So that is something everybody should have. But if you come to the point of like how you negotiate in terms of the salary when the HR is asking to you.

the one thing I would say that. Before getting to that point, at least we should have our homework done, so we should be very well aware of that. I'm applying for, let's say, a senior data engineer role, so I should be very clear on that. Okay. That first senior data engineer role in this specific organization or, in most of the organization of this level.

In this specific country and in this specific city, this is the range. So at least I should have an overall range in front of me, okay. Before negotiating, because that's how I can get an idea that, okay, this could be the right price or not. So first that needed to be done. Second, once you have done that, so you have some range is in your mind that, or you get to that, okay, like let's say a hundred to 150 K is an average salary for, let's say for a senior engineering, just an example.

then we, you get into the, the negotiation round or that a discussion then rather than asking for something directly that, okay, I'm looking for a hundred, or I'm looking for 150, maybe you start seeing that like, okay. I have done some research and I feel that the knowledge, which I have it, and the kind of a profile which I have it, the market standard or the industry standard is up to this range.

So then you talk about that range, So that you've opened up it. So rather this getting outta one figure, you have opened the range. Now, most of the time. There is always a scope on your initial offer. if HR is, offering you, let's say a hundred K, he always and always, most of the time he is a cap of at least 10% plus doing on top of that.

So it's there. It's only that few people. Because of some fear that probably they're going to lose the job. If they say, the moment they ask to hire, I mean, ask a little more. And then probably the HR will say no to it. I feel that that's a little bit misconception because most of the time this does not happen.

Okay. Because think from the organization's perspective, organization has already spent a lot of time on you. They might have taken 3, 4, 5 technical rounds and after that you got selected. So in their eyes, you are a right candidate first. So they have already put a lot of efforts on you. And if you feel they're just asking for money, they'll leave you.

Probably they're not. Maybe they'll say that, okay, this is the best we can offer. Please tell us that. Okay. Whether you are okay with that or not, but asking more, I think it's not a problem, it's not gonna lose your job, so you should give a range. And try to ask to the top of that range, If you feel that that discussion is not going very well, then you say that, okay, this is the top of the range, which I'm expecting, but I'm a little flexible because I want to work with you.

Something like that. And then you do it. So at least it's right that probably you not accept their number, they're not going to accept your number, but it is midway something, you can break the deal and that would be plus for you. So get into that mode. I feel that that is something, going, help you a lot to negotiate in better manner.

Richie Cotton: I like the idea that there is a lot more upside to negotiating than there is like downside,

Deepak Goyal: Indians our best negotiator.

Richie Cotton: Yeah. The worst that's gonna happen is they're gonna say no. The same offer before. They're not

Deepak Goyal: Yeah.

Richie Cotton: offer unless you like. 

Deepak Goyal: And also, just to add one more thing. Association power get, multiply the moment you have a multiple offer. If you have a multiple offer, then definitely you can negotiate very hard. So that is also the thing which you can think of it like the moment you have a one offer. Then you can also put that point very.

Very smartly there. Okay. Saying that, okay, I'm looking for this some X to Y range and I'm already having in the final round with another big short company. So I'm expecting an offer from there as well. But although I'm still love to join your company, we, I get the higher side of this range like that.

So you can smartly put that point as well.

Multiple jobs and the moment you've got a one offer, the next offer is going to be, you know, you have a very, very strong negotiation power with you.

Richie Cotton: brilliant. Okay, so suppose this hold goes well and you get the job. What's a likely first project for data engineer?

Deepak Goyal: I think if you are a experienced professional, then definitely the project would be is like. Whatever their highest priority project, to be honest, because they might be hiring you directly for some of the projects. So most of the organization will not going to be so choosy on what to give you and what not to give you.

So because they already have something in front of them and they probably is hiring you directly for that requirement. maybe it could be your. Some data engineering, building the data lakehouse project or maybe like streaming based project could be depend whatever the requirement they have. On the second side, let's say if you are someone who is a beginner or fresher, for the fresher, probably there is some chance that.

They might not directly put you into the, one of the critical project or maybe, the most important project for the organization. They might start you with little, the smaller project with some smaller client or the project, which they've started out trying to do some kind of a POC and all, so that's where probably you can place in, in case of the fresher, but mostly in case of an experienced professional, probably they know that you are going to get fit there.

Probably that could be the project, which you going to get it. So they'll not be so choosy in that case because, as an hiring manager, I would always try to, hire the best people for the best project. for whatever project I have it. So they'll not choose like that mostly.

Richie Cotton: So it's gonna depend a bit on seniority. I'll say at data camp there's like a policy of like anytime someone gets hired, like an win in the first. Just make sure things smoothly.

Deepak Goyal: Yeah, but generally data engine project is like a year long or a six month, eight month long project, so you cannot run it. Someone gives so much room to, you know, uh.

Richie Cotton: absolutely. Okay. So, more senior are kind of the, the bigger and

Deepak Goyal: So maybe in that case you can think of that. Let's say if I have a agile based project where I'm gonna run multiple sprints, so maybe for initial one or two sprints, they might not give it a lot of work in that project. You might get assigned to the bigger project, but for initial spins you get a small amount of work so that you get adjusted to the environment, you get adjusted to the work culture of the project and all.

And then slowly and gradually they will, increase the load on you.

Richie Cotton: So I like to talk about what happens after you get hired. And how we keep up to date. So as you mentioned, technology's moving pretty quickly. Like it's been big shift, like for example, like Microsoft launching of like new and rapidly how keep when.

Deepak Goyal: So I think the first thing would be is follow Richie. So Richie is bringing the direction of people. So, and that, that's an easy way, right? You are going office and you can just switch on your, the podcast and, you can keep listening. So lot of people might have a the problem that they, they don't have a time, right?

So maybe the Richie's podcast could be one of the best. So. you keep on bringing a lot of new folks, And who are like a leader into the the specific technology and the stream. So listening to those people always, give unfair advantage. So that is one way is like, if you don't get the first thing, like people if can get set and do it, that's the best.

But if they don't have that leverage or they don't have the time right? A following this kind of a podcast could be one of our easiest way probably I would say that people can do it. So that's a one way to keep yourself updated. At least you get to know the new things, and maybe this is something is coming in, which is important for you, which is not, not important for you.

So the first level of filtration is definitely going to be done here itself. Then second, I would say that. if someone can get time, probably they can take the pointers from these kind of following the leaders, listening to podcasts, following people on the LinkedIn and all, they get that pointers.

And with that pointers they can go further deep down set and, they can read a little more about it. If they get some time to do hands-on amazingly well, they can do that. If you feel that like You still need to be someone who needs some kind of support to make you aware of everything maybe they can think of taking some online courses, a quick short courses where, maybe go to the data camp and some quick course of the latest technology that could also help somehow to keep you updated with the right set of technology in an easiest manner.

Where to read, what to read and, all right, so they can just simply go to the data camp and then, pick a right course. And probably in next two, three hours they'll be ready with the best summarized content. So that could be another way, maybe. Then if you have more time, you may be reading some blogs or, following some specific art.

Let's say you work in the Snowflake or in data, I mean, Databricks and all. So maybe, you can follow their blogs and read some announcement, whatever is going on. So these are some of the ways probably where, you can keep yourself updated.

Richie Cotton: it's good that there's a, a real mix of like, maybe learning is as simple as just reading a blog or listening to a podcast. It doesn't be formal learning all. 

When taking some courses gonna help out. So I guess the tricky bit with that is you just got a new job.

How Find the learn and work.

Deepak Goyal: That's one of the toughest question. The time is critical, but as I said, like finding time, it's more of like, your priority. So maybe I would say that I. Make some routine. So as we're talking about it, right, that getting a time is tough. Like, but at least let's say you are, traveling. So at least you can be listen to podcast.

So at least the bare minimum that somebody can think of doing it. Or maybe I would say like if you are working, at least make a calendar that okay. Once in a month, I have to sit for two hours. nothing like that. I'm like, I'm just fixing my calendar a month prior that, okay, I'm going to sit for two hours a month in my calendar.

Maybe I cannot do it for two hours. In a straightforward, I could cut it down like a 30 minutes, 30 minutes, 30 minutes, 30 minutes across four slots, or maybe initial, maybe forget about two hours, at least block an hour or 30, 30 minutes of a tooth thing. I mean, two calls. You can, block for yourself.

Calendar and then, you make a habit that, okay, I'm going to go and, sit and, write it. Maybe like you do a lot of social media, right? So maybe on the social media itself, maybe you, you love to the Instagram too, or the TikTok to watch the videos and all. Maybe at least start following some other tech influencers as well.

So maybe while you, you are surfing it. For 10 seconds, 22nd, 32nd, they will talk about something and you might learn something new probably because these influencers are the one, whenever something new get launched, probably they're the first one to, start talking about it. So maybe these are not directly but indirectly, make you a little bit aware of the new thing, which is going on in cases when you have zero time dedicatedly for learning stuff.

Richie Cotton: That's hack for social media is just make sure that you are following like some relevant sort of work people as well as just like the fun stuff. Yeah.

Video is better than learning nothing at all. Uh, You certainly find out what

Deepak Goyal: Yeah.

Richie Cotton: brilliant advice for.

Deepak Goyal: the final advice. One to ETL tool, pi spark. I would say that master this. Second, if you can add on some of the certification, well, if not, that's completely fine. At least you should have a real world knowledge of hands-on practice. So though focus on more doing the hands-on because that is more needed even than the, your certification do that.

Third, I would say that that is one of the biggest hack I would say that is whatever you are learning, try to, teach in some way. So maybe by writing blogs, by posting on social media, by posting on LinkedIn or something like that. that will build your portfolio as well.

Secondly, probably whenever you try to share your knowledge in, in any of these different ways, that helps you to understand the things much, much better. So probably if you are learning, if you are sharing, and if. Probably that eventually build your entire ecosystem that will pushes you for your best and that.

Richie Cotton: Brilliant. So I guess yeah, get all those taste skills beforehand. Make sure that you've got something to demonstrate your value, the certifications, portfolio, things like that. So get the job and then yeah, just keep learning. Alright. It sounds so easy. Nice. So I are.

Deepak Goyal: The first one will be there, rich. So go ahead, follow his podcast. I mean, I come across a couple of months back, so, and I started listening to him. So that is the first recommendation I would say. The second one would be myself, so you can keep following me. If you are someone who is talking about the data engineering.

So that you can do it. Third, I would say like Zach Wilson. So Zach Wilson is stuck a lot around the data engineer, so you can find him on the LinkedIn. So he's also making a very good content on the data engineering side. You can follow that. I'm, personally someone who is following him a lot, so that would be a good people to follow.

Richie Cotton: Follow.

Deepak Goyal: Thank you.