The ability to efficiently work with big datasets and extract valuable information is an indispensable tool for every aspiring data scientist. When working with a small amount of data, we often don’t realize how slow code execution can be. This course will build on your knowledge of Python and the pandas library and introduce you to efficient built-in pandas functions to perform tasks faster. Pandas’ built-in functions allow you to tackle the simplest tasks, like targeting specific entries and features from the data, to the most complex tasks, like applying functions on groups of entries, much faster than Python's usual methods. By the end of this course, you will be able to apply a function to data based on a feature value, iterate through big datasets rapidly, and manipulate data belonging to different groups efficiently. You will apply these methods on a variety of real-world datasets, such as poker hands or restaurant tips.
Selecting columns and rows efficientlyFree
This chapter will give you an overview of why efficient code matters and selecting specific and random rows and columns efficiently.The need for efficient coding I50 xpWhat does time.time() measure?50 xpMeasuring time I100 xpMeasuring time II100 xpLocate rows: .iloc and .loc50 xpRow selection: loc vs iloc100 xpColumn selection: .iloc vs by name100 xpSelect random rows50 xpRandom row selection100 xpRandom column selection100 xp
Replacing values in a DataFrame
This chapter shows the usage of the replace() function for replacing one or multiple values using lists and dictionaries.Replace scalar values using .replace()50 xpReplacing scalar values I100 xpReplace scalar values II100 xpReplace values using lists50 xpReplace multiple values I100 xpReplace multiple values II100 xpReplace values using dictionaries50 xpReplace single values I100 xpReplace single values II100 xpReplace multiple values III100 xpMost efficient method for scalar replacement50 xp
This chapter presents different ways of iterating through a Pandas DataFrame and why vectorization is the most efficient way to achieve it.Looping using the .iterrows() function50 xpCreate a generator for a pandas DataFrame100 xpThe iterrows() function for looping100 xpLooping using the .apply() function50 xp.apply() function in every cell100 xp.apply() for rows iteration100 xpVectorization over pandas series50 xpWhy vectorization in pandas is so fast?50 xppandas vectorization in action100 xpVectorization with NumPy arrays using .values()50 xpBest method of vectorization50 xpVectorization methods for looping a DataFrame100 xp
Data manipulation using .groupby()
This chapter describes the groupby() function and how we can use it to transform values in place, replace missing values and apply complex functions group-wise.Data transformation using .groupby().transform50 xpThe min-max normalization using .transform()100 xpTransforming values to probabilities100 xpValidation of normalization100 xpWhen to use transform()?50 xpMissing value imputation using transform()50 xpIdentifying missing values100 xpMissing value imputation100 xpData filtration using the filter() function50 xpWhen to use filtration?50 xpData filtration100 xpCongratulations!50 xp
In the following tracksPython Programming
PrerequisitesData Manipulation with pandas
Leonidas SouliotisSee More
PhD @ University of Warwick
Leonidas Souliotis is a PhD student at the University of Warwick, UK. His research interests lie in the field of bioinformatics, machine learning, and deep learning. Before that, he completed his MSc in Statistics degree from Imperial College London, UK, and his BSc in Statistics and Insurance Science from the University of Piraeus. He has worked in different areas of applied statistics and machine learning, both inside and outside academia. This includes stock trading, epidemiology and biology.