Tutorials
data manipulation
+1

Data Science for Search Engine Marketing (SEM)

In this tutorial, data science meets Search Engine Marketing: learn to create Google AdWords campaigns by generating keywords and ad templates with Python!

As a heavy user of DataCamp since two years, I learned quite a lot. My main business is in online marketing, and I have benefited a lot from the data science skills I learned. I'm now able to manage way larger and more complex campaigns, and efficiently maintain them. Since I'm quite familiar now with the content and the different topics and packages in R and Python, I thought about how I might create search campaigns for DataCamp, just as an exercise.

In this tutorial, you'll tackle the following topics:

  • You'll first learn more about Search Engine Marketing (SEM) and search campaigns;
  • Then, you'll take a closer look at the business case at hand;
  • Next, you'll generate keywords for DataCamp's instructors, technologies, courses and topics.
  • With the keywords at hand you can start generating your ads. You'll first come up with a strategy, after which you can start to create ad templates.

Search Engine Marketing (SEM) and Campaigns

What Is SEM?

SEM is one of the most effective ways to grow your business in an increasingly competitive marketplace. It is the process of gaining website traffic by purchasing ads on search engines. Google AdWords is one of the most popular forms of online advertising: you bid for ad placement in a search engine's sponsored links for keywords related to your business, then you pay the search engine a fee for each click.

Paying per click might become quite the cost, and so it is very important to plan and group your keywords and ads, in such a way that you control your cost, and hopefully have profitable campaigns.

Although the exact process of determining the price per click is a well-guarded secret and depends on many factors, the main idea is that the more relevant your ads are, the lower your cost per click.

Setting Up a Search Campaign

Before setting up a campaign, you'll typically come up with a budget, pick keywords, and check out the competition. Also, you'll make sure that your landing page is in order. Then, you can set up your campaign: you will write your first ad, fix its details and then set up conversion tracking. This tutorial will mainly cover the process of generating a large number of keywords and ads, based on the different landing pages that we have.

But, before you start, let's take a look at what these search campaigns exactly are!

Ad Groups

To get started with a search campaign, you first need to develop a plan for your ad groups. This is the basic unit of organization where you map keywords, ads and landing pages. But what are these three components exactly?

  1. Keywords: the user's intention, what they are looking for.
  2. Ads: your promise to the user for that intention.
  3. Landing pages: your delivery of the promise.

Mapping those three elements properly is around 70-80% of the job. If you get this right, then maintenance and changes become much easier.

This means that, in order to be specific and relevant to users, you need to promote the product that you have, and mainly focus on sending the right person to the right page, through the right message or expectation. A good campaign structure reflects and follows the structure of the website, which is essentially reflecting the business strategy.

Note that every ad group needs to be part of a campaign. At the campaign level, you have settings that govern all ad groups in the campaign, such as language targeting, locations, devices, time of day, etc.

Search Campaign, Step-by-Step

In the previous section, you looked at search campaigns from a high level: you saw that keywords, ads, and landing pages are part of an ad group and that you need to come up with a plan for this basic unit of organization where you map these three components in such a way that they reflect your business strategy. In this section, you'll dive deeper into what you can to do set up a campaign, focusing on the steps that you will go through in this tutorial.

Search Engine Marketing Campaigns Data Science

As shown above, this is the mapping that you want to achieve. As output, you want to end up having two main DataFrames: one for keywords and another for ads. You can see how this would look like below.

Keywords
Campaign Ad Group Keyword
campaign 1 adgroup 1 keyword 1
campaign 1 adgroup 2 keyword 2
campaign 2 adgroup 1 keyword 3
campaign 2 adgroup 1 keyword 4
campaign 2 adgroup 2 keyword 5


Ads
Campaign Ad Group Display URL Final URL Headline 1 Headline 2
campaign 1 adgroup 1 datacamp.com datacamp.com/a Headline 1 Headline 1a
campaign 1 adgroup 2 datacamp.com datacamp.com/b Headline 1 Headline 1a
campaign 2 adgroup 1 datacamp.com datacamp.com/c Headline 1 Headline 1a
campaign 2 adgroup 2 datacamp.com datacamp.com/d Headline 2 Headline 2a
campaign 2 adgroup 3 datacamp.com datacamp.com/e Headline 3 Headline 1a


Once these are done, you would be ready to upload them as Comma-separated Value (CSV) files to Google AdWords or even other platforms that work exactly the same way. Google AdWords is by far the most popular paid search platform used by search marketers, in addition to Bing Ads, Yandex, Baidu, and others.

Note that if you want to import your data to other platforms and not Google Adwords or Bing Ads, you will need to do minor tweaks!

The Business Case: DataCamp

As you read in the introduction, you want to create search campaigns for DataCamp. Looking at the different ways in which someone can express interest in the product that this company provides, that is, data science education, you can easily come up with an initial list of some of those ways:

  1. Instructors: 'Hadley Wickham course', 'courses by Garrett Grolemund', etc.
  2. Technologies: 'python course', 'r courses', 'sql data science course', etc.
  3. Courses: names of specific courses that we have; 'unsupervised learning in python', 'deep learning in python', etc.
  4. Topics: 'machine learning courses', 'data visualization course', etc.
  5. Packages/libraries: 'learn ggplot', 'matplotlib tutorial', etc.

In this tutorial, you will not go through the traditional keyword research phase, and that's because you have the power of the programming language to generate all possible combinations and group them in their respective campaigns and ad groups.

Therefore, you need two main things to know to do this setup:

  • Product knowledge: knowing what courses and topics DataCamp has, how they are organized on the website, and how they are grouped. You can find this out by browsing, and you can definitely get better insights with internal knowledge (especially if there are changes planned!).
  • Relevant keywords: all the possible ways in which someone might express desire in the things that you provide. In DataCamp's case, that's 'course', 'learn', 'education', 'tutorial', etc. You can do this by brainstorming, looking into keyword tools, and other methods.

Keyword Generation For Your SEM Campaign

Let's start by generating the keywords!

1. Generate Instructor Keywords

This campaign will be targeting people who search for courses by any of the instructors at DataCamp.

You can have the keywords restricted to "course by <instructor name>" or you can go for a broad set of keywords targeting the instructor name only. At the beginning, it's better to target the names with 'course' or 'courses' and see the effect.

You will also need to check if any of the names that you have also happens to be the name of another famous person, or a very common name, and then you will need to restrict the keywords for that instructor.

You begin by getting the names of the instructors and the URLs for each:

import requests
from bs4 import BeautifulSoup


instructors_page = 'https://www.datacamp.com/instructors?all=true'
instructor_link_selector = '.instructor-block__description .instructor-block__link' # CSS class of the link
instructor_name_selector = '.mb-sm'  # CSS class of the name

instructor_resp = requests.get(instructors_page)
soup = BeautifulSoup(instructor_resp.text, 'lxml')

instructor_urls = [url['href'] for url in soup.select(instructor_link_selector)]
instructor_names = [name.text.strip() for name in soup.select(instructor_name_selector)]
instructor_urls = ['https://www.datacamp.com' + url for url in instructor_urls]

You put them in a DataFrame for later use. The URLs will be used later for generating ads.

instructor_df = pd.DataFrame({
    'name': instructor_names,
    'url': instructor_urls
})
print(instructor_df.shape)
instructor_df.head()
(72, 2)
name url
0 Filip Schouwenaars https://www.datacamp.com/instructors/filipsch
1 Jonathan Cornelissen https://www.datacamp.com/instructors/jonathana...
2 Hugo Bowne-Anderson https://www.datacamp.com/instructors/hugobowne
3 Greg Wilson https://www.datacamp.com/instructors/greg48f64...
4 Nick Carchedi https://www.datacamp.com/instructors/nickyc


Now that you have the names of instructors, you will be using a template whereby you combine each name with a set of keywords related to your topic, which is mainly courses by the instructor.

Variables used in the code below:

col_names: This is a list of the header names of the table that we will end up uploading to Google AdWords.
words: The words that you will be combining with the instructor names to generate the full keywords / phrases.
match_types: More details can be found on AdWords help center, but here are the basics.
[data science course], "data science course", and data science course are technically three different keywords.

  • exact match (in brackets), will only trigger ads if a user searches for exactly that keyword, 'data science course', for example, written exactly like this.
  • phrase match (with quotes) , will trigger your ads if a user searches for the exact string together with anything before, or after it. So "best data science course", or "data science course online" would trigger our ads.
  • broad match (no punctuation) would trigger our ads if someone searches for anything similar to or related to 'data science course'. This is up to Google's algorithms, and you have to be careful with it, as it might trigger ads when someone searches for 'data science platform' for example, which is not exactly what we are trying to promote. I like to use the modified broach match more, because it restricts targeting a little more. This is basically triggering ads if someone searches for any derivative of the word, and not any similar word in meaning. This is denoted with a '+' sign at the beginning of the word. So, '+game' would trigger ads by 'gaming', 'gamers', but not by 'play'.
col_names = ['Campaign', 'Ad Group', 'Keyword', 'Criterion Type']
instructor_keywords = []

words = ['course', 'courses', 'learn', 'data science', 'data camp', 'datacamp']
match_types = ['Exact', 'Phrase', 'Broad']
for instructor in instructor_df['name']:
    for word in words:
        for match in match_types:
            if match == 'Broad':
                keyword = '+' + ' +'.join([instructor.replace(' ', ' +').lower(), word])  # modified broach match
            else:
                keyword = instructor.lower() + ' ' + word
            row = ['SEM_Instructors',  # campaign name
                   instructor,  # ad group name
                   keyword, # instructor <keyword>
                   match]  # keyword match type
            instructor_keywords.append(row)

# do the same by having the keywords come before the instructor name
for instructor in instructor_df['name']:
    for word in words:
        for match in match_types:
            if match == 'Broad':
                keyword = '+' + ' +'.join([word, instructor.replace(' ', ' +').lower()])
            else:
                keyword = word + ' ' + instructor.lower() 
            row = ['SEM_Instructors',  # campaign name
                   instructor,  # ad group name
                   keyword, # <keyword> instructor 
                   match]  # keyword match type
            instructor_keywords.append(row)


instructor_keywords_df = pd.DataFrame.from_records(instructor_keywords, 
                                                   columns=col_names)
print('total keywords:', instructor_keywords_df.shape[0])
instructor_keywords_df.head()
total keywords: 2592
Campaign Ad Group Keyword Criterion Type
0 SEM_Instructors Filip Schouwenaars filip schouwenaars course Exact
1 SEM_Instructors Filip Schouwenaars filip schouwenaars course Phrase
2 SEM_Instructors Filip Schouwenaars +filip +schouwenaars +course Broad
3 SEM_Instructors Filip Schouwenaars filip schouwenaars courses Exact
4 SEM_Instructors Filip Schouwenaars filip schouwenaars courses Phrase


Basically, what you are doing is looping over the instructor names, all the different keywords, and all the match types that we have, so that we have all possible combinations. We are doing it twice, once to have the template 'instructor name keyword' and 'keyword instructor name'.

Now you simply repeat the same for all of our segments with minor modifications on keywords, and how we extract the data.

A good guideline that I learned from R for Data Science is that if you are going to copy and paste more than twice, it's time to write a function!

def generate_keywords(topics, keywords, match_types=['Exact', 'Phrase', 'Broad'],
                     campaign='SEM_Campaign'):
    col_names = ['Campaign', 'Ad Group', 'Keyword', 'Criterion Type']
    campaign_keywords = []

    for topic in topics:
        for word in keywords:
            for match in match_types:
                if match == 'Broad':
                    keyword = '+' + ' +'.join([topic.lower().replace(' ', ' +'), word.replace(' ', ' +')])
                else:
                    keyword = topic.lower() + ' ' + word
                row = [campaign,  # campaign name
                       topic,  # ad group name
                       keyword, # instructor <keyword>
                       match]  # keyword match type
                campaign_keywords.append(row)

    # I said more than twice! :)             
    for topic in topics:
        for word in keywords:
            for match in match_types:
                if match == 'Broad':
                    keyword = '+' + ' +'.join([word.replace(' ', ' +'), topic.lower().replace(' ', ' +')])
                else:
                    keyword = word + ' ' + topic.lower()
                row = [campaign,  # campaign name
                       topic,  # ad group name
                       keyword, # <keyword> instructor
                       match]  # keyword match type
                campaign_keywords.append(row)

    return pd.DataFrame.from_records(campaign_keywords, columns=col_names)

Let's give it a try:

topics = ['Data Science', 'Machine Learning']
keywords = ['course', 'tutorial']
generate_keywords(topics, keywords).head(10)
Campaign Ad Group Keyword Criterion Type
0 SEM_Campaign Data Science data science course Exact
1 SEM_Campaign Data Science data science course Phrase
2 SEM_Campaign Data Science +data +science +course Broad
3 SEM_Campaign Data Science data science tutorial Exact
4 SEM_Campaign Data Science data science tutorial Phrase
5 SEM_Campaign Data Science +data +science +tutorial Broad
6 SEM_Campaign Machine Learning machine learning course Exact
7 SEM_Campaign Machine Learning machine learning course Phrase
8 SEM_Campaign Machine Learning +machine +learning +course Broad
9 SEM_Campaign Machine Learning machine learning tutorial Exact


This looks good. Now let's generate the relevant topics and keywords for each of your segments!

2. Technology Keywords

topics = ['R', 'Python', 'SQL', 'Git', 'Shell']  # listed on the /courses page
keywords = ['data science', 'programming', 'analytics', 'data analysis', 'machine learning',
            'deep learning', 'financial analysis', 'data viz', 'visualization', 'data visualization',
            'learn', 'course', 'courses', 'education', 'data import', 'data cleaning', 
            'data manipulation', 'probability', 'stats', 'statistics', 'course', 'courses',
           'learn', 'education', 'tutorial']  # @marketing_team: this list can / should be refined or 
                                              # expanded based on the strategy and how specific the 
                                              # targeting needs to be
tech_keywords = generate_keywords(topics, keywords, campaign='SEM_Technologies')
print('total keywords:', tech_keywords.shape[0])
tech_keywords.head()
total keywords: 750
Campaign Ad Group Keyword Criterion Type
0 SEM_Technologies R r data science Exact
1 SEM_Technologies R r data science Phrase
2 SEM_Technologies R +r +data +science Broad
3 SEM_Technologies R r programming Exact
4 SEM_Technologies R r programming Phrase

3. Generating Course Keywords

This is probably the most specific, and therefore the most relevant of the segments to target people. If someone is searching for "data visualization with r" and DataCamp has that course, then the ad would be extremely relevant, because you would have the right landing page that exactly satisfies that user's need.

There's one small problem, though. Some of the course names don't correspond to what a user would typically search for: 'Machine Learning with the Experts: School Budgets', 'Sentiment Analysis in R: The Tidy Way'. These are not bad course names, but they will need some attention as to selecting the proper keywords that people might use to search for them.

Again, you can scrape the names and corresponding URLs as you did with the instructors' campaign:

courses_page = 'https://www.datacamp.com/courses/all'
course_link_selector = '.courses__explore-list .course-block'

course_resp = requests.get(courses_page)
soup = BeautifulSoup(course_resp.text, 'lxml')

course_urls = [link.contents[1]['href'] for link in soup.select(course_link_selector)] 
course_urls = ['https://www.datacamp.com' + url for url in course_urls]
course_names = [link.h4.text for link in soup.select(course_link_selector)]
course_df = pd.DataFrame({
    'name': course_names,
    'url': course_urls
})
course_df['name_clean'] = course_df.name.str.replace('\(.*\)', '').str.strip()  # remove (part x)
print('total keywords:', course_df.shape[0])
course_df.head()
total keywords: 104
name url name_clean
0 Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Intro to Python for Data Science
1 Introduction to R https://www.datacamp.com/courses/free-introduc... Introduction to R
2 Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-... Intermediate Python for Data Science
3 Introduction to Git for Data Science https://www.datacamp.com/courses/introduction-... Introduction to Git for Data Science
4 Intermediate R https://www.datacamp.com/courses/intermediate-r Intermediate R


You will do the same (use the generate_keywords() function) for the courses, but you need to be careful, as they need to be reviewed because as mentioned above, some of the names are not really what people would look for, and you just need to account for that case-by-case. The following should be good enough for a start, and then you can see the data and make decisions.

Please note that using the empty character below is not a mistake. The names of course are long and specific enough that they are fit to be keywords in and of themselves, without having to add other qualifier keywords like 'learn' or 'course'. So you will be using the course names alone, as well as with the qualifier keywords.

keywords = ['', 'learn', 'course', 'courses', 'tutorial', 'education']
course_keywords = generate_keywords(course_df['name_clean'], keywords, campaign='SEM_Courses')
print('total keywords:', course_keywords.shape[0])
course_keywords.head(10)
total keywords: 3744
Campaign Ad Group Keyword Criterion Type
0 SEM_Courses Intro to Python for Data Science intro to python for data science Exact
1 SEM_Courses Intro to Python for Data Science intro to python for data science Phrase
2 SEM_Courses Intro to Python for Data Science +intro +to +python +for +data +science + Broad
3 SEM_Courses Intro to Python for Data Science intro to python for data science learn Exact
4 SEM_Courses Intro to Python for Data Science intro to python for data science learn Phrase
5 SEM_Courses Intro to Python for Data Science +intro +to +python +for +data +science +learn Broad
6 SEM_Courses Intro to Python for Data Science intro to python for data science course Exact
7 SEM_Courses Intro to Python for Data Science intro to python for data science course Phrase
8 SEM_Courses Intro to Python for Data Science +intro +to +python +for +data +science +course Broad
9 SEM_Courses Intro to Python for Data Science intro to python for data science courses Exact

4. Keywords For Topics

These are basically generic topics that people might be interested in searching for. They are covered by the 'tracks' section, which has skills and career as sub-sections. For your purpose, they can be grouped under the same campaign. The process is again the same, just like you have done in the previous sections!

Skills

skills_page = 'https://www.datacamp.com/tracks/skill'
skills_link_selector = '#all .shim'

skills_resp = requests.get(skills_page)
skill_soup = BeautifulSoup(skills_resp.text, 'lxml')

skills_urls = [link['href'] for link in skill_soup.select(skills_link_selector)] 
skills_names = [skill.replace('/tracks/', '').replace('-', ' ') for skill in skills_urls]
skills_urls = ['https://www.datacamp.com' + url for url in skills_urls]

Careers

career_page = 'https://www.datacamp.com/tracks/career'
career_link_selector = '#all .shim'

career_resp = requests.get(career_page)
career_soup = BeautifulSoup(career_resp.text, 'lxml')

career_urls = [link['href'] for link in career_soup.select(career_link_selector)] 

career_names = [career.replace('/tracks/', '').replace('-', ' ') for career in career_urls]
career_urls = ['https://www.datacamp.com' + url for url in career_urls]
tracks_df = pd.DataFrame({
    'name': skills_names + career_names,
    'url': skills_urls + career_urls
})
tracks_df['name'] = [x.title() for x in tracks_df['name']]
tracks_df.head()
name url
0 R Programming https://www.datacamp.com/tracks/r-programming
1 Importing Cleaning Data With R https://www.datacamp.com/tracks/importing-clea...
2 Data Manipulation With R https://www.datacamp.com/tracks/data-manipulat...
3 Python Programming https://www.datacamp.com/tracks/python-program...
4 Importing Cleaning Data With Python https://www.datacamp.com/tracks/importing-clea...
tracks_keywords = generate_keywords(tracks_df['name'], keywords, campaign='SEM_Tracks')
print('total keywords:', tracks_keywords.shape[0])
tracks_keywords.head()
total keywords: 720
Campaign Ad Group Keyword Criterion Type
0 SEM_Tracks R Programming r programming Exact
1 SEM_Tracks R Programming r programming Phrase
2 SEM_Tracks R Programming +r +programming + Broad
3 SEM_Tracks R Programming r programming learn Exact
4 SEM_Tracks R Programming r programming learn Phrase


Finally, you concatenate all keywords that are listed in instructor_keywords, tech_keywords, course_keywords, and tracks_keywords so that they are all gathered into one big DataFrame:

full_keywords_df = pd.concat([instructor_keywords_df, tech_keywords, course_keywords, tracks_keywords])
print('total keywords:', full_keywords_df.shape[0])
print('total campaigns:', len(set(full_keywords_df['Campaign'])))
print('total ad groups:', len(set(full_keywords_df['Ad Group'])))
full_keywords_df.to_csv('keywords.csv', index=False)
full_keywords_df.head()
total keywords: 7806
total campaigns: 4
total ad groups: 191
Campaign Ad Group Keyword Criterion Type
0 SEM_Instructors Filip Schouwenaars filip schouwenaars course Exact
1 SEM_Instructors Filip Schouwenaars filip schouwenaars course Phrase
2 SEM_Instructors Filip Schouwenaars +filip +schouwenaars +course Broad
3 SEM_Instructors Filip Schouwenaars filip schouwenaars courses Exact
4 SEM_Instructors Filip Schouwenaars filip schouwenaars courses Phrase


Now you're ready to go with your keywords! If you want to take a look at the full data set, get it here. Next, you need to generate ads for each of your ad groups.

Let's generate the ads.

Ad Generation and Copy Writing

You currently have the following:

  • Campaigns and ad group names properly mapped to each other in the full_keywords_df DataFrame
  • The ad group name and the corresponding URL from your previous scraping (courses, instructors, and tracks), and you need to generate one for technologies.

With this at hand, you can follow the following plan:

  1. Create ad templates to use (2-3 should be good to start with)
  2. Create a Campaign column and add it to the ('name', 'url') DataFrames
  3. Merge all the ('name', 'url') DataFrames into one big one
  4. Generate all ads for all ad groups; this consists of the following fields for each ad (these would be new columns in the same DataFrame you are working with):

    • Headline 1: maximum 30 characters
    • Headline 2: maximum 30 characters
    • Display URL: automatically inferred from the final URL
    • Final URL: the full path where the user will end up (has to be the same domain as the display URL
  5. Make sure the work is consistent with the keywords DataFrame

  6. Upload and launch campaigns!

Generating ads simply means replacing the topics name (course, tech, instructor, etc) where it belongs in the template, and making 2-3 ad variations for each of the ad groups that you have.

Python SEM Campaign

Ad Templates

An important thing to take note of is that although plugging in the ad group names where they belong is a straightforward process, the problem is that you need to have the length of the fields under the limits mentioned above. And since the names of your courses and topics vary a lot, you need to see what you can do about it.

Let's see how much of a problem that is:

%matplotlib inline
import matplotlib.pyplot as plt
adgroup_lengths = pd.Series([len(adgrp) for adgrp in full_keywords_df['Ad Group'].unique()])
long_adgroups = sum(adgroup_lengths > 30)
plt.figure(figsize=(9,6))
plt.hist(adgroup_lengths, rwidth=0.9, bins=50)
plt.vlines(x=30, ymin=0, ymax=10, colors='red')
plt.title(str(long_adgroups) + ' ad group name lenghts > 30 (don\'t fit in a headline)',  fontsize=17)
plt.xlabel('Ad Group Name Lengths', fontsize=15)
plt.ylabel('Count', fontsize=15)
plt.yticks(range(11))
plt.xticks(range(0, 51, 5))
plt.grid(alpha=0.5)
plt.show()

Search Engine Marketing Data Science Python

It seems your problem is not a trivial one, and there is no straightforward way of dealing with it. Ideally, you would like to have the two headlines contain the full title of the product (course) that we are promoting.

I thought of different ways around it, and decided to write a simple algorithm that splits the name into two phrases, each containing at most thirty characters.

The algorithm is not suitable for general use, and needs some tweaks to make it general, but it works well enough for this data set:

def split_string(string, splits=2, max_len=60):
    """Split `string` into `splits` words, each shorter than `max_len` / `splits`"""
    if len(string) < max_len / splits:
        return string, ''
    str_words = string.split(' ')
    result = ''
    for i, word in enumerate(str_words):
        if len(result + ' ' + word) <= max_len / splits:
            result += word + ' '
        else:
            break
    spaces = result.strip().count(' ')
    result2 = string[string[len(result):].index(word) + len(result):]
    return result.strip(), result2
print(split_string('this is a very long course name that needs splitting', 2, 60))
print(split_string('short course name', 2, 60))
('this is a very long course', 'name that needs splitting')
('short course name', '')

This is not something that I do usually, but I like this technique, and I think I'll be using it when creating other campaigns. That's a nice side effect of writing a tutorial!

Now let's think about the templates that you want to write.

The general ad template that I use consists of the following elements (typically in this order).

  • Product: if I'm searching for 'data science course' on Google, I need to see 'data science course' in the links / ads somewhere.
  • Benefits: this is the emotional / psychological thing that people are really after, beyond just completing the course, 'boost your career', 'stand out from the crowd'.
  • Features: now that you've promised me the moon, show me how you are going to get me there! 'over 100 data science courses', 'learn from top experts', 'get instant feedback on your coding skills'.
  • Call to action: OK, so you told me what you have, you motivated me to buy it, and you showed me how I'm going to get there. I'm sold. What do I do now? 'sign up for a free trial', 'sample first chapters for free', 'save 20% on annual subscriptions'

It's not easy to put all these in one ad which is a tweet long, and some of them may overlap. But you will try your best.

  • Headline 1: This will always contain the name of the course (or the first half).
  • Headline 2: The second half of the course name, or one of the following:
    • Boost Your Data Science Career
    • Stand Out From the Crowd
    • Tackle Complex Questions
  • Description: each ad group will have the three variations below and they will rotate.
    • Learn Directly From the Top Experts in the Field. 20% off Annual Subscriptions
    • Get Ahead of the Curve, Master Data Science Skills. $29 / Month. Cancel Anytime
    • Choose From a Wide Variety of Topics Tuaght by the Best in the World. Start Now

Create Campaign Columns

full_keywords_df.Campaign.unique()  # just to make sure you have consistent naming conventions
array(['SEM_Instructors', 'SEM_Technologies', 'SEM_Courses', 'SEM_Tracks'], dtype=object)
course_df['Campaign'] = 'SEM_Courses'
course_df = course_df.rename(columns={'name_clean': 'name', 'name': 'old_name'})
course_df.head()
old_name url name Campaign
0 Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Intro to Python for Data Science SEM_Courses
1 Introduction to R https://www.datacamp.com/courses/free-introduc... Introduction to R SEM_Courses
2 Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-... Intermediate Python for Data Science SEM_Courses
3 Introduction to Git for Data Science https://www.datacamp.com/courses/introduction-... Introduction to Git for Data Science SEM_Courses
4 Intermediate R https://www.datacamp.com/courses/intermediate-r Intermediate R SEM_Courses
instructor_df['Campaign'] = 'SEM_Instructors'
instructor_df.head()
name url Campaign
0 Filip Schouwenaars https://www.datacamp.com/instructors/filipsch SEM_Instructors
1 Jonathan Cornelissen https://www.datacamp.com/instructors/jonathana... SEM_Instructors
2 Hugo Bowne-Anderson https://www.datacamp.com/instructors/hugobowne SEM_Instructors
3 Greg Wilson https://www.datacamp.com/instructors/greg48f64... SEM_Instructors
4 Nick Carchedi https://www.datacamp.com/instructors/nickyc SEM_Instructors
tracks_df['Campaign'] = 'SEM_Tracks'
tracks_df.head()
name url Campaign
0 R Programming https://www.datacamp.com/tracks/r-programming SEM_Tracks
1 Importing Cleaning Data With R https://www.datacamp.com/tracks/importing-clea... SEM_Tracks
2 Data Manipulation With R https://www.datacamp.com/tracks/data-manipulat... SEM_Tracks
3 Python Programming https://www.datacamp.com/tracks/python-program... SEM_Tracks
4 Importing Cleaning Data With Python https://www.datacamp.com/tracks/importing-clea... SEM_Tracks
tech_domain = 'https://www.datacamp.com/courses/tech:'
tech_domain_list = []
for tech in ['R', 'Python', 'SQL', 'Git', 'Shell']:
    tech_domain_list.append((tech, tech_domain + tech))
tech_df = pd.DataFrame.from_records(tech_domain_list, columns=['name', 'url'])
tech_df['Campaign'] = 'SEM_Technologies'
tech_df
name url Campaign
0 R https://www.datacamp.com/courses/tech:R SEM_Technologies
1 Python https://www.datacamp.com/courses/tech:Python SEM_Technologies
2 SQL https://www.datacamp.com/courses/tech:SQL SEM_Technologies
3 Git https://www.datacamp.com/courses/tech:Git SEM_Technologies
4 Shell https://www.datacamp.com/courses/tech:Shell SEM_Technologies

Merge All ('name', 'url') DataFrames

full_ads_df = pd.concat([course_df[['Campaign', 'name', 'url']],
                        instructor_df,
                        tracks_df,
                        tech_df], ignore_index=True)
full_ads_df = full_ads_df.rename(columns={'name': 'Ad Group', 'url': 'Final URL'})
print('total rows:', full_ads_df.shape[0])
n_adgroups = full_ads_df.shape[0]
full_ads_df.head()
total rows: 201
Campaign Ad Group Final URL
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc...
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-...
3 SEM_Courses Introduction to Git for Data Science https://www.datacamp.com/courses/introduction-...
4 SEM_Courses Intermediate R https://www.datacamp.com/courses/intermediate-r

Generate Ads

Just to keep track of where you are. You now have the ads data frame containing Campaign, Ad Group, and Final URL columns properly mapped. You need to add the Headline 1, Headline 2, and Description Fields.

Keep in mind that for each ad group that you will be adding three different ad variations. This is a good practice mainly for testing purposes, to see what people click on, which ads convert more, etc. You should end up with a DataFrame that has three times the number of rows of the current one.

Let's start by duplicating each row three times, in the full_ads_df:

full_ads_df = full_ads_df.iloc[[x  for x in range(n_adgroups) for i in range(3)], :] 
print('total rows:', full_ads_df.shape[0])
full_ads_df.head(9)
total rows: 603
Campaign Ad Group Final URL
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth...
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth...
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc...
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-...
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-...
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-...


You now add the three different descriptions that you created above.

Description = [
    'Learn Directly From the Top Experts in the Field. 20% off Annual Subcriptions',
    'Be Ahead of the Curve, Master Data Science Skills. $29 / Month. Cancel Anytime',
    'Choose From a Wide Variety of Topics Tuaght by the Best in the World. Start Now'   
]
Description = [x for i in range(n_adgroups) for x in Description ]
full_ads_df['Description'] = Description
full_ads_df.head()
Campaign Ad Group Final URL Description
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Learn Directly From the Top Experts in the Fie...
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Be Ahead of the Curve, Master Data Science Ski...
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Choose From a Wide Variety of Topics Tuaght by...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc... Learn Directly From the Top Experts in the Fie...
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc... Be Ahead of the Curve, Master Data Science Ski...


Next, you add Headline 1 and Headline 2 to the DataFrame. You're almost done!

benefits = [
    'Boost Your Data Science Career',
    'Stand Out From the Crowd',
    'Tackle Complex Questions'    
]

The benefits will be included as Headline 2 in the cases where the length of the ad group is 30 or less. In this case, you would have an empty field, which you will fill with one of the benefits that you have.

benefits = [x for i in range(n_adgroups) for x in benefits]
headlines = [split_string(x) for x in full_ads_df['Ad Group']]
full_ads_df['Headline 1'] = [x[0] for x in headlines]
full_ads_df['Headline 2'] = [x[1] if x[1] else benefits[i] for i, x in enumerate(headlines)]
print('total ads:', full_ads_df.shape[0])
full_ads_df.head(9)
total ads: 603
Campaign Ad Group Final URL Description Headline 1 Headline 2
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Learn Directly From the Top Experts in the Fie... Intro to Python for Data Science
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Be Ahead of the Curve, Master Data Science Ski... Intro to Python for Data Science
0 SEM_Courses Intro to Python for Data Science https://www.datacamp.com/courses/intro-to-pyth... Choose From a Wide Variety of Topics Tuaght by... Intro to Python for Data Science
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc... Learn Directly From the Top Experts in the Fie... Introduction to R Boost Your Data Science Career
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc... Be Ahead of the Curve, Master Data Science Ski... Introduction to R Stand Out From the Crowd
1 SEM_Courses Introduction to R https://www.datacamp.com/courses/free-introduc... Choose From a Wide Variety of Topics Tuaght by... Introduction to R Tackle Complex Questions
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-... Learn Directly From the Top Experts in the Fie... Intermediate Python for Data Science
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-... Be Ahead of the Curve, Master Data Science Ski... Intermediate Python for Data Science
2 SEM_Courses Intermediate Python for Data Science https://www.datacamp.com/courses/intermediate-... Choose From a Wide Variety of Topics Tuaght by... Intermediate Python for Data Science


As you can see, "Intro to Python for Data Science" was long enough to take two fields, so you keep it as is, to keep it clear. In the second case "Introduction to R" fits in the first headline, so you insert the three different benefits that you created in Headline 2.

And you're done!

Double Check The Details

Just a quick check, to make sure that all campaign and ad group names in both DataFrames are the same.

ads_check = (full_ads_df[['Campaign', 'Ad Group']]
             .drop_duplicates()
             .sort_values(['Campaign', 'Ad Group'])
             .reset_index(drop=True))
keywords_check = (full_keywords_df[['Campaign', 'Ad Group']]
                  .drop_duplicates()
                  .sort_values(['Campaign', 'Ad Group'])
                  .reset_index(drop=True))

all(ads_check == keywords_check)
True
full_ads_df.to_csv('ads.csv', index=False)

Upload and Launch

As discussed, this tutorial only covered specific steps within SEM campaigns and therefore requires a basic understanding of how search engine marketing works. If you want to know more about the mechanics and details about uploading and launching your ad groups, you can check the AdWords' guide for how to get started. To access the full courses in different areas, you can check out the Google Academy for Ads.

Data Science and SEM: There's Much More to Discover!

With the two CSV files you have now, you can upload them and start running your campaigns. You will need to set daily budgets, geo-targeting and a few other parameters, but for the most part, you are good to go.

If you want to check out the CSV files, you can find them here:

I hope that you found this tutorial useful. Thanks for reading and if you want to share any growth projects that you did with the help of Python, don't hesitate to reach out to me on Twitter @eliasdabbas.

Want to leave a comment?