https://datacamp-community-prod.s3.amazonaws.com/65076e3c-9df1-40d5-a0c2-36294d9a3ca9

This PySpark SQL cheat sheet is your handy companion to Apache Spark DataFrames in Python and includes code samples. 

PySpark Cheat Sheet: Spark DataFrames in Python

https://datacamp-community-prod.s3.amazonaws.com/4d91fcbc-820d-4ae2-891b-f7a436ebefd4

This PySpark cheat sheet with code samples covers the basics like initializing Spark in Python, loading data, sorting, and repartitioning.

PySpark Cheat Sheet: Spark in Python

https://www.datacamp.com/community/podcast/buzzfeed-digital-media

<p>How does data science help Buzzfeed achieve online virality? What type of mass online experiments do data scientists at BuzzFeed run for this purpose? What products do they develop to make all of this easy and intuitive for content producers?&nbsp;</p>

How does data science help Buzzfeed achieve online virality? What type of mass online experiments do data scientists at BuzzFeed run for this purpose? What products do they develop to make all of this easy and intuitive for content producers? 

Data Science at BuzzFeed and the Digital Media Landscape

https://www.datacamp.com/community/podcast/data-science-astronomy

<p>Data science, astronomy, the open source development world and the importance of interdisciplinary conversations to data science.&nbsp;</p>

Discover how data science impacts modern astronomy, the importance of the open source development world and how essential interdisciplinary conversations are to data science. 

Data Science, Astronomy and the Open Source

https://www.datacamp.com/community/news/cricpy-takes-a-swing-at-the-odis-nohaj97lw3

No computer has ever been designed that is ever aware of what it’s doing; but most of the time, we aren’t either.” Marvin Minksy “The competent programmer is fully aware of the limited size of his own skull. He therefore approaches his task with full humility, and avoids clever tricks like the...

Cricpy takes a swing at the ODIs

https://www.datacamp.com/community/news/introducing-cricpya-python-package-to-analyze-performances-of-cricketers-nxcr6rpgaik

Full many a gem of purest ray serene, The dark unfathomed caves of ocean bear; Full many a flower is born to blush unseen, And waste its sweetness on the desert air. Thomas Gray, An Elegy Written In A Country Churchyard Introduction It is finally here! cricpy, the python avatar , of my R...

Introducing cricpy:A python package to analyze performances of...

https://www.datacamp.com/community/news/data-science-nuclear-engineering-and-the-open-source-with-katy-huff-dye1i4ngg8e

Data science, nuclear engineering, the importance of interdisciplinary data science and the open source.

Data Science, Nuclear Engineering and the Open Source (with Katy Huff)

https://www.datacamp.com/community/news/data-science-at-buzzfeed-and-the-digital-media-landscape-50nqxmvpwed

https://www.datacamp.com/community/news/the-10-algorithms-machine-learning-engineers-need-to-know-9jasu2c6ici

It is no doubt that the sub-field of machine learning / artificial intelligence has increasingly gained more popularity in the past couple of years. So if you want to learn more about machine learning, how do you start?

The 10 Algorithms Machine Learning Engineers Need to Know

<p><center><a href="https://www.datacamp.com/groups/business"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1603718763/banner_style_1_cruuf7.png" /></a><br></p>
<p>Data engineering continues to be a top priority for enterprises, and in 2021, there will be exciting developments in the data infrastructure space. In a <a href="https://www.datacamp.com/resources/webinars/operationalizing-data-within-large-organizations">recent webinar</a>, Maksim Percherskiy, Data Engineer at The World Bank and former Chief Data Officer of the City of San Diego, highlighted three trends to watch out for in particular: data orchestration platforms, data discovery engines, and data lakehouses.</p>
<h2 id="data-orchestration-platforms">Data orchestration platforms</h2>
<p>Although orchestration platforms have been around for many years to manage computer systems and software, <a href="https://dzone.com/articles/data-orchestration-its-open-source-but-what-is-it">data orchestration</a> is a relatively new concept that abstracts data access across storage systems, virtualizes data, and presents data to data-driven applications. Data orchestration platforms help companies become more data-driven by combining siloed data from multiple data storage locations and making them usable. Examples include <a href="https://airflow.apache.org/">Apache Airflow</a>, <a href="https://www.prefect.io/">Prefect</a>, <a href="https://luigi.readthedocs.io/en/stable/">Luigi</a>, and <a href="https://www.stitchdata.com/">Stitch</a>, which are compatible with modern software approaches like version control, DevOps, and continuous integration.</p>
<p>DataCamp’s course <a href="https://www.datacamp.com/courses/introduction-to-airflow-in-python">Introduction to Airflow in Python</a> is a great place to start to learn how to implement and schedule ETL and data engineering workflows in an easy and repeatable way.</p>
<h2 id="data-discovery-engines">Data discovery engines</h2>
<p>It makes sense that as data increases, companies will invest more time in enabling their teams to find the data they need, document it, and reduce rework. Data discovery engines like <a href="https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-engine-62d27254fbb9?gi=a4af847eb7fe">Lyft’s Amundsen</a> and <a href="https://eng.uber.com/databook/">Uber’s Databook</a> aim to improve the productivity of data users by providing a search interface for data. These tools rely on metadata, which support productivity and compliance by allowing data infrastructure to scale as companies grow. The goal of data discovery is to make data more FAIR: findable, accessible, interoperable, and reusable.</p>
<p><center><a href="https://www.datacamp.com/groups/business"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1610377093/datadiscoveryengines_pi2uj7.png" /></a><br></p>
<h2 id="data-lakehouse">Data lakehouse</h2>
<p>There are different use cases for <a href="https://www.datacamp.com/community/blog/data-lakes-vs-data-warehouses">data lakes versus data warehouses</a>. Each is designed for different purposes and for different end users. For example, data warehouses are typically used by analysts for the purpose of collecting business intelligence—they contain flat files that have been explicitly processed for their work. On the other hand, data lakes are set up and maintained by data engineers who integrate them into data pipelines. Data scientists work more closely with data lakes as they contain data of a wider and more current scope.</p>
<p>Data lakehouses are exactly what they sound like: a combination of data lakes and data warehouses. Companies like <a href="https://delta.io/">Delta Lake</a> and <a href="https://www.snowflake.com/guides/what-data-lakehouse">Snowflake</a> have created solutions to provide a single source of truth for data that combines the benefits of data lakes and data warehouses. Data lakehouses implement data warehouses’ data structures and management features for data lakes, which are typically more cost-effective for data storage. At the same time, they retain the schema and versioning of the data. With data lakehouses, data scientists will be able to conduct both machine learning and business intelligence.</p>
<p><center><a href="https://databricks.com/blog/2020/01/30/what-is-a-data-lakehouse.html"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1610377091/datalakehouse_dv35im.png" /></a></p>
<h5 id="source-databricks-https-databricks-com-blog-2020-01-30-what-is-a-data-lakehouse-html-">Source: <a href="https://databricks.com/blog/2020/01/30/what-is-a-data-lakehouse.html">Databricks</a></h5>
<p>All three of these trends will be crucial for enterprises to scale data infrastructure in 2021. They’re useful for companies that were built around data as well as for those that are struggling to sustainably integrate data in their workflows. <a href="https://www.datacamp.com/community/blog/data-driven-enterprise">Find out more</a> about the challenges large companies face to operationalize their data and become data-driven.</p>
<p><center><a href="https://www.datacamp.com/business/demo"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1603718763/banner_style_2_m573ef.png" /></a></p>


https://www.datacamp.com/community/blog/data-infrastructure-trends

Get your data engineering function ahead of the curve with orchestration platforms, data discovery engines, and data lakehouses.

Top 3 Trends in Data Infrastructure for 2021 | DataCamp

Top 3 Trends in Data Infrastructure for 2021

<center><a href="https://www.datacamp.com/tracks/tableau-fundamentals"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1610375720/TableauCTA_v2qbt7.png" /></a></center><br>

<p>Tableau is a business intelligence (BI) platform that functions like a supercharged Microsoft Excel. Its user-friendly, drag-and-drop functionality makes it possible for anyone who interacts with data to analyze and create presentation-ready visualizations—from data newbies to analysts, and even business leaders. You definitely don’t need to be a seasoned data scientist to use Tableau!</p>

<h2>Which companies use Tableau?</h2>
<p>Tableau’s roster of customers is a who’s who of the business world, including well-known brands such as Amazon, Experian, and Unilever. But it’s not just global giants using Tableau to democratize their data—over <a href="https://techcrunch.com/2019/06/10/salesforce-is-buying-data-visualization-company-tableau-for-15-7b-in-all-stock-deal/">86,000 businesses</a> are using Tableau to extract and share insights from their data.</p>

<h2>Certification overview</h2>

<center><CALLOUT BOX>
<a href="https://www.tableau.com/learn/certification/desktop-specialist">Tableau Desktop Specialist</a><br>
Fee: $100<br>
Title valid for: No expiration<br>
</CALLOUT BOX></center><br>


<p>Tableau recommends that you have at least three months’ product experience before taking the Desktop Specialist exam. If you don’t yet have this experience, DataCamp recommends that you prepare for the exam by completing our online <a href="https://www.datacamp.com/tracks/tableau-fundamentals">Tableau Fundamentals</a> skill track.</p>

<p>The exam will take place on GoToMeeting or Zoom, and you will be asked to set up your workspace using Google Chrome and confirm that your connection speed meets Tableau’s requirements. Additionally, you will be required to:</p>

<ul>
<li>Present photo identification to the exam proctor via webcam</li>
<li>Close programs, tabs, and remove any paperwork that may help during your exam</li>
<li>Demonstrate to the proctor that you are alone in your room via webcam</li>
<li>Agree to being monitored via webcam for the duration of the exam</li>
</ul>

<p><a href="https://mkt.tableau.com/files/Tableau-Certification-4-steps-to-exam-success.pdf">View the full exam setup requirements</a></p>

<p>Just like the Tableau learning experience <a href="https://www.datacamp.com/tracks/tableau-fundamentals">on DataCamp</a>, the exam is completed using a Windows virtual machine that has Tableau Desktop installed—this may involve a certain amount of toggling between the question and virtual desktop screens which may result in a slightly smaller screen size than you may be used to. It’s also worth noting that you may not have any other windows or programs open for the duration of the test other than those required for the assessment—just like exams in school, you can’t have any notes hidden in your calculator sleeve. The exam is scored automatically and a passing score of 70% or more is required to obtain your certification.</p>

<p>You can register and book your exam on the <a href="https://www.tableau.com/learn/certification/desktop-specialist">Tableau Product Certification Exam center</a>, managed by Loyalist Exam Services (LES). You can also reschedule your exam using this site.</p>

<h2>Training Resources</h2>
<p>At DataCamp, all our instructor-led courses are fully interactive, with no installation or prior experience required. You will also receive a statement of accomplishment which you can also share on your LinkedIn profile upon completion. Please note that DataCamp’s statement of accomplishment is not the same as the Tableau Desktop Specialist certification:</p>

<p><a href="https://www.datacamp.com/tracks/tableau-fundamentals">Tableau Fundamentals</a>: 14 hours. 4 Courses</p>
<p>Gain the essential skills you need to confidently use Tableau and prepare to pass the Tableau Desktop Specialist certification. No prior experience required.</p>

<ul>
<li><a href="https://www.datacamp.com/courses/introduction-to-tableau">Introduction to Tableau</a>: 4 hours<br>
Learn how to use Tableau’s user-friendly drag-and-drop functionality to quickly clean, analyze, and visualize your team’s data.</li>
<li><a href="https://datacamp.com/courses/analyzing-data-in-tableau">Analyzing data in Tableau</a>: 4 hours<br>
Create detail-rich map visualizations, configure date and time fields to show trends over time, and extend your data using Calculated Fields.</li>
<li><a href="https://datacamp.com/courses/creating-dashboards-in-tableau">Creating Dashboards in Tableau</a>: 4 hours<br>
Apply dashboard best practices, add interactive or explanatory elements, and use actions to make your dashboard fully interactive.</li>
<li><a href="https://www.datacamp.com/courses/connecting-data-in-tableau">Connecting Data in Tableau</a>: 4 hours<br>
Discover how  to connect Tableau to different data sources and prepare the data for a smooth analysis.</li>
</ul>

<p>Tableau also provides an excellent choice of learning resources you can use to prepare for the exam:</p>

<ul>
<li><a href="https://www.tableau.com/learn/training/20204">Tableau Training Videos</a></li>
<li><a href="https://mkt.tableau.com/files/DesktopSpecialist_ExamGuide.pdf">Desktop Specialist Exam Prep Guide (PDF)</a></li>
<li><a href="https://community.tableau.com/s/explore-forums?_ga=2.195627941.886887347.1610106578-1596736181.1607685378">Tableau community forums</a> and <a href="https://community.tableau.com/s/groups?_ga=2.195627941.886887347.1610106578-1596736181.1607685378">user groups</a></li>
</ul>

<p>Found this article useful? If so please share on <a href="https://www.linkedin.com/company/datacampinc/">LinkedIn</a>, <a href="https://twitter.com/DataCamp">Twitter</a>, or <a href="https://www.facebook.com/datacampinc/">Facebook</a>.</p>

<br><center><a href="https://www.datacamp.com/tracks/tableau-fundamentals"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1610375720/TableauCTA_v2qbt7.png" /></a></center>


https://www.datacamp.com/community/blog/Tableau-desktop-specialist-certification

Add Tableau to your data analyst skill set. Learn to use this BI platform and pass the Desktop Specialist Certification exam with DataCamp.

Pass Tableau’s Desktop Specialist Certification | DataCamp

Tableau Desktop Specialist Certification: Learn How to Use Tableau and Ace the Exam

https://www.datacamp.com/community/tutorials/generate-data-trumania

Why do data scientists and data engineers work with synthetic data? How do they obtain it? Discover Trumania, a scenario-based random dataset generator library.

Generating Synthetic Data with Trumania

Generating Realistic Random Datasets with Trumania

https://www.datacamp.com/community/tutorials/apache-spark-tutorial-machine-learning

Apache Spark tutorial introduces you to big data processing, analysis and ML with PySpark.

Apache Spark tutorial introduces you to big data processing, analysis and Machine Learning (ML) with PySpark.

community

datacamp

big data

News (36)

Cricpy takes a swing at the ODIs

Data Science at BuzzFeed and the Digital Media Landscape

Data Science, Nuclear Engineering and the Open Source (with Katy Huff)

Tutorials (3)

Generating Realistic Random Datasets with Trumania

Apache Spark Tutorial: ML with PySpark

Cheat Sheets (2)

PySpark Cheat Sheet: Spark DataFrames in Python

Open Courses (0)

Podcast (5)

Data Science at BuzzFeed and the Digital Media Landscape (with Adam Kelleher)

Blog Posts (24)

Top 3 Trends in Data Infrastructure for 2021

Tech Thoughts (0)