Apache Spark Mitap

April 27 at the Attic RAMBLER & Co will be held the first mitap dedicated to working with Apache Spark.

Apache Spark has already established itself as one of the main frameworks for working with big data and has been successfully used in such large companies as Amazon, Baidu, IBM, Databricks, NASA JPL and TripAdvisor. We know that in Russia Spark is used in many small and in some large companies, and very effectively.

At Rambler & Co, we have been using Spark for almost all the tasks of the department of advertising technologies related to ETL and machine learning for about a year. Moreover, at the beginning of the year we successfully upgraded to version 2.1.0.
')
At the mitap, we would like to share our experience in implementing Spark in production, talk about the problems we have encountered, and discuss the solutions that have been applied. Find out what new and cool features appeared in Spark 2, and what bugs successfully migrated from previous versions. And, of course, meet other enthusiasts and practitioners of this wonderful tool and make our event regular! Come, it will be interesting!

Topics for presentations:

1. Pavel Klemenkov (Head of Machine Learning)
Pipeline machine learning on apache spark
What did we have before Spark, how did we get to it, and what does mathematics-programmers have to do with it?

2. Konstantin Kolokolov (mathematician-programmer) and Vladimir Shtanko (mathematician-programmer)
How to program correctly on PySpark?
A brief introduction to the framework architecture. What can go wrong, where to look and how to fight? How not to shoot yourself in the foot?

3. Dmitry Nosov (mathematician-programmer)
Criteo 1TB benchmark
Test Vowpal Wabbit, XGBoost and Spark ML on the dataset Criteo

4. Shorin Alexander (devops development engineer)
Minutes from life with Spark
How Spark lives with us, how we live with it, history of exploitation, support, struggle with underwater rakes.

Collection of guests at 18.30.
The beginning of the first report at 19.00.

Registration: rambler-co-e-org.timepad.ru/event/470664
Broadcast link: www.facebook.com/afishamansarda