What experts in Data Science and Big Data are discussing today

Today we decided to go through the ranking of experts on Data Science at Quora and see what the most active members of the community are discussing.

William Chen, who works at Quora as an analyst [Eng. data scientist] shares his experience regarding his toolkit. He says his team uses Python and SQL. Many others also use the statistical package R, but due to the fact that the main Quora code is written in Python, William’s approach significantly speeds up the work of his colleagues.
')
The Chen team regularly uses iPython Notebook and Jupyter to record the results of their calculations. For data analysis, they most often use the Pandas, Seaborn, Numpy and SciPy packages. William's colleagues in the field of activity for the most part use Sublime Text for development and Unison for file synchronization. Just like the Quora developers, his team uses the Phabricator for version control and code analysis.

It is widely believed that every data scientist must know R, Matlab and Hadoop. Ricardo Vladimiro ( Ricardo Vladimiro ), a specialist in this area and an employee of Miniclip, believes that this is not the case. In his opinion, in order to really dive into the study of data, you must be well versed in statistics and probability theory, be able to conduct experiments and test your hypotheses, and also know at least one programming language that allows you to process "big data."

Ricardo adds that you need to understand the field of knowledge itself, where the data for analysis come from. In addition, the name “data science” itself says that knowledge of methods of storing, managing, processing and transmitting data will not prevent a specialist from doing so. Among personal qualities, he highlights the desire for new knowledge: whether it is algorithms, programming languages ​​or business communication skills.

Data scientist will not be able to do without programming skills:

“You will constantly limit yourself, instead of achieving the desired result. You can grow only if you leave your comfort zone. Get over it. Programming is not such a complicated thing. ”

If you work in the field of Big Data and, say, you want to solve problems of dynamic pricing, then you should be an expert in at least one of these areas: economics, econometrics, finance, statistics or industrial engineering. So says Laszlo Korsos (Laszlo Korsos), a senior analyst at Uber. William Chen adds that having programming skills will be a huge advantage.

If you think your programming skills leave a lot to be desired, remember that you still have a chance. Paul DeVos from IBM Watson Health recommends paying attention to job openings with a focus on analytics. Among the requirements for such positions usually indicate skills in SQL, Excel, SAS and SPSS. It is good if you know how to work with the R package or analytical tools for Python (Numpy, Pandas, Scipy, Scikit Learn, Seaborn, Plotly, Matplotlib). They are slightly more complicated than SPSS or Excel, but they can be mastered quickly enough without significant programming experience.

When studying the topic of Data Science, as Joe Blitzstein writes, practice is the most important thing. Of course, you will learn something if you take video courses, but this activity is too passive. Practical skills are acquired only in the course of laboratory work and homework. It makes no sense to watch the video all day: most barely stand an hour lecture.

Regarding additional literature, Pandora Research Director Michael Hochster recommends reading Scott McCloud's Comic Mechanics . Quite a large part of the work on data analysis, in his opinion, is communication with words and pictures, as in comics. The book is filled with deep reasoning and many examples and, according to Hochster, it will be more useful and more interesting than the standard literature on data visualization.



Today, experts in the field of Bid Data can be divided, for example, into those who analyze data in Excel, and those who write models in R or Python. Dmitry Korolev ( Dima Korolev ), who worked at Google, Facebook and Microsoft, believes that employees with universal expertise like the concept of a “universal Full-Stack Developer” will soon be in demand.

Apple developer and founder of several IT start-ups Shane Ryoo talks about how he hires data science specialists. First of all, a person should be able to program well in Python, C / C ++ and / or Java: he is not interested in R, Matlab and other languages. The candidate must be able to compose algorithms and it is advisable to understand machine learning. In addition, he should be able to tell in detail about the maximum likelihood method, Bayes theorem, Viterbi algorithm and regularization, ideally - to write an article on these topics.

Many are interested in the level of earnings in this area. Paul DeVos claims that last year, for example, in Dallas, the average salary was about 130 thousand dollars. He is familiar with the three professionals who receive such a salary. "Each one has a different experience, and each has a master's degree," says Paul.

Elias Abu-Haydar ( Elias Abou Haydar ), a data scientist at iGraal, believes that the most successful colleagues are distinguished by effective communication skills, in particular, working with the media. He notes that this does not mean at all that other, less visible specialists are somehow worse.

“We don’t have to be sweet, especially when there are a lot of those who boast more of what they do,” writes Elias. Of course, not the last role is played by the experience and skills of solving complex analytical problems.

Working with data forces you to communicate with people from different departments of your company. As a result, you find yourself in the center of events, so you need to understand in which areas the business works, what employees do and how you can interact with them. Therefore, work in the field of Big Data gives you a clear advantage over specialists from nearby fields of activity and more opportunities for career growth.