Course program and materials on Scala

Good day.

My name is Golovach Ivan, I am a Java Tech Lead practitioner with 10+ years of programming experience (Java EE, J2ME, C, C ++, M-language, Delphi), who switched to Scala.

I prepared and read both regular programming courses ( Java Core + Junior Java Developer ) and special courses (Multicore Programming for JVM ( one and two times )).
')
At the moment I am starting a special course on Scala and in this topic I want to share the materials that I found most interesting / informative (the course has been prepared for over a year).

Course program
Materials: "Miscellaneous"
Materials: Object Oriented Programming in Scala
Materials: Functional Programming in Scala
Materials: Higher kinded types
Materials: Parser combinators
Materials: Metaprogramming / Reflection
Materials: Metaprogramming / Macroses
Materials: Scalaz
Materials: Netty
Materials: Akka
Materials: Finagle
Materials: Zookeeper
Materials: Using FP in the financial industry

Course program

The special course on Scala sets its goals
1. Deeply study the language, including such "dark corners" as macroses, path dependent types, generics of higher kind. Learn to create internal and external DSL. find out why functional programming languages ​​are so popular in the financial industry.
2. Understand why when writing code in functional programming languages ​​(and Scala in particular) often use mathematical libraries referring to algebra and category theory.
3. See why modern startups (like Twitter and LinkedIn) often write their infrastructure on Scala.

The special course starts on February 27, 2015 and consists of 16 webinars for 2-2.5 hours each. Duration 3 months (1-2 lessons per week). All lectures are recorded on video and made available to listeners. All necessary literature is provided in electronic form. I answer questions both during the webinar and the rest of the time. You have the opportunity to communicate with 15-20 other students who are learning Scala in parallel.

• Module # 1 (4 lectures): Basic Scala
  The task of the module is to review the syntax and ways to solve “everyday” tasks on Scala
  
  • "Miscellanea"
    
    • Local Type Inference, unified types
    • Structural types, tuples
    • Generics: variance, upper / lower bounds, existential types
    • Lazy values, DelayedInit trait, App / Application
    • Implicits, view, view bounds
    • Sequence omprehensions
    • Annotations, Exceptions
    • Work with XML
    • Scala's Collections Library, Arrays
    
    
  • Object Oriented Programming
    
    • Classes, objects, traits
    • Type members, abstract types
    • Explicitly Typed Self References
    • Mixin class composition, hierarchy linearization
    • Cake Pattern
    
    
  • Functional programming
    
    • Functional types, anonymous functions, nested functions, curring, partial application
    • Methods / Functions, Operators (prefix, infix, postfix; associativity), non-partial + pointless recording
    • Call-by-name / call-by-value, named and default params
    • Pattern matching, classes, extractor objects
    
    
  
  
• Module # 2 (4 lectures): Advanced Scala
  The task of the module is to examine in detail the capabilities of Scala, which are not in Java.
  
  • Dependent types
  • Higher kinded types
  • Parser combinators
  • Metaprogramming = Reflection + Macros
  
  
• Module # 3 (4 lectures): Type-acrobatic libraries in Scala
  The task of the module is to consider the most advanced libraries (in the sense of using the type system and language features), how they are written, what can be done on them.
  
  • Scalaz - some category theory (monad, applicative functor, catamorphism, ...)
  • Shapeless - a type class and dependent type based generic programming library
  • Algebird - or like Twitter uses algebra (monoid, group, half-ring, ...)
  
  
• Module # 4 (4 lectures): Multi-threaded, network and distributed programming
  The task of the module is to consider the most popular network libraries for JVM, which are often the “building blocks” of large distributed systems. Understand why Twitter is written on Scala.
  
  • Netty
  • Akka
  • Finagle
  • Zookeeper
  
  

Materials: "Miscellaneous"

Scala introductory articles

These articles are brief reviews of the language (15-20 pages). They are more likely ideological / historical in nature - as the author himself (Oder) sees his brainchild, which he considers the most important / distinctive / characteristic.

• A Tour of the Scala Programming Language " , Odersky + ...
• "An Overview of the Scala Programming Language" , 2ed, Odersky + ...
• "A Scala Tutorial for Java programmers" , Michel Schinz, Philipp Haller

Scala Style Guide

• PayPal Style Guide
• Twitter.com: "Effective Scala"
• scala-lang.org: Style Guide

Scala main articles

These articles highlight and clarify the main characteristics of the language.

• "Scalable Component Abstractions" , Odersky + .... For the construction of the reusable components: Together, you can use ... ... We can do this.
• Odersky + ..., Type classes as objects and implicits , they were a disciplined alternative to ad-hoc polymorphism. For example, retroactive extension of programs. They are also recognized as a mechanism for concept-based generic programming and, more recently, have been evolved into a mechanism for type-level computation . This paper presents a lightweight approach.
• Matching Objects With Patterns , Odersky + ..., Pattern Matching in all its glory with Case classes / Extractors + comparison with more classical solutions ( Visitor , Type-Test / Type-Cast , ...)

Courses / Scala

The scope of the material on a small book, but not the book. From the "classics."

• Learning Scala
• Learning Scala in Small Bites
• Twitter.com: Scala School!
• Odersky + Coursera = "Functional Programming Principles in Scala"
• Odersky + Coursera = "Principles of Reactive Programming"

Yes, the Coursera itself is written in Scala .

Books

There are a large number of books on the market (about 20) on Scala, but I most liked the following 4

• “Programming in Scala. 2ed " - a book from Oder, a classic, must be read
• "Scala for the Impatient" - Comrade Horstman and then managed. If you liked his two-volume book on Java, you may be worth a try.
• “Scala in Depth” - good coverage: from the basics to the advanced moments
• "DSLs in Action" - how to implement your DSL on Scala

Materials: Object Oriented Programming in Scala

• “Independently Extensible Solutions to the Expression Problem” , Odersky +…. Proposed Scala-solution for the "classic" Expression Problem. Partial solutions to this problem are inheritance (expansion of entities) and the Visitor pattern (expansion of operations), but Odersky solves a more interesting problem - the layout of independent extensions and entities and operations simultaneously.
• "Scala's Selfless Trait Pattern" , Bill Venners - just a named template for working with inheritance in Scala
• "Scala's Stackable Trait Pattern" , Bill Venners - just a named template for working with inheritance in Scala

Materials: Functional Programming in Scala

“Functional Programming in Scala” is a powerful book from one of the distributors in Scalaz.

Materials: Higher kinded types

• "Fighting Bit Rot with Types" , Odersky + ... - described the refactoring of Scala collections for version 2.8 with good immersion in higher-kind types , implicit parameters and implicit conversions

Materials: Parser combinators

Scala includes a package ( scala.util.parsing ), which provides tools for describing External DSL grammar on Scala in a format close to EBNF. Those. This is an “internal DSL” for creating an “external DSL”.

In Scala, the parsers are, therefore, the defining combinators.

In a number of subject areas, you can build an API in the form of some primitive elements (in this case, parsers) and methods of combining these elements in a certain way (combinators). In this case, the very core of the language has everything you need to build external DSLs.

• “Parser Combinators in Scala” , Odersky + ..., - the article is gradually writing its own Parser Combinators library and comparing it with scala.util.parsing. Parsers are built for simple arithmetic expressions, xml, lambda calculus that tracks variable binding
• Combinator Parsing, Chapter 31 of Programming in Scala, First Edition - a basic tutorial from Martin Odersky, Lex Spoon, and Bill Venners. Parsers for arithmetic expressions, json, are built. There is a second edition, but it is not available in the public domain.
• Monadic Parser Combinators , Hutton + Meijer
  - one of the articles on which Oder and his comrades relied when designing parser combinators for Scala
• "How to Replace Your Resignation"
• "Higher-order functions for parsing" , Hutton
• "Functional parsers" , Fokker
• "Deterministic, Error-Correcting Combinator Parsers" , Swierstra + Duponcheel
• "Parsec: Direct Style Monadic Parser Combinators: For The Real World"
• Packrat Parsing: Simple, Powerful, Lazy, Linear Time
• "Monadic Parsing in Haskell"
• "Pickler Combinators (functional Pearl)" - Haskell pickler calls it "serializers", specialized functions for converting objects in memory into byte arrays. It is proposed to construct pickler for complex data structures to receive as combinations of pickler for basic types. SML version of the SML.NET compiler. Pickling and unpickling
• "Instant Pickles: Generating Object-Oriented Pickler Combinators for Fast and Extensible Serialization" Miller, Haller, Burmako, Odersky - a similar approach (previous article) for Scala
• "If you want to use a Sixth-Order Function?" (functional pearl) ” , Chris Okasaki - the parser is a first-order function that accepts input data and returns the result of the parsing to some format. Parser combinators are second-order functions that take first-order functions (parsers) and return first-order functions (parsers). Chris Okasaki was able to find a reasonable function of the sixth order !!! associated with parsers.

Materials: Metaprogramming / Reflection

Scala's runtime reflection has been used, provided in the package scala.reflect.api. This enables the sharing of generic code between macros and implementations that utilize runtime reflection.

Until 2.10, Scala has not had any reflection. Instead of using one of these APIs, it can be used to identify its members. However, there are many types of scala-specific elements that can be used for Java reflection, which can be used for Java elements (no functions, no traits) and types (no existential, higher-rate, path-dependent and abstract types). In addition, java reflection is not possible. a restriction that is carried through to scala.

In the Scala 2.10, there has been a direct scattering of knowledge and awareness of the Scala. With full-featured runtime reflection for Scala types and generics ...

• "Scala-lang.org: Reflection: Overview" Miller, Burmako, Haller
• "Scala-lang.org: Reflection: Environment, Universes, and Mirrors" Miller, Burmako, Haller
• "Scala-lang.org: Reflection: Symbols, Trees, and Types" Miller, Burmako, Haller
• "Scala-lang.org: Reflection: Annotations, Names, Scopes, and More" Miller, Burmako, Haller
• "Scala-lang.org: Reflection: TypeTags and Manifests" Miller, Burmako, Haller

Materials: Metaprogramming / Macroses

Scala's runtime reflection has been used, provided in the package scala.reflect.api. This enables the sharing of generic code between macros and implementations that utilize runtime reflection.

Our flavor of macros is a reminiscent of Lisp macros, adapted to incorporate safety and rich syntax. Unlike infamous C / C ++ preprocessor macros, Scala macros: 1) are written in full-fledged Scala, 2) work with expression trees, not with raw strings, 3) cannot be changed syntax of Scala. [ here ]

Macros are functions that are called compiler during compilation. Programmer has access to compiler APIs. For example, it is possible to generate, analyze and typecheck code. You can learn more about macros from the documentation. [ here ]

Scala since version 2.10.0 . It is a scant of the scala.

Experimental feature . Do not use either scala.language.experimental.macros per sample or a -language: experimental.macros (providing a compiler switch) on per-compilation basis. You can, however, don’t need to make any changes.

• "Scala-lang.org: Def Macros" Burmako - Macro Review at scala-lang.org
• "Scala-lang.org: Quasiquotes" Shabalin - Quasicatum Review
• "Scala-lang.org: Macro Bundles" Burmako - review of macro bundles
• Scala Macros, a Technical Report , Burmako + Odersky
• "What Are Macros Good For?" , Burmako - macros are good for Code generation, Static checks, Domain-specific languages
• Scala Macros: Let Our Powers Combine! Syntax and Static Types of Work with Metaprogramming » Burmako
• "Scala Macros, a Technical Report" Burmako, Odersky
• "Quasiquotes for Scala, a Technical Report" Shabalin, Burmako, Odersky - to introduce quasiquotes for the Scala programming language. Included case studies show that quasiquotes are easier to use. It would be the ultimate tool for the tree manipulation.
• "Yin-Yang: Concealing the Deep Embedding of DSLs" Jovanovic, Shaikhha, Stucki, Koch, Odersky, Nikolaev - a description of the Yin-Yang library, which is a tool for building "deeply embedded domain-specific languages" through the use of macros

Materials: Scalaz

• Learning Scalaz is a great online tutorial with 22 lessons.
• "Scalaz - Resources For Beginners" - a large collection of links on Scalaz
• "Introduction to Category Theory in Scala"
• "Applicatives are generalized functors"
• Apocalisp blog - OLD blog of "Functional Programming Scala" author
• Higher Order. Philosophy and functional programming. - NEW blog of "Functional Programming Scala" author
• "Scalaz: For the Rest of Us" - what can be done on Scalaz WITHOUT an understanding of category theory

Materials: Netty

Netty - an asynchronous network event Netty is the de fakto standard for using java.net. *, NIO and NIO.2 (Akka and Finagle use Netty, Zookeeper directly under the hood NIO / NIO.2). Although it is possible to directly implement the asynchronous message processing templates of the Reactor / Proactor / Asynchronous Completion Token / Acceptor-Connector, however, this involves a large number of template code and shading of the domain functional. The advantage of Netty is both the declarativeness of the code and the fact that the library is a protocol constructor (http, ftp, smtp, websockets, ...).

• Full code of example: io.netty.example.echo. * - simple echo client and echo server on Netty
• Full code of example: io.netty.example.telnet. * - simple telnet client and telnet server on Netty
• Full code of example: io.netty.example.factorial. *
• Full code of example: io.netty.example.http. * - simple http-client and http-server on Netty
• Full code of example: io.netty.example.socksproxy. * - simple SOCKS5 proxy on Netty
• "Netty Best Practices" - tips from the author of "Netty in Action"
• Netty 4 at Twitter: Reduced GC Overhead
• Related articles - a list of more than 100 articles recommended by the authors of Netty

Materials: Akka

Akka is the de facto framework standard for multi-threaded and distributed, artist-based architectures for JVM. Starting with the version of Scala 2.10.0, Akka supplanted the "native" implementation of actors .

• Terminology, Concepts - Concurrency vs. Parallelism, Asynchronous vs. Synchronous, Non-blocking vs. Blocking, Deadlock vs. Starvation vs. Live-lock, Race Condition, Non-blocking Guarantees (Progress Conditions)
• Actor Systems - Hierarchical Structure, Configuration Container, Actor Best Practices
• What is an Actor? - Actor Reference, State, Behavior, Mailbox, Children, Supervisor Strategy
• Supervision and Monitoring
• Actor References, Paths and Addresses
• Ka Java Java
• Message Delivery Guarantee
• Scala API: Actors
• Scala API: Typed Actors
• Scala API: Event Bus
• Scala API: Scheduler
• Scala API: Futures
• Scala API: Dataflow Concurrency
• Scala API: Fault Tolerance

Materials: Finagle

Finagle - A Protocol-Agnostic extensible RPC System for the JVM, used to construct high-concurrency servers. Written and used in twitter.com. The basic principle is "We must program locally, communicate globally . " Part of the Twitter stack (along with Ostrich, Zipkin, Mesos, Iago, ZooKeeper, Scalding). Finagle includes load balancing, connection pooling, a call with timeout, monitoring, statistics collection.

• A brief description of the components Finagle: - Protocols , Futures , Services & Filters , Servers , Clients , Names , Metrics
• "Finagle: A Protocol-Agnostic RPC System" - a post from the Twitter team about Finagle
• "How to build a simple distributed search engine using the Scala and Finagle framework" - a lecture on the project Twitter.com / Scala school.
• "Thrift: Scalable Cross-Language Services Implementation" is a reference article using the Thrift protocol. This protocol is the most popular "carrier protocol" for Finagle.

Materials: Zookeeper

Zookeeper is a fail-safe centralized service for managing configuration information, naming, distributed synchronization and other group services, an integral part of the Twitter stack . It can be used to create basic data structures (cluster) - queues, locks, barriers. Suitable for distributed algorithms such as two-phase-commit, leader election and others.

• The ZooKeeper Programmer's Guide is a good description (Data Model, Sessions, Watches, access control, Consistency Guarantees, Common Problems and Troubleshooting).
• "Programming with ZooKeeper - A basic tutorial" - examples of the implementation of barriers and producer-consumer architecture.
• ZooKeeper Internals - Atomic Broadcast, Leader Activation, Quorums
• “ZooKeeper Recipes and Solutions” - we create a distributed barrier, a queue, a lock on the basis of Zookeeper. We do two-phase-commit and leader election.
• "The Chubby lock service for loosely coupled distributed systems" - a description of the library from Google. Zookeeper was made based on its architecture, drawn, including, from this article.
• "Zab: High-performance broadcast for primary-backup systems" - a description of the Zab protocol, which lies at the heart of Zookeeper.
• “ZooKeeper: Wait-free Coordination for Internet-scale Systems.” Is a basic article from Yahoo (and Zookeeper was developed there to coordinate the HDFS / Hadoop cluster).
• "ZooKeeper - A Reliable, Scalable Distributed Coordination System" - article at infoq.com

Materials: Using FP in the financial industry

For a number of reasons (the ability to describe a subject area in the language of combinators), functional programming languages ​​are used in the financial industry, which creates a demand for "functionalists" in New York and London.

• “Composing Contracts: An Adventure in Financial Engineering” , Simon Peyton Jones + ... is a basic article by Haskell about the introduction of Haskell financial contracts. "We introduce a combination of the compositional composition and denotational semantics that can be described. We sketch an implementation of our combinator library in Haskell. Interestingly, lazy evaluation plays a crucial role. ” "It has been a great deal for economic combinations of financial contracts (swaps, futures, caps, oors, swaptions, spreads, straddles, captions, European options, ... options list on on)". “We couldn’t use a process of contracting. It is notable that we want to In Section 4 A fundamentally-important property of this semantics is that it is compositional; This is a subcontracted contract. ”
• “Commercial Uses: Going functional on exotic trades” - a DSL description used by Barclays for working with financial instruments (Haskell). “The Functional Payout Framework (fpf) is a process of express financial derivatives. There are multiple scenarios that can be used.
• “Scala at EDF Trading. Implementing a Scala language. ” -“ For Carrying Out Pricing and Scala. ” The DSL consists of a combinator of data types with case classes. ”
• "Compositional specification of commercial contracts"
• "An Overview of Financial Products" - " Algebraic Accounting and Financial Products" This language, which is called RISLA, is used by a bank. A RISLA description fixes the cash flows (amounts of money coming in or out), resulting from a product…. The language has been developed with the use of algebraic specifications.
• “Domain-Specific Languages ​​versus Object-Oriented Frameworks: A Financial Engineering Case Study” - “ Alternatively, it can be achieved by setting up an object-oriented framework. The framework has been developed. The benefits of mutual benefits. ”
• Adventures in financial and software engineering
• "Financial Domain-Specific Language Listing - a large collection of API / DSL to work with finance

PS We will be happy to answer any questions by
skype: GolovachCourses
email: GolovachCourses@gmail.com