Play RSS with PlayFramework 2.2 and Scala

Good day, dear habravchane.

We, the ~~pogrom~~ programmers, very often encounter the same problem when learning a new language X or the framework Y - what to write after the intro Yet Yet Hello World? Anything that can show some of the advantages and disadvantages of X / Y, but it would not take a lot of time.
')
My friends and I often asked this question. As a result, a simple thought was born - write an RSS reader. Here you have to work with the network, and XML parser, and the database can be connected, look at the template engine. But you never know.

So, here begins the fascinating journey to the stack of the Play Framework 2.2 + Scala + MongoDB on the backend and AngularJS + CoffeeScript on the frontend.

TL; DR

The whole project was placed in ~ 250-300 lines on Scala with documentation and ~ 150 lines on CS. Well, some HTML.
Code is available on Bitbucket

And the first stop is the question - why Scala, and not Java? And why Play, and not the same Lift?

The answers are very simple and subjective.
Scala provides a higher level of abstraction and less code for the sake of code. When I saw the documentation on the standard List with its 200 methods for all occasions ... Seriously, try it yourself.
As for the choice of framework - a simple example on Lift gave me a page on localhost for ~ 150 ms , and this without using a database. At the same time on the same machine and the same JVM Play managed for ~ 5-10 ms. I do not know, maybe the stars are so formed.
And in the console console sweetheart.

I will miss the part on how to install and start working with Play, since everything is quite thoroughly chewed in the official documentation (up to the generation of the project for your favorite IDE ), and we will go further.

Query path

The most obvious way to disassemble the application is to follow the client's request.
Today it is better to skip the black box for processing the request by the framework itself, especially since it is built on Netty, and it would have to dig deep. Perhaps to China.
As each river begins with a streamlet, any application in Play begins with routing, which is quite clearly described in

conf / routes

 # Routes
 # This file defines all application routes (Higher priority routes first)
 # ~~~~

 # Get news
 GET / news controllers.NewsController.news (tag: String? = "", PubDate: Int? = (System.currentTimeMillis () / 1000) .toInt)

 # Parse news
 GET / parse controllers.NewsController.parseRSS

 # Get tags
 GET / tags controllers.TagsController.tags

 # Map static resources from the URL path
 GET / assets / * file controllers.Assets.at (path = "/ public", file)

 # Home page
 GET / controllers.Application.index

Marking in the fields:
Separately, I want to highlight the fact that in addition to the very possibility of setting default values for the arguments passed to the specified method, you can specify expressions. For example - getting the current timestamp.
By the way, the routing in Play is quite functional, up to regexp when processing a request.

Show your ticket!

As you can guess from the title - the story continues with the controllers. In Play, user controllers are included in the package controllers , use the Treyt Controller and are objects whose methods accept and respond to user requests in accordance with the routing.
Since the application receives data from the server via AJAX, the controller for drawing the main page is as square as it is necessary only for loading HTML / CS / JS scripts.

Not typed and 20 lines

 package controllers import play.api.mvc._ /** * playRSS entry point */ object Application extends Controller { /** * Main page. So it begins... * @return */ def index = Action { Ok(views.html.index()) } }

Ok returns the play.api.mvc.SimpleResult instance, which contains the headers and body of the page. The response from the server will be equal, as they could have guessed especially attentive, 200 OK .

but
If a full-fledged controller for the entire application fits in 20 lines, then it is very likely that you write in Ruby.

So, what is the best way to send an AJAX request to a client to receive news? Right, JSON.
NewsController deals with NewsController

object NewsController

 package controllers import play.api.mvc._ import scala.concurrent._ import models.News import play.api.libs.concurrent.Execution.Implicits.defaultContext import models.parsers.Parser import com.mongodb.casbah.Imports._ object NewsController extends Controller { /** * Get news JSON * @param tag optional tag filter * @param pubDate optional pubDate filter for loading news before this UNIX timestamp * @return */ def news(tag: String, pubDate: Int) = Action.async { val futureNews = Future { try { News asJson News.allNews(tag, pubDate) } catch { case e: MongoException => throw e } } futureNews.map { news => Ok(news).as("application/json") }.recover { case e: MongoException => InternalServerError("{error: 'DB Error: " + e.getMessage + "'}").as("application/json") } } /** * Start new RSS parsing and return first N news * @return */ def parseRSS = Action.async { val futureParse = scala.concurrent.Future { try { Parser.downloadItems(News.addNews(_)) News asJson News.allNews() } catch { case e: Exception => throw e } } futureParse.map(newsJson => Ok(newsJson).as("application/json")).recover { case e: MongoException => InternalServerError("{error: 'DB Error: " + e.getMessage + "'}").as("application/json") case e: Exception => InternalServerError("{error: 'Parse Error: " + e.getMessage + "'}").as("application/json") } } }

Future . Async . Here it becomes interesting for the first time.
Let's start with the fact that Play is asynchronous and in principle you don’t need to work with streams at all. But when we ~~urgently~~ need ~~to calculate the number π~~ to access the database, read data from a file or perform another slow I / O procedure, Future comes to the rescue, which allows you to perform an operation asynchronously without blocking the main thread. To perform Future uses a separate context, so you should not worry about threads.
Since the function now returns not Future[SimpleResult] , but the Future[SimpleResult] , the async method of the ActionBuilder trait (which uses the Action object) is used

Landscapes

Let us break with this asynchronous nightmare and turn to the patterns that are nice to us. Play provides the ability to work with plain HTML. Normal such HTML with Scala code inserts. The template is automatically compiled into a rolling source and is a normal function where you can transfer parameters or connect (call) other templates. By the way, many people disliked the new template engine because of the relatively slow compilation time of that HTML into the code. Thats OK for me.

index.scala.html

 <!DOCTYPE html> <html> <head> <title> playRSS </title> <link rel="shortcut icon" href='@routes.Assets.at("images/favicon.png")' type="image/png"> <link rel="stylesheet" href="http://netdna.bootstrapcdn.com/bootstrap/3.0.0/css/bootstrap.min.css"/> <link rel="stylesheet" href='@routes.Assets.at("stylesheets/main.css")'> @helper.requireJs(core = routes.Assets.at("javascripts/require.js").url, module = routes.Assets.at("javascripts/main").url) </head> <body> <div class="container" id="container" ng-controller="MainCtrl"> <a href="/"><h1>playRSS</h1></a> @control() <div class="row"> <div class="col-lg-12"> @news() </div> </div> </div> </body> </html>

As can be seen from the source - little magic. @helper connects the requireJS supplied by the framework itself and indicates the path to main.js, where the frontend is initialized. @news() and @control() are the news.scala.html and control.scala.html templates, respectively. Perform the function and display the result inside the current template. Nicely.

And also
you can work with loops, if / else, etc. There is detailed documentation

Mount Kasbah

Let's continue, perhaps, work from a DB. In my case, Mongo was chosen. ~~Since I'm too lazy to create tables :)~~
Casbah is the official driver for working with MongoDB in a rolling pin. Its advantage is simultaneous simplicity and functionality. And the main drawback will be considered at the end.

The driver is connected quite plainly:

Add a line to libraryDependencies inside the build.sbt:
"org.mongodb" %% "casbah" % "2.6.3"
Add to our code:
import com.mongodb.casbah.Imports._
Play when launching the project itself deflates dependencies
???
PROFIT

And a little about the code. Since my reader is not uncomplicated, an object was created that distributes to the needy collections from MongoDB. The right word, the fence DAO or DI is simply superfluous.

object database

 package models import com.mongodb.casbah.Imports._ import play.api.Play /** * Simple object for DB connection */ object Database { private val db = MongoClient( Play.current.configuration.getString("mongo.host").get, Play.current.configuration.getInt("mongo.port").get). getDB(Play.current.configuration.getString("mongo.db").get) /** * Get collection by its name * @param collectionName * @return */ def collection(collectionName:String) = db(collectionName) /** * Clear collection by its name * @param collectionName * @return */ def clearCollection(collectionName:String) = db(collectionName).remove(MongoDBObject()) }

Marking in the fields:
In Scala, objects are in fact singletons. If you turn on the bore mode, an anonymous class is created and instantiated with static methods (in the Java / JVM view). So our connection will rise during the creation of the object and will be available throughout the application's working cycle.

It is time to demonstrate the work with the base on Scala and Casbah:

object news

 /** * Default news container * @param id MongoID * @param title * @param link * @param content * @param tags Sequence of tags. Since categories could be joined into one * @param pubDate */ case class News(val id: String = "0", val title: String, val link: String, val content: String, val tags: Seq[String], val pubDate: Long) /** * News object allows to operate with news in database. Companion object for News class */ object News { .... /** * Method to add news to database * @param news filled News object * @return */ def addNews(news: News) = { val toInsert = MongoDBObject("title" -> news.title, "content" -> news.content, "link" -> news.link, "tags" -> news.tags, "pubDate" -> news.pubDate) try { col.insert(toInsert) } catch { case e: Exception => } } .... /** * Get news from DB * @param filter filter for find() method * @param sort object for sorting. by default sorts by pubDate * @param limit limit for news select. by default equals to newsLimit * @return */ def getNews(filter: MongoDBObject, sort: MongoDBObject = MongoDBObject("pubDate" -> -1), limit: Int = newsLimit): Array[News] = { try { col.find(filter). sort(sort). limit(limit). map((o: DBObject) => { new News( id = o.as[ObjectId]("_id").toString, title = o.as[String]("title"), link = o.as[String]("link"), content = o.as[String]("content"), tags = o.as[MongoDBList]("tags").map(_.toString), pubDate = o.as[Long]("pubDate")) }).toArray } catch { case e: MongoException => throw e } } }

Familiar to everyone who worked with MongoDB, the API and the trivial filling of the case class News instance. While all elementary. Even too much.
Need something more interesting. How about aggregation ?

Pulling out tags

 /** * News tag container * @param name * @param total */ case class Tags(name: String, total: Int) /** * Tags object allows to operate with tags in DB */ object Tags { /** * News collection contains all tag info */ private val col: MongoCollection = Database.collection("news") /** * Get all tags as [{name: "", total: 0}] array of objects * @return */ def allTags: Array[Tags] = { val group = MongoDBObject("$group" -> MongoDBObject( "_id" -> "$tags", "total" -> MongoDBObject("$sum" -> 1) )) val sort = MongoDBObject("$sort" -> MongoDBObject("total"-> -1)) try { col.aggregate(group,sort).results.map((o: DBObject) => { val name = o.as[MongoDBList]("_id").toSeq.mkString(", ") val total = o.as[Int]("total") Tags(name, total) }).toArray } catch { case e: MongoException => throw e } } }

.aggregate allows .aggregate to work wonders without mapReduce. And the principle of work in Scala is the same as from the console. A kind of pipeline-way, only separated by commas. Grouped by tags, summed up the same in total and sorted the whole thing. Fine.

By the way, Casbah is a citadel.

You're JSON-XMLed

Never gonna give you up
Never gonna let you down

Because for a statically typed language, working with XML / JSON in this case looks like a joke. Suspiciously brief.
And in fact, parsing XML in Scala is a delight for my eyes (after massive factories in Java).

XML Parser

 package models.parsers import scala.xml._ import models.News import java.util.Locale import java.text.{SimpleDateFormat, ParseException} import java.text._ import play.api.Play import collection.JavaConversions._ /** * Simple XML parser */ object Parser { /** * RSS urls from application.conf */ val urls = try { Play.current.configuration.getStringList("rss.urls").map(_.toList).getOrElse(List()) } catch { case e: Throwable => List() } /** * Download and parse XML, fill News object and pass it to callback * @param cb */ def downloadItems(cb: (News) => Unit) = { urls.foreach { (url: String) => try { parseItem(XML.load(url)).foreach(cb(_)) } catch { case e: Exception => throw e } } } /** * Parse standart RSS time * @param s * @return */ def parseDateTime(s: String): Long = { try { new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss zzz", Locale.ENGLISH).parse(s).getTime / 1000 } catch { case e: ParseException => 0 } } /** * For all items in RSS parse its content and return list of News objects * @param xml * @return */ def parseItem(xml: Elem): List[News] = (xml \\ "item").map(buildNews(_)).toList /** * Fill and return News object * @param node * @return */ def buildNews(node: Node) = new News( title = (node \\ "title").text, link = (node \\ "link").text, content = (node \\ "description").text, pubDate = parseDateTime((node \\ "pubDate").text), tags = Seq((node \\ "category").text)) }

I agree
At first, methods with the name of the form \ or \\ are driven into a stupor. However, this makes some sense when you remember BigInteger from Java.

And what about JSON? Native JSON in Scala is no subjective yet. Slow and scary.
In a difficult moment, Play and his Writes / Reads from the play.api.libs.json package play.api.libs.json . Does anyone know the JsonSerializable interface from PHP 5.4? So in Play is still easier!

JSON Writes

 case class News(val id: String = "0", val title: String, val link: String, val content: String, val tags: Seq[String], val pubDate: Long) /** * News object allows to operate with news in database. Companion object for News class */ object News { /** * Play Magic * @return */ implicit def newsWrites = Json.writes[News] /** * Converts array of news to json * @param src Array of News instances * @return JSON string */ def asJson(src: Array[News]) = { Json.stringify(Json.toJson(src)) } }

The someObjectWrites -line method someObjectWrites in simple cases of serialization removes all the questions. Implicit conversions in Scala are powerful and convenient tools used in practice.
But this is a very commonplace case. When you want something special or complex, then functionalists and combinators come to the rescue.

Through hardship to the stars

While the user is bored and waiting for a response to the request that was sent to the server by the script ... Wait a minute. Still the same frontend.
As promised, CoffeeScript and AngularJS were used. After we started using this bundle in production, the number of pains just below the back when developing user interfaces decreased by 78.5% percent. Like the amount of code.
It is for this reason that I decided to use these stylish, fashionable and youth technologies in the reading room. And also because the framework chosen by me has the compiler CoffeeScript and LESS on board.
In fact, experienced developers do not learn anything new and interesting, so I will show only a couple of interesting tricks.

Often it is necessary to exchange data between the angular controllers. And what sophistries do not go to just some gentlemen (such as writing to localStorage) ...
A casket just opens.

It is enough to create a service and implement it into the necessary controllers.

Announcing

 define ["angular","ngInfinite"],(angular,infiniteScroll) -> newsModule = angular.module("News", ['infinite-scroll']) newsModule.factory 'broadcastService', ["$rootScope", ($rootScope) -> broadcastService = message: {}, broadcast: (sub, msg)-> if typeof msg == "number" then msg = {} this.message[sub] = angular.copy msg $rootScope.$broadcast(sub) ] newsModule

We send

 define ["app/NewsModule"], (newsModule)-> newsModule.controller "PanelCtrl", ["$scope", "$http", "broadcastService", ($scope, $http, broadcastService)-> $scope.loadByTag = (tag) -> if tag.active tag.active = false broadcastService.broadcast("loadAll",0) else broadcastService.broadcast("loadByTag",tag.name) ]

Get

 define ["app/NewsModule","url"], (newsModule,urlParser)-> newsModule.controller "NewsCtrl", ["$scope", "$http", "broadcastService", ($scope, $http, broadcastService)-> #recieving message $scope.$on "loadAll", ()-> $scope.after = 0 $scope.tag = false $scope.busy = false $scope.loadByTag() ]

In angular
Services are singletons. Therefore, we can drive messages back and forth, without fruiting instances.

All arrived

After such a chaotic journey to the depths and back is worth summing up.
Advantages and disadvantages, fatal and not so much, everyone should single out for himself. We still use the tool where it fits, and do not cultivate the cargo, right?

I like it:

Short Scala before Java. For a statically typed language, there is somehow little code, and errors are caught in compile time. That would not speak about tests, etc., and languages of dynamic typification in this plan concede.
The functionality of the framework itself, which provides many solutions, but does not impose a certain structure and style of writing code.

Did not like:

Yet hit refresh workflow works tight due to constant recompilation of modified sources. Not deadly, but slows the flight of thought
The windproblems (\\ vs /) of the framework together with the JVM do not allow to fully run the compiled and running application under the Win system. Only in dev mode. And here, especially the performance can not be measured. Fortunately there are several nix servers at hand
No logging in tests. Allegedly due to wild memory leaks in the logger. Well, I already find fault with it.

Also during development it is worth being careful when working with blocking operations using Future. However, there is one thing but. In spite of the fact that the main thread will not be blocked, another will be blocked. And it is good if you have enough flows and there will not be many competitive requests. But what if? In this case, Play developers recommend using asynchronous in nature drivers for the same databases. ReactiveMongo instead of Casbah, for example. Or at least tune the actors and thread pools. But that's another story ...

Thank you for attention.

PS
If this writings did not seem enough - here is the Bitbucket repository .

Source: https://habr.com/ru/post/197930/

All Articles