Haskell, as something very close, or we get comites from github api

Too late - 'cause I got it now
there are monads all around
IO, State and lists abound
It's easy, like those people say
but my program got abstracted all away!
Maybe ooo
It's a monad too, I know
Why should I use another language at all?

Again a crazy adept Haskell, and another attempt to prove its practicality. Timeless classics.
I will try to tell a smart story (do not get fooled by pretentious advertising) , which will have all the necessary components of a blockbuster (I’m serious, do not get fooled) - familiar characters, a well thought out universe and an open ending (well ...) .

A little seriousness never hurts. Therefore, first, without the slightest hint of humor, I will tell the logic of writing this text. I wanted (first of all, for myself, but I hope someone will be interested too) to implement some kind of painfully close, incredibly practical task on Haskell. A positive result of this task would give an extra reason to be proud of yourself, skills and one more argument in favor of choosing this programming language. As an experimental task, I chose to receive and process information about commits to the repository on github. Actually, it will contain work with github api - loading and parsing json.

I believe that it is worth deciding on steps, therefore we will begin with an initial position, namely an empty directory in the file system.
')

Module creation

First, create a new module for our purposes.

cabal init 

Inquisitive cabal will ask a few questions, and as a result you will receive a module stub with the configuration file project_name.cabal . For more aesthetics, add the src directory to the module, and specify it in the configuration

 executable project-name hs-source-dirs: src main-is: Main.hs 

Of course, Main.hs needs to be created)

Next, a few words about dependency hell . This is a sore subject Haskell, in which progress is planned. There are several options for solving the problem of dependencies, but we are young and love everything that is fashionable, so we will use the fresh cabal-1.18 feature - sandoxes.

Actually, for use you need to initialize the sandbox and install dependencies

 cabal sandbox init cabal install --only-dependencies 

In the future, to assemble the module, you can use the command as usual

 cabal build 

If there is a keen desire to debug something, and indeed, to see how it works from the inside (and, according to the laws of the genre, such a desire will necessarily arise), you can run ghci in the sandbox created by the command

 cabal repl 

Everything, the fear of the empty catalog is overcome, we move on.

http-conduit

The first task that needs to be solved is to download information about commit in json format. Actually, the source is obvious , but simple things end there. So, at this stage we will use the http-conduit package authored by the sun-faced Edward Snow Michael Snoyman. In general, conduit is a great solution for working with data streams. I hardly get to talk about it well, so welcome to the blog of a man by the name of eax . I will tell quite a bit and on the periphery.

First, add the necessary dependencies to the build-depends section of the configuration file.

 bytestring >= 0.10, conduit >= 1.0, http-conduit >= 1.9, 

and update the sandbox with the command described above.

Now we can anxiously proceed to the code. To begin with, in order to simplify your life and work with strings, add an extension

 {-# LANGUAGE OverloadedStrings #-} 

We connect the necessary modules

 import Data.Conduit import Network.HTTP.Conduit import qualified Data.Conduit.Binary as CB import qualified Data.ByteString.Char8 as BS 

All json download code will look something like this.

 main = do manager <- newManager def req <- parseUrl "https://api.github.com/../.." let headers = requestHeaders req req' = req { requestHeaders = ("User-agent", "some-app") : headers } runResourceT $ do res <- http req' manager responseBody res $$+- CB.lines =$ parserSink 

As I recall, api github requires the User-agent header, so I had to expand the request a bit. The main action takes place in the last two lines, where we get an answer with json. Since if the result is wrapped in a transformer ResourceT , then the functions for getting it must be called using runResourceT. After receiving the response body, we send it to the stock, which is designed for parsing json and it looks like this

 parserSink :: Sink BS.ByteString (ResourceT IO) () parserSink = do md <- await case md of Nothing -> return () Just d -> parseCommits d 

If successful, the stock will simply parse the received json and display it on the screen (this part of the magic is hidden in the parseCommits function).

Aeson

We continue to distort the thinking of programmers and proceed to the parsing. For it, we will use an extremely powerful package called Aeson . In fact, everything is quite simple here, but there are a few points that are used to introduce into a stupor:

• Since Haskell is strongly typed, we will need types that will describe the data structure embedded in json
• If I didn’t confuse anything, then Aeson uses lazy bytestring, while strict bytestring is in stock, so I’ll have to demonstrate the skills of juggling with types

So, first define the types. You can not bother, and define them only partially by sending some of the information from json to the firebox. We leave only the url, hash and commit message.

 import qualified Data.ByteString.Char8 as BS import Data.Aeson (FromJSON(..)) data CommitInfo = CommitInfo { message :: BS.ByteString } deriving (Show) data Commit = Commit { sha :: BS.ByteString, url :: BS.ByteString, commit :: CommitInfo } deriving (Show) 

Further, it would be canonical to use applicative functors to match json and fields from data structures, but we will all deceive and use Generic.

 {-# LANGUAGE DeriveGeneric #-} import GHC.Generics (Generic) 

and add inheritance from Generic to existing data structures

 deriving(Show, Generic) 

It remains only to declare the possibility of creating Commit & CommitInfo from json

 instance FromJSON Commit instance FromJSON CommitInfo 

Only a few steps left to finish, we are almost there

 parseCommits :: BS.ByteString -> Sink BS.ByteString (ResourceT IO) () parseCommits rawData = do let parsedData = decode $ BL.fromChunks [rawData] :: Maybe [Models.Commit] case parsedData of Nothing -> liftIO $ BS.putStrLn "Parse error" Just commits -> liftIO $ printCommits commits 

As you can see, you have to create lazy bytestring to return to decoding. If the parsing was successful, with the help of liftIO we raise the values ​​obtained and output them to the console.

Finish

Everything, the red carpet, fanfare and solemn end of the evening. A full example is located here . The code is not an example of computer science ideals, so comments from the gurus are welcome. I hope everyone else has learned something, or at least enjoyed it and became closer to the world of Haskell. May the force be with you!