Cross a hedgehog (Marathon) with a snake (Spring Cloud). Episode 2

In the first episode , we managed to pull information from Mesos Marathon straight into the bins of Spring Cloud . However, we had the first problems, one of which we will examine in the current part of the story. Let's recall our connection configuration to Marathon :

spring: cloud: marathon: scheme: http #url scheme host: marathon #marathon host port: 8080 #marathon port 

What problems do we see here? The first is that we have no authorization when connecting, which is strange for industrial use. The second is that we can specify only one host and port. In principle, it would be possible to try several masters for one balancer or DNS, but I would like to avoid this additional point of failure. How to do this is written under the cut.

Password all head

There are two authorization schemes available to us: Basic and Token . Basic authorization is so trivial that almost every developer has met with it. Its essence is painfully simple. We take the login and password. We glue them through : We encode in Base64. Add an Authorization header with the value Basic <Base64> . Profit

With a token is somewhat more complicated. In the open-source implementation, it is not available, so this method is suitable for those who use DC / OS . To do this, just add a slightly different authorization header:

 Authorization: token=<auth_token> 

Thus, we can add several properties we need to our configuration:

 spring: cloud: marathon: ... token: dcos_acs_token username: marathon password: mesos 

And then we can be guided by simple priorities. If a token is specified, then we take it. Otherwise, we take the login and password and do the basic-authorization. Well, in the absence of both, we create a “naked” client.

 Feign.Builder builder = Feign.builder() .encoder(new GsonEncoder(ModelUtils.GSON)) .decoder(new GsonDecoder(ModelUtils.GSON)) .errorDecoder(new MarathonErrorDecoder()); if (!StringUtils.isEmpty(token)) { builder.requestInterceptor(new TokenAuthRequestInterceptor(token)); } else if (!StringUtils.isEmpty(username)) { builder.requestInterceptor(new BasicAuthRequestInterceptor(username,password)); } builder.requestInterceptor(new MarathonHeadersInterceptor()); return builder.target(Marathon.class, baseEndpoint); 

The client for Marathon is implemented using the Feign http client, to which you can easily add any interceptors . In our case, they add the necessary http headers to the request. After this, the builder constructs an object for us on the interface, in which possible requests are declaratively declared:

 public interface Marathon { // Apps @RequestLine("GET /v2/apps") GetAppsResponse getApps() throws MarathonException; //Other methods } 

So, the warm-up is over, now let's do a more difficult task.

Failover Client

If we have an industrial installation of Mesos-a and Marathon -a, then the number of masters from which we can read data will be more than one. Moreover, one of the masters may accidentally be unavailable, slow down, or be able to upgrade. Failure to obtain information will either lead to obsolescence of information on customers, and, therefore, at some point to shots into the milk. Or, let's say when updating the application software, we will not get a list of instances at all and will refuse to service the client. All this is not good. We need client request balancing.

It is logical that Ribbon should be selected as a candidate for this role, as it is used in client-side balancing of requests within Spring Cloud . We will talk more about strategies for balancing requests in the following episodes, but for now we will limit ourselves to the most basic functionality that we need to solve a problem.

The first thing we need to do is to implement the balancer in our feign client:

 Feign.Builder builder = Feign.builder() .client(RibbonClient.builder().lbClientFactory(new MarathonLBClientFactory()).build()) ...; 

Looking at the code begs a logical question. What is lbClientFactory and why are we doing our own? In short, this factory constructs a client request balancer. By default, the created client is deprived of one feature we need: a repeated request in case of problems on the first request. To be able to retry , we will add it when constructing an object:

 public static class MarathonLBClientFactory implements LBClientFactory { @Override public LBClient create(String clientName) { LBClient client = new LBClientFactory.Default().create(clientName); IClientConfig config = ClientFactory.getNamedConfig(clientName); client.setRetryHandler(new DefaultLoadBalancerRetryHandler(config)); return client; } } 

Do not be afraid that our retry handler has the Default prefix. Inside it is everything that we need. And here we come to the configuration of all this stuff.

Since we can have several clients in the application, and the client for Marathon is only one of them, the settings have a specific template of the form:

 client_name.ribbon.property_name 

In our case:

 marathon.ribbon.property_name 

All properties for the client balancer are stored in the configuration manager - Archarius . In our case, these settings will be stored in memory, because we add them on the fly. To do this, in our modified client, add a helper method setMarathonRibbonProperty , inside which we will set the value of the property:

 ConfigurationManager.getConfigInstance().setProperty(MARATHON_SERVICE_ID_RIBBON_PREFIX + suffix, value); 

Now, before creating the feign client, we need to initialize these settings:

 setMarathonRibbonProperty("listOfServers", listOfServers); setMarathonRibbonProperty("OkToRetryOnAllOperations", Boolean.TRUE.toString()); setMarathonRibbonProperty("MaxAutoRetriesNextServer", 2); setMarathonRibbonProperty("ConnectTimeout", 100); setMarathonRibbonProperty("ReadTimeout", 300); 

What is there interesting. First is the listOfServers . In fact, this is an enumeration of all possible host:port pairs on which the Marathon -a masters are located, separated by commas. In our case, we simply add the ability to specify them in our configuration:

 spring: cloud: marathon: ... listOfServers: m1:8080,m2:8080,m3:8080 

Now each new request to the master will go to one of these three servers.

In order for retry to work at all, we must set the value of OkToRetryOnAllOperations to true .

We set the maximum number of repetitions using the MaxAutoRetriesNextServer option. Why does it have an indication for NextServer ? It's simple. Because there is another option MaxAutoRetries , which indicates how many times you need to try to re-call the first server (the one that was selected for the very first request). By default, this property is set to 0 . That is, after the first failure, we will go to ask for the data from the next candidate. It is also worth remembering that MaxAutoRetriesNextServer indicates the number of attempts minus attempts to request data from the first server.

Well, in order not to hang on the line indefinitely, we set the ConnectTimeout and ReadTimeout properties within reasonable limits.

Total

In this part we have made our client to Marathon more customizable and fault tolerant. And taking advantage of ready-made solutions, we did not have to make too many gardens. But we are still far from the completion of the work, because we still do not have the most interesting part of the ballet - client balancing requests for application software.

To be continued