We rewrite the home project on microservices (Java, Spring Boot, Gradle)

Introduction

In recent years, the topic of microservices has become very popular. I didn’t get into projects with microservices, so I naturally wanted to know more about this concept of architecture.

Previously, I had a home project (although rather even its prototype), which was decided to be rewritten into microservices. The project was an attempt to make an educational Java game. That is, the player has a field, in this field he can control some unit with the help of a code. Writes the code, sends it to the server, there it runs and returns the result that is displayed to the user.

All this was implemented as a prototype - there were users, one lesson and one task for it, the ability to send code that was compiled and executed. Some frontend, but in the article about him will not be. Technologies - Spring Boot, Spring Data, Gradle.

The article will be implemented the same prototype, but on microservices. Implementation will be the easiest way (more precisely, the simplest one I know). The implementation will be available to anyone familiar with Spring.

In the process of studying the information, I found a good article , where a certain small monolith on microservices was similarly broken. But there everything was done on the basis of Spring Cloud, which is definitely more correct, but I wanted to write a bicycle first, so that in practice I could understand from what problems these solutions are curing. From this article I used only Zuul .

Microservices

About what microservices have been written many times, for example here . In short: instead of one big application, we have many small ones, which have a very narrow area of ​​responsibility and communicate with each other.

The first stage, it is necessary to break the logic into a number of microservices:

• user-service : service with users (creation, viewing, authorization is possible)
• lesson-service : service with lessons (creating, viewing lessons and tasks)
• result-service : service with answers (sending completed tasks, storing results)
• task-executor-service : code execution service (compilation and execution of tasks)

At this stage, the idea appears that with all this zoo you need to somehow communicate the frontend and individual microservices. It seems inconvenient if everyone knows each other's APIs and addresses.
From here, another service appears - gateway-service - a common entry point.

The project outline will look like this:

Gateway service

As I walk along the simplest path, the first thought was to simply make the controller for each microservice, which would redirect all requests to the correct addresses using RestTemplate . But, a little googling, I found Zuul . It has integration with Spring Boot and the configuration is very simple.

The build.gradle service looks like this:

 plugins { id 'java' id 'war' } apply plugin: 'spring-boot' springBoot { mainClass 'gateway.App' } dependencies { compile('org.springframework.cloud:spring-cloud-starter-zuul:1.2.0.RELEASE') compile('org.springframework.boot:spring-boot-starter-web') } 

And the whole microservice code consists of one class, App.java:

 @SpringBootApplication @EnableZuulProxy public class App { public static void main(String[] args) { SpringApplication.run(App.class, args); } } 

This includes ZuulProxy. Routing is described in the config, I have this application.properties :

 zuul.routes.lesson-service.url=http://localhost:8081 zuul.routes.user-service.url=http://localhost:8082 zuul.routes.task-executor-service.url=http://localhost:8083 zuul.routes.result-service.url=http://localhost:8084 zuul.prefix=/services 

Thus, requests for /services/lesson-service/... will be sent to http://localhost:8081/... etc. It turns out very convenient and simple solution for the entry point.
Zuul has many other different filter type features, but we don’t need anything else from it.

Frontend, as it seems to me, in our case should be given to the client from here. We put everything that is needed in the gateway-service/src/main/webapp/... and that's it.

Other services

The remaining services will be very similar to each other and their implementation is not much different from the usual approach. But there are a few points:

1. Database.
2. Interaction between microservices.

Database

Each microservice can now decide for itself how to store data. The traditional advantage of microservices is the freedom to choose technologies independently of other parts of the application.

It is possible for everyone to use a new type of database. But I just had three MySQL databases instead of one for user-service , lesson-service and answer-service . A task-executor-service must store some task code into which user code is inserted to perform the task. It will be stored without a database, just as files.

At the moment of splitting the scheme into three bases, I had a question: how are foreign keys, data integrity at the database level and so on. As it turned out nothing. More precisely - all at the level of business logic.

Interaction between microservices

At the moment when we begin to implement the first microservice, the question arises how to request data from another. It is clear that such functionality is needed in all services, which means you need something like a library.

Create a new module in the project, call it service-client . In it there will be, first, classes for interaction with services, secondly, general classes for data transfer. That is, each service has its own Entity , corresponding to the internal logic or database schema, but they should only give out copies of objects from the common library.

For client classes, we write the abstract Client class:

 abstract class Client { private final RestTemplate rest; private final String serviceFullPath; private final static String GATEWAY_PATH = "http://localhost:8080/services"; Client(final String servicePath) { this.rest = new RestTemplate(Collections.singletonList(new MappingJackson2HttpMessageConverter())); this.serviceFullPath = GATEWAY_PATH + servicePath; } protected <T extends Result> T get(final String path, final Class<T> type) { return rest.getForObject(serviceFullPath + path, type); } protected <T extends Result, E> T post(final String path, final E object, final Class<T> type) { return rest.postForObject(serviceFullPath + path, object, type); } } 

GATEWAY_PATH - better to set from the config or something else, and not hardcodes in this class.
And an example of the inheritance of this class for lesson-service :

 public class TaskClient extends Client { private static final String SERVICE_PATH = "/lesson-service/task/"; public TaskClient() { super(SERVICE_PATH); } public Task get(final Long id) { return get(id.toString(), TaskResult.class).getData(); } public List<Task> getList() { return get("", TaskListResult.class).getData(); } public List<Task> getListByLesson(final Long lessonId) { return get("/getByLesson/" + lessonId, TaskListResult.class).getData(); } public Task add(final TaskCreation taskCreation) { return post( "/add", taskCreation, TaskResult.class).getData(); } } 

There may be a question, what is the Result and why we return the result from getData() for it. Each controller returns not just a certain requested object in json, but also additional meta-information, which can be useful later, so I didn’t remove it when rewriting into microservices. That is, an object of the Result<T> class is returned, where T is the requested object itself:

 @Data public class Result<T> { public String message; public T data; public static <T> Result<T> success(final T data) { return new Result<>(null, data); } public static <T> Result<T> error(final String message) { return new Result<>(message, null); } public static <T> Result<T> run(final Supplier<T> function ) { final T result = function.get(); return Result.success(result); } } 

There is no getData() method, although it is previously used in the code. All this thanks to the @Data annotation from lombok , which I actively used. Result convenient because further you can easily add some meta-information (for example, the execution time of the request), and somehow use it.

Now, to use the code we have written in other modules, it is enough to add a dependency ( compile project(':service-client') to the dependencies block) and create such a bin. Here is the result-service configuration:

 @SpringBootApplication(scanBasePackages = "result") @EnableJpaRepositories("result.repository") @Configuration public class App { public static void main(String[] args) { SpringApplication.run(App.class, args); } @Bean public UserClient getUserClient() { return new UserClient(); } @Bean public TaskClient getTaskClient() { return new TaskClient(); } @Bean public ExecutorClient getTaskExecutor() { return new ExecutorClient(); } } 

Its controller is:

 @RestController @RequestMapping public class ResultController { @Autowired private ResultService service; @RequestMapping(value = "/submit", method = RequestMethod.POST) public Result<TaskResult> submit(@RequestBody final SubmitRequest submit){ return run(() -> service.submit(submit)); } @RequestMapping(value = "/getByTask/{id}", method = RequestMethod.GET) public Result<List<AnswerEntity>> getByTask(@PathVariable final Long id) { return run(() -> service.getByTask(id)); } } 

It is seen that everywhere the controller returns a Result<T> . And a fragment of the service:

 @Service @Transactional public class ResultService { @Autowired private AnswerRepository answerRepository; @Autowired private TaskClient taskClient; @Autowired private ExecutorClient executorClient; public TaskResult submit(final SubmitRequest submit) { val task = taskClient.get(submit.getTaskId()); if (task == null) throw new RuntimeException("Invalid task id"); val result = executorClient.submit(submit); val answerEntity = new AnswerEntity(); answerEntity.setAnswer(submit.getCode()); answerEntity.setTaskId(task.getId()); answerEntity.setUserId(1L); answerEntity.setCorrect(result.getStatus() == TaskResult.Status.SUCCESS); answerRepository.save(answerEntity); return result; } ... 

answerEntity.setUserId(1L) - for now, it’s just a constant, for now it’s totally incomprehensible how to do authorization.

In general, the main part was done, we model all other services according to the model and everything should work. But it remains to deal with users and their authorization. This turned out to be the most difficult part for me.

Authorization

Previously, before breaking down into microservices, authorization was standard - by username and password, the user was authorized within the application context.

Now the task is expanding, and each of the services must understand whether the user on whom the request came to the service is authorized. And this is despite the fact that requests come not only directly from the user, but also from other services.

The initial search led me to various articles showing how to fumble sessions using Redis , but what I read seemed too complicated for a hello-world home project. After a while, returning to the question, I already found information about JWT - JSON Web Token. By the way, repeating the search attempts while writing this article, I immediately ran across JWT.

The idea is simple - instead of cookies, which usually hold authorization, the authorizing service will issue a certain token, which includes user data, issuing time and other information you need. Then, for any access to the services, the client must pass this token in the header (or in some other way as convenient). Each service can decipher it and understand what kind of user it is, and it does not need to go into the database and all that.

This raises many problems, such as how to revoke a token. There are ideas with several tokens (long and short, the second is used for normal requests, the first for receiving a new token of the second type and just the first can be withdrawn and to check it you need to go into the database).

Many articles have been written on this topic, for example this one , and there are already ready-made libraries for use.

But we have a hello-world project, so we do not need a serious and very correct authorization, but we need something that can be quickly implemented, but that nonetheless will work quite well.

So, having read the Internet a little, for example this article, we decide that there will be only one token and it will be issued by the user-service . Add dependencies:

 compile('org.springframework.boot:spring-boot-starter-security') compile('io.jsonwebtoken:jjwt:0.7.0') 

The second is needed just to generate the token itself. We generate the token as follows for the request with the correct login and password:

 private String getToken(final UserEntity user) { final Map<String, Object> tokenData = new HashMap<>(); tokenData.put(TokenData.ID.getValue(), user.getId()); tokenData.put(TokenData.LOGIN.getValue(), user.getLogin()); tokenData.put(TokenData.GROUP.getValue(), user.getGroup()); tokenData.put(TokenData.CREATE_DATE.getValue(), new Date().getTime()); Calendar calendar = Calendar.getInstance(); calendar.add(Calendar.DATE, tokenDaysAlive); tokenData.put(TokenData.EXPIRATION_DATE.getValue(), calendar.getTime()); JwtBuilder jwtBuilder = Jwts.builder(); jwtBuilder.setExpiration(calendar.getTime()); jwtBuilder.setClaims(tokenData); return jwtBuilder.signWith(SignatureAlgorithm.HS512, key).compact(); } 

key here is the secret key of the token that all services must know to decode the token. I did not like the fact that you need to write it to the config of each service, but other options are more complicated.

Next, we need to write a filter that with each request will check the token and authorize if everything is ok. But the filter will no longer be in the user-service , but in the service-client , since This is a common code for all services.

Filter itself:

 public class TokenAuthenticationFilter extends GenericFilterBean { private final TokenService tokenService; public TokenAuthenticationFilter(final TokenService tokenService) { this.tokenService = tokenService; } @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { final String token = ((HttpServletRequest) request).getHeader(TokenData.TOKEN.getValue()); if (token == null) { chain.doFilter(request, response); return; } final TokenAuthentication authentication = tokenService.parseAndCheckToken(token); SecurityContextHolder.getContext().setAuthentication(authentication); chain.doFilter(request, response); } } 

If the token is not sent, we do nothing, otherwise we try to authorize the client. Verification of authorization is no longer carried out by us, but later in another (standard) filter from spring security. TokenService , where a direct token check occurs:

 public class TokenService { private String key; public void setKey(String key) { this.key = key; } public TokenAuthentication parseAndCheckToken(final String token) { DefaultClaims claims; try { claims = (DefaultClaims) Jwts.parser().setSigningKey(key).parse(token).getBody(); } catch (Exception ex) { throw new AuthenticationServiceException("Token corrupted"); } if (claims.get(TokenData.EXPIRATION_DATE.getValue(), Long.class) == null) { throw new AuthenticationServiceException("Invalid token"); } Date expiredDate = new Date(claims.get(TokenData.EXPIRATION_DATE.getValue(), Long.class)); if (!expiredDate.after(new Date())) { throw new AuthenticationServiceException("Token expired date error"); } Long id = claims.get(TokenData.ID.getValue(), Number.class).longValue(); String login = claims.get(TokenData.LOGIN.getValue(), String.class); String group = claims.get(TokenData.GROUP.getValue(), String.class); TokenUser user = new TokenUser(id, login, group); return new TokenAuthentication(token, true, user); } } 

TokenData is an enum for convenience, from which you can take string representations of fields. There are also two classes - TokenUser (this is a class with three fields) and TokenAuthentication :

 public class TokenAuthentication implements Authentication { private String token; private Collection<? extends GrantedAuthority> authorities; private boolean isAuthenticated; private TokenUser principal; public TokenAuthentication(String token, boolean isAuthenticated, TokenUser principal) { this.token = token; this.authorities = Collections.singletonList(new SimpleGrantedAuthority(principal.getGroup())); this.isAuthenticated = isAuthenticated; this.principal = principal; } @Override public Collection<? extends GrantedAuthority> getAuthorities() { return authorities; } @Override public Object getCredentials() { return null; } @Override public Object getDetails() { return null; } @Override public String getName() { if (principal != null) return principal.getLogin(); else return null; } @Override public Object getPrincipal() { return principal; } @Override public boolean isAuthenticated() { return isAuthenticated; } @Override public void setAuthenticated(boolean isAuthenticated) throws IllegalArgumentException { this.isAuthenticated = isAuthenticated; } public String getToken() { return token; } } 

The user-service config will now look like this:

 @SpringBootApplication @EnableJpaRepositories("user.repository") @Configuration @ComponentScan(value = "user") @EnableWebSecurity @EnableGlobalMethodSecurity(securedEnabled = true, prePostEnabled=true) public class App extends WebSecurityConfigurerAdapter { @Value("${token.key}") private String tokenKey; public static void main(String[] args) { SpringApplication.run(App.class, args); } @Bean public BCryptPasswordEncoder bCryptPasswordEncoder() { return new BCryptPasswordEncoder(); } @Override protected void configure(HttpSecurity http) throws Exception { http .headers().frameOptions().sameOrigin() .and() .csrf() .disable() .addFilterAfter(tokenAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class); } @Bean(name = "tokenAuthenticationFilter") public TokenAuthenticationFilter tokenAuthenticationFilter() { return new TokenAuthenticationFilter(tokenService()); } @Bean(name = "tokenService") public TokenService tokenService() { TokenService tokenService = new TokenService(); tokenService.setKey(tokenKey); return tokenService; } } 

The key here is .addFilterAfter(tokenAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class) , register the filter and specify when it should be run. I like to restrict access to resources not in the config, but in annotations over the controller methods, for example @Secured("ROLE_ADMIN") .

The appearance of the token makes it necessary to receive it not only from the client, but also from other services, respectively, you need to be able to send it further. To do this, I simply accept the tokens from the headers in the controllers, where it is needed, and pass it to the methods of the service client. That is, the Client class gets two get and post methods, for the case with and without a token, an example:

 protected <T extends Result> T get(final String path, final Class<T> type) { return rest.getForObject(serviceFullPath + path, type); } protected <T extends Result> T get(final String path, final Class<T> type, final String token) { HttpHeaders headers = new HttpHeaders(); headers.set(TokenData.TOKEN.getValue(), token); HttpEntity entity = new HttpEntity(headers); return rest.exchange(serviceFullPath + path, HttpMethod.GET, entity, type).getBody(); } 

Concrete client classes change accordingly. And in the controller we get a token using annotations, for example:

 @PreAuthorize("isAuthenticated()") @RequestMapping(value = "/submit", method = RequestMethod.POST) public Result<TaskResult> submit(@RequestBody final SubmitRequest submit, @RequestHeader("token") String token){ return run(() -> service.submit(submit, token)); } 

Although this is a simple solution, it seems inconvenient - constant fussing with the token, its reception, transfer to the methods. It would seem to me more correct to make all the same automatic.

When authorizing by a token, remembering it (this is almost done already, you can look at the TokenAuthentication class), and when using the Client classes, automatically retrieve the token, if it exists, and transfer it to the next service.

In addition, it may be necessary to distinguish requests directly from the client and requests from another service. I think, in this case, the service to which the request came from the user can regenerate the token by inserting a flag into it and then use it to access other services.

Launch

There is not a single finished product that allows you to deploy and manage microservices, monitor their condition and so on, but it seems to me that this already goes beyond the limits of hello-world. To get started, I just run the bootRun for each service and enjoy the result.

Conclusion

In general, it was an interesting and rewarding experience (writing bicycles is always interesting), but if I wanted to develop my project further, I would roll back all the changes and continue working on the classic monolith, because on the scale of such a small project, the complexity of managing this all has greatly increased, and there are few advantages.

As many people have already said in various articles, the choice of such an architecture is not appropriate everywhere and should be taken sensibly. Or purely for the sake of self-study to write such bikes, as described above.

Hope the article was helpful.

→ Full source code of the project can be found here .