Docker Compose + Consul + Spring Boot + FeignClient

This article shows an example of how to raise the local development environment using Docker Compose, Consul, Make for Spring Boot applications (and not only), using, for example, PostgreSQL and Browserless.

Plan:

1. Setting up services in Docker Compose
2. Registering services in the Consul'e and adding variables to the Consul'a repository
3. Creating a Makefile
4. PostgreSQL configuration
5. Using FeignClient
6. Conclusion

The application is absolutely useless: a link to the page returns a link to the largest image from this page. The image will be retrieved by Browserless, and in PostgreSQL this file will be saved.

Link to the project: https://bitbucket.org/maximka777/consul-docker-spring-cloud/src/master/ .

1. Setting up services in Docker Compose

The first thing to do is to create a file with the configuration of docker containers docker-compose.yml :

 touch docker-compose.yml 

This file contains the version of docker-compose:

 version: '3.4' 

Network configuration:

 networks: lan: 

And the configuration of the necessary services, in this case Consul, Browserless and PostgreSQL:

 services: consul: image: consul:1.1.0 hostname: localhost networks: - lan ports: - 8500:8500 postgres: image: postgres:11.0 hostname: localhost networks: - lan ports: - 5432:5432 environment: POSTGRES_PASSWORD: password POSTGRES_DB: example_app browserless: image: browserless/chrome hostname: localhost networks: - lan ports: - 3000:3000 

POSTGRES_PASSWORD is the password for the user’s default database postgres , POSTGRES_DB is the automatically created database in the container.

To start the services you need to run the command:

 docker-compose up 

We are waiting for the end of loading of container images and launch of containers. To stop containers, use the docker-compose down . After launching all containers, you can go to the address in the browser localhost:8500 - the Consul web client should open (Fig. 1).

Picture 1

2. Registering services in Consul and adding variables to Consul’s storage

Registration of services in Consul can be done by sending several post requests to localhost:8500/v1/agent/service/register , for example, using curl.

Put all calls to curl in the bash script.

 #!/bin/bash curl -s -XPUT -d"{ \"Name\": \"postgres\", \"ID\": \"postgres\", \"Tags\": [ \"postgres\" ], \"Address\": \"localhost\", \"Port\": 5432, \"Check\": { \"Name\": \"PostgreSQL TCP on port 5432\", \"ID\": \"postgres\", \"Interval\": \"10s\", \"TCP\": \"postgres:5432\", \"Timeout\": \"1s\", \"Status\": \"passing\" } }" localhost:8500/v1/agent/service/register curl -s -XPUT -d"{ \"Name\": \"browserless\", \"ID\": \"browserless\", \"Tags\": [ \"browserless\" ], \"Address\": \"localhost\", \"Port\": 3000, \"Check\": { \"Name\": \"Browserless TCP on port 3000\", \"ID\": \"browserless\", \"Interval\": \"10s\", \"TCP\": \"browserless:3000\", \"Timeout\": \"1s\", \"Status\": \"passing\" } }" localhost:8500/v1/agent/service/register curl -s -XPUT -d"{ \"Name\": \"example.app\", \"ID\": \"example.app\", \"Tags\": [ \"example.app\" ], \"Address\": \"localhost\", \"Port\": 8080, \"Check\": { \"Name\": \"example.app HTTP on port 8080\", \"ID\": \"example.app\", \"Interval\": \"10s\", \"HTTP\": \"example.app:8080/actuator/health\", \"Timeout\": \"1s\", \"Status\": \"passing\" } }" localhost:8500/v1/agent/service/register 

chmod +x register-services.sh - to make the file run.

After executing the script, our PostgreSQSL and Browserless will appear in the list of registered services in Consule'e (Fig. 2).

Figure 2

The figure shows that the PostgreSQL check passes with an error - (it will not affect the essence) .

Add the configuration to the Consul key / value storage. Create a test.property variable in the example.app directory:

 curl --request PUT --data TEST \ localhost:8500/v1/kv/example.app/test.property 

If there are many variables, it is better to use a bash script.

3. Creating a Makefile

To simplify the launch of all this, we will write Makefiles:

 docker_up: @docker-compose up -d consul_up: @./register-services.sh && \ ./register-variables.sh compile: @cd example.app && mvn package run: @cd example.app && java -jar target/example.app-1.0-SNAPSHOT.jar up: docker_up consul_up compile run down: @docker-compose down 

Warning: The Makefile uses a special type of indentation!

The make up command will launch the entire environment.

4. PostgreSQL configuration

Next, the base Spring Boot project (Maven) was generated using the initializer of Spring Boot applications https://start.spring.io/ .

The following dependencies have been added to pom.xml :

  <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-consul-config</artifactId> <version>2.0.1.RELEASE</version> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-consul-discovery</artifactId> <version>2.0.1.RELEASE</version> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-openfeign</artifactId> <version>2.0.2.RELEASE</version> </dependency> <dependency> <groupId>org.postgresql</groupId> <artifactId>postgresql</artifactId> <scope>runtime</scope> </dependency> 

From the names of packages it is clear what they are for.
Let's write the configuration for the DataSource. In the bootstrap.properties file we drop configs:

 spring.cloud.consul.host=localhost spring.cloud.consul.port=8500 spring.cloud.consul.config.enabled=true spring.cloud.consul.config.prefix= spring.cloud.consul.config.defaultContext=example.app spring.cloud.consul.discovery.register=false spring.cloud.service-registry.auto-registration.enabled=false spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults = false spring.jpa.database-platform=org.hibernate.dialect.PostgreSQL9Dialect spring.jpa.show-sql=false spring.jpa.hibernate.ddl-auto=create 

In application.yml :

 example.app: db: name: 'example_app' feign: client: config: default: connectTimeout: 20000 readTimeout: 20000 loggerLevel: basic management: endpoint: health: show-details: always endpoints: web.exposure: include: '*' 

And the configuration class itself:

 @Configuration public class PersistenceConfiguration { @Value("${example.app.db.name}") private String databaseName; @Autowired private DiscoveryClient discoveryClient; @Bean @Primary public DataSource dataSource() { var postgresInstance = getPostgresInstance(); return DataSourceBuilder .create() .username("postgres") .password("password") .url(format("jdbc:postgresql://%s:%s/%s", postgresInstance.getHost(), postgresInstance.getPort(), databaseName)) .driverClassName("org.postgresql.Driver") .build(); } private ServiceInstance getPostgresInstance() { return discoveryClient.getInstances("postgres") .stream() .findFirst() .orElseThrow(() -> new IllegalStateException("Unable to discover a Postgres instance")); } } 

The getPostgresInstance() method takes the first instance of the service with a postgres tag registered in Consul. The dataSource() method is a DataSource bin.

Next, we will declare a repository with basic operations on the Image entity, which stores the page address and image address:

 @Repository public interface ImageRepository extends JpaRepository<Image, Long> { } 

5. Using FeignClient

Next, in the resources, let's cast a JS script that will pull the largest image from the page.

 module.exports = async ({page, context}) => { const {url} = context; await page.goto(url); await page.evaluate(_ => { window.scrollBy(0, window.innerHeight); }); const data = await page._client.send('Page.getResourceTree') .then(tree => { return Array.from(tree.frameTree.resources) .filter(resource => resource.type === 'Image' && resource.url && resource.url.indexOf('.svg') == -1) .sort((a, b) => b.contentSize - a.contentSize)[0]; }); return { data, type: 'json' }; }; 

Define the BlowserlessClient interface:

 @FeignClient("browserless") //   browserless      public interface BrowserlessClient { @PostMapping("/function") ImageInfo findLargestImage(LargestImageRequest request); //       Browserless'     class ImageInfo { private String url; public String getUrl() { return url; } } //  ,    Browserless  ,    class LargestImageRequest { private String code; private BrowserlessContext context; public LargestImageRequest(String code, BrowserlessContext context) { this.code = code; this.context = context; } public String getCode() { return code; } public BrowserlessContext getContext() { return context; } } //        class BrowserlessContext { private String url; public BrowserlessContext(String url) { this.url = url; } public String getUrl() { return url; } } } 

The service method that requests the image and saves it to the database:

 public Image findLargestImage(String url) { var browserlessContext = new BrowserlessContext(url); var largestImageRequest = new LargestImageRequest(getLargestImageScript, browserlessContext); var imageInfo = browserlessClient.findLargestImage(largestImageRequest); var image = new Image(); image.setSourceUrl(url); image.setImageUrl(imageInfo.getUrl()); return imageRepository.save(image); } 

Functionality Controller:

 public class MainController { private static Logger log = LoggerFactory.getLogger(MainController.class); @Autowired private ImageService imageService; @Value("${test.property}") private String testProperty; @GetMapping("/largest-image") public ResponseEntity<Image> getTitle(@RequestParam("url") String url) { return ResponseEntity.ok(imageService.findLargestImage(url)); } @GetMapping("/property") public ResponseEntity<String> getProperty() { return ResponseEntity.ok(testProperty); } } 

Here, the testProperty field testProperty pulled from the testProperty key / value store.

6. End

Everything !

I hope I was able to show the possible configuration of the presented tools and this article will be useful to someone.

This article is not very much explanation, as I believe that in this case it is easier to understand the code.

Useful links:
1) https://docs.docker.com/compose/overview/ - Docker Compose documentation
2) https://www.consul.io/intro/index.html - introduction to Consul
3) http://matt.might.net/articles/intro-to-make/ - introduction to make
4) https://cloud.spring.io/spring-cloud-netflix/multi/multi_spring-cloud-feign.html - Feign documentation