Why a Spring Cloud Config Server is Crucial to a Good CI/CD Pipeline (Pt 1)

Introduction

Before I began developing large-scale, enterprise level software, I didn’t fully comprehend the value of things like integration tests, automated build processes and shared libraries.

For the small applications I built during my coding bootcamp, it was easy for me to do things like hard code URLs so my frontend application could talk to its backend counterparts and their databases.

I only had two environments: my local development environment on my laptop and my AWS production environment where I hosted my projects portfolio, and it was a fairly simple, albeit entirely manual, process (with a good bit of trial and error) to get the pieces of the puzzle connected.

Fast forward several months to my software engineering job of maintaining and improving upon an application backed by no less than 13 microservices. No, I’m not joking. Yes, all 13 are critical to this application working.

Did I mention that in my team’s test driven development (TDD), agile process we also have 4 different development environments before a new feature makes it to production? Yep: a local development space, a QA space, a Q1 space, an acceptance space and finally our production space.

So, think about it: does it make sense to have to manually update each and every one of these services as they move through the development process on their way to production? Of course it doesn’t.

You need a centralized, easy-to-automate way to update those environment variables. This is where a cloud configuration server can shine.

To give you a quick overview, cloud config servers are designed to:

“Provide server and client-side support for externalized configuration in a distributed system. With the Config Server you have a central place to manage external properties for applications across all environments.”

- Spring Cloud Config

Basically, a config server allows you to externally store variables your application will need to run in all environments, regardless of lifecycle, and update them in one, centralized place.

The first config server I was introduced to is the Spring Cloud Config server, since my team is responsible for a full stack application composed of many Java Spring Boot backend microservices and a JavaScript Node.js frontend microservice.

Setting Up Your Config Server

Setting up the config server is actually pretty easy.

To get started, you can go the Spring Initializr site, add Config Server as a dependency to the project and click the "Generate Project" button in the browser.

I’ll also add that, at least at the time I’m writing this, the Spring Cloud Config Server is not compatible with Spring Boot version 2 (I found this out the hard way), so choose the highest Spring Boot version 1 snapshot you can.

Once you’ve downloaded the project and opened it up in your IDE of choice (mine is IntelliJ), you can go straight to the main application file, add the @EnableConfigServer and @SpringBootApplication Spring Boot annotations, and you’re almost ready to go.

Below is my actual main method file, truly, that’s all it needs.

package com.myExampleConfigServer; 
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer; 

@EnableConfigServer
@SpringBootApplication

public class MyExampleConfigServerApplication {  
  public static void main(String[] args) {
    SpringApplication.run(MyExampleConfigServerApplication.class,  args); 
    }
}

The other file you’ll need to configure is your application.yml file in your resources/ folder. This file will be where you set up your cloud config server’s access to GitHub (and the GitHub files it will use to provide application configuration variables).

As you can see, there’s a spot for your GitHub URL (which I’ll come back to in a minute), username and password.

Since our applications are hosted on Pivotal Cloud Foundry, we use what are called VCAP services (it stands for VMWare Cloud Application Platform). It’s a fancy way of saying a service where environment variables are stored that only people (and applications) who have access to that space within PCF can access. This lets us put sensitive information like service accounts and passwords somewhere besides a public GitHub repo where the credentials are available for the config server, but not available for anyone without authorization.

Here’s my application.yml. With these two files configured and a cloud platform to host your server, you’ll be ready to start using your config server.

application.yml

server:  
  port:
    8888

spring:  
  application:    
    name: myexample-config-server  

cloud:    
  config:      
    server:        
      git:          
        uri: ${vcap.services.config-service.credentials.url}
        username: ${vcap.services.config-service.credentials.user}          
        password: ${vcap.services.config-service.credentials.token}

If you’d like to see an example of the config server app, I’ve put a starter project here in GitHub.

Ok, so you’ve got your config server. Great, now it’s time to set up your config server properties — the files the config server will actually access to pull environment variables for your projects.

Setting Up Your Config Server Properties

This part is a cinch. Create a totally empty new repo in GitHub, and then create a new file with the following naming conventions:

[application name]-[life cycle profile].yml

ex. my-app-to-config-QA.yml

Within this YAML file, you’ll be able to add your configuration properties for your application. Here’s some sample info you might include in the YAML.

my-app-to-config-QA.yml

configurations:
  featureToggleFlag: true
  my-custom-flag: false
  sampleCronJob: "0 0 1 * * *"
  sampleUrl: http://google.com

Commit this file to a GitHub repo, copy that repo URL and place it in the Cloud Config Server URI in your config server’s application.yml. Now the server knows where to look in GitHub for the config files you want to use.

You can see an example of a config properties repo here in GitHub.

Connecting Your Java Spring Boot Application to Your Config Server

With both your config server and your properties you want the server to provide to your project, you need your project to be able to get those properties when your Spring Boot service starts up.

Again, there’s not a lot of overhead to getting this working. In the build.gradle file of your app that needs these config server values, you’ll need the following dependencies:

build.gradle

compile group: 'org.springframework.cloud', name: 'spring-cloud-starter-config', version:'1.3.3.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-configuration-processor', version: "${springBootVersion}"

Then, you’ll create a bootstrap.yml file which will live alongside your application.yml file in your src/main/resources/ folder, and it will contain information pointing to the config server’s location wherever it’s being hosted. It will look something like this:

bootstrap.yml

spring:
  application:
    name: ##APPLICATION NAME GOES HERE##
  cloud:
    config:
      uri: https://myexample-config-server.non-prod.com
---
spring:
  profiles: production
  cloud:
    config:
      uri:  https://myexample-config-server.prod.com

Next, you’ll go to whichever files actually need the config properties and add the annotations @Configuration and @ConfigurationProperties(prefix=”configurations") (or whatever you’ve named your config properties in the config properties YAML).

Here's an example:

XyzProperties.java

@Configuration
@ConfigurationProperties(prefix = "configurations")
public class ConfigProperties {
    
}

If you notice, the properties are prefixed by configurations. To bind these properties to the ConfigProperties class, you'll first need to add a "prefix" to the @ConfigurationProperties annotation like so:

To bind the “featureToggleFlag” property, add the following member variable. along with the getter and setter. Or use Lombok’s @NoArgsConstructor and skip the getters and setters - your choice.

XyzProperties.java

@Configuration
@ConfigurationProperties(prefix = "configurations")
public class ConfigProperties {
   private boolean featureToggleFlag;
   
   //Add Getters and Setters here if desired...
    
}

If you match the member variable name with the actual property name in the configuration properties, Spring will automatically bind it to the member variable. So far so good.

The very last thing you must do before starting up your service is ensure that you set SPRING_PROFILES_ACTIVE to the correct config properties environment. The current values would be “QA” for the QA environment, “Q1” for the Q1 environment, “prod” for production and so on. Be warned, these values are case sensitive, so however you’ve named them in your config properties file, it must be exactly the same in the "Active Profiles" input in IntelliJ's start script setup.

In IntelliJ, this can be done by clicking the "Edit Configurations" property when setting up a Spring Boot project to run, and adding the correct value to the "Active Profiles" input midway down the config modal.

And that’s it.

Now, when you start up the Spring Boot service, you should see Spring Profile being set in the logs, and to be sure, you might also put in some System.out.println messaging to let you know if it’s successfully reached the config server and acquired the properties needed.

Fantastic, we've built a config server, we've moved environment variables out of your Spring Boot project to a centralized location, so changes can be made quickly and we've configured a Java service to pull those variables from the config server. Nicely done.

Conclusion and Part Two: The Config Server and Node.js

But, how do you do the same thing with a JavaScript / Node.js project? Can you use the same Spring Boot config server? Or do you have to set up a separate Node-based config server?

You can use the existing one, and I’ll show you how to set it up, and how you might use it in a JavaScript project to enable and disable feature toggles for faster feature deployments in my next post.

Thanks for reading!

Further References & Resources

• Spring Cloud Config Initializr
• Sample Cloud Config Server Repo
• Sample Cloud Config Properties Repo