As more businesses prepare to make a digital transformation, containers have become the choice cloud computing architecture for faster, more portable and reliable deployments. With the growing interest in containerization, the question arises about how containers integrate with existing infrastructure. In this post, we will look at how containerization affects service discovery and present a network routing solution that allows NetflixOSS Eureka to provide unified discovery between both containers and with VM-based services.

Kenzan specializes in cloud technologies with extensive experience in Amazon Web Services (AWS). We have adopted a number of tools from the NetflixOSS stack for use in AWS, such as Zuul, Ribbon, and Eureka. The discovery service feature of Eureka allows us to build dynamically scalable AWS environments without the need to setup fixed routing and load balancing infrastructure.

Docker introduced a new networking layer that changes everything we know about networking in the cloud. This makes discovery and routing with Eureka challenging. Containers have their own IP addresses, belong on a different subnet, and are only routable from the host running the Docker daemon.

We have experimented with tools like flanneld that create virtual networking layers between Docker hosts, allowing for cross-host communication between Docker containers. Flanneld is easy to setup and does as advertised, but requires hosts wishing to network with containers to be running the flannel daemon. All-in-one Docker solutions like Kubernetes do everything from networking containers to orchestrating and managing multiple Docker hosts, but still cannot network with containers from outside the Kubernetes cluster.

What we are looking for is to spin up containers like we do with EC2 instances, have them register with a discovery service, and allow us to send traffic from anywhere in the VPC. Let’s start simple with a cluster of Docker hosts, which is something that EC2 Container Service (ECS) will provide us. We launch a few applications as Docker instances using ECS, but we get containers that can’t talk to each other and can’t be reached from any external service.

As an application in a discovery-based world, you need to tell the discovery service who you are and how others can reach you. This is easy on EC2 instances because the application can provide the host IP address and the port it’s listening on. Containers in Docker are given IP addresses that are only routable to containers running on the same Docker daemon. We can expose internal container ports as host ports, but that port (which port? That = ?)has to be static and non-conflicting with other containers on the same host. e want to be dynamic and not have to remember which ports are used versus which ports are free.

ECS provides an option to expose containers through an Elastic Load Balancer (ELB). Issues with ELB’s include consuming several VPC IP addresses, requiring management of limits on how many ELB’s can be created, and adding additional AWS costs. Our applications now have to remember a series of hostnames representing ELB endpoints for each environment, adding more configuration overhead. ELB’s  have the advantage of security groups, which is something we may expect to lose in a container world. The new networking layer on top of Docker does not play nicely with network based firewalls like security groups in AWS.

The goal is to find a non-conflicting dynamic way for containers deployed to a cluster of Docker hosts to identify themselves to a discovery service and have their identity be reachable. To achieve this, we need a tool that will find other containers on the host and route traffic based on a series of rules. Traefik is that tool. Traefik is a discovery based HTTP reverse proxy and load balancer that can discover Docker containers with minimal configuration, as well as several other means of discovery.

Let’s look at what it takes to get Traefik running with a Docker based backend:

$ docker run -d -p 80:80 -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock traefik –web –docker –docker.domain=docker.localhost

That’s it.  The front end is now listening on port 80. Notice what we did here is we mounted the docker.sock file as a volume launching the container. This gives Traefik API access to the Docker daemon so that it can find other containers.

Alright, now it is time to add some containers for Traefik to find:

$ docker run -d –name nginx -l traefik.port=80 nginx

We added an Nginx container, and added a Label of traefik.port=80. Traefik will use the Docker metadata API exposed through the mounted unix socket to find container labels and bind the listener to port 80 that Nginx is listening on.

The Nginx container can be seen on the Traefik admin page listening on port 8080

Notice how the Rule is Host:nginx.docker.localhost. This is a combination of the container name we provided to Nginx with the –name argument. The docker.localhost part of the domain came from the –docker.domain=docker.localhost we gave to Traefik at startup.

Running a curl to the Docker host with a Host header of nginx.docker.localhost returns the Nginx welcome page.

$ curl -H “Host: nginx.docker.localhost” 192.168.33.10
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
….

Traefik is routing requests with this Host header to our Nginx container. We didn’t tell Traefik to do that, but it did (and that’s alright).

We can use other labels attached to the Nginx container to control how Traefik routes requests to it. Let’s try a path based rule.

$ docker run –name nginx -l traefik.port=80 -l traefik.frontend.rule=PathPrefix:/nginx nginx

Now all requests with the path prefix of /nginx will go to our Nginx container

$ curl 192.168.33.10/nginx/test
<html>
<head><title>404 Not Found</title></head>
….

We of course get a 404 since our nginx doesn’t have a /test resource, but we can see in the Nginx logs that the requests are coming through.

172.17.0.2 – – [10/Jan/2017:22:48:37 +0000] “GET /nginx/test HTTP/1.1” 404 169 “-” “curl/7.49.1” “192.168.33.1”

That was a quick proof-of-concept to show how quickly we got Traefik running as a discovery based routing service with Docker. By putting a Traefik container on every Docker host, we can  dynamically setup routes to our containers running on those hosts. All we need to do now is identify our containers to the discovery service as the host IP address and the port that Traefik is listening on. The applications calling the services have to remember to include the path prefix in the request. Below is an architecture diagram showing Traefik set up on multiple Docker hosts.

Darren Bathgate is a technical architect at Kenzan. Over the course of his 5+ years at Kenzan, Darren has worked extensively with Java, MySQL, PHP, Cassandra, Node.js, oracle, Jenkins, Netflix OSS and Docker.

In our first post of this series, we looked at how microservices help you build applications with an eye towards the future. And in our second post, we took a deep dive into how microservices work. But what if you currently have a monolithic application? How do you know if it’s time to move to microservices? And if so, how do you get there?

As we mentioned before, knowing why to build a microservice is only half the battle. Knowing when (or when not) to build one can be a trickier question. There’s no one size fits all. That said, there are a few questions you can ask to help decide if you’re ready for microservices. We’ll explore them in this final post of our series.

Reading the Signs

Is your organization or team experiencing exceptional growth in both employee base and technical stack?
As a codebase grows to a certain size, more people are needed to maintain it. But anyone who has worked on a single codebase with ten other engineers knows the struggle of dealing with merge conflicts during code reviews. If a team is losing valuable time trying to resolve conflicts, it might be time for microservices. Microservices help distribute development teams more efficiently, as complex backend systems can be built faster by dividing workloads into multiple small teams, with each team responsible for a given set of microservices.

Is your application down often?
Every time you deploy an application, it breaks. Or the backend system is seemingly down all the time for maintenance. If these scenarios sound all too familiar, it might be time to shift from a monolithic architecture to microservices.

Are you having difficulty scaling hardware to optimize application performance?
Larger and more complex applications require additional hardware to perform well. Some parts of an application might put more demand on the hardware than others, and the whole application can face degraded performance due to a single poorly-performing feature. Microservices allow individual units of business logic to be assigned their own dedicated hardware, and they can be scaled independently of other services. A slower-performing process can be isolated and capped to a fixed about of CPU, memory, disk, and network bandwidth, which means it can’t steal resources from other features.

Choosing the Moment

Once you’ve decided to build a microservice, the next questions are when to build it and when to switch over. That’s a hard question to answer for any organization. The monolith architecture works for small applications and small engineering teams. But when does the monolith stop working?

It turns out growth is often the death of the monolith. Organizations in the process of undergoing exceptional growth, in both the employee base and in technology stack, can achieve that growth by switching to microservices. If the engineering team is losing valuable time resolving merge conflicts, then it might be time for microservices. If the growing backend system is down all the time for maintenance, then microservices might be the answer. The answer of when to build a microservice is when growth is expected.

Hardware provides a great example of this, as it can only be scaled horizontally so much. Maybe those high-CPU and high-memory servers are too expensive and too underutilized during non-peak hours. The cloud is becoming more appealing with the ability to provision hardware as needed, and smaller servers with lightweight workloads become easier to bring up and tear down in response to changing demands. Microservices have lower hardware requirements, and they can start up quickly to meet peak demands.

Making the Move

Monoliths present a number of challenges that you’ll need to tackle in the move to microservices. A monolith has a tendency toward tight coupling of components, as well as stateful behavior. The application was never intended to live as separate and isolated components working in concert to make a system. The monolith may also be rooted to the infrastructure it lives on, depending on system resources such as a local filesystem and network. What’s more, you need to keep everything up and running during the transition. So how do you move to microservices? Let’s break it down.

Step 1: Determine Your Domains

Your first task is to look at all the features of the application and organize them into logical business units, or domains. For example, your application may have major features related to login/logout, user data, and session management. These features can be logically organized into a single business domain called Authentication. Repeat this step for all of the other domains in your application.

Step 2: Prepare the Monolith

Next, get your application ready for the big break up. To do this, decouple components within the application along the lines of the business domains you came up with. This will result in a number of edge and middle modules within the application, each dedicated to a particular domain, like our Authentication example. You’ll also need to move stateful in-memory stores into shared datastores. Finally, put a router in front of the monolith to smooth the rollout of microservices.

Step 3: Work From the Bottom Up

As you begin breaking out microservices from the monolith, it’s always best to start at the bottom and work your way up. First, create a separate database to store data for the domains you are moving to microservices. Next, break out the data access modules that access this data into middle microservices. Finally, break out edge modules that consume this data into edge microservices. When everything’s ready to go, use the router to toggle redirection to the new edge services. Keep repeating this same process for each domain until all of your modules are broken out and the monolith is no more.

In the diagram below, the Login, Users, and Sessions modules were decomposed into modules within the monolith, and then broken out as separate microservices.

One challenge you’ll encounter during the transition is the need to support both the microservices and the monolith side by side. That also means duplicating work, as you’ll often have to make changes or fixes in both places. The good news is that organizing your monolith similarly to your microservices makes it easier to copy-and-paste code between them.

Conclusion

If your application and organization are both set for growth, microservices can help you meet the challenge of building modular, scalable, highly-available solutions. When you’re ready, you need to organize your application’s features into domain-specific modules, then break them out one by one.

Just remember that transitioning from a monolith won’t happen in a single overnight deployment. You need to strategize as you cherry-pick domains out of the monolith. The key is to complete the transition to microservices and not leave your application in limbo. If you’d like to learn more about how to make the move to microservices, or if you need some help, feel free to ask us. That’s what we’re here for.

In our first post in this series, we showed how microservices can help you architect applications with the future in mind.

Knowing what a microservice is, and what purpose it serves, is a big part of building a successful architecture. But to truly fit your business needs and meet your goals, understanding the variations of microservices is key.

While there is no precise way to define the architectural style of microservices, by looking at specific characteristics, we can better understand what makes an application a microservice. In this post, we’ll go into more detail about how microservices work, and how to make them work for you.

How Big is Micro?

You can build what you think is a microservice, but actually, what you have just created is a distributed monolith. Individual applications become so coupled together that we start referring to them as a single noun. In other cases, we end up seeing microservices get so large that they themselves become monolithic, resulting in a monolith army.

The size of a microservice isn’t determined by the number of lines of code or the amount of functionality, but by the amount of volatility a microservice has—the amount of change that is expected to occur over the life of a microservice. If changing one microservice also requires changing another microservice, it means those microservices have been incorrectly decoupled from one another and should instead be combined. On the other hand, if a microservice is composed of features that are fundamentally dissimilar and volatile, it means those features have been incorrectly coupled together, and they should instead be split up.

In a nutshell, features of a microservice should be grouped by similarity and the likeliness of change. If changes to a microservice keep causing backwards incompatibility or constantly require refactoring, it probably needs to be decomposed into multiple microservices.

Life on the Edge (and in the Middle)

Another part of defining a microservice is understanding what the types of microservices are and the rules they should follow. These can vary depending on your architecture. One architectural solution we’ve employed with success defines two types of microservices: data-driven services in the middle and business-driven services on the edge.

Middle Tier: Driving Data

Data-driven services are called middle tier services, which are solely assigned and are responsible for a single data source. In this architecture, the only way to access any given data source is through the corresponding middle tier service. These are assigned to a single data source, so that if one data source goes down, it will not take the other services with it in the event of an outage. The only responsibility of a middle tier service is to make data available to other microservices. This could also mean applying caching or fallback scenarios to keep the data flowing.

Edge Tier: Driving Business

We refer to business-driven services as edge tier services, which drive the business logic of an application. Edge tier services typically exhibit the most amount of volatility-based decomposition, as business logic for each edge tier service can vary greatly depending on the systems it supports.

While choosing when to build a middle tier service is easy (if you have a data source, you need a microservice to go with it), choosing edge tier services requires more attention to the amount of volatility in the business logic. Functionality is driven by edge services, and in this type of architecture the number of edge tier services will always be greater than the number of middle tier services. Edge tier services connect with one or more middle tier services, but typically won’t connect with another edge tier service. In this particular solution, proper decomposition of microservices based on volatility shouldn’t require edge services to depend on other edge services, or middle services to depend on more than one data source.

While we’ve put this type of architecture to good use, other solutions are certainly possible—more on that in a bit.

Finding Each Other in the Cloud

Deploying microservices to the cloud enables clusters of microservices to be scaled up as load increases, and scaled down after load diminishes. This means that IP addressing of machines running microservices is constantly changing. So the challenge becomes: how do we route fixed traffic to a moving target?

In the past, a common pattern was to reference hosts using domain names rather than IP addresses. In this case, when a host IP changes, the DNS record is updated with the new IP address. Tools such as Consul can be employed to propagate DNS changes. Alternately, redundantly-deployed services can register with a load balancing appliance. The group of services is then referred to using the address of the load balancer. However, in a microservices architecture, this can rapidly become costly due to the large number of load balancers required.

To better solve this issue, we must get creative and route traffic in a more dynamic way. Service discovery is a pattern that lets us identify a group of microservices by name rather than by IP address. With service discovery, a centralized registry is used to store the locations of all services in the surrounding environment.

The most common pattern we implement is to have microservices push their own information to the registry on startup and say, “Hey, my name is service-a, and here is my IP address and port”. Another service can ask the registry for all IP addresses for services named “service-a”. The requesting service can then strategize which one of the services named “service-a” to talk to. This type of push-based discovery pattern can be implemented with Eureka (part of the NetflixOSS stack), and it requires discovery-enabled apps to use a client library to talk to the discovery service.

API Gateways performing reverse proxy operations can also take advantage of this service discovery feature, and can route all traffic for a specific path to a group of microservices of the same name. These gateways are utilized as a router of all inbound HTTP requests, which consolidates Internet-facing traffic under a single domain name. The router can also apply rules or filters to enforce security for edge services, for example, to require authentication. This gives you a fine degree of control over how different types of traffic are routed.

Exploring Alternate Designs

At Kenzan, we are always thinking of ways we can improve on our architectural patterns. Lately we have been exploring alternatives to the edge/middle microservice design pattern described in this post. The details of this alternate design will hopefully make an appearance in a future blog post, but in the meantime here’s a sneak peek.

We like to refer to this design as the one-edge/many-middle pattern. It might even become a three-tiered design as it evolves, with edge services, middle services, and data services. In this scenario, data services provide the data abstraction layer, middle services drive business logic, and edge services drive domain logic. The edge services have similarities to an API gateway, but they also make domain-specific decisions on which middle services to use. The result is a better decoupling of microservices and a clearer design with fewer edge services.

As we are learning, a particular microservice architecture may work for one organization but not for another. We will continue to explore different microservice designs, and evaluate the benefits and challenges of each.

Conclusion

To make microservices work for your business, it’s important to compose them correctly and place the right functionality in the right tier according to your chosen architecture. There are many ways to architect and develop microservices, and we are constantly talking about ways to improve the design.

When starting with a microservice architecture, plan a consistent strategy and define guidelines for the entire organization to use. Share best practices and lessons learned with the other teams, and continue to evolve microservice patterns. Consider the scaling capabilities, and build in capabilities like service discovery.

Going with microservices from the start can be a great approach. But what if you have an existing application you want to migrate to microservices? In our next post, we’ll talk about how to break up a monolith into manageable chunks.

Architecting for the Future

At Kenzan, we often work with large, consumer-facing companies whose customers are demanding richer, more interactive, and friendlier experiences on all of their devices. As trends in software architecture evolve to adjust to these expectations, organizations need to create applications that are both highly available and highly scalable.

We’ve been building web applications with microservices for a while now and have found they’re easy to manage, can be scaled independently or across servers, and offer flexibility to quickly deliver features.

In this blog series, we’ll look at how you can architect for the future using microservices, some of the challenges you may face along the way, and strategies for mitigating these challenges.

What Are Microservices?

Microservices have been around for a few years, but you may not have encountered them yet.

In a traditional architecture, all functionality resides in a single, monolithic server-side application. This makes it easier for one developer to understand the whole system or run it locally, but it can quickly lead to issues as the application grows in size.

In a microservices architecture, functionality is distributed across small, self-contained, modular services that communicate with one another over the network.

What Are the Advantages?

Adopting a microservices architecture brings a lot of benefits – let’s check them out.

Up and Running Today

High availability and uptime are two big selling points for microservices. That’s because they allow for smaller components or services to be upgraded without risk of taking down an entire system.

Imagine a customer-facing application that has one monolith for all services, including account information, identity data, and login portals. Maintaining and deploying updates to just one of those services means you’re at risk of bringing down the entire application if there’s an issue. With microservices, if one service goes offline, the rest of the application doesn’t have to go down with it.

Scale to Meet the Future

The first thought that comes to mind when talking about scale is the amount of load a system can handle before falling down. But that’s not the only type of scale to consider when you’re building a microservice. When you’re dealing with big data, a microservices architecture lends itself to scale for velocity (number of active users on a given system) and for volume (how much data you have). With microservices, you can scale your applications just at the point of demand. You don’t have to scale everything together. For example, edge services that communicate directly with web clients often have higher load requirements than back end services, especially if caching strategies are used to reduce communication with the back end.

Microservices can also help scale your organization along with your application. Using microservices allows for distributing development teams more efficiently. Complex backend systems can be built faster by dividing workloads into multiple small teams, with each team responsible for a given set of microservices. Clear domain ownership means you always know which team to go to when there’s a problem with a microservice. Likewise, teams are empowered to operate and deploy independently at a delivery cadence that makes sense for them rather than coordinating large releases across many teams.

Every Service For Itself!

Let’s look at our imaginary monolithic application again. The many features of this application share resources like CPU, memory, network latency, and I/O, and as a result could easily overwhelm these resources, causing all the services to suffer. In a microservices architecture, many of these resources are isolated to the individual service and can be optimized more easily. Through containerization with frameworks like Docker, or serverless solutions like AWS Lambda and API Gateway, resource isolation is a much more cost-efficient practice in microservices. Services scale according to their own needs rather than to the needs of the busiest component.

Use of common protocols like HTTP, and standard interfaces like REST for communication with other services, allows microservices to be technology agnostic. Teams can choose to build microservices in the language they are most comfortable developing in. Having a microservice stack including both Node.js and Java applications is becoming more common. This lets organizations tap into a wider range of skillsets. (Choose technologies wisely, though, as some may not play nicely with other technologies in the microservice stack.)

What Are the Challenges?

Knowing why you should build a microservice is only half the battle. Knowing when to (or when not to) build one can be a trickier question. Every engagement is different, and there’s no one size fits all. Here are some of the key factors to consider:

• Infrastructure – Adopting a microservice stack can require a bit of scaffolding to get up and running. Considerations such as infrastructure requirements, deployment strategies, monitoring, and configuration management come into play when building a custom microservices solution. Platform-as-a-Service products like Amazon’s Elastic Beanstalk can reduce the time required to develop a microservice platform, and they streamline the process of developers deploying code.
• Code Compatibility – Microservices can suffer from code compatibility issues like monoliths can, but the single codebase nature of a monolith does allow code compatibility issues to be caught at compile time. In contrast, changing the interface of a microservice may go unnoticed until another microservice breaks after deployment. Maintaining API Contract Tests is a good way to ensure consistency of a microservice’s external API and avoid unintentional changes.
• Shared Libraries – There will be many microservices doing similar things, and it often becomes apparent to put common patterns into libraries to be shared with other teams. Sharing code with other teams is like sharing open source code on GitHub. The code must be useful enough to share and intended to accept contributions from other teams.
• Coding Practices – Many different teams working in siloed environments can result in vastly diverged coding practices. Whether or not this is an issue can be left to the discretion of the organization and their development culture, but establishing company-wide standards and periodic discussions of new patterns can help keep teams in sync.

Finally, microservices aren’t right for every application, like those with a limited scope, a small number of users, or a single data repository.

Conclusion

When applied to the right problems, microservices are a flexible solution that many companies are adopting to architect for the future. Given the complexities, you might decide to have an experienced guide on your journey.

In our next post, we’ll talk about how to put microservices to work to solve challenges now and prepare for future needs.

Have a question about building and implementing microservices at your company? Email us at info@kenzan.com

Darren Bathgate is a technical architect at Kenzan/ Over the course of his 5+ years at Kenzan, Darren has worked extensively with Java, MySQL, PHP, Cassandra, Node.js, oracle, Jenkins, Netflix OSS and Docker.