Category Archives: Tech Skills

The front-end development landscape has changed dramatically over the past few years. The release of the latest EMCAScript specification in 2015 brought a new sense of maturity to JavaScript. It also introduced two key features: modules and classes. This has led to a proliferation of tools, libraries, and frameworks that have accelerated the development of front-end applications. But with so many great choices, how do you decide what’s best for your application and team?

In Kenzan’s latest series for Linux.com, we set out to answer that question. We examine in depth how a stack focusing on Yarn for package management, webpack for application bundling, and TypeScript for writing application source code can give developers and organizations the building blocks to write more maintainable and better optimized applications.

For the next few weeks, Linux.com will be rolling out the five-part series on their blog. The first part, posted today, gives you a short history lesson on JavaScript and delves into the core philosophies of front-end development. Throughout the rest of the series, you’ll learn more about the main components of our modern day front-end development stack. We’ll finish up with a case study of a working hello-world application that shows all the elements of the stack in action. By the end, we hope you’ll be able to take what you’ve learned and apply it to your own infrastructure.

As with everything at Kenzan, this project wouldn’t have come to fruition without our incredibly talented team:

Development

  • Marie Schmidt
  • Chris Pruyne
  • Chris Joslyn
  • Paul Barry
  • Carlos Hernandez

Tech Writers

  • Ryan Daugherty
  • Mark Anthony

Leadership

  • Owen Buckley
  • Nick Wester
Read the first part of our series on Linux.com: Set Up a CI/CD Pipeline with Kubernetes.

What’s an eight letter word that starts with K and is the name of a powerful container orchestration tool?

If you just shouted out Kubernetes (or if you were curious about the answer), then you’re ready to tackle the Kenzan Kr8sswordz Puzzle.

The Kr8sswords Puzzle is a crossword puzzle app that runs in pods on Kubernetes. It takes advantage of a full CI/CD pipeline in Jenkins that also runs in a pod, automating the entire build-and-deploy process. And it includes a bunch of other cool components that we don’t want to spoil for you because it’s much more fun to boot up the app yourself and see the different pieces in action. (Okay, here’s a hint: React frontend, Etcd caching, and MongoDB persistence.)

Kenzan designed the app as a four part blog series that is launching on Linux.com today. Part 1 kicks things off by walking you through setting up a local Kubernetes development environment. Each weekly blog post will add components to show the Kr8sswordz Puzzle in action, how to spin up many pod instances for a load test, and how you can easily incorporate Kubernetes into a CI/CD pipeline.

The Kr8sswords Puzzle app was a lot of fun to pull together. We hope you have just as much fun  building it out as the blog series goes on.

Kenzan is a family, and we would like to give a shout out to the many folks that made the app and blog series possible:

Developers
Chad Moon, Platform Engineer
Sean Korten, Platform Lead Engineer
Evan Yeager, Technical Architect
Geiser Menoia, Back End Developer
Justin Tomlinson, Director of Engineering
Marie Schmidt, Front End Developer
Scott Pullano, Director of Engineering
Rona Kilmer, Director of Engineering

Tech Writers
Ryan Daugherty
Mark Anthony

Testers
Oscar Santamaria
Matthew Gardner
Ben Sawyer
PJ Pannoni
Robin Morrison
Matt Poirier

Leadership
Elana Krasner
Nick Wester
Craig Martin

Download our guide to continuous delivery with Spinnaker and Kubernetes.


The path to digital transformation is one that many companies, regardless of industry, are taking in order to bring products and services to market faster. As part of this transformation from traditional enterprise to modern software company, the role of IT is now a core part of business strategy with the responsibility of delivering software to users faster, safer and more efficiently.

Over the past few years, technology has evolved to enable this kind of delivery. The practice of continuous delivery has emerged as a way to enhance the ability of technical teams to deploy software to the end user faster, safer and more efficiently. Kenzan has paired two tools, Netflix’s Spinnaker as the preferred continuous delivery framework and Google’s Kubernetes as the preferred container management platform, in order to provide a continuous delivery solution that can support any enterprise that is pursuing digital transformation. 

Download our guide to continuous delivery

Since it was created in 2007, GitHub has not only become the primary destination for open source development and distribution, but also one of the largest collaborative platforms on the internet. GitHub’s open source practices have established the platform as one to the most important tools for developers and gives coders of all levels a place to learn new skills, contribute to a vast number of projects and work alongside other programmers. Kenzan is a company that leverages the benefits of open source software by opening many of our  projects to the community and contributing to others in order to support the technology landscape at large.

More than just a GUI on top of Git, the GitHub platform  offers many powerful tools  and features that can help maintainers of OSS projects provide a more useful, transparent, and open experience for their project and their community. In this post, we will walk through some of the best practices that can help make an OSS project successful and inspire community participation and contribution.

Getting Organized

In order to ensure a consistent path to PR integrity, acceptance and delivery, it is important to make sure team permissions are setup appropriately.  At Kenzan, we identify maintainer and contributor teams for our projects.  The primary goal is to establish a protected branch (typically master) for a given repo and those who should have write access to it. Protected branches allow you to give read access to everyone else in the organization or an internal contributing team, but only permit a small team to actually be able to manage the main line of development. Roles can be defined as:

  • Maintainers – Act as the stewards of the project with sole access to protected branches like master
  • Contributors – Community team within the organization that are allowed to read and push branches within the repo

Below is an example of a sample configuration that protects the master branch with options like:

  1. Ensuring only the maintaining team can push to master
  2. Requiring PR approvers
  3. Requiring PRs pass a Continuous Integration build

Pro-Tip!
Always use teams for delegating repo access, for both maintainers and contributors. This will ensure easy addition and removal of members, as well as a simple and streamlined permissions strategy for your organization and repositories.

Welcoming Contributions

Great software is developed by many people contributing more than just code like testing, refactoring, documentation, and more.  However, each project is generally run in a particular way, and so clearly communicating these specific processes and workflows can help make a big difference in the growth and success of your project.  Guiding developers and contributors through the following recommendation will  help ensure understanding and onboarding from the start.

  • README.md – First impressions are everything, so a README that includes important information like an overview (your “elevator pitch”), installation/setup, usage and example, license, etc., will ensure first timers to your project know exactly what it is, what it does, and if it is right for them.
  • CONTRIBUTING.md – by including a CONTRIBUTING.md in your project’s root, you can define the Contributing Guidelines for your project. With this, GitHub can prompt all users who open an issue or PR to your project with a link to read it.
  • LICENSE – Having a LICENSE file is important to disclose to users how your project can be consumed, used, and distributed. In addition, be aware of the license expectations of any dependencies your project has.
  • Webhooks – Webhooks allow “push notification” integration with 3rd party services such as a Continuous Integration server like Jenkins.  Anything that can be automated in your project (linting, static analysis, style guide, tests, etc) should be scripted and run against every PR to provide fast turnaround and feedback to contributors (i.e. pass or fail).  Kenzan uses GitHub PR Builder.
  • Issues / PRs – GitHub has done a lot to improve the features around these workflows, in particular Pull Request Templates, fields for reviewers, assignees and labels, reactions and many others.  These are great features for identifying/establishing ownership and status.

Below is an example of a PR making use of some of the available PR fields like

  1. Contributing guidelines (the yellow banner)
  2. PR templates (the content inside the textarea)
  3. Reviewers / Assignees

Pro-Tip!
Always create an issue for any non-trivial change. This promotes discussion of the feature to happen independently of the code review itself.

Bonus!
If you link to the issue in the PR, using #XX (where XX is the issue number) GitHub will close the issue when the PR is merged.

Steering the Project

As a project grows and matures, planning out goals and improving documentation will naturally require some additional organization.  No worries, GitHub has you covered!

  • Project Management – Leverage features like milestones and projects to group issues into related buckets of work.  At Kenzan, we see projects as sprints (in agile terms with one or more projects making up a milestone. This allows a project’s mantainer to communicate their roadmap clearly and can help direct contributors to where help is most immediately needed.  It also helps establish feature / bug delivery expectations.
  • Release Management – Whenever a Git tag is pushed to your repository, GitHub will create a link to a page for that tag, thus enabling  the easy creation of release notes for every tag.  This is useful for tracking progress over time and to keep a record of features and bugs completed.
  • Wiki – The wiki is a great place to organize and manage supplemental information for consumers and developers. Rather than pack everything into the README, the wiki can be used to establish separate documentation for onboarding, migration guides, API docs and more.
  • GitHub PagesGitHub Pages are a simple way to generate web content from source code or even just markdown and deploy that to a domain. This is a simple way to create a nice product, marketing, or API documentation for users with minimal fuss and overhead.
  • GitterGitter is a chat platform that integrates with popular development/OSS tools like GitHub, Jenkins, Bitbucket, Trello, and others that can allow you to foster a real time community around your project, assist others or let the community jump in and help!


In the below screenshot, we can see an example of the project feature in action, tracking issues sourced from the issue tracker.

Closing Thoughts

Hopefully this has helped shine a light on some of the great features and workflows GitHub  offers to support the development of your own OSS project.  It is important to remember that Open Source is an inclusive process by nature, so make sure to be open to feedback and respectful of all contributions. OSS will undoubtedly attract contributors of all skills levels, so make sure to engage openly and without prejudice. Likewise, if there are small bugs or documentation tasks, you can recommend those to newcomers as part of your onboarding documentation. With robust documentation around your project and workflows, developers are more likely to contribute and newcomers can self-onboard, with minimal guidance from you.

Most importantly, developing software should be fun and rewarding.  When the right people get together with the right tools and in the right way, great things can happen. Happy coding!

JavaScript still reigns as king for many developers. In Stack Overflow’s recent survey of 64,000 developers across the globe, JavaScript came out on top for the fifth year in a row as the most common programming language.

But with so many tools, libraries and frameworks in the Javascript ecosystem, how does a developer make sense of it all?

Kenzan’s director of engineering, Owen Buckley, shared his insight into the world of JavaScript during one of Kenzan’s latest tech meetups in Providence, Rhode Island. Owen cover’s some important aspects including language & specification, libraries & frameworks, and development & tooling.

If you didn’t get a chance to hear it live, check out the recording below:

Visit our meetup page to RSVP for Kenzan’s next meetup! 

Continuous delivery has quickly become the choice method for faster, safer and more frequent software deployments. As more and more tools come into play in this arena, developers are looking for new ways to enable this kind of software delivery and maximize its benefits. This guide will show how I created a bare-metal continuous-delivery appliance using Spinnaker, running on a Kubernetes cluster of “mini pcs”.

Why build a bare-metal cluster?

Because it’s fun! Many times when using a cloud platform, much of the magic gets abstracted behind dashboards and APIs. When you set up your own cluster from scratch, it really helps to connect the dots and learn about how the pieces fit together.

The cloud is not cheap. Running Spinnaker in the cloud is quite pricey due to the resource requirements. When we run this on our own hardware, we pay up front around as much as one month in the cloud, but we can run it forever!

Free up resources. There are tools like minkube that allow you to set up your own single node Kubernetes cluster on your laptop. However this ties up resources. It’s very nice being able to use an always running cluster on your network without needing to constantly “spin up and spin down” environments.

Total control. Running a cluster via minikube or GKE is very convenient. However with bare-metal we can tweak our setup to our heart’s content. Want to install an nfs server for persistent volumes on a node? Go for it! Want to experiment with the Ubuntu Kubernetes distribution? Install the iso on a node! There is less magic and more “nitty gritty”. It really helps you understand how things work from the core.

Put your spare compute to work. It’s really satisfying to have your own “on prem” equipment. How many of us have raspberry pis or old desktops/laptops just laying around? Instead of just collecting dust, we can add these nodes to our fleet. We can then run jobs or applications that distribute this load. You can put the cluster behind your router and host web sites or run home automation applications. All without the overhead of the cloud. Plus it looks really cool sitting on your desk!

What is Spinnaker?

http://www.spinnaker.io/ is set of microservices that make it easy to build continuous delivery pipelines. Contributors to the project include Netflix, Google, Microsoft and Kenzan.

The project brings together best practices and patterns for easily deploying immutable infrastructure style software. Deploy targets can be instances or containers running on a multitude of platforms including AWS, GCP, Azure and Kubernetes.

Why Kubernetes?

Spinnaker can be run from any of the above platforms, however due to the nature of the resources needed it can be quite expensive. Kubernetes allows us to set up our own “cloud” on bare metal. We can then use our Spinnaker instance to easily deploy to other cloud platforms or clusters. It’s also pretty neat having a self contained “appliance” running Spinnaker. Hardware prices are constantly falling and it is pretty fun experimenting with software on our own “datacenter”.

Choosing hardware

While I was able to get the cluster turned up with the first version of this guide, The “Stick pcs” proved to be too weak on the memory requirement. I needed nodes that had at least 4gb memory. After some searching I decided on three “nexbox” pcs.

https://www.aliexpress.com/store/product/1Set-Nexbox-T9-Smart-TV-Box-Z8300-1-84GHz-4-Cores-Win-10-Mini-PC-4GB/2130214_32658221265.html

  • 4GB memory
  • Quad core atom processor
  • 64GB SSD
  • Ethernet port

It took a while for the boxes to arrive from Aliexpress but I was excited to get started when they did.

Installing Ubuntu

Unlike the tv-sticks, these boxes came with windows installed. That was not good for the cluster, so I began by trying to install Ubuntu server. Unfortunately since the chipset in these machines was “cherry trail” it had limited Linux compatibility. The nic did not work at all during install.

Thankfully, after some searching I was able to find an Ubuntu image with a modified kernel to support the chipset: http://linuxiumcomau.blogspot.com/search?updated-min=2017-01-01T00:00:00%2B11:00&updated-max=2018-01-01T00:00:00%2B11:00&max-results=2

Burning the image to a thumb drive, I was then able to hold the ESC key and boot from the drive to install.

With Ubuntu installed on the boxes I was now able to install docker on each node.

Installing Kubernetes

apt-get update
apt-get install docker.io

I leveraged the docker-multinode scripts to get Kubernetes installed along with heapster and the dashboard.

https://github.com/kubernetes/kube-deploy

Installation is relativly simple, you run the master.sh script on one node and worker.sh on the remaining two nodes.

Tunneling 8080 into the master node and the dashboard displayed like a charm.

Installing Spinnaker

I then was able to make some minor modifications to the “spinikube” specs and get Spinnaker installed.

https://github.com/kenzanlabs/spinikube

Overall, I’m very happy with how the cluster turned out. It’s great to be able to have a dedicated cluster without wasting resources on vms. I’m looking forward to running more workloads and monitoring performance. The next step will be to experiment with getting persistent volume storage in place with ceph or gluster. It will be great to take advantage of all the storage on the nodes. Stay tuned for part 2 of this guide, where we leverage Spinnaker to do a deploy on the cluster, along with some other advanced functionality.


Chad Moon is a platform engineer at Kenzan, based out of the Denver office. Specialties include crafting continuous delivery pipelines and containerizing all the things. Current work includes integrating Jenkins, Spinnaker and Kubernetes for large enterprise clients.


Have questions about building your own Spinnaker-Kubernetes cluster, or just about Continuous Delivery in general? Comment here or tweet at us: @kenzanmedia.

For more information about Kenzan services, contact info@kenzan.com

 

As more businesses prepare to make a digital transformation, containers have become the choice cloud computing architecture for faster, more portable and reliable deployments. With the growing interest in containerization, the question arises about how containers integrate with existing infrastructure. In this post, we will look at how containerization affects service discovery and present a network routing solution that allows NetflixOSS Eureka to provide unified discovery between both containers and with VM-based services.

Kenzan specializes in cloud technologies with extensive experience in Amazon Web Services (AWS). We have adopted a number of tools from the NetflixOSS stack for use in AWS, such as Zuul, Ribbon, and Eureka. The discovery service feature of Eureka allows us to build dynamically scalable AWS environments without the need to setup fixed routing and load balancing infrastructure.

Docker introduced a new networking layer that changes everything we know about networking in the cloud. This makes discovery and routing with Eureka challenging. Containers have their own IP addresses, belong on a different subnet, and are only routable from the host running the Docker daemon.

We have experimented with tools like flanneld that create virtual networking layers between Docker hosts, allowing for cross-host communication between Docker containers. Flanneld is easy to setup and does as advertised, but requires hosts wishing to network with containers to be running the flannel daemon. All-in-one Docker solutions like Kubernetes do everything from networking containers to orchestrating and managing multiple Docker hosts, but still cannot network with containers from outside the Kubernetes cluster.

What we are looking for is to spin up containers like we do with EC2 instances, have them register with a discovery service, and allow us to send traffic from anywhere in the VPC. Let’s start simple with a cluster of Docker hosts, which is something that EC2 Container Service (ECS) will provide us. We launch a few applications as Docker instances using ECS, but we get containers that can’t talk to each other and can’t be reached from any external service.

As an application in a discovery-based world, you need to tell the discovery service who you are and how others can reach you. This is easy on EC2 instances because the application can provide the host IP address and the port it’s listening on. Containers in Docker are given IP addresses that are only routable to containers running on the same Docker daemon. We can expose internal container ports as host ports, but that port (which port? That = ?)has to be static and non-conflicting with other containers on the same host. e want to be dynamic and not have to remember which ports are used versus which ports are free.

ECS provides an option to expose containers through an Elastic Load Balancer (ELB). Issues with ELB’s include consuming several VPC IP addresses, requiring management of limits on how many ELB’s can be created, and adding additional AWS costs. Our applications now have to remember a series of hostnames representing ELB endpoints for each environment, adding more configuration overhead. ELB’s  have the advantage of security groups, which is something we may expect to lose in a container world. The new networking layer on top of Docker does not play nicely with network based firewalls like security groups in AWS.

The goal is to find a non-conflicting dynamic way for containers deployed to a cluster of Docker hosts to identify themselves to a discovery service and have their identity be reachable. To achieve this, we need a tool that will find other containers on the host and route traffic based on a series of rules. Traefik is that tool. Traefik is a discovery based HTTP reverse proxy and load balancer that can discover Docker containers with minimal configuration, as well as several other means of discovery.

Let’s look at what it takes to get Traefik running with a Docker based backend:

$ docker run -d -p 80:80 -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock traefik –web –docker –docker.domain=docker.localhost

That’s it.  The front end is now listening on port 80. Notice what we did here is we mounted the docker.sock file as a volume launching the container. This gives Traefik API access to the Docker daemon so that it can find other containers.

Alright, now it is time to add some containers for Traefik to find:

$ docker run -d –name nginx -l traefik.port=80 nginx

We added an Nginx container, and added a Label of traefik.port=80. Traefik will use the Docker metadata API exposed through the mounted unix socket to find container labels and bind the listener to port 80 that Nginx is listening on.

The Nginx container can be seen on the Traefik admin page listening on port 8080

Notice how the Rule is Host:nginx.docker.localhost. This is a combination of the container name we provided to Nginx with the –name argument. The docker.localhost part of the domain came from the –docker.domain=docker.localhost we gave to Traefik at startup.

Running a curl to the Docker host with a Host header of nginx.docker.localhost returns the Nginx welcome page.

$ curl -H “Host: nginx.docker.localhost” 192.168.33.10
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
….

Traefik is routing requests with this Host header to our Nginx container. We didn’t tell Traefik to do that, but it did (and that’s alright).

We can use other labels attached to the Nginx container to control how Traefik routes requests to it. Let’s try a path based rule.

$ docker run –name nginx -l traefik.port=80 -l traefik.frontend.rule=PathPrefix:/nginx nginx

Now all requests with the path prefix of /nginx will go to our Nginx container

$ curl 192.168.33.10/nginx/test
<html>
<head><title>404 Not Found</title></head>
….

We of course get a 404 since our nginx doesn’t have a /test resource, but we can see in the Nginx logs that the requests are coming through.

172.17.0.2 – – [10/Jan/2017:22:48:37 +0000] “GET /nginx/test HTTP/1.1” 404 169 “-” “curl/7.49.1” “192.168.33.1”

That was a quick proof-of-concept to show how quickly we got Traefik running as a discovery based routing service with Docker. By putting a Traefik container on every Docker host, we can  dynamically setup routes to our containers running on those hosts. All we need to do now is identify our containers to the discovery service as the host IP address and the port that Traefik is listening on. The applications calling the services have to remember to include the path prefix in the request. Below is an architecture diagram showing Traefik set up on multiple Docker hosts.


Darren Bathgate is a technical architect at Kenzan. Over the course of his 5+ years at Kenzan, Darren has worked extensively with Java, MySQL, PHP, Cassandra, Node.js, oracle, Jenkins, Netflix OSS and Docker.

From the cloud, to the Internet of Things, to big data, companies are embracing digital transformation to help them quickly react to marketplace shifts, consumer demands and new opportunities. The roadmap for digital transformation is different for all companies and often calls for customized software, which can be a significant investment of time, money, attention and resources.

Software development can sometimes be seen as a slow and expensive process, especially at the enterprise level. Managing these factors are fundamental priorities and few people are better equipped to keep things under control than project managers.

Many organizations that are on a path towards digital transformation don’t fully recognize the value PM’s add to software development teams. According to a 2016 report from the Project Management Institute titled “The High Cost of Low Performance”, less than two in five companies surveyed place a high priority on creating cultures that recognize the importance of project management as a driver of better project performance.

That’s not the case at Kenzan, where our project managers play a central role on each development team and are vital for success. We’re not the the only ones that have seen the positive results of project managers. Organizations that invest in project management waste 13 times less money because strategic initiatives are completed more successfully.

But for those who might disagree, we’re here to dispel some myths about Project Managers:

Myth: Project managers are clueless.

Busted: Technical comprehension is as critical to the success of a project as leadership and strategic business management. While a PM doesn’t write code, they spend their days working with those that do. In order for a PM to adapt quickly to changing conditions, assign and re-prioritize tasks, communicate effectively and spot risks, they need an understanding of  relevant tools and technologies that the team uses.    

Myth: Project Managers just schedule meetings and manage calendars.

Busted: Project managers guide the structure, scope, quality and budget of a project while also representing the interests of the product owner. The real value of a project manager is as a leader, liaison and mentor. With a big picture point of view, the project manager balances and guides software development teams while defining requirements and goals to ensure the teams meet expectations — on time and on budget.

Myth: Project managers are just paper pushers.

Busted: Project managers define policies and procedures that enable success. Without the structure of process, a project can easily fall apart. A project manager knows the operational requirements (things like time sheets, budgets and resource allocation) and can navigate communication channels with clients so technical team members can focus on what they do best – developing software.

Myth: A Project manager’s top priority is execution.

Busted: Project managers support the team from start to finish. While the software delivery lifecycle (SDLC) is the guiding framework for development, it doesn’t stand alone in supporting and achieving business objectives. Project management methodologies work in tandem with the SDLC for the initiating, planning, monitoring and delivery of projects, which provides consistency and stability of process through every phase of the SDLC.

Now that we’ve dispelled some of these major misconceptions about project managers, it’s (hopefully) clear how important they really are. Think about all the projects you or your organization have worked on that went over budget, missed a deadline or derailed entirely. It may be too late to go back and right those wrongs, but it’s not too late to consider how future projects — and your company as whole —  will benefit from the guidance of project managers.

For companies that are looking to get or stay competitive, strong project management practices can play a crucial role in driving the business forward through digital transformation. Contact info@kenzan.com to learn more about our project management and other services.


As a certified Scrum Master in Agile methodologies, Jennifer Aczualdez leads Kenzan’s project management and business analysis team . She manages the scope, budget and timelines of projects and acts as a central point of contact for both internal and client teams. She is involved in the full software delivery cycle, from initiation and planning to monitoring, delivering and closing.

Kenzan has created many microservice applications over the years, with many of those running thousands of instances. Our experience is that a lot of organizations want to take advantage of the increased scalability and other benefits that microservices offer, only they don’t know where to start. How do they set up data? How will microservices affect their deployments? What technologies should they use? The task can feel quite daunting! The reality is that it’s much simpler than it seems. To prove this point, Kenzan created an open source microservices project called Million Song Library (MSL).


Not familiar with the benefits of microservices yet? No problem! Check out our blog series on microservices for a deeper dive. In a nutshell, microservices are a set of small services that make up a full application stack. The services are typically broken down by functional area within the business. This brings several core benefits:

  • Targeted scalability where and when needed
  • Decoupling of the functional areas of an application
  • Facilitation of continuous delivery
  • Cleaner code management

This is just a high level overview of the benefits, so be sure to read Microservices for a Macro World if you’d like a more detailed discussion.


So what is Million Song Library? At a basic level, MSL is a microservices-based application that lets users navigate through large sets of music (albums, artists, and songs) while also tracking and rating their favorites. That said, the functionality of the application is actually secondary to the main goal: to show how easy it can be to create microservices and run them both locally and up in the cloud. As you’ll see, with the help of good, solid architectural patterns, it is simple to create and maintain a fully functioning microservices application.

Over the next couple of weeks, you can look forward to a series of blog posts covering the different aspects of MSL. At the end of the series, our hope is that you’ll have a good familiarity with microservices, the technologies used in MSL, and its core architectural patterns. You’ll also be able to run MSL locally as well as within your own AWS environment.

At this point you probably want to know more about the architecture of MSL and what makes it tick. So let’s get to it!

For MSL, it’s best to review the stack from a bottom up approach. The data layer leverages Cassandra NoSQL data stores, and there is a separate data collection client for each functional area. The services are also broken up into functional areas, each one related to managing a library with a million songs. These include things like catalog services, login services, and even ranking services. Each of these services are fully RESTful, and each (ideally) has a very small set of responsibilities. The task of managing access to these services belongs to a proxy layer (Zuul) that handles all the API traffic coming into the application as well as discovery of the correct microservice.

As it stands now, MSL is easily deployed into AWS, but it was intentionally built to be deployed into other environments. Want to drop the routing layer logic, the services layer, and the data tier into another cloud or data center environment? You can do that!

From a front-end perspective, MSL is comprised of a few core technologies. Our goal was to use a technology stack that we find easy to work with and is something common in the marketplace:

  • AngularJS 1.4 – Front end framework
  • ES6 – JavaScript environment
  • Less – CSS preprocessing
  • Gulp – Build management
  • NPM – Front end package manager
  • Webpack – Bundler for modules and dependencies
  • Karma – Test runner for JavaScript
  • ESlint – Style guide linter tool
  • Material Design – Standard design toolkit

The back-end technology was architected with the same simplicity in mind. Given the robustness of the Netflix OSS stack, we decided to stick with Java and gain the benefits of a solid Netflix OSS ecosystem:

  • Java 8 – Server side language
  • Jersey – Web service layer that extends JAX-RS
  • JUnit – Server side unit testing
  • Datastax – Database driver (Cassandra)
  • Netflix OSS – Components for building microservices applications:
    • Eureka – Enables application discovery within the microservices environment
    • Hystrix – Handles circuit breaking within the application
    • Zuul – Routes API calls into the environment (proxy server)
    • Ribbon – Manages software load balancing (client library)
    • Archaius – Offers dynamic properties management
    • Karyon – Provides a base container for all microservices

At Kenzan, we believe that documentation is as important as the code we write. For that reason, the MSL project uses some critical tools to facilitate documentation:

  • Swagger – Framework for generating API documentation alongside code
  • KSS – Specification for generating CSS style guides
  • AsciiDoc – Markup language for generating general user documentation

Currently, you can deploy MSL in several ways, with additional methods on the horizon:

  • Maven – Local builds
  • Docker – Containerized deployments
  • RPM – Manual deployments into AWS
  • Spinnaker – Pipelines for continuous integration (CI) and continuous delivery (CD)

This all probably seems like a lot, but don’t worry! By the end of this blog series, you will be familiar with all these technologies, along with the recommended architectural patterns to use for the microservices. We are excited to release MSL into the open source community and watch it take flight. And, we’re glad you came along on the adventure!


Craig Martin is the Vice President of Engineering at Kenzan. He is based out of Denver, and oversees all engineering activities within Kenzan. Many of his current responsibilities have been focused on architecting, and leading projects to create, highly scalable cloud-based microservice applications.

In our first post of this series, we looked at how microservices help you build applications with an eye towards the future. And in our second post, we took a deep dive into how microservices work. But what if you currently have a monolithic application? How do you know if it’s time to move to microservices? And if so, how do you get there?

As we mentioned before, knowing why to build a microservice is only half the battle. Knowing when (or when not) to build one can be a trickier question. There’s no one size fits all. That said, there are a few questions you can ask to help decide if you’re ready for microservices. We’ll explore them in this final post of our series.

Reading the Signs

Is your organization or team experiencing exceptional growth in both employee base and technical stack?
As a codebase grows to a certain size, more people are needed to maintain it. But anyone who has worked on a single codebase with ten other engineers knows the struggle of dealing with merge conflicts during code reviews. If a team is losing valuable time trying to resolve conflicts, it might be time for microservices. Microservices help distribute development teams more efficiently, as complex backend systems can be built faster by dividing workloads into multiple small teams, with each team responsible for a given set of microservices.

Is your application down often?
Every time you deploy an application, it breaks. Or the backend system is seemingly down all the time for maintenance. If these scenarios sound all too familiar, it might be time to shift from a monolithic architecture to microservices.

Are you having difficulty scaling hardware to optimize application performance?
Larger and more complex applications require additional hardware to perform well. Some parts of an application might put more demand on the hardware than others, and the whole application can face degraded performance due to a single poorly-performing feature. Microservices allow individual units of business logic to be assigned their own dedicated hardware, and they can be scaled independently of other services. A slower-performing process can be isolated and capped to a fixed about of CPU, memory, disk, and network bandwidth, which means it can’t steal resources from other features.

Choosing the Moment

Once you’ve decided to build a microservice, the next questions are when to build it and when to switch over. That’s a hard question to answer for any organization. The monolith architecture works for small applications and small engineering teams. But when does the monolith stop working?

It turns out growth is often the death of the monolith. Organizations in the process of undergoing exceptional growth, in both the employee base and in technology stack, can achieve that growth by switching to microservices. If the engineering team is losing valuable time resolving merge conflicts, then it might be time for microservices. If the growing backend system is down all the time for maintenance, then microservices might be the answer. The answer of when to build a microservice is when growth is expected.

Hardware provides a great example of this, as it can only be scaled horizontally so much. Maybe those high-CPU and high-memory servers are too expensive and too underutilized during non-peak hours. The cloud is becoming more appealing with the ability to provision hardware as needed, and smaller servers with lightweight workloads become easier to bring up and tear down in response to changing demands. Microservices have lower hardware requirements, and they can start up quickly to meet peak demands.

Making the Move

Monoliths present a number of challenges that you’ll need to tackle in the move to microservices. A monolith has a tendency toward tight coupling of components, as well as stateful behavior. The application was never intended to live as separate and isolated components working in concert to make a system. The monolith may also be rooted to the infrastructure it lives on, depending on system resources such as a local filesystem and network. What’s more, you need to keep everything up and running during the transition. So how do you move to microservices? Let’s break it down.

Step 1: Determine Your Domains

Your first task is to look at all the features of the application and organize them into logical business units, or domains. For example, your application may have major features related to login/logout, user data, and session management. These features can be logically organized into a single business domain called Authentication. Repeat this step for all of the other domains in your application.

Step 2: Prepare the Monolith

Next, get your application ready for the big break up. To do this, decouple components within the application along the lines of the business domains you came up with. This will result in a number of edge and middle modules within the application, each dedicated to a particular domain, like our Authentication example. You’ll also need to move stateful in-memory stores into shared datastores. Finally, put a router in front of the monolith to smooth the rollout of microservices.

Step 3: Work From the Bottom Up

As you begin breaking out microservices from the monolith, it’s always best to start at the bottom and work your way up. First, create a separate database to store data for the domains you are moving to microservices. Next, break out the data access modules that access this data into middle microservices. Finally, break out edge modules that consume this data into edge microservices. When everything’s ready to go, use the router to toggle redirection to the new edge services. Keep repeating this same process for each domain until all of your modules are broken out and the monolith is no more.

In the diagram below, the Login, Users, and Sessions modules were decomposed into modules within the monolith, and then broken out as separate microservices.

breaking-up-the-monolith

One challenge you’ll encounter during the transition is the need to support both the microservices and the monolith side by side. That also means duplicating work, as you’ll often have to make changes or fixes in both places. The good news is that organizing your monolith similarly to your microservices makes it easier to copy-and-paste code between them.

Conclusion

If your application and organization are both set for growth, microservices can help you meet the challenge of building modular, scalable, highly-available solutions. When you’re ready, you need to organize your application’s features into domain-specific modules, then break them out one by one.

Just remember that transitioning from a monolith won’t happen in a single overnight deployment. You need to strategize as you cherry-pick domains out of the monolith. The key is to complete the transition to microservices and not leave your application in limbo. If you’d like to learn more about how to make the move to microservices, or if you need some help, feel free to ask us. That’s what we’re here for.