A Modern Day Front-End Development Stack

Tech Skills

The front-end development landscape has changed dramatically over the past few years. The release of the latest EMCAScript specification in 2015 brought a new sense of maturity to JavaScript. It also introduced two key features: modules and classes. This has led to a proliferation of tools, libraries, and frameworks that have accelerated the development of…

The front-end development landscape has changed dramatically over the past few years. The release of the latest EMCAScript specification in 2015 brought a new sense of maturity to JavaScript. It also introduced two key features: modules and classes. This has led to a proliferation of tools, libraries, and frameworks that have accelerated the development of front-end applications. But with so many great choices, how do you decide what’s best for your application and team?

In Kenzan’s latest series for Linux.com, we set out to answer that question. We examine in depth how a stack focusing on Yarn for package management, webpack for application bundling, and TypeScript for writing application source code can give developers and organizations the building blocks to write more maintainable and better optimized applications.

For the next few weeks, Linux.com will be rolling out the five-part series on their blog. The first part, posted today, gives you a short history lesson on JavaScript and delves into the core philosophies of front-end development. Throughout the rest of the series, you’ll learn more about the main components of our modern day front-end development stack. We’ll finish up with a case study of a working hello-world application that shows all the elements of the stack in action. By the end, we hope you’ll be able to take what you’ve learned and apply it to your own infrastructure.

As with everything at Kenzan, this project wouldn’t have come to fruition without our incredibly talented team:

Development

  • Marie Schmidt
  • Chris Pruyne
  • Chris Joslyn
  • Paul Barry
  • Carlos Hernandez

Tech Writers

  • Ryan Daugherty
  • Mark Anthony

Leadership

  • Owen Buckley
  • Nick Wester

Crossword Puzzles, Kubernetes and CI/CD

Tech Skills, ,

Read the first part of our series on Linux.com: Set Up a CI/CD Pipeline with Kubernetes. What’s an eight letter word that starts with K and is the name of a powerful container orchestration tool? If you just shouted out Kubernetes (or if you were curious about the answer), then you’re ready to tackle the…

Read the first part of our series on Linux.com: Set Up a CI/CD Pipeline with Kubernetes.

What’s an eight letter word that starts with K and is the name of a powerful container orchestration tool?

If you just shouted out Kubernetes (or if you were curious about the answer), then you’re ready to tackle the Kenzan Kr8sswordz Puzzle.

The Kr8sswords Puzzle is a crossword puzzle app that runs in pods on Kubernetes. It takes advantage of a full CI/CD pipeline in Jenkins that also runs in a pod, automating the entire build-and-deploy process. And it includes a bunch of other cool components that we don’t want to spoil for you because it’s much more fun to boot up the app yourself and see the different pieces in action. (Okay, here’s a hint: React frontend, Etcd caching, and MongoDB persistence.)

Kenzan designed the app as a four part blog series that is launching on Linux.com today. Part 1 kicks things off by walking you through setting up a local Kubernetes development environment. Each weekly blog post will add components to show the Kr8sswordz Puzzle in action, how to spin up many pod instances for a load test, and how you can easily incorporate Kubernetes into a CI/CD pipeline.

The Kr8sswords Puzzle app was a lot of fun to pull together. We hope you have just as much fun  building it out as the blog series goes on.

Kenzan is a family, and we would like to give a shout out to the many folks that made the app and blog series possible:

Developers
Chad Moon, Platform Engineer
Sean Korten, Platform Lead Engineer
Evan Yeager, Technical Architect
Geiser Menoia, Back End Developer
Justin Tomlinson, Director of Engineering
Marie Schmidt, Front End Developer
Scott Pullano, Director of Engineering
Rona Kilmer, Director of Engineering

Tech Writers
Ryan Daugherty
Mark Anthony

Testers
Oscar Santamaria
Matthew Gardner
Ben Sawyer
PJ Pannoni
Robin Morrison
Matt Poirier

Leadership
Elana Krasner
Nick Wester
Craig Martin

5 Ways to Improve Employee Development

Tech Culture

As the tech skills gap grows and more IT roles remain unfilled, the search for top tech talent has never been more competitive. Companies are upping the ante with perks like yoga classes, nap pods, free meals, unlimited vacation days and more in order to attract and retain employees. But according to Stack Overflow’s annual…

As the tech skills gap grows and more IT roles remain unfilled, the search for top tech talent has never been more competitive. Companies are upping the ante with perks like yoga classes, nap pods, free meals, unlimited vacation days and more in order to attract and retain employees. But according to Stack Overflow’s annual developer survey, there is one factor that potential employees in the tech sector value more than anything — even more than pay and compensation — and that’s opportunities for professional development and continued learning.

developer learning and education
Source: Stack Overflow

The response from more than 22,000 developers that Stack Overflow asked shouldn’t come as a surprise to anyone who has ever worked with developers and programmers. With 90% of the survey’s respondents also stating that they are at least partially self-taught, it’s clear that the industry is a community of learners. Many of our leaders have been self taught and driven by pure passion for technology — Steve Jobs, Bill Gates and Mark Zuckerberg all famously left prestigious universities in their pursuit of technological innovations.

Investing in continued learning and education isn’t just a good way to attract new talent, but can also be used as a tool to keep current employees engaged, productive and feeling empowered — making them less likely to take their skills elsewhere. The impact of disengaged employees can be staggering, affecting everything from staff retention to morale to a company’s bottom line. Gallup estimates that a disengaged employee costs an organization about 34% of their salary per year, but re-engaging these employees offers an opportunity to improve performance, profitability and customer experience.

At Kenzan, continued development and learning is not only a big part of our culture, but part of our business. As a software engineering and digital consulting firm that is tasked with helping clients make the most out of technology, it’s crucial that our teams stay current on the tools and tech that are leading the industry.

We are continuously improving how we enable our employees to skill up. Here’s a few things that we’ve learned that help ensure a well-supported learning community within an organization:

Develop a continuous feedback loop of learning.
Managers who recognize progress, tangible goals, and teach-back opportunities will have stronger relationships with their reports and a clear roadmap for their team.

Provide a structure for training requests.
A clear process allows all staff to understand what is considered a reasonable request and that requesting training is not considered self-serving. Providing structure opens the door for new or shy team members to take opportunities because they know it’s appropriate and valued.

Share a clear strategic vision with your team.
Most employees want to make sure their training aligns with the organization’s future. Setting the goal posts gives employees the agency to plan their path forward.

Implement a way to track skills and progress.
Tracking progress means you value the effort that your team is putting in. Unless the team prioritizes training the way you approach other tasks, it will always be at the bottom of the list. At Kenzan, we’re developing a Badge Board to assess team members’ skills and growth based on the organization’s goals and technical vision.

Encourage developers to manage up.
Managers have a lot on their plate, and in the tech world, are usually more than just managers. It’s often up to the developer to make sure that their learning needs are part of the regular conversation. For employees who aren’t quite comfortable approaching their manager about training opportunities, we’ve got some tricks for them too:

Set up a clear need that keeps the core of your request in focus. Make sure you have an answer to questions like: “What is your learning goal? How will growing in this area make you a better employee?”

Respect your supervisor’s time. Follow set procedures and follow up with your needs. Even the most thoughtful, learning-minded supervisor is going to forget about the registration deadline. It doesn’t mean they don’t want you to go, but just that they probably had a few fires to fight this week.

Be Flexible. Sometimes your request will be denied, perhaps due to timing or cost. If this is the case, there is likely another resource that offers growth in this skill, but is less expensive or time bound. If the no is because your supervisor doesn’t see a connection to your learning goals or a value to the organization, make sure you’re continuing the conversation to align your goals with your company’s strategic vision.


Whether your goals are in recruiting, employee retention or tied directly into business drivers, the value that learning and development brings to an organization is clear. Far too often professional development is overlooked, and far too many talented employees leave a company because their success is stymied by lack of support.


Kate Pisano is the Training and Development Manager at Kenzan, based out of our Rhode Island office. She is responsible for managing Kenzan’s internship program and supporting Kenzanites through continued learning opportunities. With a background in AmeriCorps and program development, Kate’s always excited to help team members grow and advance.

Image is Everything: Continuous Delivery with Spinnaker and Kubernetes

Tech Skills, , , , ,

Download our guide to continuous delivery with Spinnaker and Kubernetes. The path to digital transformation is one that many companies, regardless of industry, are taking in order to bring products and services to market faster. As part of this transformation from traditional enterprise to modern software company, the role of IT is now a core…

Download our guide to continuous delivery with Spinnaker and Kubernetes.


The path to digital transformation is one that many companies, regardless of industry, are taking in order to bring products and services to market faster. As part of this transformation from traditional enterprise to modern software company, the role of IT is now a core part of business strategy with the responsibility of delivering software to users faster, safer and more efficiently.

Over the past few years, technology has evolved to enable this kind of delivery. The practice of continuous delivery has emerged as a way to enhance the ability of technical teams to deploy software to the end user faster, safer and more efficiently. Kenzan has paired two tools, Netflix’s Spinnaker as the preferred continuous delivery framework and Google’s Kubernetes as the preferred container management platform, in order to provide a continuous delivery solution that can support any enterprise that is pursuing digital transformation. 

Download our guide to continuous delivery

Managing an Open Source Project in GitHub

Tech Skills,

Since it was created in 2007, GitHub has not only become the primary destination for open source development and distribution, but also one of the largest collaborative platforms on the internet. GitHub’s open source practices have established the platform as one to the most important tools for developers and gives coders of all levels a…

Since it was created in 2007, GitHub has not only become the primary destination for open source development and distribution, but also one of the largest collaborative platforms on the internet. GitHub’s open source practices have established the platform as one to the most important tools for developers and gives coders of all levels a place to learn new skills, contribute to a vast number of projects and work alongside other programmers. Kenzan is a company that leverages the benefits of open source software by opening many of our  projects to the community and contributing to others in order to support the technology landscape at large.

More than just a GUI on top of Git, the GitHub platform  offers many powerful tools  and features that can help maintainers of OSS projects provide a more useful, transparent, and open experience for their project and their community. In this post, we will walk through some of the best practices that can help make an OSS project successful and inspire community participation and contribution.

Getting Organized

In order to ensure a consistent path to PR integrity, acceptance and delivery, it is important to make sure team permissions are setup appropriately.  At Kenzan, we identify maintainer and contributor teams for our projects.  The primary goal is to establish a protected branch (typically master) for a given repo and those who should have write access to it. Protected branches allow you to give read access to everyone else in the organization or an internal contributing team, but only permit a small team to actually be able to manage the main line of development. Roles can be defined as:

  • Maintainers – Act as the stewards of the project with sole access to protected branches like master
  • Contributors – Community team within the organization that are allowed to read and push branches within the repo

Below is an example of a sample configuration that protects the master branch with options like:

  1. Ensuring only the maintaining team can push to master
  2. Requiring PR approvers
  3. Requiring PRs pass a Continuous Integration build

Pro-Tip!
Always use teams for delegating repo access, for both maintainers and contributors. This will ensure easy addition and removal of members, as well as a simple and streamlined permissions strategy for your organization and repositories.

Welcoming Contributions

Great software is developed by many people contributing more than just code like testing, refactoring, documentation, and more.  However, each project is generally run in a particular way, and so clearly communicating these specific processes and workflows can help make a big difference in the growth and success of your project.  Guiding developers and contributors through the following recommendation will  help ensure understanding and onboarding from the start.

  • README.md – First impressions are everything, so a README that includes important information like an overview (your “elevator pitch”), installation/setup, usage and example, license, etc., will ensure first timers to your project know exactly what it is, what it does, and if it is right for them.
  • CONTRIBUTING.md – by including a CONTRIBUTING.md in your project’s root, you can define the Contributing Guidelines for your project. With this, GitHub can prompt all users who open an issue or PR to your project with a link to read it.
  • LICENSE – Having a LICENSE file is important to disclose to users how your project can be consumed, used, and distributed. In addition, be aware of the license expectations of any dependencies your project has.
  • Webhooks – Webhooks allow “push notification” integration with 3rd party services such as a Continuous Integration server like Jenkins.  Anything that can be automated in your project (linting, static analysis, style guide, tests, etc) should be scripted and run against every PR to provide fast turnaround and feedback to contributors (i.e. pass or fail).  Kenzan uses GitHub PR Builder.
  • Issues / PRs – GitHub has done a lot to improve the features around these workflows, in particular Pull Request Templates, fields for reviewers, assignees and labels, reactions and many others.  These are great features for identifying/establishing ownership and status.

Below is an example of a PR making use of some of the available PR fields like

  1. Contributing guidelines (the yellow banner)
  2. PR templates (the content inside the textarea)
  3. Reviewers / Assignees

Pro-Tip!
Always create an issue for any non-trivial change. This promotes discussion of the feature to happen independently of the code review itself.

Bonus!
If you link to the issue in the PR, using #XX (where XX is the issue number) GitHub will close the issue when the PR is merged.

Steering the Project

As a project grows and matures, planning out goals and improving documentation will naturally require some additional organization.  No worries, GitHub has you covered!

  • Project Management – Leverage features like milestones and projects to group issues into related buckets of work.  At Kenzan, we see projects as sprints (in agile terms with one or more projects making up a milestone. This allows a project’s mantainer to communicate their roadmap clearly and can help direct contributors to where help is most immediately needed.  It also helps establish feature / bug delivery expectations.
  • Release Management – Whenever a Git tag is pushed to your repository, GitHub will create a link to a page for that tag, thus enabling  the easy creation of release notes for every tag.  This is useful for tracking progress over time and to keep a record of features and bugs completed.
  • Wiki – The wiki is a great place to organize and manage supplemental information for consumers and developers. Rather than pack everything into the README, the wiki can be used to establish separate documentation for onboarding, migration guides, API docs and more.
  • GitHub PagesGitHub Pages are a simple way to generate web content from source code or even just markdown and deploy that to a domain. This is a simple way to create a nice product, marketing, or API documentation for users with minimal fuss and overhead.
  • GitterGitter is a chat platform that integrates with popular development/OSS tools like GitHub, Jenkins, Bitbucket, Trello, and others that can allow you to foster a real time community around your project, assist others or let the community jump in and help!


In the below screenshot, we can see an example of the project feature in action, tracking issues sourced from the issue tracker.

Closing Thoughts

Hopefully this has helped shine a light on some of the great features and workflows GitHub  offers to support the development of your own OSS project.  It is important to remember that Open Source is an inclusive process by nature, so make sure to be open to feedback and respectful of all contributions. OSS will undoubtedly attract contributors of all skills levels, so make sure to engage openly and without prejudice. Likewise, if there are small bugs or documentation tasks, you can recommend those to newcomers as part of your onboarding documentation. With robust documentation around your project and workflows, developers are more likely to contribute and newcomers can self-onboard, with minimal guidance from you.

Most importantly, developing software should be fun and rewarding.  When the right people get together with the right tools and in the right way, great things can happen. Happy coding!

JavaScript in 2017

Tech Skills,

JavaScript still reigns as king for many developers. In Stack Overflow’s recent survey of 64,000 developers across the globe, JavaScript came out on top for the fifth year in a row as the most common programming language. But with so many tools, libraries and frameworks in the Javascript ecosystem, how does a developer make sense…

JavaScript still reigns as king for many developers. In Stack Overflow’s recent survey of 64,000 developers across the globe, JavaScript came out on top for the fifth year in a row as the most common programming language.

But with so many tools, libraries and frameworks in the Javascript ecosystem, how does a developer make sense of it all?

Kenzan’s director of engineering, Owen Buckley, shared his insight into the world of JavaScript during one of Kenzan’s latest tech meetups in Providence, Rhode Island. Owen cover’s some important aspects including language & specification, libraries & frameworks, and development & tooling.

If you didn’t get a chance to hear it live, check out the recording below:

Visit our meetup page to RSVP for Kenzan’s next meetup! 

The Digital Skills Gap: Rethinking Tech Talent

Tech Culture

For companies looking to grow their digital footprint, an increased investment in new tools and technology is a no-brainer. This may also mean expanding technical teams in order to support the needs of an organization. That said, finding and retaining top tech talent can be a major concern for businesses. Despite thousands of new jobs…

For companies looking to grow their digital footprint, an increased investment in new tools and technology is a no-brainer. This may also mean expanding technical teams in order to support the needs of an organization. That said, finding and retaining top tech talent can be a major concern for businesses.

Despite thousands of new jobs opening up in software development, engineering and architecture, companies are hitting a hiring wall, unable to fill the roles they need to take their digital game to the next level. Failure to meet hiring needs has managers and CTOs alike nervous about meeting business goals. 83% of hiring managers cite the inability to fill roles as having a negative effect on revenue, market expansion, product development and employee turnover.

Likewise, job seekers are struggling to meet the criteria set by hiring companies. With a laundry list of requirements, many applicants simply don’t check off all the technical boxes in a job description.

What’s standing between companies and job seekers?
On February 23, Kenzan and Dev Bootcamp hosted an event, “Rethinking Tech Talent” to address that question. We brought together speakers from our own organizations, as well as from Uncubed and Andela, to discuss the current tech recruiting climate and actionable solutions to close the skills gap.

digital skills gap
“Rethinking Tech Talent on February 23 at Dev Bootcamp. Panelists included: Uncubed, Andela, Dev Bootcamp and Kenzan

We wanted to share some key takeaways  that came out of that conversation.

Alternative Education
One reason for the skills gap: Education just can’t keep up with technology. The tools and skills needed to develop software are evolving quicker than most colleges and universities can teach them. By the time a course is complete in one technology, another has emerged.

Instead of turning to higher education, job seekers are now starting to approach learning differently. Alternative training, like the immersive coding courses that Dev Bootcamp offers, are becoming an increasingly popular choice among those wishing to get into technology. In 2016, there were almost 18,000 graduates from coding bootcamps across the US and Canada, with that number to likely increase in 2017.

Training and Mentorship
It’s not just jobseekers that are seeking more education. According to Stack Overflow, 70% of working developers say that learning new technology is a priority. Companies looking to retain top tech talent would do well to look at continued learning opportunities for their current workforce and to invest in programs that help employees skill up. By introducing more employer-sponsored education, companies will not only be able to keep workers happy, but will also be able to provide less-experienced developers with on-the-job training.

Rethinking Recruiting
Our panel was lucky to be joined by Andela, an organization that is helping companies look beyond the usual recruiting sources in order to tap into a market with plenty of tech talent: the African continent. Rather than focusing on education, the organization vets developers based on skills, putting applicants through a rigorous assessment before presenting them to hiring companies.

While organizations like Andela are getting more attention, many companies are still hindered by their limited definition of what it means to be highly-qualified, looking solely at candidates from specific colleges or with experience at a well-known brand. Even as the number of people gaining skills from coding bootcamps and similar technical schools increases, more than half of employers still say that a computer science degree is the most important qualification. Instead of focusing on an applicant’s education, companies could benefit more by shaking off that narrow criteria in favor of a more holistic, inclusive hiring policy.  

More than technical skills
Gone is the image of the hoodie-wearing developer, secluded behind his computer, headphones blaring, locked into a coding marathon. In 2017, developers work on cross-functional teams, connect with clients, and give demos and presentations in public venues. Collaboration and communication are among crucial soft skills developers need to possess.

Bring hiring companies and education together
Despite the growing popularity of alternative education and a change in recruiting policies, the biggest change can come from companies and educational sources working together.

Uncubed is an organization helping to facilitate that kind of dialogue and also, as it turns out, was a panelist at our event. As a video-first jobs platform, they know all too well the challenges both companies and job seekers face. Beyond their recruiting tools, Uncubed addresses the tech skills gap by bringing together educators and companies to develop more effective education that meets the needs of hiring companies and better prepares students for a career in the digital economy.

Creating a Spinnaker Appliance with Kubernetes

Tech Skills

Continuous delivery has quickly become the choice method for faster, safer and more frequent software deployments. As more and more tools come into play in this arena, developers are looking for new ways to enable this kind of software delivery and maximize its benefits. This guide will show how I created a bare-metal continuous-delivery appliance…

Continuous delivery has quickly become the choice method for faster, safer and more frequent software deployments. As more and more tools come into play in this arena, developers are looking for new ways to enable this kind of software delivery and maximize its benefits. This guide will show how I created a bare-metal continuous-delivery appliance using Spinnaker, running on a Kubernetes cluster of “mini pcs”.

Why build a bare-metal cluster?

Because it’s fun! Many times when using a cloud platform, much of the magic gets abstracted behind dashboards and APIs. When you set up your own cluster from scratch, it really helps to connect the dots and learn about how the pieces fit together.

The cloud is not cheap. Running Spinnaker in the cloud is quite pricey due to the resource requirements. When we run this on our own hardware, we pay up front around as much as one month in the cloud, but we can run it forever!

Free up resources. There are tools like minkube that allow you to set up your own single node Kubernetes cluster on your laptop. However this ties up resources. It’s very nice being able to use an always running cluster on your network without needing to constantly “spin up and spin down” environments.

Total control. Running a cluster via minikube or GKE is very convenient. However with bare-metal we can tweak our setup to our heart’s content. Want to install an nfs server for persistent volumes on a node? Go for it! Want to experiment with the Ubuntu Kubernetes distribution? Install the iso on a node! There is less magic and more “nitty gritty”. It really helps you understand how things work from the core.

Put your spare compute to work. It’s really satisfying to have your own “on prem” equipment. How many of us have raspberry pis or old desktops/laptops just laying around? Instead of just collecting dust, we can add these nodes to our fleet. We can then run jobs or applications that distribute this load. You can put the cluster behind your router and host web sites or run home automation applications. All without the overhead of the cloud. Plus it looks really cool sitting on your desk!

What is Spinnaker?

http://www.spinnaker.io/ is set of microservices that make it easy to build continuous delivery pipelines. Contributors to the project include Netflix, Google, Microsoft and Kenzan.

The project brings together best practices and patterns for easily deploying immutable infrastructure style software. Deploy targets can be instances or containers running on a multitude of platforms including AWS, GCP, Azure and Kubernetes.

Why Kubernetes?

Spinnaker can be run from any of the above platforms, however due to the nature of the resources needed it can be quite expensive. Kubernetes allows us to set up our own “cloud” on bare metal. We can then use our Spinnaker instance to easily deploy to other cloud platforms or clusters. It’s also pretty neat having a self contained “appliance” running Spinnaker. Hardware prices are constantly falling and it is pretty fun experimenting with software on our own “datacenter”.

Choosing hardware

While I was able to get the cluster turned up with the first version of this guide, The “Stick pcs” proved to be too weak on the memory requirement. I needed nodes that had at least 4gb memory. After some searching I decided on three “nexbox” pcs.

https://www.aliexpress.com/store/product/1Set-Nexbox-T9-Smart-TV-Box-Z8300-1-84GHz-4-Cores-Win-10-Mini-PC-4GB/2130214_32658221265.html

  • 4GB memory
  • Quad core atom processor
  • 64GB SSD
  • Ethernet port

It took a while for the boxes to arrive from Aliexpress but I was excited to get started when they did.

Installing Ubuntu

Unlike the tv-sticks, these boxes came with windows installed. That was not good for the cluster, so I began by trying to install Ubuntu server. Unfortunately since the chipset in these machines was “cherry trail” it had limited Linux compatibility. The nic did not work at all during install.

Thankfully, after some searching I was able to find an Ubuntu image with a modified kernel to support the chipset: http://linuxiumcomau.blogspot.com/search?updated-min=2017-01-01T00:00:00%2B11:00&updated-max=2018-01-01T00:00:00%2B11:00&max-results=2

Burning the image to a thumb drive, I was then able to hold the ESC key and boot from the drive to install.

With Ubuntu installed on the boxes I was now able to install docker on each node.

Installing Kubernetes

apt-get update
apt-get install docker.io

I leveraged the docker-multinode scripts to get Kubernetes installed along with heapster and the dashboard.

https://github.com/kubernetes/kube-deploy

Installation is relativly simple, you run the master.sh script on one node and worker.sh on the remaining two nodes.

Tunneling 8080 into the master node and the dashboard displayed like a charm.

Installing Spinnaker

I then was able to make some minor modifications to the “spinikube” specs and get Spinnaker installed.

https://github.com/kenzanlabs/spinikube

Overall, I’m very happy with how the cluster turned out. It’s great to be able to have a dedicated cluster without wasting resources on vms. I’m looking forward to running more workloads and monitoring performance. The next step will be to experiment with getting persistent volume storage in place with ceph or gluster. It will be great to take advantage of all the storage on the nodes. Stay tuned for part 2 of this guide, where we leverage Spinnaker to do a deploy on the cluster, along with some other advanced functionality.


Chad Moon is a platform engineer at Kenzan, based out of the Denver office. Specialties include crafting continuous delivery pipelines and containerizing all the things. Current work includes integrating Jenkins, Spinnaker and Kubernetes for large enterprise clients.


Have questions about building your own Spinnaker-Kubernetes cluster, or just about Continuous Delivery in general? Comment here or tweet at us: @kenzanmedia.

For more information about Kenzan services, contact info@kenzan.com

 

Hacking Docker: Discovering Containers

Tech Skills

As more businesses prepare to make a digital transformation, containers have become the choice cloud computing architecture for faster, more portable and reliable deployments. With the growing interest in containerization, the question arises about how containers integrate with existing infrastructure. In this post, we will look at how containerization affects service discovery and present a…

As more businesses prepare to make a digital transformation, containers have become the choice cloud computing architecture for faster, more portable and reliable deployments. With the growing interest in containerization, the question arises about how containers integrate with existing infrastructure. In this post, we will look at how containerization affects service discovery and present a network routing solution that allows NetflixOSS Eureka to provide unified discovery between both containers and with VM-based services.

Kenzan specializes in cloud technologies with extensive experience in Amazon Web Services (AWS). We have adopted a number of tools from the NetflixOSS stack for use in AWS, such as Zuul, Ribbon, and Eureka. The discovery service feature of Eureka allows us to build dynamically scalable AWS environments without the need to setup fixed routing and load balancing infrastructure.

Docker introduced a new networking layer that changes everything we know about networking in the cloud. This makes discovery and routing with Eureka challenging. Containers have their own IP addresses, belong on a different subnet, and are only routable from the host running the Docker daemon.

We have experimented with tools like flanneld that create virtual networking layers between Docker hosts, allowing for cross-host communication between Docker containers. Flanneld is easy to setup and does as advertised, but requires hosts wishing to network with containers to be running the flannel daemon. All-in-one Docker solutions like Kubernetes do everything from networking containers to orchestrating and managing multiple Docker hosts, but still cannot network with containers from outside the Kubernetes cluster.

What we are looking for is to spin up containers like we do with EC2 instances, have them register with a discovery service, and allow us to send traffic from anywhere in the VPC. Let’s start simple with a cluster of Docker hosts, which is something that EC2 Container Service (ECS) will provide us. We launch a few applications as Docker instances using ECS, but we get containers that can’t talk to each other and can’t be reached from any external service.

As an application in a discovery-based world, you need to tell the discovery service who you are and how others can reach you. This is easy on EC2 instances because the application can provide the host IP address and the port it’s listening on. Containers in Docker are given IP addresses that are only routable to containers running on the same Docker daemon. We can expose internal container ports as host ports, but that port (which port? That = ?)has to be static and non-conflicting with other containers on the same host. e want to be dynamic and not have to remember which ports are used versus which ports are free.

ECS provides an option to expose containers through an Elastic Load Balancer (ELB). Issues with ELB’s include consuming several VPC IP addresses, requiring management of limits on how many ELB’s can be created, and adding additional AWS costs. Our applications now have to remember a series of hostnames representing ELB endpoints for each environment, adding more configuration overhead. ELB’s  have the advantage of security groups, which is something we may expect to lose in a container world. The new networking layer on top of Docker does not play nicely with network based firewalls like security groups in AWS.

The goal is to find a non-conflicting dynamic way for containers deployed to a cluster of Docker hosts to identify themselves to a discovery service and have their identity be reachable. To achieve this, we need a tool that will find other containers on the host and route traffic based on a series of rules. Traefik is that tool. Traefik is a discovery based HTTP reverse proxy and load balancer that can discover Docker containers with minimal configuration, as well as several other means of discovery.

Let’s look at what it takes to get Traefik running with a Docker based backend:

$ docker run -d -p 80:80 -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock traefik –web –docker –docker.domain=docker.localhost

That’s it.  The front end is now listening on port 80. Notice what we did here is we mounted the docker.sock file as a volume launching the container. This gives Traefik API access to the Docker daemon so that it can find other containers.

Alright, now it is time to add some containers for Traefik to find:

$ docker run -d –name nginx -l traefik.port=80 nginx

We added an Nginx container, and added a Label of traefik.port=80. Traefik will use the Docker metadata API exposed through the mounted unix socket to find container labels and bind the listener to port 80 that Nginx is listening on.

The Nginx container can be seen on the Traefik admin page listening on port 8080

Notice how the Rule is Host:nginx.docker.localhost. This is a combination of the container name we provided to Nginx with the –name argument. The docker.localhost part of the domain came from the –docker.domain=docker.localhost we gave to Traefik at startup.

Running a curl to the Docker host with a Host header of nginx.docker.localhost returns the Nginx welcome page.

$ curl -H “Host: nginx.docker.localhost” 192.168.33.10
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
….

Traefik is routing requests with this Host header to our Nginx container. We didn’t tell Traefik to do that, but it did (and that’s alright).

We can use other labels attached to the Nginx container to control how Traefik routes requests to it. Let’s try a path based rule.

$ docker run –name nginx -l traefik.port=80 -l traefik.frontend.rule=PathPrefix:/nginx nginx

Now all requests with the path prefix of /nginx will go to our Nginx container

$ curl 192.168.33.10/nginx/test
<html>
<head><title>404 Not Found</title></head>
….

We of course get a 404 since our nginx doesn’t have a /test resource, but we can see in the Nginx logs that the requests are coming through.

172.17.0.2 – – [10/Jan/2017:22:48:37 +0000] “GET /nginx/test HTTP/1.1” 404 169 “-” “curl/7.49.1” “192.168.33.1”

That was a quick proof-of-concept to show how quickly we got Traefik running as a discovery based routing service with Docker. By putting a Traefik container on every Docker host, we can  dynamically setup routes to our containers running on those hosts. All we need to do now is identify our containers to the discovery service as the host IP address and the port that Traefik is listening on. The applications calling the services have to remember to include the path prefix in the request. Below is an architecture diagram showing Traefik set up on multiple Docker hosts.


Darren Bathgate is a technical architect at Kenzan. Over the course of his 5+ years at Kenzan, Darren has worked extensively with Java, MySQL, PHP, Cassandra, Node.js, oracle, Jenkins, Netflix OSS and Docker.

Closing the Tech Skills Gap

Tech Culture

As more and more business are going digital, the question many companies are asking themselves isn’t how they’ll make this transformation, but who will help them do it. That question is becoming increasingly more difficult to answer. While technology is evolving with lightening speed, the demand for highly-skilled employees who will support this work is…

As more and more business are going digital, the question many companies are asking themselves isn’t how they’ll make this transformation, but who will help them do it.

That question is becoming increasingly more difficult to answer. While technology is evolving with lightening speed, the demand for highly-skilled employees who will support this work is far outpacing the supply.

Kenzan and General Assembly are bringing together Denver-area technology companies and educational resources to address the tech skills gap in Colorado. On January 19, at Kenzan’s office in downtown Denver, we’ll discuss actionable solutions to the challenges both job seekers and hiring companies are facing. Our panel of industry experts from Galvanize, Skillful, Istonish and GoSpotCheck will talk about:

  • The challenges tech companies face when looking for tech talent;
  • Why you shouldn’t disregard a candidate that doesn’t have a degree in computer science;
  • In-demand skills and where you can find employees that have them;
  • How tech companies and educational organizations can work together to grow the talent pipeline;
  • Supporting tech workers through continued learning and development programs

RSVP for Closing the Skills Gap:
Join the conversation on January, 19th at Kenzan:
1743 Wazee St, Suite 200 at 6pm


Rona is a Director of Engineering for Kenzan where she leads technical teams in the production of web applications. She is active in the hiring of development resources and helps drive initiatives that encourage the growth of Kenzan’s employees.