Docker and Kubernetes, IoT, Elasticsearch as a Cache and the CI/CD Process – Perspectives from our Senior Developer
June 4, 2019|
Welcome back to our digitalML spotlight series! Last month we spoke to Platform Director Mark Whitehead who gave his perspectives on Multi-cloud, Blockchain, DevOps and Digital Transformation Strategy.
This month we’re chatting with Ryan Jones, Senior Developer, who shares his views on:
- Docker, Kubernetes and containerization for simplifying processes and enabling scalability
- The IoT and privacy concerns it brings
- How problem solving can be the most rewarding part of his role
- Implementing Elasticsearch as a cache
- Why it’s worth setting up a CI/CD process driven by ignite to give one source of truth for service designs
Kubernetes and containerization help with service scalability and availability
Gemma: What are 1 or 2 industry trends that are interesting to you right now and why?
Ryan: Definitely Docker, Kubernetes and containerization. As a developer, I find the most frustrating part of my job is installing and configuring app servers, databases and other things within my development environment. Docker allows me to do this once and reuse the same image time and time again.
Kubernetes and containerization are interesting areas with a lot of traction right now as it enables services to be easily scaled. This is vital for a business looking to rapidly expand.
Kubernetes also offers container health checks; it’ll restart a service considered unhealthy if a restart policy is configured. This is especially useful as it acts as a quick response to ensure high availability.
Gemma: What emerging tech are you currently following?
Ryan: I’m really interested in the IoT and have lots of connected devices around the home, all linked through the Google Assistant.
Privacy is still a big concern for a lot of people though and it looks to be something the likes of Google and Amazon are trying to sort out. Some of these devices are essentially always listening.
I’ve read recently that Google is looking to rebrand their connected home offerings to Google Nest and with this, plan to clear up the concerns users like myself have about what data is collected and when.
The IoT is really exciting though, and there are countless ways it can help – Through IFTTT and Nest, both linked to my phone, I am able to set up an automatic instruction to turn on the heating when I leave work – so I never come home to a cold house!
On the rewarding nature of problem solving and implementing Elastic Search as a cache
G: What’s your favorite thing about being a Developer?
R: I really enjoy the problem-solving side of things. Sometimes you get complex cases, which take a lot of investigation and problem solving to find the root cause. The code change is usually the easiest part, its tracking down the faulty code that takes the time. That sense of accomplishment you get when a tricky case has been solved is great.
G: Having been at digitalML for 4+ years now, what’s been the most interesting project you’ve worked on?
R: It has to be implementing Elasticsearch as a cache.
Within ignite, we already maintained an in-memory cache for certain artifacts to avoid unnecessary database load. Of course, there are quick database consistency checks done to ensure the cache entry is up to date.
As this is an in-memory cache, it only works on a single JVM and only for the lifespan of that JVM. We wanted something more permanent that could be accessed across multiple JVMs, while still avoiding loading the data directly from the database.
Using Elasticsearch for this really made sense as we already had knowledge of it within the team; it is also used in Inventory Explorer for searching the service catalog.
I set out to use Elasticsearch as a JSON document store with mapping disabled, so not to make the document content searchable. There is no point in allowing Elastic Search to waste resources indexing the content when it would never be queried against.
Retrieving certain artifacts within ignite now consists of a 3-step process:
1 – Retrieve from the in-memory cache
2 – If it doesn’t exist or it fails the database consistency check, retrieve from Elastic Search
3 – If it still does not exist or fails the database consistency check, load the object from the database
G: Sounds like a project that’s made a huge difference to the ignite platform! What’s your overall favorite ignite feature, and why?
R: Developer+. It’s a fantastic way to turn your API designs into code. First, you have to set up a set of Output Templates* within ignite, but thankfully ignite ships with a standard set of Java-based templates to get you going.
*Note to reader: output templates are expression-based templates which pull information from any abstracted service design, to enable auto-generation of runtime artifacts for any technology.
G: What’s the best part about working at digitalML?
R: Hearing the customer success stories and knowing that you are making a real difference in their productivity!
Setting up a CI/CD process driven by ignite gives you one source of truth for your service designs
G: Finally, have you got any other advice for our customers?
R: Spend the time setting up a CI/CD process which is driven by ignite. When a service design meets any particular governance state, ignite can generate the code and push it directly to your SCM system. This can then trigger a build in your CI/CD environment and feedback into ignite.
Setting this up ensures ignite is the source of truth – any improvements or change requests can be made to the service design within ignite and will always flow through into your development environment.
Taking a code-first approach risks losing valuable documentation relating to your service design but maintaining in ignite prevents this.
A big thanks to Ryan for sharing his view this month. Coming up in future installments we’ll sit down with DevOps and Customer Success team members, so be sure to come back.
See you in July!