Insen

Write, Code, Dream, Travel, Play !

© 2018. All rights reserved. Personal opinions. Hefty pinches of salt !!.

site based on hyde by @mdo

'Not all those who wander are lost' - J.R.R Tolkien

DevOps - Getting off the blocks.

27 Sep 2017

DevOps is everywhere these days. As it should be, yes. Yet that doesn't take anything away from the fact that DevOps is a difficult concept to understand and implement. The principles are fair and easy enough to grasp, but then the where to begin question comes in, and that is a very difficult question for something as nebulous as DevOps.

So how does one start being 'DevOps'y? What drives DevOps?

Context, Culture, Tools, Change, Automation, Feedback Loops and Collaboration !

The above set summarizes the key drivers of DevOps anywhere. Additionally, there are a small but definite set of foundational DevOps practises, which have been proven enablers of DevOps over time.

Foundational DevOps Practises

Essentially, the practises above together cumulate into a CI/CD pipeline. How sophisticated/refined your CI/CD pipeline is, is one of the key technological measures of a DevOps oriented team's maturity level.

So then the question driving a group trying to adopt a DevOps-y approach to software service delivery is, most likely, this

Our DevOps Question

Given our current context, what culture, practises and tools should I adopt which will enable

Note : Remember this question, We'll come back to this time and again.

If we formulate an answer to the question above, we should have a workable Devops Strategy 101. Let us try an exercise and see.

Setting the 'Context'

Context is all about NOT sacrificing global optima to achieve local optima. But then my junior dev comes up, the one I hired last month, saying he doesn't know what the ceo knows so he doesn't have all the context so he can't code a-la DevOps.

Oops!

That didn't sound legit, did it? Cos it wasn't. In DevOps context itself is context-sensitive, i.e. it means take the biggest picture from where you stand. That's all you can do anyways.

Since context is king, we will set a context within which to start out on our DevOps analysis. Assume the following organizational scenario first.

A team trying to provide custom software development and integration services on Azure. A team trying to a create a distributed processing layer for the .NET Stack system. A team working on big-data analytics and data presentation system on Azure. A management team trying to manage all these.

Also assume that all of the teams have various degrees of maturity on the build, test, deploy aspects of their individual systems - so effectively, automated builds, automated tests, and CI/CD exists, but to various degrees. Which is usually the case in diverse organizations.

So the context here is a bunch of software engineers trying to provide system development services on Azure.

Next.

Consider 'Culture'

Now what can we do about Culture here?

Well, in a software development group, there is one 'Culture' culture, or group culture or organization culture.

Now this top-level culture is probably not something everybody can start off with. But in every culture, there are sub-cultures. And there are sub-cultures across development tools, eco-systems or resources. (Java vs .Net, anyone?). So for a start, let's just pick a culture by eco-system. Since Azure is common across, let's pick Azure.

Enter Azure Culture. But what does Azure Culture in a DevOps organization mean? Let's go back and see how our original question changes.

What practises and tools should I adopt which will enable

Note - Should we limit ourselves to Azure specific tools. What if we work on Scala, Node, .NET? So perhaps, we need to analyse tooling more comprehensively.

The above is the kind of question that completely derails initiatives if we sit down to exhaustively analyse the options. Sometime you just pick the first option and go - just so that it sets an operational context, if nothing else.

So we will just set one - the .NET stack. Keeping things within the family, you see. You pick scala, or hadoop or node as per your needs. After all, you need multiple iterations of this.

So our task has now decomposed into a search for tooling options on the Azure platform and the .NET software development stack that enables foundational DevOps practises while achieving one or more of our goals - faster change adoption, more automation,more shared process information, more closed feedback loops. Once we identify tools and technologies, we assess our maturity levels on identified tools and practises to identify gaps which can be plugged.

The gaps are what we attack in our DevOps strategy 101. The number of iterations of this process you go through, and the reviews with all concerned stakeholders should pare the list to items with highest overall priority.

Now we will do a non technical map-reduce. In map phase, we list out every possible tool, practise or activity that looks like it might help. In reduce phase - we prioritize items from list.

And in map phase our goals are simple.

Now, the initial list will probably be large, as both Azure and Microsoft .NET are huge eco-systems, but remember two things,

But we do need to manage, classify and process this list. Wikipedia, in its DevOps page, suggests the SDLC stages - Code, Test, Build/Package/Relase (merged these as boundaries between them are overlapping, especially in the matter of tooling), and Monitor.

Azure/.NET Stack | SDLC Stage - Code

A basic DevOps practise here is Version Control. Distributed VCS are now standard, so we pick Git.

Architectural support targeting DevOps enablement for Azure PaaS systems is a much-needed eco-system centric practise. Enter Microservice architectures which by encouraging small pieces and plug-and-play composition help keeping pieces small and nimble. Smaller pieces offer much greater ability to respond to change, better automation and easier deployment options.

Test-Driven Design (TDD) and Domain Driven Design (DDD) are other standard practises that help in better code and decoupled pieces.

Thus

Tools

Practises

Azure/.NET Stack | SDLC Stage - Test

Test automation has several flavours

Additionally, we can also automate the test generation process through Behavior Driven Design Tools like SpecFlow, and test execution automation is usually done through build pipelines and Continuous Integration (CI)/Continuous Delivery(CD).

Tools

Some possible activities. The one around logging formats and test reports is especially interesting -

Activities

Azure .NET Stack | SDLC Stage - Build/Package/Release

In this section, practises are common across software development verticals and horizontals.

Practises

Tools

These include

Azure/.NET Stack | SDLC Stage - Monitor

This is probably a big problem area. For now we will gloss over it, but in practise, we would now probably take just monitoring as a problem area for DevOps and resort to the same technique we used throughout this post to break that down into tools, practises and activities.

Practises

What is needed here is feedback loops and operation trend monitoring through shared dashboards. To large extents, these feedback loops across localized scope (from a DevOps point of view) can be provided by ALM tools like JIRA on the scope and requirements and features perspective, while build and test feedback loops can be provided by CI servers and build-pipelines.

The difficult part is actually setting up on what to record and monitor. Sifting noise from signal here is a significant and not-always-technical step. And once past this, we have the technically demanding part in setting up a system which integrates separate local scopes into integrated dashboards. That is hard and definitely not DevOps 101. Maybe later.

Tools include VSTS - for application life-cycle management, Azure AppInsight for application operation management, standard external tools like NewRelic, and ELK, advanced services like Azure OMS (native) and Google Analytics (external).

Activities

Following activities are required.

At some point, build a Data Collection and Aggregation Tooling/Implementation.

Big List of Tools, Practises and Activities

What we have been doing so far, is basically running the map part of a map-reduce analysis. Aggregating all of the tools, practises and activities found above, we get the following big list.

Tools / Practises

Practises

Activities

This may not be exactly the list you come up with, but if you followed along on the exercise model, you have your own list.

I am leaving the reduce part of this operation out, as that will probably diverge for everyone. At its basic premise, evaluating your group's maturity model on each of these items and identifying the highest priority items should get you there. Multiple rounds of reviews from multiple stake-holders is the way forward now, but the no. of items should 'reduce' in reduce phase.

So there you are, with a set of starting points for a DevOps 101 Strategy. All we have to do now is Go forth, and Iterate.

Oh, and of course, implement !

Related Posts