DevOps

Many people have a very fuzzy idea of what DevOps is. They understand it is about getting software products to market quickly but they have little idea about what this entails in the real world.

DevOps is a process you generally put together to suit your business. Like so many new concepts there is not a one size fits all solution and it is certainly not a product offering. Sure there are tools out there to help you along your way, but fundamentally you will need to change the way you work and requires a high level of rigour and discipline.

To put things into perspective the companies who have done this really well have software development as the cor of their business. The people who come to mind and are oft quoted are people like Netflix, Amazon, Google and Spotify. They have the scale, focus and money to drive a strong DevOps culture.

This does not mean there is no benefit from taking a DevOps approach, its just going to be more difficult, especially in a mixed IT environment. In reality many smaller and medium sized companies operate largely 3^rd party systems and their development is focused on small apps, integration and reporting. The areas where DevOps is most likely to be of the most relevance is within Digital teams which are often engaged in Customer or Partner facing web applications, which lend themselves well to this approach.

DevOps is a journey it is unlikely that you will be able to get there in one go First of all you really need to be living and breathing Agile development and that means having properly implemented Scrum teams typically with a 2 week delivery cadence.

Under a DevOps model, development and operations teams are no longer “siloed.” Sometimes, these two teams are merged into a single team where the engineers work across the entire application lifecycle, from development and test to deployment to operations, and develop a range of skills not limited to a single function. Quality assurance and security teams may also become more tightly integrated with development and operations and throughout the application lifecycle. This is a far more collaborative and natural way to work and runs counter to the highly structured and siloed approach of many organisations often driven by ITIL process thinking.

There are 3 phases to adopting a DevOps approach:

• Automated Testing

Even to get started you need to have automated testing in place and this needs to be getting up to 90% test code coverage levels. That is a huge challenge in its own right but a necessary part of the sprint delivery cycle. To do this you will need to be building testing alongside your code and once these tests have been run for the first time, they need to be included in your automatic testing suite. One approach to consider for this is Test Driven Development (TDD) where by tests are written first with code initially failing and code built until the tests fully pass. It may feel like an unnatural way to work but it means that all of the requirements are properly baked into the design right from the start.

• Continuous Integration

Once you have done this you can build a continuous integration suite that will run the full suite of tests whenever code is submitted, which should be at least daily. In this way bugs are identified and can be fixed extremely fast before they even get close to a user.  This does of course require you to have a high level of code coverage to be useful. An example of a tool that will help with this is Jenkins. This is essentially a server running in the background that builds and tests the software.

• Continuous Delivery

This effectively is the full cycle of DevOps. There are a range of tools that can support this work. An example would be a combination of GIT Hub for storing code and version control, Jenkins for continuous integration testing, Puppet to automatically deploy the software, AWS Webservices for running the software operationally and New Relic for monitoring the business outcomes of the software.

There are of course lots of alternative tools but you will need to select ones that fit your development teams and technology stack. A comment here is to allow the teams to select the tools they want to use but then be consistent with their usage across the board. This tooling later in the cycle is only really relevant once you have all the other DevOps disciplines in place.

Ref:

TDD – Test Driven Development - write a failing test for code that is yet to exist we then write the code and really the code passes the test. The test is based on the development requirements.

BDD – Behaviour driven development – you write behaviours that drives your development. Write a test that describes a behaviour for code that may or may not exist. If the code does not exist it fails until it is working correctly. It is a way that the product owner BA and tester can work together to initially define the requirements, write tests for them and then write the code that will make the tests pass. Avoids lost in translation issues.

The need for automated deployment has been driven by

1. Need to scale and deploy to large numbers of servers
2. The process of migrating from test to production – often systems work in test and then fail when they hit production because the environments are not identical you need a tool to:
   • Deliver a consistent environment through the lifecycle
   • Enable roll back to previous states
   • Allow scaling

Infrastructure As Code (IaC)

Just as software deployment can be deployed automatically so can infrastructure. This is done by provisioning of infrastructure by code instead of a manual process.

Optimally both infrastructure and software would be provisioned together.

The advantages of this are:

• Just need to configure once and then deploy out, all environments will be identical
• Gives you a record of exactly what was deployed because it was automated, this means you can quickly rollback to the previous stable version
• Can increase the release frequency

Deployment Tooling Comparison

All of this can of course can be done using shell scripts such as PowerShell but using Configuration Management tools such as Puppet 80% of the functionality is already there to use. CM tools also provide a UI.

Common Tools compared include Puppet, SaltStack, Chef and Ansible

• Scalability – all tools highly scalable
• Ease of Set Up – Puppet, SaltStack and Chef are Master Agent which means they have a Master Server and Clients. Ansible in contrast is Master Node with just a Server and the Client VM needs no special set up and is simpler to set up
• Availability – all have good failover capability with multiple masters with secondary instances
• Management – Puppet is not easy to learn with its own language called Puppet DSL and Chef requires a programmer to configure using Ruby DSL Puppet and Chef use pull configuration. These products are cryptic and may need to go on courses to get the training which causes staff dependency issues. SaltStack and Ansible much easier and use push configuration and use YAML which is a human-readable language commonly used for configuration files e.g. Install: Package X

Puppet and Chef use Pull configuration and Ansible and use SaltStack use Push configuration.

• • Push config – configuration will be pushed onto the others servers (nodes) from the central server.
  • Pull configuration - the clients poll the central server and pull the configurations from the central server without any command.

• Interoperability – All solutions need to run on a Linux server but clients can all also operate on Windows machines.
• GitHub Activity – SaltStack and Ansible biggest active communities
• Popularity – Chef and Puppet are the oldest so the most used but Ansible has caught up and ease of use is a major factor

Ref: https://www.youtube.com/watch?v=OmRxKQHtDbY

Ansible

Configuration Management, Application Deployment including groups of web, app servers and databases and Service Orchestration.

Works by client connects to server via SSH. Ansible gathers Server information.

Playbooks – define deployment steps and config in YAML and can have variables enabling them to be used to orchestrate steps across multiple machines.