GitOps, SRE, DevOps - A Look Back and into the Future

Hi,

Ten years ago, most companies were still dealing with manually managed infrastructures. Today, many medium-sized businesses still find themselves in the same situation. But that comes at a cost - and I’m feeling it right now.

The Ghosts of Differences

It’s astonishing how often we chase ghosts simply because our environments differ. This ranges from obvious things like available CPU or RAM to subtler issues like different kernel versions or configuration disparities in applications. It gets especially tricky with external application servers, where configuration files are spread throughout the system and can easily drift apart.

• Different permissions on the file system
• Differences in user names and rights
• Variable sudoers rights

All these nuances turn the hunt for apparent problems into reality.

The Challenge of Harmony

It just happened to me: we identified and fixed a problem in production. But when validating the fix on our test platform, we encountered a configuration difference that cost us several days. These discrepancies don’t always become apparent immediately.

The solution lies in harmonizing the environments. This requires a multi-layered strategy:

• Hypervisor Provisioner: For consistent VM resources across all environments, for example, Terraform.
• Configuration Management: To install programs in the same versions and have folders with identical permissions, for example, Ansible, Puppet, or Chef.
• Containerization: To ensure a uniform application configuration, with Docker being the most well-known application.

Storing all these configurations in a VCS like Git allows us to track changes, automatically audit, and ensure the compatibility of our software. GitOps is the keyword here.

One Step at a Time

Integrating these changes into an existing project is not easy, but it’s not impossible either. It doesn’t have to be all done at once. If you don’t have harmonized test environments yet, there’s a significant optimization opportunity ahead of you. Every step towards a unified infrastructure will move you forward.

Rule the Backend,

~ Marcus