Established in the early 2010s without a proper focus on DevOps, Blocksi ran its services on manually managed Google Cloud VMs, using PM2 and git pull for deployments. This approach made it hard for the system to scale, keeping it vulnerable to human errors and cutting ways to automate processes or track changes.

Putting product growth and features development first, the client missed some key security practices in their system. For example, Docker images had hardcoded secrets in them, database endpoints were publicly accessible without IP restrictions, and a rollback strategy was missing, meaning the in-house team had to fix all failed deployments manually.

With no time nor expertise to fix it, Blocksi has a few critical issues in monitoring, including broken Grafana dashboards, gaps in Prometheus metrics, and missing alerts in some critical services. With all that, the team has to act reactively, often finding out about issues only after they have already impacted users.

Supporting Blocksi's growth, OpsWorks Co. started by rearchitecting the infrastructure setup. Opting out of VMs, the team migrated services to a containerized environment using Google Kubernetes Engine. This setup made scaling and higher availability possible for the whole system. Taking it to the next level, the team configures the GKE cluster to span across multiple availability zones to avoid single points of failure.

To improve deployments and minimize risk during each of them, we introduced a Jenkins-based CI/CD pipeline with Helm for Kubernetes templating. This allowed the team to version, track, and safely deploy application changes using Infrastructure as Code practices. The pipeline was designed to include blue-green deployments and automated rollback mechanisms, meaning any issue in production could be instantly reverted with no hands-on involvement.

Knowing that security was a top priority for the client, the OpsWorks Co. team removed hardcoded secrets from Docker images and moved the management of all sensitive data to Kubernetes Secrets under role-based access control. Then we configured NGINX Ingress with TLS support to encrypt communication for all public endpoints. Additionally, database endpoints moved behind private IPs and were limited to a strict set of IP whitelists. These both resolved critical vulnerabilities and improved compliance with industry security standards.

Right after handling core infrastructure and delivery pipeline changes, OpsWorks Co. focused on observability. We upgraded Prometheus and Grafana, fixing broken dashboards and adding the ability to gather other insights. To achieve full visibility of application performance, database health, memory usage, traffic, and latency, we integrated Datadog. All critical alerts were configured and routed to Slack and email channels, making it easier for the team to detect and resolve production issues.