Cloud Infrastructure Setup & Operations

Choose the cloud on the workload, not on the brand

GCP, Azure, AWS and DigitalOcean all do the basics, and much the same basics. What actually differs are the details that touch your particular workload: whether the organisation already lives in Microsoft 365 or Google Workspace, how close to your users in Thailand the region needs to be, whether the work is processing on a schedule or traffic all day, whether you need GPUs or a managed database, and which vendor's tooling the team taking it over already knows. We work through those questions first and pick the provider afterwards. Some workloads sit better on a small server at DigitalOcean than on a managed service bought against load that has not arrived.

• Start from the workload: traffic volume, processing schedule and GPU needs
• Check what the organisation is already tied to, such as Microsoft 365 or Google Workspace
• Weigh up data centre location and any requirement about where data must live
• Judge which toolset your own team can realistically run
• More than one provider is fine, when splitting the work costs less than keeping it in one place

Only the doors that have to be open

The network layout is a decision about what can see what. Databases, job queues and internal services go on a private network with no public address, leaving one way in from the internet: a reverse proxy, which is also where the TLS certificate lives. Administrative ports such as SSH are open only from named addresses. Certificates come from Let's Encrypt with automatic renewal, and the renewal itself gets checked, because a certificate expiring at two in the morning is an outage that was entirely avoidable.

The mistake we see most often is an application bound to every interface. If it listens on all interfaces instead of 127.0.0.1, that port is reachable straight from the internet, without TLS, no matter what the reverse proxy is configured to do. So every application binds to loopback, the proxy really is the only door, and the same deny rules go into every virtual host: for the API documentation pages frameworks publish by default, and for dot-files.

• Databases and internal services on a private network, with no public address
• One way in from the internet: the reverse proxy, which is where TLS lives
• Applications bound to 127.0.0.1, not to every interface
• SSH and administrative ports open only from named addresses
• Let's Encrypt certificates renewed automatically, one certificate covering several names
• The same deny rules on every domain, for API docs pages and dot-files

Backups you have actually restored, and alerts before the customer calls

A backup nobody has ever restored is a hope, not a plan. So we set the backup schedule together with how far back you are able to go, keep copies somewhere other than the system they came from, and then rehearse a restore onto a separate machine, timing it and checking the data is complete, before anyone relies on it.

For monitoring, health checks go down to the port and confirm the service still accepts a connection, rather than only checking that a process exists: a hung process is still alive by that measure while answering nothing. Alongside that we watch disk space, memory and certificate expiry dates, and gather the logs from every service in one place so an incident can be traced backwards.

For access, we keep a register of who holds which key, use named accounts rather than one shared login, keep secrets out of the code, and write the revocation steps down in advance, because the day you need them is the day there is no time to work them out.

• A backup schedule with a stated how-far-back, stored away from the system it came from
• Restores rehearsed on a separate machine, so recovery time and completeness are known
• Health checks at the port, not just a process that exists
• Disk space, memory and certificate expiry all watched
• Logs from every service gathered in one place for tracing incidents
• Named accounts, a key register, and revocation steps written before they are needed

On cloud, cost is a design decision, not an invoice you receive

Cloud bills that get out of hand rarely do so because of unit prices. They do it because of decisions made at design time: instances left running for work that takes a few hours, egress traffic nobody counted, a managed service bought against load that has not arrived, and disks and snapshots nobody deleted. So we cost the architecture as we design it, separate what is fixed from what varies with use, tag resources so a line on the bill can be traced back to a system, and set budgets with alerts when spend runs above its usual trend.

Once the system has settled, we hand over the scripts that actually built it, the architecture documentation, and procedures your team can follow. Scripted deployment matters more than it sounds: a manual procedure drifts a little each time somebody repeats it, while a script gives the same result every time and can be read back to see exactly what was done.

• Costs estimated alongside the design, not after the first bill arrives
• Fixed spend separated from spend that varies with use
• Resources tagged, so a cost can be traced back to a system
• Budgets with alerts when spend runs above its usual trend
• Idle resources shut down or sized down, and stray disks and snapshots cleared
• Handover with deployment scripts, architecture documentation and runbooks