Exoscale – Zak Abdel-Illah https://zai.dev Automation Enthusiast Fri, 13 Dec 2024 14:14:40 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 https://zai.dev/wp-content/uploads/2024/06/android-chrome-512x512-1-150x150.png Exoscale – Zak Abdel-Illah https://zai.dev 32 32 Overview: Extending my home network to the cloud https://zai.dev/2024/09/11/overview-extending-my-home-network-to-the-cloud/ Wed, 11 Sep 2024 22:02:52 +0000 https://zai.dev/?p=915 As a frequent traveller, I found it impractical to maintain a physical system infrastructure, so I relocated my home infrastructure to the cloud.

Establishing a VPN Connection

To begin, I set up a VPN connection from my OpenWRT router to the cloud provider using WireGuard. I created two VPCs in the cloud provider – one public and one private – to mimic the “WAN-LAN” scenario of at-home routers.

This setup provides isolation similar to a home network, where the resources on the private network can only be access by other resources on the same network, but they are also able to communicate with the outside world.

The intention is to have the private network as an extension to my “home” (at any given time).

Deploying a Cloud Router

I deployed a virtual machine that will act as a router spanning both networks. This needs to be across both networks as I need an endpoint to connect to (which requires an internet-exposed network) while still being able to access private resources.

I chose VyOS as the cloud router’s operating system because it is configuration-driven, allowing for an Infrastructure-as-Code (IaC) approach for easy re-deployment on any cloud provider.

Utilizing Object Storage for Plex Media Server

I adopted object storage to take advantage of the “unlimited” data offered by the cloud provider, and configured s3fs to mount the object storage on a specific node. With this, Plex can access data directly from the object storage bucket without many configuration changes or plugins to Plex.

The VPN connection allows me to access the Plex server securely as if it were local on both my PS5 and laptop. This setup ensures that the Plex interface remains non-accessible to the public and bypasses the bandwidth limit when proxying via the official Plex servers.

Securely Pushing Metrics from In-House Devices

By using the VPN connection, I can push metrics from my in-house devices directly, such as weather sensors without exposing my Prometheus instance to the public internet.

The VPN’s security layer wraps around all traffic, eliminating the need for implementing a CA chain for Prometheus when using platforms such as AWS IoT or Grafana Cloud (where devices are expected to communicate with a public HTTPS endpoint)

Automating At-Home Devices with HomeAssistant

I use HomeAssistant within the cloud provider to automate my at-home devices without worrying about downtime or maintaining a device inside my home. HomeAssistant is scriptable, easily re-deployable, and can bridge a wide range of IoT devices under a single platform, such as HomeKit and Hue.

I can now utilize my old infrastructure without worrying about maintaining hardware, and plan to deploy many services to the private cloud. Keep an eye out for a deeper breakdown on how I deployed and configured each element of my private cloud

]]>
Exoscale Exporter for Prometheus https://zai.dev/2024/09/04/exoscale-exporter-for-prometheus/ Wed, 04 Sep 2024 11:18:20 +0000 https://zai.dev/?p=830

I’d built a Prometheus exporter for Exoscale, allowing me to visualize cloud spending and resource usage from a central location alongside AWS and DigitalOcean.

The Exoscale exporter is built using Go and leverages the latest version of Exoscale’s Go API, egoscale v3 and includes basic integration tests and automatic package building for all major platforms and architectures.

Some of the metrics exported are;

  • Organization Information: Usage, Address, API Keys
  • Compute Resource Summary: Instances, Kubernetes, Node Pools
  • Storage Resource Summary: SOS Buckets & Usage, Block Volumes
  • Networking Resource Summary: Domain & Records, Load Balancers

By integrating organizational data from Exoscale into the Prometheus ecosystem, I can now configure alerts for spending or resource usage on either Exoscale specifically or for all platforms using AlertManager.

I can also identify where I may have left resources behind using Grafana, in the event I’m manually creating them or my IaC executions didn’t do a proper clean-up.

Metric Browser in Grafana; Showing some values exported from the Exporter

I decided to deploy the exporter to my Kubernetes cluster, scraping based on the default interval of 2 minutes. This is roughly a good balance between;

  • When a new billing amount gets updated (hourly)
  • How often infrastructure elements themselves gets updated (could be on a minutely-basis)
  • How much data gets consumed by the time-series

I chose Kubernetes cluster rather than a server-less solution or a dedicated VM so that I can optimize the costs of running the exporter by sharing resources, in addition to abstracting the cloud provider away from the application.

]]>