Google Cloud

Recover a Corrupt Google Cloud VM with Disk Space Issues Caused by Docker

by
Google Cloud VMs can sometimes get stuck or become inaccessible due to disk space exhaustion — often from leftover Docker images, containers, and volumes accumulating after CI/CD pipelines. Here’s a clear, practical step-by-step guide to restore your VM and clean up Docker efficiently. Scenario Step 1: Create a Rescue VM and Attach the Corrupt Disk … Read more

Translating and Deploying a docker-compose project to Kubernetes using Kompose

by
I’ve chosen a LAMP stack-based demo app just because it’s a multi-tiered application that demonstrates well the power of docker-compose as a container orchestration tool. You can apply this to any docker-compose project. The idea behind this topic is to translate docker-compose.yml file to a single kubemanifests.yaml resource file using Kompose, apply that YAML file … Read more

Google Cloud IoT core

by
Cloud IoT Core is a fully managed service. This ensures that you don’t need to do autoscaling, replication configuration, database partitioning, or pre-provisioning of resources. One or millions of devices can be connected and Cloud IoT Core can scale to suit your needs. Cloud IoT combines the highest security level of the MQTT protocol (TLS … Read more

IoT – Communicating with Devices

by
MQTT is an industry-standard IoT (Message Queue Telemetry Transport) protocol. It is a communication protocol for publish/subscribe (pub/sub). The model for publish/subscribe is event-driven. Signals are pushed to customers who subscribe to the subject. The Broker is the contact point. Clients are sending messages to the broker and the broker is getting messages out to … Read more

The Internet of Things IoT

by
The Internet of Things, or “IoT” for simple, is about expanding the power of the Internet to a whole range of other devices, systems, and environments beyond computers and smartphones. The amount of support IoT receives is exponentially increasing. In reality, it was not until 1999 that the word “Internet of Things” was invented. Since … Read more

Challenges facing modern data pipelines

by
Introduction Software engineers have a central responsibility for building efficient, scalable, and secure pipelines. Data may come in from all various data sources inside modern organizations. Message-oriented architectures with Cloud Pub/Sub The first part of the pipeline puzzle that manages vast amounts of streaming data that do not come from a single organized database. Alternatively, … Read more

Compute Power for Analytic and ML Workloads

by
Google trains its output machine learning models on its large data center network and then deploys smaller trained versions of these models to your phone’s hardware for video predictions for example. You can use pre-trained AI building blocks to exploit Google’s AI work. For example, if you’re a film trailer producer and want to quickly … Read more

An Introduction to Google Cloud Data Engineering

by
By 2025, we’ll have more than 21 billion active IoT devices connected (Statista, 2020). According to Wikipedia: “The Internet of things (IoT) is a system of interrelated computing devices, mechanical and digital machines provided with unique identifiers (UIDs) and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction.”

Google Cloud DataProc

by
DataProc
Cloud Dataproc is a managed Spark and Hadoop service that allows you to use open-source data resources for batch processing, querying, downloading, and machine learning. Automation with Cloud Dataproc lets you rapidly build clusters, spins up instances of the cluster’s compute engine, handle them efficiently, and save money by shutting off clusters that aren’t necessary. … Read more