Building a 100k RPS Architecture on GCP: Bigtable and Real-Time Bidding

Part 1: The 50-Millisecond Bloodbath and the Core Engine

Welcome to the digital bloodbath known as Real-Time Bidding (RTB). Let’s set the scene: you have exactly 50 milliseconds to receive a bid request, cross-reference it with a user’s historical profile, run a pricing algorithm, and send a response. If you take 51 milliseconds, the auction is over, the slot is given to your competitor, and your infrastructure just burned CPU cycles for absolutely nothing. Now, multiply this by 100,000 requests per second. Every single second of every day.

Most marketing brochures will tell you that the cloud is magic and scales infinitely to handle this. They casually forget to mention that “infinite scaling” comes with an infinite monthly bill. Today, we are going to design an RTB architecture on Google Cloud Platform that actually survives production, doesn’t bankrupt the company, and keeps European regulators reasonably happy.

Before we choose the core database engine, let’s look at the usual suspects and why they will inevitably fail you under this load.

The Illusion of Choice: Evaluating Alternatives

Option 1: Cloud Spanner. Ah, Spanner. Google’s globally consistent, fully relational masterpiece. It is a fantastic database if you are a bank processing high-value financial transactions where ACID compliance is a matter of life and death. But at 100,000 writes per second, running an AdTech platform on Spanner is like commuting to the grocery store in a Boeing 747. It works, it’s impressive, but your FinOps manager will have a heart attack when the invoice arrives. You are paying a massive premium for relational consistency that an advertising bid simply does not need.

Option 2: Memorystore (Redis). The latency here is beautiful. Sub-millisecond response times. But we are dealing with terabytes of historical user profiles, segment mappings, and bidding logs. Storing petabytes of persistent, durable data in RAM is a privilege reserved for startups that are actively trying to burn through venture capital before they go bankrupt. It is fundamentally the wrong storage tier.

Option 3: Cloud Bigtable. The undisputed winner for this specific nightmare. Bigtable is a wide-column NoSQL database designed specifically for extreme throughput and single-digit millisecond latency at the petabyte scale. It doesn’t have standard SQL. It doesn’t have JOINs. It doesn’t care about your feelings. It forces you to denormalize absolutely everything. But if you treat it right, it will chew through 100,000 RPS and calmly ask for more.

The Entry Point: F# and The Shock Absorber

The 100k RPS traffic doesn’t just magically appear in the database. It hits a load balancer and flows into Google Kubernetes Engine (GKE). Here, your lightweight F# microservices act as the gatekeepers. F# is mathematically precise and handles concurrent asynchronous operations beautifully, which is exactly what we need when every thread is fighting for milliseconds.

The microservice receives the bid request, instantly queries Bigtable to fetch the user profile, calculates the bid, and fires the response back. But what about recording the event itself? We do not write the log synchronously. Writing takes time. Instead, the F# service drops the raw event payload into Cloud Pub/Sub and forgets about it. Pub/Sub acts as our massive, resilient shock absorber. From there, a separate background worker picks up the messages and executes the actual write to Bigtable. This asynchronous decoupling is the only reason the frontend API can consistently reply within the 50ms deadline.

Engineering the Perfect Row Key

If Bigtable is a Formula 1 car, the Row Key is the steering wheel. If you design it wrong, you will instantly crash into a wall.

How NOT to do it (The Anti-Pattern): Let’s say a junior engineer decides to use the event timestamp as the Row Key. The logic seems innocent: you want to sort events by time. Congratulations, you just created a “hotspot.” Bigtable stores data lexicographically across multiple physical nodes. If your keys are sequential (e.g., 2026-07-23-12:00:01, 2026-07-23-12:00:02), every single write request is being hammered into one single server at the end of the cluster. That specific node’s CPU hits 100%, latency skyrockets to 5000ms, and the rest of your expensive cluster sits there doing absolutely nothing, sipping virtual coffee.

How to actually do it (The Engineered Solution): To survive, we need to evenly distribute the 100,000 writes across every single node in the cluster. We achieve this by hashing the keys and introducing salting.

The ideal Row Key structure for our user profiles and bids looks like this: [hash(user_id)]#[shard_id]#[reverse_timestamp]

Why does this work? First, the hash(user_id) ensures that user A and user B are physically stored on entirely different nodes, instantly killing the hotspotting problem. Second, the shard_id (a random number from 1 to N) further divides extremely active users so they don’t overwhelm a single data block. Finally, the reverse_timestamp (calculated as Long.MAX_VALUE - timestamp) is the absolute magic trick. Bigtable only sorts in ascending order. By reversing the time, the absolute newest event for a user is always physically stored at the very top of their data block. When your service needs to read the user’s latest bid history, it grabs the first few rows instantly and ignores the rest of the terabytes below it.

Part 2: The Analytical Bridge and the GDPR Fortress

You have successfully built a massive, extremely fast Bigtable engine. It is happily digesting 100,000 requests per second. The engineering team is celebrating. And then, a data analyst from the marketing department opens their laptop and types SELECT * FROM bids WHERE country = 'France'.

If you let them run this query directly against Bigtable, your cluster will die. Bigtable does not have secondary indexes. To find all the French bids, it must perform a full table scan across petabytes of data. Your CPU will spike to 100%, latency will jump from 50 milliseconds to 5 seconds, and your RTB platform will lose every single auction.

Giving analysts direct access to Bigtable is like letting a toddler play with a chainsaw. You need to physically separate your fast transactional system (OLTP) from your heavy analytical system (OLAP). Here is how you build that bridge.

Feeding the Analysts: How to Export Data

Approach 1: The Dual-Write (The Naive Way).

Your F# microservices in GKE simply send the data twice: once to Bigtable for the auction, and once to BigQuery for the analysts.

Why it is bad: You are paying for double processing. Worse, networks are chaotic. If the Bigtable write succeeds but the BigQuery write fails, your systems are physically out of sync. You will spend weeks arguing with the finance department about why the production database shows different numbers than the analytical dashboard.

Approach 2: Nightly Batch Processing (The Cheap Way).

You leave Bigtable alone during the day. At 2:00 AM, you spin up a cheap Dataflow (Apache Beam) job that dumps yesterday’s data into Cloud Storage as Parquet files.

Why it is bad: Your analysts will hate you. In the modern AdTech world, waiting 24 hours to see if a marketing campaign is profitable is completely unacceptable. The business needs near-real-time data to adjust bidding strategies.

Approach 3: Change Data Capture (The Enterprise Standard).

This is the architecture we are actually going to implement. We enable Bigtable Change Streams. This is a native GCP feature where the database itself silently generates a continuous, low-level log of every single insert, update, and delete.

We attach a lightweight Dataflow pipeline to read this stream and pipe it directly into BigQuery.

The Result: The analytical data in BigQuery is only a few seconds behind production. The RTB cluster does not experience any CPU load from analytics because reading the stream is a background operation.

The FinOps Hack: Storing petabytes of raw bid logs in BigQuery is a fast way to get fired. We set up strict partitioning in BigQuery. We keep granular, row-level data for exactly 30 days in active storage. On day 31, an automated script aggregates the historical data into daily summaries and throws the massive raw logs into cold Cloud Storage.

The Legal Nightmare: Surviving GDPR

Now that the data flow is perfect, European lawyers enter the chat. Under GDPR, a user has the “Right to be forgotten.” They can demand that you delete their personal profile immediately.

How do you delete one specific user from a denormalized NoSQL database that has no secondary indexes? If you try to run a script to search for “User_12345” across petabytes of distributed shards, you will ruin your cluster’s performance.

We solve this not with heavy database queries, but with cryptography. It is a pattern called Crypto-Shredding.

Instead of writing raw user profiles into Bigtable, we encrypt the sensitive payload (like location, browsing history, and demographics) before it ever leaves our F# microservice. We use Cloud KMS (Key Management Service) to generate encryption keys for user segments.

When a European citizen exercises their right to be forgotten, we do not touch Bigtable at all. We simply go to Cloud KMS and permanently destroy their specific encryption key.

Instantly, in less than a millisecond, all of their personal data in Bigtable becomes mathematically unreadable. It is officially cryptographic garbage, fully compliant with GDPR regulations.

To keep our database clean, we don’t leave that encrypted garbage sitting there forever. We configure Bigtable’s native Garbage Collection feature (TTL – Time to Live) at the column-family level. Any record older than 90 days is automatically and silently purged by the database engines in the background, without any performance penalty.

Part 3: Surviving the Apocalypse, Paying the Bill, and Looking Ahead

If you have survived this far, your architecture is fast, analytically friendly, and legally bulletproof. But in the cloud, everything fails eventually. A marketing campaign goes viral, a network switch in a Google data center dies, or a junior developer deploys a malformed JSON payload.

Hope is not an engineering strategy. Let’s design the automated defenses that will keep your platform alive while you are asleep, and then look at the actual cost of running this monster.

Disaster Recovery: Designing for Chaos

Scenario 1: The Traffic Tsunami.

Suddenly, traffic spikes from 100,000 to 300,000 RPS. If you manually provisioned your Bigtable cluster to handle exactly 100k, the CPU will hit 100%, and the system will collapse.

The Solution: We implement native Bigtable Autoscaling. We set a strict rule: if the cluster CPU utilization crosses 70%, GCP automatically adds more physical nodes. When the traffic storm passes, it removes them.

The Trade-off: Autoscaling takes a few minutes to react. During those initial minutes of a sudden spike, latency will increase. You are trading a few seconds of slow responses for the guarantee that the database will not physically melt.

Scenario 2: The Database Hiccup (The Circuit Breaker).

What happens if Bigtable suddenly takes 60 milliseconds to respond instead of 10? If your F# microservices politely wait for the database, the incoming 100,000 RPS will quickly consume all available memory and threads in your GKE cluster. The entire Kubernetes environment will crash.

The Solution: We implement the Circuit Breaker pattern in our F# code. We set a hard timeout of 40 milliseconds. If Bigtable does not answer by then, the microservice aggressively cuts the connection. It stops trying to read the user’s profile and immediately sends a “default bid” (a safe, low-priced bid) back to the auction. It is better to lose a few cents on a default bid than to bring down the entire infrastructure.

Scenario 3: Poison Pills.

A partner sends a corrupted bid request with missing IDs. If your system tries to process it, it might throw an unhandled exception and crash the node.

The Solution: The Dead Letter Queue (DLQ). Before data enters the main pipeline, a validation layer checks the schema. If the data is broken, it is immediately thrown into a separate Pub/Sub topic called the DLQ. The main system stays healthy, and your engineers can inspect the DLQ the next morning to see what went wrong, rather than waking up at 3:00 AM to fix a crashed server.

FinOps: The Cold Hard Math

Architectures are beautiful on paper, but the finance department only cares about one thing: the monthly invoice. Let’s look at the reality of running this architecture.

• 10,000 RPS (The Startup Phase): You can run a minimal Bigtable cluster (1-3 nodes) and a small GKE cluster. The cost is manageable, roughly the equivalent of leasing a nice car.
• 100,000 RPS (The Enterprise Reality): This is where you need serious hardware. You will need a multi-node Bigtable cluster, high network egress fees, and a massive Pub/Sub throughput budget. You are now paying the equivalent of a monthly mortgage on a luxury house.
• 500,000 RPS (The Global Player): At this scale, you are spending hundreds of thousands of dollars a year. However, because Bigtable scales linearly, your cost per transaction actually becomes highly predictable.

Time to Build: This is not a weekend project. Building the F# microservices, configuring the Row Keys correctly, setting up the Change Streams to BigQuery, and testing the Circuit Breakers will take a team of three senior engineers at least three to four months.

The Future: Is This Forever?

Is this Bigtable architecture just a temporary patch, or is it a long-term foundation?

For AdTech, IoT logs, and high-speed analytics, this is a long-term foundation. Bigtable is designed to hold petabytes of data for years. As long as your queries are strictly key-based, this system will easily scale to a million RPS without requiring a rewrite.

When do you abandon this architecture?

There is exactly one scenario where you must destroy this system and start over: when the business demands strict, multi-region financial transactions. If, in the future, your company pivots from placing ad bids to processing actual credit card payments where a user in Paris and a user in New York must deduct money from the same exact bank account simultaneously without conflicts, Bigtable will fail you.

When you need absolute ACID compliance and relational integrity across the globe, you will have to pack your bags, say goodbye to Bigtable, and finally pay the massive premium for Cloud Spanner. But until that day comes, enjoy the speed.

Part 4: Day 2 Operations, The Logging Trap, and Deploying Without Crying

Welcome to “Day 2.” The architecture is deployed, the traffic is flowing, and the CEO is happy. Now comes the hardest part: keeping it alive when you actually need to update the code or debug an error. Running a 100,000 RPS system requires a fundamental shift in how you think about observability and deployments.

The Cloud Logging Trap (How to Burn $50,000 in a Weekend)

Let’s imagine a standard debugging scenario. A developer decides they want to see the incoming payload for every bid request. They add a simple logger.info(request) line in the F# microservice and push it to production.

At 100,000 requests per second, you are generating about 8.6 billion log entries per day. Google Cloud Logging charges by the gigabyte. If your JSON payload is just 1 kilobyte, you are writing 8.6 terabytes of text logs every single day. By Monday morning, the finance team will be standing at your desk asking why you just spent the entire quarterly budget on text files nobody will ever read.

When you operate at this scale, traditional logging is dead. You cannot log events; you must log metrics.

Observability: Metrics and Aggregation

Instead of writing text logs, we use Google Cloud Monitoring and OpenTelemetry. We don’t care about one specific bid; we care about the mathematical health of the entire system.

• Percentiles, not Averages: Never look at average latency. The average lies. If 99 requests take 1ms and 1 request takes 5000ms, the average looks fine, but one system just crashed. We monitor the p95 and p99 latencies. If the 99th percentile crosses 45 milliseconds, the system automatically triggers a critical alert.
• Trace Sampling: We still need to know where the time is spent (is the GKE pod slow, or is Bigtable slow?). We use distributed tracing, but we set a sampling rate of 0.01%. We only record the full journey of 1 out of every 10,000 requests. This gives us perfect statistical visibility into system bottlenecks without paying for petabytes of tracing data.

The CI/CD Minefield: Canary Deployments

How do you update the F# bidding algorithm when 100,000 requests are hitting it every second? You cannot just shut down the old version and start the new one. Even a 5-second downtime means dropping half a million auctions.

We use an advanced deployment strategy in Kubernetes called a Canary Release.

When the engineering team merges a new feature, the CI/CD pipeline builds the container and deploys it to the GKE cluster, but the load balancer is instructed to route exactly 1% of the total traffic to the new version. The other 99% continues to hit the old, stable code.

For the next ten minutes, an automated script watches the metrics of that 1%. Does the new F# code return errors? Did the latency jump to 60ms? Did Bigtable CPU usage suddenly spike?

• If any metric turns red, the load balancer instantly kills the Canary deployment and routes 100% of traffic back to the old version. The engineers get an alert, and no money is lost.
• If the metrics stay green, the pipeline slowly scales the new version: 10%, then 50%, and finally 100%.

Testing in production is a terrible idea—unless you only test on 1% of your traffic and have automated rollbacks. Then, it is called “modern engineering.”

Final Conclusion: The Tech Macro Philosophy

Designing architecture on Google Cloud Platform is not about blindly trusting managed services. It is about physics. Bigtable is a beast, but it requires extreme discipline with your Row Keys. Change Streams give you perfect analytics, but only if you actively manage your BigQuery partitions.

The cloud will gladly give you infinite scale, but it will also gladly send you an infinite bill. True engineering is finding the exact intersection between sub-millisecond latency, strict legal compliance, and aggressive cost control. Everything else is just marketing.

art 5: The Post-Mortem (Or How We Almost Bankrupted the Company)

Welcome to the most important phase of any engineering project: the reality check. Real software architecture is not about drawing perfect, undisputed diagrams on a whiteboard. It is about actively trying to break your own design before the production environment does it for you.

After finalizing the first four parts of this architecture, we sat down for a brutal, honest review of our own logic. And we found three critical design flaws. If we had deployed the original design exactly as written, we would have killed our database performance, burned through the quarterly budget, and probably updated our CVs.

Before you start writing F# code, you must apply these three structural patches to the architecture.

Bug 1: The Row Key Bullet to the Foot

The Mistake: In Part 1, we proudly engineered our Bigtable Row Key as [hash(user_id)]#[shard_id]#[reverse_timestamp]. We added a random shard_id to spread a single, highly active user across multiple nodes to avoid hotspotting. The Reality: We outsmarted ourselves. By adding a random shard, we completely broke the magic of the reverse_timestamp. If a user’s data is randomly spread across five different shards, the database no longer knows where their absolute newest bid is physically located. To find their latest event, our microservice would have to query all five shards simultaneously and merge the results. At 100,000 RPS, this instantly destroys our 50-millisecond latency budget. The Patch: Unless a single user is generating thousands of bids per second (which means they are a bot, and we should block them at the load balancer), sharding a single user is unnecessary and harmful. We drop the shard_id. The optimal key is simply [hash(user_id)]#[reverse_timestamp]. The hash distributes different users across the cluster, and the reverse timestamp guarantees that we can grab a user’s freshest data in a single, instant read.

Bug 2: The KMS Bankruptcy

The Mistake: In Part 2, discussing GDPR compliance, we stated that we would use Google Cloud KMS to generate encryption keys for user segments to enable “crypto-shredding.” The Reality: Google Cloud KMS charges a monthly fee for every single active key version. If you have 50 million European users and you generate a unique KMS key for each one, Google will send you an invoice for tens of thousands of dollars every month just to store the keys. You will go bankrupt before a single user ever asks to be forgotten. The Patch: We must use Envelope Encryption (via a library like Google Tink). We create exactly one Master Key (KEK) in Cloud KMS. Then, our F# microservice generates a free, local Data Encryption Key (DEK) for each user. We encrypt the user’s Bigtable profile with the local DEK, and then we encrypt the local DEK itself using the KMS Master Key. When a user requests deletion under GDPR, we simply delete their encrypted DEK from our metadata table. The data in Bigtable turns into cryptographic garbage, and our monthly KMS bill remains exactly a few cents.

Bug 3: The Autoscaling Illusion

The Mistake: In Part 3, we claimed that if a traffic tsunami hits and we jump from 100k to 300k RPS, native Bigtable Autoscaling would save us by automatically adding nodes when CPU hits 70%. The Reality: Bigtable is a massive distributed database, not a lightweight Cloud Run container. Provisioning new physical nodes and rebalancing the data (moving tablets between nodes) takes anywhere from 15 to 20 minutes. If traffic triples in one second, Autoscaling will simply watch your database burn to the ground for 20 minutes while it tries to rebalance. The Patch: Autoscaling is fantastic for predictable, slow daily trends (like traffic rising at 8:00 AM and dropping at midnight). It will not save you from a sudden spike. To survive a true traffic tsunami, you only have two options. First, Overprovisioning: you must always pay for an extra 30% of unused CPU capacity just to absorb sudden shocks. Second, Load Shedding: you must configure strict Rate Limiting at your API Gateway or GKE ingress. It is infinitely better to successfully process 100,000 requests and aggressively drop the remaining 200,000 than to let 300,000 requests into the system and crash the entire company.

Consider yourselves warned. Copy-pasting architecture is easy; surviving production is hard.

As data systems evolve, they naturally accumulate architectural debt, leading to fragile pipelines and escalating cloud costs. Before applying superficial fixes or adding new tools, the most effective step is a methodical, engineering-first review of your current setup. My GCP Architecture Assessment & Modernization Roadmap is designed to deeply diagnose your infrastructure, isolate bottlenecks, and trace data lineage without any marketing noise. You will receive a prioritized, objective blueprint for building idempotent, mathematically sound data systems on Google Cloud, complete with an honest breakdown of all technical compromises. If you are looking for a calm, rigorous approach to stabilize your data ecosystem, I invite you to explore the details of the assessment.