Retail Markdown Optimization: Dynamic Clearance Pricing Algorithms

Introduction: The Context of the Problem

Every season, fashion retail faces the same financial disaster: the clearance sale. When the season ends, warehouses are full of unsold clothes. To get rid of them, retailers usually press a single red button and apply a flat 30% or 50% discount to everything.

This approach is a massive problem. It is a linear solution applied to a highly non-linear reality. By discounting everything at once, the business effectively burns its own money.

Why is this important? Because retail profit margins are already incredibly thin. When you apply a “blind” 50% discount to a popular size (like Medium) that would have easily sold at a 10% discount, you are destroying your profit. At the same time, the unpopular sizes (like Extra Small or Extra Large) will not sell even at a 70% discount, meaning they will just sit in the warehouse and cost money to store.

This article is designed for data engineers, technical architects, and retail analysts who want to stop guessing and start calculating. We will explore how to build a dynamic pricing engine that calculates the exact discount curve for a specific item, in a specific store, to sell the exact remaining stock right before the season ends.

The Essence of the Task

From the perspective of Operations Research, this is a classic Revenue Management problem. You have a limited resource (the physical inventory) and a hard deadline (the end of the season).

The goal is not just to sell everything. The goal is to maximize the total money earned while making sure the inventory reaches zero by a specific date.

To solve this, we cannot look at the product as a whole. A “blue winter jacket” is not one product. A blue winter jacket in size Medium in a store in the city center is an entirely different product than the same jacket in size XXL in a suburban mall. They have different demand curves, and they need different prices.

The Mathematical Foundation

Before we look at the software solutions, we must understand the core mathematics. Every solution below is based on the principles of a Markov Decision Process (MDP) and the Bellman equation.

Do not worry, the logic is very practical. At any given moment, the algorithm looks at the time left until the end of the season (T), the current stock level (I), and a list of allowed discounts (for example: 0%, 10%, 20%, 30%).

The goal is to find the maximum expected revenue (V) for our current stock and time.

The formula looks like this:

V(time, Inventory) = Maximum of [ Price(discount) * Expected_Sales(discount) + V(time + 1, Inventory – Expected_Sales(discount)) ]

In simple words: The algorithm calculates if it is better to take a small amount of money today (by offering a big discount) or wait and hope to sell the item for a higher price tomorrow, knowing there is less time left.

Solution 1: The Sell-Through Rate (STR) Heuristic

This is the simplest and cheapest way to stop using blind discounts. Instead of trying to predict the future with complex artificial intelligence, we simply build a system that reacts to how fast items are selling right now. This speed is called the Sell-Through Rate (STR).

The Logic

The business sets a target STR for each week. For example, by week 4 of the season, 40% of the total stock should be sold. The system checks the database daily. If a specific item has only sold 20% (it is moving too slowly), the script automatically drops the price one step down (e.g., a 10% discount). If the item is selling faster than the target, the price stays at full retail.

Technology Stack

You do not need a complex backend for this. The entire logic can live inside your data warehouse. You can use BigQuery as the storage engine and dbt (Data Build Tool) with standard SQL to write the daily check rules.

Cost and Time to Implement

• Cost: Minimal. You only pay for the data processing queries in BigQuery, which is usually under $500 a month for a mid-sized retailer.
• Time: 2 to 4 weeks.

Real-World Behavior

This solution works perfectly for basic, predictable items like plain white t-shirts or socks. However, it reacts slowly. It only drops the price after the item has already failed to sell. It also does not understand outside factors. The algorithm might drop the price of winter boots in July, thinking they are unpopular, without realizing it is simply too hot outside.

Ease of Adaptation

Extremely easy. Business managers understand SQL rules and target percentages. If the logic is wrong, a data engineer can fix the dbt model in ten minutes.

Probability of Scenario-Solution Success

85%. If a retail company is currently doing all discounts manually in Excel, this simple automated database logic will immediately save them millions and is very likely to be successfully deployed.

Solution 2: Classic Dynamic Programming (Backward Induction)

When rules and target percentages are no longer enough, we move to pure mathematics. This approach solves the Bellman equation directly using a method called Backward Induction. Instead of guessing the future, the algorithm starts at the last day of the season (when the leftover inventory is worth zero or goes to recycling) and calculates the optimal price backwards to the current day.

The Logic

We assume that customer demand follows a mathematical distribution (like the Poisson distribution) depending on the price. The algorithm creates a massive grid of all possible situations: (Every possible inventory level) multiplied by (Every remaining day in the season). It calculates the best price for each box in this grid to maximize the final money earned.

Technology Stack

This is a heavy mathematical task. You should not run this in Python or SQL. This requires a fast, strongly typed backend language capable of parallel processing. A microservice written in F# hosted on Google Cloud Run is perfect here. F# handles the complex tree-search algorithms beautifully, while Cloud Run scales automatically to calculate thousands of products at the same time. Google BigQuery is used to feed the historical sales data into the calculation.

Cost and Time to Implement

• Cost: Medium. You are paying for CPU time. Running intensive F# math on Cloud Run for a large catalog will cost around $1,500 to $3,000 a month in Google Cloud billing.
• Time: 2 to 3 months.

Real-World Behavior

Mathematically, this solution is beautiful. Practically, it suffers from the “curse of dimensionality.” If you try to calculate this matrix for every specific combination (Product + Size + Specific Store Location), the number of calculations grows exponentially. A calculation that should take ten minutes can suddenly take ten hours. It also struggles if a product has never been sold at a discount before, because the algorithm has no historical data to calculate price elasticity.

The Anti-Pattern: The Mega-Matrix

The biggest mistake engineers make here is trying to compute everything globally. Do not calculate the path for “Store 452”. Group your stores into clusters (e.g., “Premium City Malls” or “Outlet Centers”) and calculate the dynamic programming grid for the cluster. Otherwise, your cloud bill for compute time will destroy any extra profit the algorithm generates.

Probability of Scenario-Solution Success

60%. It is a solid mathematical step, but it often breaks down in the real world when demand suddenly changes because of factors the math does not see (like a sudden change in weather).

Solution 3: Machine Learning + Deterministic Optimizer (The Champion)

This is the industry standard for enterprise retail. We divide the complex problem into two separate, manageable parts: an AI predicts what will happen, and a mathematical engine decides what to do about it.

The Logic

1. The Demand Predictor: A machine learning model predicts the demand curve. The formula is:Expected Sales = Function(Price, Size, Days Left, Weather Forecast, Competitor Price).
2. The Solver: A deterministic mathematical optimizer takes that expected demand and calculates the exact price needed to hit zero inventory on the last day, while maximizing the profit margin.

Technology Stack

We need a robust data pipeline.

• Machine Learning: Google Vertex AI Pipelines running Python (using XGBoost or LightGBM) to train the demand model every week.
• The Optimizer Engine: A high-performance API written in F# running on Cloud Run. It downloads the weights from Vertex AI, looks at the current inventory, and calculates the optimal prices daily.
• Serving Layer: The final calculated prices are pushed into a fast NoSQL database like Google Firestore or Amazon DynamoDB. The physical cash registers in stores and the e-commerce frontend read prices instantly from this database.

Cost and Time to Implement

• Cost: High. You are running ML training pipelines, hosting inference endpoints, and running solvers. Expect $5,000 to $10,000 a month in cloud infrastructure costs.
• Time: 4 to 6 months of dedicated data engineering and data science work.

Real-World Behavior

This solution is incredibly adaptive. Because the ML model looks at outside factors, it knows that nobody buys winter coats in a warm November, even with a 20% discount. The optimizer then knows it must drop the price harder now, rather than waiting for December when it is too late.

The Anti-Pattern: Ignoring Ghost Inventory

The most dangerous anti-pattern is trusting your inventory data blindly. If the database says there are five shirts in the store, but they were actually stolen or lost (Ghost Inventory), the ML model sees that the shirts are not selling. The optimizer thinks the price is too high and drops it to zero. You must build strict dbt (Data Build Tool) data quality tests to exclude items that have not registered a single scan in 30 days before feeding the data to the algorithm.

Probability of Scenario-Solution Success

90%. As long as your historical data is reasonably clean, this architecture delivers massive financial returns and is highly reliable in production.

Solution 4: Deep Reinforcement Learning (DRL)

This is the cutting edge of artificial intelligence, often discussed at tech conferences but rarely surviving in actual retail environments. Here, we build a simulator of the retail market and let an AI “agent” play it like a video game.

The Logic

Using algorithms like Proximal Policy Optimization (PPO), the agent looks at the state of the warehouse. It tries applying different discounts in the simulation. If the simulation results in high profit and zero remaining stock, the agent receives a “reward.” Over millions of games, it learns the ultimate pricing strategy.

Technology Stack

This requires massive parallel processing. You need frameworks like Ray RLlib and TensorFlow, running on clusters of Google Compute Engine instances equipped with heavy GPUs (like NVIDIA T4 or A100).

Cost and Time to Implement

• Cost: Extreme. Renting GPU clusters for continuous reinforcement learning can easily exceed $15,000 a month.
• Time: 9 to 12+ months. This is basically a corporate research project.

Real-World Behavior

It acts like a black box. The agent might discover a mathematical loophole in your data and decide to discount your best-selling premium shoes by 90% on the first day of the season. It might actually be a brilliant long-term mathematical move to attract foot traffic, but your merchandising directors will have a heart attack and shut the system down immediately.

The Anti-Pattern: No Guardrails

Giving a DRL agent direct control over production pricing without strict business limits is professional suicide. If you must build this, you need a “Safety Layer”—a hardcoded script that says, “No matter what the AI suggests, the maximum allowed discount in week one is 15%.”

Probability of Scenario-Solution Success

15%. It is over-engineered for the problem. The cost of development and compute power usually destroys the extra margin it creates.

Summary Comparison: The Matrix of Success

To make a final architectural decision, we must look at all four solutions side-by-side. We evaluate them based on cost, adaptability to strange events (like sudden weather changes), implementation risk, and the realistic probability of success (Scenario-Solution Probability).

Solution	Complexity & Cloud Cost	Adaptability to Anomalies	Business Risk	Probability of Success
1. STR Heuristic	Low (Under $500/mo)	Very Low	Low (Easy to understand)	85% (Best Quick Win)
2. Dynamic Programming	Medium ($1,500 – $3k/mo)	Medium	Medium	60%
3. ML + Optimizer	High ($5k – $10k/mo)	Maximum	Medium (Requires clean data)	90% (The True Champion)
4. Deep RL	Extreme ($15k+/mo)	High	Critical (Black Box behavior)	15% (Science Project)

Critical Bottlenecks and Trade-Offs

Even the Champion architecture (ML + Optimizer) will fail if you ignore the specific physics of retail. When designing the system, you must engineer trade-offs to handle the following bottlenecks.

1. The Broken Size Run

This is the biggest risk for algorithmic pricing. Imagine you have a jacket, but the only size left in the store is Extra Small (XS). If you apply the ML elasticity formula, the algorithm will see that sales have dropped to zero. It will assume the price is too high and drop it aggressively. But the price is not the problem; the problem is that people who wear size XS are rare. Demand is zero because the target audience is missing.

• The Trade-off: The algorithm must group inventory at the “Colorway” level (all sizes of the blue jacket together) to calculate demand, but it must apply a mathematical penalty coefficient if the size run is “broken” (missing sizes M and L).

2. Price as a Quality Signal

In the fashion industry, price is a signal of quality. If you apply a 70% discount to a premium item too early, you can actually kill the demand. The consumer starts to think, “This item must be defective,” or “This brand is losing its premium status.” A pure mathematical solver does not understand human psychology. It only sees numbers.

• The Trade-off: You must hardcode minimum price floors based on the brand tier. Premium brands should never fall below a 40% discount, even if the math says a 60% discount will clear the inventory faster.

3. Cannibalization (Cross-Price Elasticity)

If the optimizer decides to drop the price of a blue t-shirt by 40%, it will sell out quickly. However, the sales of the exact same t-shirt in black (which is still at full price) will suddenly drop to zero. The blue shirt “cannibalized” the sales of the black shirt. Calculating the exact mathematical cross-elasticity for millions of product combinations requires too much CPU time and will explode your cloud compute bill.

• The Trade-off: Do not calculate cross-elasticity inside the heavy ML model. Instead, apply a lightweight “business rule” filter at the very end of the pipeline: Rule: Products in the same family cannot have a price difference greater than 20%.

The Architect’s Recommendation Block: Actionable Strategy

If you are leading the technical transformation for a retail brand, here is the exact step-by-step strategy you should recommend to the business.

1. Do not build the Champion immediately.

Never start by building the complex ML + Optimizer (Solution 3). Your first goal is to build the delivery pipeline. Start with the basic SQL-based STR Heuristic (Solution 1). Spend your first month proving that a script can successfully change a price in the central database, push that price to the physical cash registers, and update the website price tags in under 15 minutes. If your basic data pipes are slow or broken, an advanced AI engine is completely useless.

2. Stop discounting the SKU; start discounting the Barcode.

Change the business mindset. Retailers traditionally discount the entire SKU (Stock Keeping Unit). You must re-architect the database to apply discounts at the exact Barcode (EAN/UPC) level. Size M is a different barcode than Size XXL. Size M should sell at full retail price. Size XXL should be discounted by 30%. This single architectural change often increases clearance profit margins by 10% to 15% before you even introduce machine learning.

3. Isolate the Mathematics.

Decouple your architecture. Do not build the mathematical optimizer inside your ERP system or your e-commerce backend. Build the optimization engine as an isolated, stateless microservice written in a fast language like F# (hosted on Google Cloud Run or AWS ECS). It should have only one job: take an input file of inventory, and output a file of prices. By doing this, if you decide to change your ML model from XGBoost to a Neural Network two years from now, the F# solver does not care. It just takes the new numbers and continues maximizing the integral.

4. Data Quality is more important than the Algorithm.

No dynamic programming equation can save you if the physical stores have terrible inventory accuracy. This is known as “Ghost Inventory.” The database says there are 10 items on the shelf, but they were actually stolen three months ago. The algorithm will see that the items are not selling, and it will aggressively lower the price of an item that does not exist.

Before feeding any data into your pricing engine, you must use a tool like dbt (Data Build Tool) to write a strict filter: If an item has an inventory count greater than zero, but has not registered a single scan, return, or movement in 30 days, flag it as Ghost Inventory and remove it from the pricing model.

Closing Thought

Clearance optimization is not a magical AI project. It is a strict mathematical logistics problem. Start small, clean your data, isolate your calculation engine, and never let the algorithm forget the basic physics of the retail floor.