Garbage In, Garbage Out: We Fed AI With Dirty Data So You Don’t Have To

You can hire the best data scientists, create GPU clusters with outmatched performance, and deploy the most advanced language models… but if the data you’re feeding your system with is wrong, the results will be wrong. With AI, the results become wrong at scale and at speed.

The “Garbage in, garbage out” principle has never been more literal than today. Modern analytics and AI pipelines turn bad inputs into costly, detrimental decisions, such as mispriced products, misrouted shipments, or chatbots that give users wrong advice.

To demonstrate this, we ran several experiments to show you how minor mistakes can render the dataset useless and the insights unreliable.

What you’ll learn

• Spot hidden data landmines fast: run five tests and watch charts expose silent errors.
• See how bad data drains value: tie each failure to real losses: skewed pricing, false predictions, and user mistrust.
• Learn how to deal with garbage: get actionable tips to make the most of your data.

The Real Cost of “Garbage In, Garbage Out” in AI Systems

Bad data has always carried a price tag, but the numbers are now more forbidding than ever.

Gartner still pegs the direct hit at $12.9 million per company, per year, a figure that hasn’t moved since before the GenAI boom, meaning it is now a floor, not a ceiling.

Unity Software is a great example of how a lack of data observability and a corrupted dataset led to significant damage. The firm’s ad targeting tool ingested one corrupted file that led to inaccurate target profiles and incorrect ad placements. The company suffered $110 million revenue loss.

Revenue leaks are only one side of the story. Trust evaporates even faster than cash. PwC’s 2024 Trust Survey shows a 60-point gap between how much executives think customers trust them (90%) and how much customers actually do (30%). This gap is widening with each year and mishandling data with AI will only accelerate this process.

LLMs open new possibilities to disrupt trust, such as misscoring loyal shoppers, suggesting content that doesn’t appeal to users, or censoring someone over a comment that meant no harm. With AI being applied to more and customer-facing features, it’s essential to train it to perfection, and doublecheck the training data for anomalies or false entries.

Regulators are no longer waiting for the next breach. Italy’s Garante fined OpenAI €15 million for training ChatGPT on personal data without a lawful basis. The Dutch DPA hit Clearview AI with €30.5 million for a similar offence: scraping faces without consent. These are no-audit, no-appeal numbers that land because the training data could not be proven clean.

Forrester adds that one in four analytics teams now burn $5 million or more every year just keeping low-grade data from breaking production models. This money never reaches feature work or user value.

Whatever weaknesses lurked in the raw data are now multiplied with the emergence of AI pipelines. Money, trust, compliance, and speed all collapse on the same fault line: GIGO. And the fix is no longer optional.

Hands-On Experiments That Make GIGO Impact Obvious

Amid the noise surrounding AI, companies and thought leaders share contradictory takes. Some say that data is crucial and without it, you can’t expect production-ready outputs. Others claim that the data issue isn’t that serious and creating a simple RAG system doesn’t require much data work.

To get to the root of the matter, we decided to feed AI with dirty data and see what happens firsthand. To expose the real-world cost of garbage in, garbage out, we designed five data tests.

Each experiment is based on a clean dataset and a corrupted dataset. We run those datasets through GPT-o3 with advanced data analysis (code interpreter) to see how it reacts and what results it outputs.

Experiment 1: Numeric Analysis Trap + Schema Typo

We started the research with a simple experiment. Our dataset contains the following fields:

• employee_id
• month
• units_sold
• unit_price_usd.

The clean dataset has regular data, with the number of units sold ranging from 32 to 55. However, a single row from our corrupted dataset has 5000 units sold, which is 11 times more than the dataset's average, disrupting the holistic revenue picture. We also made a small typo in the dataset schema. The fourth column is now called “unit_pce_usd”.

This numeric trap is a textbook case of garbage in, garbage out skewing revenue metrics.

Let’s feed the clean dataset to GPT-o3, ask it to calculate total revenue per employee, and display the results on a chart:

In our clean dataset, the revenue per employee is nearly the same across all three employees. Now, let’s see how GPT processes the corrupted dataset:

GPT fails to spot the anomaly and processes the dataset as it is. As a result, employee 3 generates 30 times more revenue than the other employees. However, it successfully corrects the schema typo, so it doesn't affect the final calculation.

If we send GPT a follow-up request to spot anomalies in the dataset, it notifies us that one value is more than five times higher than the median and suggests checking for a misplaced decimal or an extra zero.

Verdict: Test failed ❌

Another reminder that GIGO can ruin analytics.

Even though GPT easily fixed the schema typo, it failed to identify the revenue anomaly during the initial processing. This leads to compromised final analysis results. Alternatively, you can ask GPT to check for anomalies in the dataset, but without access to the source data, it can’t understand whether it’s truly an anomaly or valid data. Additional API costs add up with every single action you want AI to perform. This slows down the response and creates more room for AI to hallucinate.

A simple fix

An alerting system connected to your data warehouse could easily solve this problem at a fraction of the cost compared to verifying the credibility of data using AI. Using these systems, you can create a rule (e.g., revenue per product should be less than or equal to $200) that the system will check with every new data entry and notify you whenever this rule is violated. This basically costs zero dollars and ensures that your data stays clean.

Experiment 2: How Wrong Data Labelling Deteriorates the Output of Review Classifier

In this run, we investigated what happens when customer sentiment is wrongly labeled.

For context, e-commerce teams often infer sentiment from the numeric score that sits next to the text, with one and two stars indicating negative, and four and five indicating positive. Everything rated three stars is either thrown out or, worse, pushed into whichever bucket keeps class sizes even. However, this is completely wrong, as three- or even four-star reviews aren’t always positive, especially when they come with a written review, which often holds more context.

Everyone is now chasing context from unstructured data, so we created two datasets to show how easily it can be skewed. The first contains correctly labeled reviews, and the second contains mixed labels. In the corrupted dataset, some reviews that are clearly positive might be labeled negative or neutral.

Mislabelled sentiment is simply GIGO dressed as customer insight.

We feed the clean file first and ask GPT to process it and train a miniature sentiment classifier. Nothing fancy, it simply learns which reviews sound positive, neutral, or negative. Then we do the same with the corrupted file. We split the datasets into 80 percent training data and 20 percent test data.

After processing the dataset, we ask our LLM to create a confusion matrix for both datasets. Here are the results:

For clean data, the matrix is flawless and actual labels match the predicted labels. However,  for the corrupted dataset, the accuracy of sentiment identification falls to 75 percent. Negatives now look linguistically positive to the model, so two of four are misread as positive. Some neutral examples drift into negative.

This means that on average, one out of four review sentiments will be identified incorrectly.

Verdict: Test failed ❌

There is nothing that LLM can do in this situation because the problem is the faulty data that it was trained on.

However, this issue can be avoided with simple annotation hygiene.

How to avoid mislabeling:

• For critical data, have two or more people label your data. Accept the label only if they agree on a uniform label for the data.
• Seed every labeling batch with benchmark items whose correct label is already known. By comparing each annotator’s answers against that ground truth, you can spot anyone with a low hit rate and send their whole batch (or just their mistakes) back for review.
• Create a short style guide that spells out exactly what counts as positive, neutral, or negative, backed by real borderline sentences. When annotators hit an ambiguous review, they consult those examples, so everyone makes the same call and the label set stays consistent.

Experiment 3: Incorrect date format

Our next step is to see how LLMs handle incorrect date formats.

As usual, we have two datasets. Each contains just two columns:

• Date
• Website sessions

The corrupted dataset includes dates in mixed formats, namely yy-mm-dd and mm/dd/yy.

We feed both datasets to our LLM and ask it to “Plot a 7-day moving-average of sessions over time and identify the busiest week”. AI executes the task flawlessly.

GPT automatically normalises all dates to yy-mm-dd format, and drops any rows it can’t read unambiguously.

With both datasets, we get the same chart that looks like this:

Verdict: Test passed ✅

Strict parsing saved us from a silent garbage in, garbage out error here.

Experiment 4: Hidden Sentinel Values

Property data is notorious for placeholders. County assessors often record an unknown lot size as 0 until a survey arrives. That placeholder looks innocent in a spreadsheet, but when you feed it to a model it behaves like a real number and drags every coefficient down with it.

The essence of our experiment stays the same: two datasets, one clean and one corrupted.

Both datasets contain four columns:

• house_id
• Bedrooms
• Lot_sqft
• Price_usd

The only difference is that the corrupted dataset has zeros in the “Lot_sqft” which illustrate a stealthier form of garbage in, garbage out.

For each file, we ask GPT to:

“Predict price_usd from bedrooms and lot_sqft, test it on a 20% hold-out, and calculate the R-squared score.”

---

💡 R-squared measures how well the model's predictions match the actual data points.

---

Unfortunately, the model fails to act on zero values in the dataset. The LLM treats them as regular rows, which worsens the quality of the final output.

On the clean file, the model achieves an R² of 0.964. The same request on the corrupted file causes R² to drop to 0.763. The model thinks a sizable share of homes have zero land and adjusts downward, so even real 2,500 sq ft homes look overpriced.

Verdict: Test Failed ❌

Why this matters in production

Real-estate portals feel this pain daily. Real-estate blogs document county records listing five-acre lots as two and “city parcels” as zero until paperwork catches up, skewing automated valuations by tens of thousands of dollars. Academic work on regression accuracy shows that unflagged missing values bias coefficients and erode predictive power. Proper NaN handling or imputation is the first fix in every textbook. Yet industry datasets still ship with sentinel codes because legacy systems can’t store blanks.

Experiment 5: Imbalanced data fooling predictions

This time, the bad file is not corrupted at the cell level. Instead, its class distribution is skewed beyond recognition: 270 positive reviews, 15 neutral, 15 negative. A model can earn around 100% accuracy by parroting “positive” every time and still miss almost everything that matters.

Class skew is yet another pathway to garbage in, garbage out.

First, we feed the model a balanced dataset, ask it to split 80/20, run a quick training pass to decide whether each review is positive, neutral or negative, and then print the accuracy for each class.

With an even dataset the accuracy scores look healthy for each category (negative 0.18, neutral 0.45, positive 0.39).

Next, we upload the imbalance dataset and repeat the exact same request. The positive accuracy leaps to ~90 %, yet the precision for neutral and negative collapses toward zero. The confusion matrix turns into a solid wall in the “positive” column, proof that the model has learned the majority class and nothing else.

Verdict: Test Failed ❌

Why does this matter outside this demo?

Ignore balance and precision, and you invite garbage in, garbage out on every prediction.

Because most production datasets are unbalanced. Fraud records account for well under 1% of credit card transactions, but a classifier that rubber-stamps every purchase as “legit” will still show 99 % accuracy while allowing criminals to stroll through the gate.

Google’s own ML Crash Course says:

For heavily imbalanced datasets, where one class appears very rarely, say 1% of the time, a model that predicts negative 100% of the time would score 99% on accuracy, despite being useless.

The takeaway is that a single metric can lie.  Unless you surface class-level precision and balance or re-sample your training data, AI will optimise for the majority, congratulate itself, and quietly ignore everything that costs the enterprise money or exposes it to risk.

How to Turn Garbage into Valuable Data

AI projects fail when data pipelines drift, labels rot, or sensitive fields leak. Closing those gaps is less about moon-shot tech and more about methodical work across six pillars that every CIO can start reinforcing today.

Treat data as a product

Inventory the datasets that drive revenue decisions, assign a named owner to each, and publish refresh rates and usage metrics on the CIO’s dashboard. When teams see a dataset’s “uptime” and business value every quarter, duplication plunges and stale spreadsheets disappear.

Unlock the 80% of data that’s unstructured

Call recordings, chat logs, PDFs, and images hold the context . Store them in object storage, auto-transcribe, run entity extraction with an LLM, and pin embeddings to a vector index. Tie every asset back to the source data so structured and unstructured views converge in one query.

Build a connected data fabric

Come up with a single semantic layer. Map “lead,” “prospect,” and “opportunity” to the same definition, then wrap it in role-based APIs so analysts and LLMs fetch data by business term, not table name. Consistent meaning eliminates silent joins that wreck model features.

For more tips on avoiding garbage and getting best output from LLMs models read our article on six pillars of data readiness for AI.

Close the AI Data Readiness Gap with Vodworks

If messy data or AI talent shortage are holding your AI programs back, Vodworks can help you close those gaps, assess your data infrastructure, and build a tailored roadmap to prepare your organization for further AI efforts.

Our team starts with a focused workshop where we identify AI use cases with quick payback periods, then runs a structured AI readiness assessment that inspects data quality, infrastructure fitness, and current state of data governance.

You receive a maturity scorecard, a gap-by-gap action plan, and realistic cost and timeline estimates, everything needed for the board to green-light next steps.

If you choose, the same specialists can stay on to implement the fixes, from data architecture and cleansing to ML Ops and compliance, so your first production model ships on schedule and under control.

Book a 30-minute discovery session with our AI solution architect. During the session, we’ll discuss the current state of your data estate, AI use cases you plan to implement, and define the next steps based on your data maturity.