Skip to content
Build an AI Agent That Finds Contradictions Across Website Pages
Article

Build an AI Agent That Finds Contradictions Across Website Pages

AI & Machine Learning

Build a workflow that pulls pricing, plan and policy claims from your own website, flags pages that may disagree and sends each case to a person with the evidence.

By MrScraper Team 14 min read

Large websites drift. The pricing page says the trial lasts 14 days. An old help article still says 30. A plan comparison table lists a feature that the docs say is Enterprise only.

Nobody wrote the wrong thing on purpose. Pages were updated at different times by different teams. But customers notice, and support teams spend time explaining the gap.

In this tutorial, you will build a workflow that finds these cases on your own website. It collects claims from your pages, finds claims that may disagree and sends each one to the right person with both quotes side by side.

One rule shapes the whole design: the workflow flags, and a person decides. It never declares which page is correct. Two different values can both be right, and only someone who knows the product can tell.

Run this on a site your team owns, or one you have permission to check.

Which claims should you compare?

Not every sentence is worth comparing. Focus on claims that are specific, that customers act on and that appear on more than one page.

Good claim types

Topic Example claim Where it often appears
price "Team costs $12 per user per month" Pricing page, plan docs, FAQs
plan_limit "Starter includes 3 projects" Pricing table, help articles
feature_availability "SSO is available on Business" Pricing table, security docs
trial_length "Start a 14-day free trial" Home page, signup page, FAQ
refund_window "Refunds within 30 days of purchase" Terms, billing help, FAQ
data_retention "Logs are kept for 90 days" Security docs, plan docs
support_hours "Support is available 24/7" Contact page, plan page

Claims to skip

  • Opinions and marketing language, like "the easiest tool for teams"
  • Claims that only appear once, since there is nothing to compare
  • Blog posts and release notes, since they describe a moment in time Start with three or four topics. You can add more once the review process works.

How do you extract claim and evidence pairs?

Each claim needs two parts: the structured fact, and the exact sentence it came from. The structured part lets code compare claims. The quote lets a reviewer see what the page actually says.

Find the pages to check

The MrScraper Scrape Sitemap endpoint collects URLs from a starting page. Use includePatterns to keep the sections where claims live.

python
import json
import os
import requests
from datetime import datetime, timezone
from urllib.parse import urlparse
 
MRSCRAPER_URL = "https://api.mrscraper.com/"
TOKEN = os.environ["MRSCRAPER_API_TOKEN"]
SITE = "www.example.com"   # Your own site
 
 
def call_mrscraper(body: dict, **params) -> dict:
    response = requests.post(
        MRSCRAPER_URL,
        params={"token": TOKEN, **params},
        headers={"x-api-token": TOKEN, "Content-Type": "application/json"},
        json={**body, "maxRetries": 2},
        timeout=330,
    )
    response.raise_for_status()
    return response.json()
 
 
def find_pages() -> list[str]:
    result = call_mrscraper({
        "url": f"https://{SITE}/",
        "agent": "map",
        "maxDepth": 3,
        "maxPages": 10,
        "limit": 300,
        "includePatterns": [".*/(pricing|plans|help|docs|security|legal|faq).*"],
    })
    urls = (result.get("data") or {}).get("urls", [])
    return [u for u in urls if urlparse(u).netloc == SITE]

Extract claims with a schema

The MrScraper Scrape JSON endpoint takes a prompt and a JSON schema and returns structured data. The response also includes the page html by default, which you will use to check each quote.

python
TOPICS = ["price", "plan_limit", "feature_availability", "trial_length",
          "refund_window", "data_retention", "support_hours"]
 
CLAIM_SCHEMA = {
    "type": "object",
    "properties": {
        "page_updated": {"type": ["string", "null"]},
        "claims": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "topic": {"type": "string", "enum": TOPICS},
                    "subject": {"type": "string"},
                    "attribute": {"type": "string"},
                    "value": {"type": "string"},
                    "conditions": {"type": ["string", "null"]},
                    "quote": {"type": "string"},
                },
            },
        },
    },
}
 
PROMPT = (
    "List every specific factual claim on this page about these topics: "
    + ", ".join(TOPICS) + ".\n"
    "subject is what the claim is about, such as a plan name or product.\n"
    "attribute is what is being stated, such as 'monthly price' or 'trial length'.\n"
    "conditions holds any limits on the claim, such as 'billed annually', a region or a date.\n"
    "quote must be the exact sentence from the page.\n"
    "If the page shows a last-updated date, put it in page_updated as YYYY-MM-DD.\n"
    "Json Schema:\n" + json.dumps(CLAIM_SCHEMA)
)
 
 
def extract_claims(url: str) -> dict:
    return call_mrscraper({"url": url, "prompt": PROMPT, "agent": "general"})

Check every quote and save the claims

Only claims with a quote that really appears on the page go into the comparison. This uses a small HTML-to-text helper that keeps inline tags like <b> from breaking a sentence apart.

python
from html.parser import HTMLParser
 
BLOCK_TAGS = {"p", "div", "li", "br", "tr", "td", "th", "h1", "h2", "h3", "h4", "section"}
 
 
class TextOnly(HTMLParser):
    def __init__(self):
        super().__init__()
        self.parts, self.skip = [], False
 
    def handle_starttag(self, tag, attrs):
        if tag in ("script", "style"):
            self.skip = True
        elif tag in BLOCK_TAGS:
            self.parts.append(" ")
 
    def handle_endtag(self, tag):
        if tag in ("script", "style"):
            self.skip = False
        elif tag in BLOCK_TAGS:
            self.parts.append(" ")
 
    def handle_data(self, data):
        if not self.skip:
            self.parts.append(data)
 
 
def normalize(text: str) -> str:
    return " ".join((text or "").split()).lower()
 
 
def page_text(html: str) -> str:
    parser = TextOnly()
    parser.feed(html or "")
    return normalize("".join(parser.parts))
 
 
def collect_claims(urls: list[str]) -> list[dict]:
    claims = []
    for url in urls:
        result = extract_claims(url)
        text = page_text(result.get("html", ""))
        data = result.get("data") or {}
        for c in data.get("claims", []):
            if c.get("quote") and normalize(c["quote"]) in text:
                claims.append({
                    **c,
                    "url": url,
                    "page_updated": data.get("page_updated"),
                    "retrieved_at": datetime.now(timezone.utc).isoformat(),
                })
    return claims

Save the claims to a file or database. Each run gives you a dated snapshot you can compare with the next one.

How do you separate contradiction from changed context?

This is the hard part. Two pages can show different values and both be correct:

  • "$12 per user per month, billed annually" and "$15 per user per month, billed monthly"
  • "Logs kept for 90 days" on the Team plan and "1 year" on Enterprise
  • "Refunds within 14 days" for one region and "30 days" for another
  • A price that changes on a future date, shown on a page that says so If the workflow flags all of these, reviewers will stop trusting it. So it must compare only claims that are truly about the same thing.

Step 1: Match claims that are about the same thing

Two claims match when they share a topic, a subject and an attribute. Pages name things in different ways, so keep a small alias list in config.

python
import re
 
ALIASES = {
    "pro": "professional", "pro plan": "professional", "professional plan": "professional",
    "team plan": "team", "teams": "team",
    "per month": "monthly price", "monthly cost": "monthly price", "price per month": "monthly price",
}
 
 
def canon(text: str) -> str:
    t = normalize(text).strip(" .")
    return ALIASES.get(t, t)
 
 
def claim_key(c: dict) -> tuple:
    return (c["topic"], canon(c["subject"]), canon(c["attribute"]))

A model can suggest new aliases when it sees names that look alike. Have a person approve each one before it goes into the list. A wrong alias creates false alarms.

Step 2: Compare values in a standard form

"$12", "12 USD" and "twelve dollars" are the same value. "14 days" and "two weeks" are too. Turn values into a simple form before you compare them.

python
WORD_NUMBERS = {"one": 1, "two": 2, "three": 3, "seven": 7, "ten": 10, "fourteen": 14, "thirty": 30}
UNIT_DAYS = {"day": 1, "days": 1, "week": 7, "weeks": 7, "month": 30, "months": 30, "year": 365, "years": 365}
 
 
def standard_value(c: dict) -> str:
    v = normalize(c["value"])
    for word, n in WORD_NUMBERS.items():
        v = re.sub(rf"\b{word}\b", str(n), v)
    if c["topic"] in ("trial_length", "refund_window", "data_retention"):
        m = re.search(r"(\d+)\s*(days?|weeks?|months?|years?)", v)
        if m:
            return f"{int(m.group(1)) * UNIT_DAYS[m.group(2)]} days"
    nums = re.findall(r"\d+(?:\.\d+)?", v)
    return " ".join(nums) if nums else v

Converting months to 30 days is a rough rule. It is fine for spotting differences, but a reviewer should always see the original text.

Step 3: Compare the conditions

Pull simple tags out of each claim's conditions and quote. If two claims have different tags, the difference may be explained by context.

python
CONTEXT_PATTERNS = {
    "billing:annual": r"\b(annual|annually|yearly|per year)\b",
    "billing:monthly": r"\b(monthly|billed monthly|month-to-month)\b",
    "from_price": r"(starting at|starts at|from \$|from usd)",
    "promo": r"\b(limited time|promo|discount|offer)\b",
    "dated": r"\b(effective|starting|from|until|as of) \w+ \d{1,2},? \d{4}\b|\b\d{4}-\d{2}-\d{2}\b",
    "region": r"\b(eu|uk|us|europe|united states|united kingdom|india|apac)\b",
}
 
 
def context_tags(c: dict) -> set:
    text = normalize(f"{c.get('conditions') or ''} {c['quote']}")
    return {tag for tag, pattern in CONTEXT_PATTERNS.items() if re.search(pattern, text)}

Step 4: Label each pair

Now compare every pair of matching claims from different pages. Each pair gets one of four labels. None of them says which page is right.

Label Meaning Review priority
consistent Same value None, not sent to review
different_context Different values, but the conditions differ Low
possible_stale Different values, same conditions, and one page is much older Medium
possible_conflict Different values, same conditions, no clear reason High
python
from datetime import date
from itertools import combinations
 
STALE_DAYS = 180
 
 
def days_between(a: str | None, b: str | None) -> int | None:
    try:
        return abs((date.fromisoformat(a) - date.fromisoformat(b)).days)
    except (TypeError, ValueError):
        return None
 
 
def label_pair(a: dict, b: dict) -> tuple[str, str]:
    if standard_value(a) == standard_value(b):
        return "consistent", "values match"
    tags_a, tags_b = context_tags(a), context_tags(b)
    if tags_a != tags_b:
        diff = sorted(tags_a ^ tags_b)
        return "different_context", f"conditions differ: {', '.join(diff)}"
    gap = days_between(a.get("page_updated"), b.get("page_updated"))
    if gap is not None and gap > STALE_DAYS:
        return "possible_stale", f"pages were updated {gap} days apart"
    return "possible_conflict", "different values with the same conditions"
 
 
def find_pairs(claims: list[dict]) -> list[dict]:
    groups = {}
    for c in claims:
        groups.setdefault(claim_key(c), []).append(c)
 
    pairs = []
    for key, group in groups.items():
        for a, b in combinations(group, 2):
            if a["url"] == b["url"]:
                continue
            label, reason = label_pair(a, b)
            if label != "consistent":
                pairs.append({"key": key, "a": a, "b": b, "label": label, "reason": reason})
    return pairs

possible_stale does not mean the older page is wrong. It means the age gap is the first thing a reviewer should look at.

Where a model can help, and where it should not

You can ask a model a narrow question about a flagged pair, such as "Do these two quotes describe the same plan and billing period?" Use its answer to sort the queue or add a note.

Do not ask it "Which page is correct?" A model does not know your current prices or policies. Treat any model output as a hint for the reviewer, not a decision.

How do you generate a review queue?

A flagged pair is only useful if it reaches the right person with everything they need.

Route each pair to an owner

Different teams own different parts of a site. Map URL paths to owners in config.

python
OWNERS = {
    "/pricing": "marketing", "/plans": "marketing",
    "/help": "support-content", "/faq": "support-content",
    "/docs": "docs", "/security": "security", "/legal": "legal",
}
TOPIC_WEIGHT = {"price": 3, "refund_window": 3, "trial_length": 2, "feature_availability": 2,
                "plan_limit": 2, "data_retention": 2, "support_hours": 1}
LABEL_WEIGHT = {"possible_conflict": 3, "possible_stale": 2, "different_context": 1}
 
 
def owner_for(url: str) -> str:
    path = urlparse(url).path
    return next((team for prefix, team in OWNERS.items() if path.startswith(prefix)), "web-team")

Export the queue

A CSV file works with almost any tool your team already uses, such as a spreadsheet or an issue tracker import. Each row holds both claims, both quotes and both URLs. The last columns are blank for the reviewer.

python
import csv
import hashlib
 
 
def pair_id(p: dict) -> str:
    quotes = sorted([normalize(p["a"]["quote"]), normalize(p["b"]["quote"])])
    return hashlib.sha256("|".join(quotes).encode()).hexdigest()[:12]
 
 
def export_queue(pairs: list[dict], already_reviewed: set, path: str = "review_queue.csv") -> int:
    rows = []
    for p in pairs:
        pid = pair_id(p)
        if pid in already_reviewed:
            continue
        a, b = p["a"], p["b"]
        rows.append({
            "id": pid,
            "priority": TOPIC_WEIGHT.get(p["key"][0], 1) * LABEL_WEIGHT[p["label"]],
            "label": p["label"],
            "reason": p["reason"],
            "topic": p["key"][0],
            "subject": a["subject"],
            "attribute": a["attribute"],
            "value_a": a["value"], "quote_a": a["quote"], "url_a": a["url"],
            "updated_a": a.get("page_updated") or "", "owner_a": owner_for(a["url"]),
            "value_b": b["value"], "quote_b": b["quote"], "url_b": b["url"],
            "updated_b": b.get("page_updated") or "", "owner_b": owner_for(b["url"]),
            "retrieved_at": max(a["retrieved_at"], b["retrieved_at"]),
            "decision": "", "notes": "",
        })
    rows.sort(key=lambda r: r["priority"], reverse=True)
    if rows:
        with open(path, "w", newline="") as f:
            writer = csv.DictWriter(f, fieldnames=list(rows[0]))
            writer.writeheader()
            writer.writerows(rows)
    return len(rows)

Give reviewers clear choices

Offer a short list of decisions, so results are easy to count:

  • Fix page A or Fix page B
  • Both correct, because the context differs
  • Needs product input
  • Not a real claim, which means the extraction got it wrong Save each decision with its pair ID. On the next run, export_queue skips pairs that were already reviewed. If either quote changes, the ID changes, and the pair comes back for a fresh look.

Track the "not a real claim" and "both correct" counts over time. If they are high, tighten the prompt, the alias list or the context patterns.

Run it on a schedule

Pages change, so contradictions come and go. Run the workflow weekly, or after big site updates. Compare each new claims snapshot with the last one, and only send new or changed pairs to review.

MrScraper offers a Schedule feature for automated runs and a Webhook integration for sending results to your services. Check the docs to see whether they fit your setup.

FAQs

Which claims should you compare?

Compare claims that are specific, that customers act on and that appear on more than one page. Prices, plan limits, feature availability, trial length, refund windows, data retention and support hours are good places to start. Skip opinions and one-off claims.

How do you generate a review queue?

Label each pair of claims that may disagree, give it a priority based on topic and label and route it to the team that owns each page. Export the queue as a CSV with both quotes, both URLs and blank columns for the reviewer's decision. Skip pairs that were already reviewed unless a quote changes.

Which parts does MrScraper provide, and which parts must my application implement?

MrScraper is a developer-focused web data API. In this workflow, it discovers URLs on your site and extracts structured claims from each page with a prompt and schema. It also returns the page HTML, which you use to check each quote.

Your application owns the rest. This includes the claim topics, quote checks, alias list, value and context rules, pair labels, owner routing, the review queue and saved decisions. Any model you use to compare claims is your own, and a person always makes the final call.

Summarize this post

Open it in your assistant of choice with the prompt ready to send.

Take a Taste of Easy Scraping!
@MrScraper_

Find more insights here

Return Verbatim Evidence Highlights in Web Research Answers
Article

Return Verbatim Evidence Highlights in Web Research Answers

Build an evidence step that picks supporting passages, checks every quote against the source page, a…

Benchmark Heading-Aware and Semantic Chunking for Web Documentation
Article

Benchmark Heading-Aware and Semantic Chunking for Web Documentation

Run a fair test of heading-aware and semantic chunking on the same documentation pages, with a retri…

How to Build a Real-Time Amazon Price Monitor for Enterprise Intelligence
Guide

How to Build a Real-Time Amazon Price Monitor for Enterprise Intelligence

Learn how to build a real-time Amazon price monitor using a website scraping tool. Scalable retail i…

Your choices

Cookie preferences

Necessary cookies keep your selection. Optional categories are disabled until you switch them on.

Strictly necessary

Remembers your privacy selection and keeps the site working.

Always on