What 6 Months of AI-Generated Product Copy Taught Us
- We replaced human-written product descriptions with AI-generated copy across 800 SKUs for 6 months and tracked conversion, SEO, and return rates — the results were not what we expected.
- Conversion rate: statistically identical for standard products, down 12% for high-consideration technical products where nuance matters.
- SEO: AI descriptions outperformed human copy on long-tail keyword capture by 23% — the AI naturally included more relevant phrase variations.
- Return rate: up 8% for products where the AI overstated specifications — the most important failure mode to monitor and prevent.
Six months ago we made a decision that felt simultaneously obvious and risky: use AI to generate product descriptions for every SKU in a client’s 800-product catalogue. The client had inconsistent copy, many products had no description at all, and hiring copywriters for 800 products was out of budget. The experiment taught us things benchmarks and think-pieces do not tell you.
Month 1–2: The Initial Results Were Good
For the first two months, the results were positive enough that we felt validated. Conversion rates held steady, organic traffic improved (Google was seeing more content to index), and the client was pleased with the speed of delivery. Then the return data started coming in.
Month 3: The Return Rate Problem
Eight specific products had elevated return rates. When we read the customer return reasons, a pattern appeared: customers were citing incorrect specifications. The AI had, in several cases, generated plausible-sounding dimensions, material compositions, or compatibility statements that were slightly wrong. Not wildly wrong — plausibly wrong. A cable listed as “compatible with USB-C 3.2 Gen 2” that was actually USB-C 3.0. A shelf described as “supporting up to 30kg” that was rated for 20kg.
This is the critical lesson: for any product with measurable, verifiable specifications (dimensions, weight capacity, compatibility, certifications), AI-generated copy requires human review of the spec claims specifically. The AI is very good at writing around the specs — it is unreliable at getting the specs right when they are not clearly provided in the input data.
The Tiered Approach We Landed On
After month 3, we implemented a tiered system: commodity products (clothing sizes, basic accessories) with no technical specifications — AI writes, no review. Products with measurable specs — AI writes, human reviews only the specification claims. High-consideration technical products — AI writes a first draft, human rewrites the key benefit statements. This reduced the error rate to zero while maintaining 70% time savings on total copy production.
References
- Claude for Content GenerationAnthropic — Claude API used for structured product description generation in the experiment.
- Google Product SchemaGoogle Search Central — structured data for product specifications that reduces AI hallucination risk in copy.
- Google Search ConsoleGoogle — Search Console data used for tracking organic keyword improvements from AI-generated descriptions.
Leave a Reply