Mary Fung
questionMay 9, 2026

How do you evaluate synthetic data when there's no blanket metric?

"Looks realistic" is the lazy proxy. Real quality is conditional on the question you're trying to answer with it.

Open question. Most quality discussions collapse into "is the marginal distribution close enough to real" — which says little about whether the data will hold up the use it's meant to support. A synthetic vendor-master that's perfect for testing a deduplication pipeline can be useless for testing a fraud detector, and vice versa. I don't yet have a clean formulation of what the bar should be when the answer is always "depends on the downstream task."

← back to the field