Five Studies From Five Firms Is Not a Shortlist

A team assembles five markets for the board. Market one has a full commissioned study. Two and three have reports from a different firm, done eighteen months apart. Four has a broker's market overview. Five has an internal deck built from public data and a site visit.

Every document is competent. The stack is still not a shortlist, because nothing in it can legitimately be ranked against anything else.

Why the comparison silently fails

The definitions differ. One report's "mid-scale corporate demand" and another's "healthy mid-market activity" may describe entirely different capacity ranges. Nothing on the page tells you that. The comparison feels apples-to-apples and isn't.

The vintages differ. An eighteen-month-old report and a current one are describing different markets, and the older one carries no marker of its own staleness. It reads exactly as confident as the fresh one.

The scoring scales differ. A "7/10 opportunity" from one firm and "high potential" from another cannot be reconciled. In practice teams reconcile them anyway, by feel.

The authors' dispositions differ. Some analysts are structurally cautious, some optimistic. Across five reports from four firms, that variance is indistinguishable from real variance between the markets themselves.

The consequence is quietly serious: the shortlist ends up ordered by which document was most persuasively written, not which market is strongest. And the most persuasive document is usually the most expensive one, which is usually the market someone already favoured — so the process returns the answer it started with, having spent six figures doing it.

Comparability is a different property from depth

The instinct when a shortlist feels shaky is to commission more depth. That doesn't fix this. Five deep reports on incompatible bases are still incomparable — you've bought more confidence in each individual market and no more ability to choose between them.

What fixes it is running every candidate through one framework: the same demand definition, the same competitive-density method, the same capacity bands, the same verdict thresholds, the same vintage. Then a difference between two cities' scores means a difference between the cities, rather than a difference between two firms' house styles.

This is why we hold the capacity bands fixed — every venue market scored against the same five delegate bands, from "up to 20" through "500+", rather than each analysis defining its own. It makes no single city's story more flattering. It makes the comparison honest, which is the only thing a shortlist actually needs.

What consistency reveals that bespoke work hides

Once every market sits on one scale, two things become visible that a stack of reports cannot show.

Base rates. Across 155 venue markets the distribution is 62 Conditional Go, 58 No Go, 30 Strong Go, 5 Hold, with a mean demand score of 60.8. That population view is what lets you judge whether a given city is genuinely strong or merely typical. A single report has no denominator and therefore no way to tell you this.

Format sensitivity. The same 208 cities score dramatically differently by category — the 5-star hotel lens returns 73 No Go and 26 Strong Go; boutique returns 43 and 32, and the verdict changes outright in 93 of the 208. Five bespoke studies commissioned against one format would never surface that 36 of your 73 rejected markets clear Conditional Go or better for another.

The reasonable objection

"A generic framework can't capture what's distinctive about each market."

Two responses. First, the framework is constant but the signals inside it are category-calibrated — delegate bands for venues, ADR/RevPAR and brand-tier gap for hotels, daypart demand and average check for restaurants. It isn't one questionnaire stretched across four businesses.

Second, and more importantly: distinctiveness is what the deep dive on the winning market is for. Screening and diligence are different jobs. Use a consistent framework to decide which market deserves the bespoke work, then do genuinely bespoke work on it. The error is using bespoke, incomparable work to perform the screening step — which is where the leverage is and where consistency matters most.

The takeaway

Look at the evidence behind your current shortlist and ask one question: if two of these markets scored differently, would that reflect a real difference between the cities, or a difference between the people who wrote the reports?

If you can't answer confidently, the ranking isn't a ranking. It's a reading order.

Book a demo and we'll put your five markets on one scale, on the call.