In public consultation, we often say we “consulted with the community”, but did we?
Any self-selecting public consultation process (e.g. online or paper surveys, public meetings) will not be statistically representative of the community. To be reliably representative, participants would need to be selected randomly (aka “sortition”) to avoid self-selection bias. Self selecting participants are more likely to be advocates with stronger opinions compared to the average resident.
That said, notwithstanding the self-selection bias, how representative a public survey is can be judged, in part, by comparing how well the demographic profile of the respondents matches the known demographics of a community, as recognized by the regional census, for example, looking at the gender and age of participants.
We can also look at the address of the participants – were they from the local community?
Specific to each consultation topic, we should also consider representation of stakeholder types. For example a consultation about a park should include all types of park users, including seniors, parents with children, organized sport players, people with and without dogs, etc. In the case of roadway reconfigurations, how people travel (walk, bike, bus, or drive) is a key stakeholder type to consider.
And importantly, is the rate of participation large enough to be significant. A survey of 50 people is likely less representative than one of 500 people.