What has the recent Canada Day “poll” commissioned and being massaged daily by the CBC to do with Social Justice? A great deal, as a matter of fact, because it serves as an example of abuse of statistical methods to create journalistic fodder, and thus to misrepresent our perceptions of our country’s beliefs and attitudes. It is an exercise in misinformation and misleading. It is not fake news, because it is reporting something that actually happened, but it is sloppy news. Canadians on the whole may or may not be “conflicted and worried”, as the July 1st headline would have it, but you can’t prove it by this “poll”, one way or the other.
This piece of work is not a “poll”, but a “survey”. A survey becomes a poll when rigour is applied to the size and composition of the sample, the wording of the questions, and the nature of the questioning. This survey meets none of the relevant standards, and therefore cannot be generalized. It is valid for what it is, a survey of 4,500 (3,000 + 3×500) Canadians not randomly selected who were asked slanted questions over the internet and confined to multiple-choice answers. It has no capacity to speak to the larger outlook. Éric Grenier, the reporter who wrote the first story, and who with his background presumably knows better, as he tries to reassure us on the reliability of his reportage, makes the following, startling statement:
Because the sample is based on those who initially self-selected for participation in the Maru Voice panel rather than a probability sample, no estimates of sampling error can be calculated. However, a comparable probabilistic national sample of 3,000 voters would have a margin of error of +/- 1.8 percentage points, 19 times out of 20, while samples of 500 voters have a margin of error of +/- 4.4 percentage points, 19 times out of 20.
Logically this is exactly the same as saying, “Since this animal before us is an ass we can’t expect it to do the same work as a horse. But if it were a horse, we could get it to do the work. So even though we know it’s an ass, we’ll put it to work as if it were a horse.” We may judge the work accordingly.
By the way, in my day we didn’t have “probability” or “probabilistic” samples. We had random samples, representative samples, small samples, large samples, etc. all of which use probabilities, each of which performs very differently. Mr. Grenier is trying to baffle us into overlooking his horse-ass problem. The CBC survey, by the way, uses a very small, even minuscule, not random, not remotely representative sample.
You can find Mr. Grenier’s story at https://www.cbc.ca/news/politics/cbc-election-poll-1.5188097. The CBC has published others since, but I haven’t read them.
Let’s look at the questions. I propose to increase the sample to 4,501, a .022% (that is, 2.2 one-hundredths of one percent) increase, by answering some of the questions myself. I realize this percentage is small. It’s considerably larger than the CBC’s .016% of the adult Canadian population, however. To have any chance of supporting the generalizations the CBC wants to make they would need something closer to 450,000 respondents, randomly selected and personally interviewed, at least by telephone. I am not saying they don’t learn anything from their almost infinitesimal, biased, internetted sample, only that it provides no support for conclusions about the adult population as a whole, especially considering the questions asked. These are not simple, like “How would you vote?” or “Do you have a job?” They are complicated questions, inviting nuanced answers. The answers are pre-set, however, thus depriving the respondents of the right to say what they believe or how they feel in their own words.
On to the questions, as described in Mr. Grenier’s story. I have enough respect for his integrity to assume he is reporting them accurately.
“When you think of you and your family, are you worried or optimistic about the future?” (Answers provided: Worried, somewhat worried, somewhat optimistic, optimistic.) My answer: I am neither worried nor optimistic. These words do not apply to me as I squint my way towards the future. Some good things are happening, and some not so good. In the future I expect that some good things will happen, and some not so good. Whether these will be the same things remains to be seen. I am reasonably sure, based on my age, that I will die some time in the next twenty-five years. I am not worried about that.
“What, if anything, are you most worried about?” (My health/health of a family member, cost of living, climate change, crime and public safety, terrorism, my job/finding a job, immigration, international relations/trade agreements, truth in the media, racism, social inequality, none of these issues worry me). I see reason for concern in what is happening and has happened in several of these areas, some more than others, but I know good people are working on them and trust their efforts will be fruitful at least to some extent. Progress is necessarily gradual. Ill-informed and self-interested opposition to those efforts is of course always a concern, but can be overcome by people of good will whose hearts are in the cause. The cause, for me, is Social Justice, which enters into all these fields and defies simplistic formulas of any kind.
I am going to skip over the next group, which concern specific issues or points of view. Besides, I am running out of room.
“In general, do you think Canada is on the right track or on the wrong track?” (Right track, wrong track). What a stupid question! Canada is a pluralistic country, moving through time in a way compatible with both its nationhood and its pluralism, neither confined to one track nor unconstrained in its modes of travel. All human possibilities, for good or ill, are within our potential, although I believe our potential for good exceeds the other.
“Thinking about the upcoming federal election in October specifically, which issues are most concerning to you?” (Health care, climate change, cost of living, jobs/the economy, housing affordability, home ownership, government mismanagement, deficit spending, gun control, nobody to vote for, immigration, terrorism, trade negotiations, rascism, the quality of life in Indigenous communities, women’s equality) I see progress and reason for concern in all these areas. When I look at the parties and their leadership (always more than one person) I am more interested in their casts of mind than the specifics of their “programs” let alone their “promises”. I will vote for whichever party appears to have the most favourable cast of mind for the cause of Social Justice in all its pluralistic, multi-faceted ways.
It wouldn’t surprise me in the least if a hefty percentage of the other 4,500 people agree with me, or will agree when we have had a good conversation about the meanings of the words we are using. Alas, I do not expect the CBC to sponsor anything like that kind of conversation, or to support it by the necessary kind of social-political journalism.
By the way, if anybody knows what a cross between a fox and a hedgehog (or a porcupine) looks like, or how it behaves, please let me know. I think we may need one to tame the Unsolved Riddle of Social Justice and put it to work. Isaiah Berlin, echoing Archilochus, tells us: “The fox knows many things, but the hedgehog knows one big thing.” I think that’s a Both-And, which may not really be all that hard to do.
Maybe Canada is such a cross: BOTH one big thing, AND many things,