To structure my evaluation of the Ontario Citizens' Assembly on Electoral Reform I use the proven criteria and suggestions from the popularly cited “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer, published in 2000 in the Science, Technology, & Human Values journal. You can download a free copy of the article PDF from socialsciences.wur.nl I also included additional indicators at the end that did not seem to quite fit in the Rowe Frower paradigm but provide useful insight. I end with some potential criticisms and suggestions but a largely positive assement.

Unless otherwise referenced, my descriptions pull directly from the “Democracy at work” final report (PDF) by the Ontario Citizens' Assembly Secretariat. I use findings from the “Citizen Deliberative Decision-making” independent evaluation report by the Canadian Institute on Governance (IoG) and quotes from assorted referenced press coverage.

Acceptance Criteria

1. Representativeness: The public participants should comprise a broadly representative sample of the population of the affected public.
Suggestions: select a random stratified sample of the affected population; involve the use of questionnaires to determine the spread of attitudes with regard to a certain issue, using this as a basis for the proportionate selection of members.

Ontario Citizens' Assembly on Electoral Reform (OCA) consisted of 103 citizens selected at random, representing Ontario's 103 electoral ridings, 52 women and 51 men. While the demographic data of thr assembly is not presented in such as way as to allow for easy comparison to the census data, it is obvious their is a representative mix of established and new Canadians, white- and blue-collar workers, students, retirees, homemakers and diverse ethnic groups. The age demographics are generally representative with a modest (under 8%) over representation of members aged 50+ .

As part of the public consultation process that informed the OCA, there were public 41 meetings held in 35 cities across the province. Eight of these meetings were bi-lingual and accommodations were made for people with disabilities at all meetings.

501 people made presentations at the 41 public consultation meetings that were held throughout the province. (IoG p.12) Of the 295 formal presentations by citizens only 23% of these presenters were women. The age data is incomplete but trends towards a representative mix. Other demographic data of presenters and authors of written submissions is not available.

Four special outreach focus groups were arranged with the help of the Social Planning Network of Ontario (SPNO) in Peel, Sudbury, Ottawa, and St. Catharines. targeting people who are homeless or living on low incomes and people who have only basic literacy or English skills.

The Maytree Foundation assisted the Secretariat in arranging a focus group to obtain viewpoints from the perspective of the immigrant community.

One meeting was specifically arranged to engage people with disabilities. The Canadian Hearing Society, the Canadian Helen Keller Centre, the Canadian National Institute for the Blind, and the Canadian Paraplegic Association of Ontario assisted with this meeting by publicizing it to their members

2. Independence: The participation process should be conducted in an independent, unbiased way.
Suggestions: steering committee or management team incorporates members from diverse bodies or neutral organizations, such as university academics; disclosure from participants of any relationship to the sponsoring body ; the use of a respected facilitator.

A select committee of Members of Provincial Parliament from all parties in the legislature created the founding terms of reference.

“The government appointed as chair a man with impeccable credentials — George Thomson, a former judge and senior bureaucrat. He, in turn, appointed a secretariat headed by Jonathan Rose, a Queen's University professor of political science with no published record on the issue of electoral reform. Both men say they are approaching their tasks with their minds open to every possibility, including the status quo.” (Toronto Star, Sept 9/06). “In the view of the members, the evaluators, and the Secretariat, the Chair did an excellent job in this role.” (IoG p.17)

The OCA members rated the neutrality of the secretariat and the facilitators on average 4.6 out of 5 in the IoG evaluation surveys.

3. Early [public] involvement: The public should be involved as early as possible in the process as soon as value judgements become salient.
Suggestions: Public debate should thus be allowed on underlying assumptions and agenda setting and not just on narrow, predefined problems.

Early in the process, the Secretariat invited key stakeholders, organizations with a specific interest in the Citizens’ Assembly process, to express an interest in the work of the Assembly. L’Assemblée de la francophonie de l’Ontario; Equal Voice; Fair Vote Canada; Fair Vote Ontario; Ontario Chamber of Commerce; Ontario Federation of Indian Friendship Centres; and Ontario Federation of Labour all participated in early discussions and engaged in outreach and public education about the initiative. \

Although the process started with terms of reference that included eight principles of electoral systems to consider (Legitimacy, Fairness of representation, Voter choice, Effective parties, Stable and effective government, Effective parliament, Stronger voter participation, a and Accountability) the Assembly was invited to add their own principles (they chose to add “Simplicity and Practicality”) and prioritize the principles as they saw fit.

The assembly discussed, amended and approved its own “rules of procedure”.

From the public consultation phase onward members of the OCA took part in advisory committees to help plan and direct the structure of their process.

4. Influence: The output of the procedure should have a genuine impact on policy.
Suggestions: ensure that there is a clear acceptance beforehand as to how the output will be used and how it might direct policy; use of the media to inform the general public about the specific ways in which the output has influenced policy

The recommendation from the OCA has gone directly to an October 10th binding referendum and thus will become lawif approved by 60% of all votes cast, plus a simple majority of more than 50% in at least 60% of the ridings.

To date the government mandated public education about the referendum has been very limited to simple flyers and promotions for the yourbigdecision.ca web site. The presents the choice in an even handed way without giving any information on about the Assembly process was or its democratic legitimacy.

5. Transparency: The process should be transparent so that the public can see what is going on and how decisions are being made.
Suggestions: releasing information on aspects of the procedure, varying from the manner of the selection of the public participants to the way in which a decision is reached to the minutes of meeting; if any information needs to be withheld from the public, for reasons of sensitivity or security, … admit the nature of what is being withheld and why.

Complete videos of all plenary sessions, slides and reference materials were posted to the public web site in a timely manner through out the process. Small group deliberations were private except for one group each session that volunteered to be open to the public and recorded on video. Copies of all public submissions were published in summary and in full on the web site.

By week one of the deliberation phase (five months in) “The Assembly had been covered on one national television program and three province-wide public affairs programs carried on ten different television stations or networks. At least twenty different radio shows had discussed the Assembly. At least fifty different newspapers had printed articles. More than 225 news articles about the Assembly were published during the consultation phase, constituting half of the media coverage (450+ articles) since the beginning of the process. That figure was increasing by six to twelve articles per day. The Assembly’s website had already attracted more than 45,000 unique visitors.” (OAC report, p. 94)

46,681 consultation guides and brochures were distributed around the province. (IoG p.10)

The Citizens’ Assembly website was visited by 58,002 unique Ontario visitors from 1 July 2006 to 9 May 2007. During the same period, there were 15,769 downloads of documents in a number of formats as well as 27,133 views of the video “Billy Ballot” (IoG p.10)

For reference, the Ontario population is about 12 million. (statcan.ca)

Process Criteria

6. Resource accessibility: Public participants should have access to the appropriate resources to enable them to successfully fulfil their brief.
Suggestions: information resources (summaries of the pertinent facts); human resources (e.g., access to scientists, witnesses, decision analysts); material resources (e.g., overhead projectors/whiteboards); and time resources (participants should have sufficient time to make decisions).

The OCA had a $6-million budget, (Rabble.ca June 20/07).

Members received compensation at the rate of $150 per day (taxable income) for Saturday and Sunday meeting days and for their participation on panels during formal consultation meetings.

The weekend learning and deliberation sessions were held in the world class Osgoode Hall Law School. Simultaneous French/English translation and American Sign Language (ASL) interpreters were available for all Assembly meetings.

The Secretariat produced or directed the production of a variety of learning materials for the Assembly and the public, including: A 60 page textbook, “From Votes to Seats: Four Families of Electoral Systems”; Summer reading list and package; Annotated bibliographies and readings at introductory, intermediate, and advanced levels; A summary of the principles and characteristics related to electoral systems; Detailed slide presentations on each learning topic; A video animation, “Billy Ballot”, explaining the families of electoral systems

Members spent 60 classroom hours learning about the topics using a variety of adult education formats. They also had additional informal learning and discussion meetings during their work weekend evenings and in their on-line members' forum.

Members formed working groups for additional examinations of special topics.

They had 22 guests speakers, mostly academics from Canada, plus from the UK, one from New Zealand, a few former MPPs, Chief Statistician of Canada, Statistics Canada and Brian Lambie from Redbrick Communications.

The Academic Reference Group was composed of thirteen leading scholars on electoral systems from Ontario universities representing a range of expertise. They were not restricted in their commentary, opinion or advocacy.

Small group sessions were conducted by a team of 11 facilitators that were also subject experts.

The members' assessment of the degree to which they felt informed about electoral systems increased substantially over the course of the learning phase. The mean on a scale from 0 (not informed) to 10 (very informed) increased from 4.32 before the learning phase to 7.68 after the learning phase. (IoG p.7)

On average across the six weekend sessions in the learning phase, more than 85% of the members considered the pace of the plenary sessions to be “just right” and more than 87% of the members considered the pace of the small group sessions to be “just right”. (IoG p.8)

Concerning the public meetings, “over 90% of the respondents to the public surveys agreed or strongly agreed that presenters were given enough time to present and answer questions.” (IoG p.13)

7. Task definition: The nature and scope of the participation task should be clearly defined.
Suggestions: [clearly define at the outset] the scope of a participation exercise, its expected output, and the mechanisms of the procedure.

The OCA terms of reference clearly established the task and scope of investigating potential changes to the Ontario electoral system and authoring a recommendation for a new system if they thought necessary.

The process, phases, mechanisms and agendas were clearly presented early in the process and reviewed and revised by the OCA as they saw fit within their schedule limitations.

8. Structured decision making: The participation exercise should use/provide appropriate mechanisms for structuring and displaying the decision-making process.
Suggestions: A variety of decision-aiding tools might be incorporated into a participation procedure, such as decision analysis, decision trees, multiattribute utility theory, and the Delphi technique; structure the decision process in groups; important to structure the decision process in [small] groups; independent decision analyst could be usefully involved; use of an [experienced] group facilitator to employ rules for effective group decision making.

“The Assembly agreed on the values and procedures for working together during the first weekend session of the learning phase. These included: equality of opportunity to participate; commitment and focus on the shared objectives; respect for each others' opinions; listening and learning from each other; and a comfortable environment of mutual trust, cooperation and fun.” (IoG p16)

Decision lists and decision trees were used and reference throughout the deliberation phase.

Experienced facilitators used explicit ground rules for supporting quality deliberation towards consensus. When consensus was not possible, various voting strategies were adopted.

Deliberation was conducted both in plenary form and 20 person break-out groups. Groups were designed to be diverse.

“The composite ratings related to the deliberation objective ranged from 4.48 to 4.80 (very satisfied) over the six weekend sessions of the deliberation phase.” (IoG p.16) This included 'ability to raise questions and express their views'

“Close to 93% of the members who completed the detailed survey at the end of the deliberation phase agreed or strongly agreed that every Assembly member had had an equal opportunity to present their views.” (IoG p.17)

9. Cost-effectiveness: The procedure should in some sense be cost-effective.
Suggestions: take account of the potential costs of the alternative methods, in both time and money, and to consider the extent to which they fulfill the other criteria.

It could be argued that the $6-million budget was a waste and a small group of experts working with a 15 person citizen panel could have created the same recommendation for under a million dollars. While it is certain that an alternative process could have been much more economical, it is unlikely it could have had the same democratic legitimacy combined with educated reasoning.

Additional Indicators

Beyond the Rowe and Frewer nine criteria it is informative to review the following process quality indicators:

Over the 12 work weekends the assembly attendance averaged 102 of 104 (including the chair).

No members dropped out of the eight month commitment.

The OCA received 1,036 written submissions – a total of more than 3,500 pages. Ninety-nine of Ontario's 103 electoral districts were heard from, in addition to some submissions from other provinces and countries. Sixty-two submissions were made on behalf of organizations, but the vast majority of the submissions came from individual Ontarians.

According to the Secretariat's records, 1,973 members of the public attended the 41 public consultation meetings held in 35 cities and towns across the province from 20 November 2006 to 25 January 2007. (IoG p.10)

The final decision to recommend the Assembly's decision to the people of Ontario was approved by 92% of assembly.

Potential Criticisms

From a citizen's point of a few the most obvious criticism of the process was its fundamental lack of scope. The Assembly was never given opportunity to address any concerns about elections or politicians other than the essential mechanics of the voting process. For example, they could not investigate or give recommendations on campaign rules, comment on politician salaries, tenure and oversight or any new techniques for increasing government accountability and transparency.

A second obvious criticism was that all roles within the secretariat were appointed without any third-party monitoring or politically balanced committee input. The chair, staff , facilitators and reference team that structured, informed and managed the process had unlimited opportunity to bias the process towards their own agenda. That said, the IoG evaluation observations and surveys did not recognize any pronounced bias in the orchestration of the process. Further investigation would be necessary to recognize potential conspiracies within the secretariat.

While the Assembly members were quite representative of the population demographics, the participant and specifically the presenters in the public consultation were predominantly men. The first-come-first-serve nature of the presenters privileged those with greater resources and confidence. That said, the outreach focus groups conducted in partnership with Maytree Foundation and the Social Planning Network of Ontario did show effort to hear the voices of the typically under-represented.

Giving the final approval of the policy recommendation to the citizens in the form of a referendum takes the obvious power out of the hands of politicians and bureaucrats, but gives unaccountable power to the editors of each media outlet and to those who can afford to conduct media campaigns to sway public support in their preferred direction. More generally, it could be argued that it is beyond the capacity of the current North American sound-bite and specatcle driven popular media to effectively inform the public on any policy topic and thus no referendum could be considered a valid expression of an informed public will.

Overall Assessment

Never in Ontario's history has such as representative group of citizens been given so much potential political influence. Nor has their been such a demonstration of adult education combined with public consultation and quality deliberation. The sheer number of hours the members committed to the relatively esoteric topic of election reform is astounding.

On reviewing some of the education materials, the plenary slides, the video recordings and reading the process report and independent evaluation, it seems quite clear that this process was of high quality and directed with care and well informed consideration. Compared to other traditional processes and even leading public participation process internationally, this was an absolutely amazing example of democracy at its best.

Reading the recommendation the Assembly created and the reasoning behind it, there is always room for debate, as their was within the Assembly itself and between the experts that informed their choices. If there was a problematic bias, hidden agenda or conspiracy within the secretariat, it has yet to be recognized.

Even if the referendum fails to approve the recommendation, I have no doubt that the Ontario Citizens' Assembly process will be a model for other governments and organizations around the world for many years to come.

Suggestions for the Future

When the next opportunity comes to conduct a similar Citizens' Assembly process, it would be recommended that their be more care put into the design of a democratic, politically balanced and overtly merit based selection process for all key secretariat positions.

Rather than using an expensive, superficial and media reliant referendum to approve the recommendation, it would be more sensible to use a panel of randomly selected superior court judges to affirm that the assembly process and recommendation are legitimate and thus approved to become policy.

Most importantly, the Ontario experience shows that given enough resources, time and due diligence, a citizens' assembly can make quality decisions on topics that are unfamiliar to the public. We should be conducting many more citizen assemblies at all levels of government.

---

 

References

Democracy At Work: The Ontario Citizens’ Assembly on Electoral Reform. A record of Otario's first citizens' assembly process (PDF)
Created by the Ontario Citizens' Assembly Secretariat, 2007.

Citizen Deliberative Decision-Making: Evaluation of the Ontario Citizens' Assembly on Electoral Reform
By the Institute On Governance (IoG), May 2007.

Beware citizens' assemblies on electoral reform: Giving back power to MPPs would solve problems.
By Ian Urquhart in the Toronto Star, September 9, 2006

Voting for change
By Tor Sandberg on rabble.ca, June 20, 2007.

 

 

CitizensAssemblyMonument.ca is a volunteer project of Jason Diceman, an independent stakeholder engagement consultant and deliberative democracy researcher in Toronto. He is neither associated to the Vote For MMP campaign nor the Ontario Citizens’ Assembly Secretariat.

Here is the story so far…

After spending six months in Venezuela learning Spanish and researching their new participatory democracy system, I came back home to Ontario in August 2007 with a newly invigorated enthusiasm for citizen driven politics. I looked into the Ontario Citizens’ Assembly on Electoral Reform and learned how amazing of an example it was of real people getting informed, deliberating and making a smart decision for government. 103 random and representative citizens from every riding in Ontario spent about 35 hours a month for 8 months researching, discussing and coming to a popular decision.

When I started to ask my friends, family and neighbours what they thought about the Citizens Assembly, I just got blank stares. In just a few weeks their will be a provincial referendum to approve the Mixed-Member Proportional election system (MMP) recommended by the Citizens’ Assembly, and no one knows who they were and how they worked. If we give our resounding support for the Citizens’ Assembly decision and thus the process they used, it could trigger a dramatic turning point in Ontario political history, but it is not likely without more popular awareness.

The problem kept me awake at night. What could I do as one person with no budget, no political influence, no access to the big media and only a month to organize? Then it came to me:

I could tour across the province asking everyday people to participate in the creation of a monument to the Citizens’ Assembly, with the aim of getting coverage by local and popular media to the amazing work that was accomplished in the Citizens’ Assembly, to the MMP recommendation they decided on and to the potential for more deliberative democracy in general.

But then it became obvious that I would need a dedicated team and substantial budget to pull such an event off without looking like a loony. When an offer for such resources fell through, I had to sadly shelf that plan.

Since then I have been focusing on building a list of political science professors who support the Assembly process. I am now drafting a press release about the issue and the supporting in hopes of getting some media coverage. It is not quite a monument, but a step in the right direction and could affect the results of the referendum, even if just a little.

If you would like to get involved, please get in contact.

cheers

– Jason Diceman

Learn more about me and what I do at www.JasonDiceman.ca

 

Starting September 2006, 103 randomly selected Ontario citizens committed to 30-40 hours a month for 8 months to get educated, hear public consultation and deliberate on the topic of electoral reform to make a unified policy recommendation that would be voted on by a binding public referendum.

The public process was mandated by government, well financed and executed in an independent and transparent manner and with great care and consideration by highly respected experts. The participating Citizens' Assembly members were selected at random and represented a cross section of the population. In the Learning Phase the citizens attended six full weekends of adult education on the topic of electoral reform. The materials, presentations and process were extensive and balanced by design and the learning was measurably very successful. In the Consultation Phase, 501 people made presentations at the 41 public consultation meetings that were held throughout the province. Additionally five outreach focus groups were held for people who are homeless, living on low income, who have only basic literacy or English skills and new immigrants. Along with making every meeting accessible, there was an extra special meeting for people with disabilities. The assembly also received and reviewed 1,036 written submissions. In the Deliberation Phase the assembly spent six complete weekends in well facilitated discussion groups, consensus driven deliberations and structured decision-making processes. Many decisions were made via consensus and participants highly rated their opportunity to participate and the neutrality of the facilitators. The final decision to recommend the Assembly's decision to the people of Ontario was approved by 92% of the assembly. The assembly's final 27 page report was a unified, clear and detailed document approved by consensus. According to the Canadian Institute on Governance who acted as third party evaluators monitoring the entire process: “The whole Assembly process was undertaken in an open and transparent manner and is well documented for the benefit of future exercises of a similar nature. It serves as a model of how to engage and empower citizens to deliberate and decide on selected public policy questions.” The Assembly's recommendation will become Ontario policy if during the October 10th 2007 referendum the proposal is approved by 60% of all votes cast, plus a simple majority of more than 50% in at least 60% of the ridings.

Visit the official site Download the 280 page “Democracy at Work” report describing the complete assembly process (PDF) Read an Evaluation of the Ontario Citizens' Assembly Process written by Jason Diceman. Download the “Citizen Deliberative Decision-making” evaluation report (PDF) prepared by the Institute on Governance

For the past few weeks I have been researching and writing about the Ontario Citizens’ Assembly on Electoral Reform. I did some outreach and have so far found 13 Ontario political science academics who agree:

“The Ontario Citizens’ Assembly was an exceptional example of a legitimately democratic public policy development process.”

See the full list supporters.

I also wrote a brief summary of the OCA, a detailed evaluation of their process, a press release and an op-ed. But I doubt any of the hundred press contacts I sent to will do much with it. The media does not seem interested in how unique the referendum recommendation process was and that it set a precedent for deliberative democracy in Ontario.

I’ll follow-up after the election when the referendum most likely fails to get the necessary 60% approval. I’ll see if I can grow a coalition of people to say that the OCA is a great model we should reproduce for other policy questions – without the referendum requirement.

Contact me to get involved.