The Ontario Citizens’ Assembly was Monumental

citizens assembly logoFor the past few weeks I have been researching and writing about the Ontario Citizens’ Assembly on Electoral Reform. I did some outreach and have so far found 13 Ontario political science academics who agree:

The Ontario Citizens’ Assembly was an exceptional example of a legitimately democratic public policy development process.”

See the full list supporters.

I also wrote a brief summary of the OCA, a detailed evaluation of their process, a press release and an op-ed. But I doubt any of the hundred press contacts I sent to will do much with it. The media does not seem interested in how unique the referendum recommendation process was and that it set a precedent for deliberative democracy in Ontario.

I’ll follow-up after the election when the referendum most likely fails to get the necessary 60% approval. I’ll see if I can grow a coalition of people to say that the OCA is a great model we should reproduce for other policy questions – without the referendum requirement.

Contact me to get involved.



More Criteria for Evaluating Participatory Processes

I found this “Practical Guide for Evaluating Participatory Processes” (PDF) by the International Observatory of Participatory Democracy (OIDP is the Spanish acronym). I’m not that impressed in the ambiguity of the criteria, the redundancy between criteria and the lack of reference to foundational research. Instead I would highly recommend “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer, whose criteria I summarized in a previous blog post.

The OIDP guide was written for municipal governance, but can be applied to other levels of government and community organizations. I believe the original document was written in Spanish and the meaning of some terms may have been a bit confused in the English translation I read.

Below I copy the criteria titles directly from the guide, although the definitions were rewritten based on my interpretations of their descriptions.

A. Criteria related to the process coordination

  1. Consensus: Agreement from political groups, social groups and government related technicians on the need and methodology of the process.

  2. Transversality: Participation in planning and implementation from relevant political and technical bodies.

  3. Initiative and leadership: Government leadership is required for a institutionally valid process. A diverse group of stakeholders and technicians should also be part of the leadership.

  4. Integration in the [government] participatory system: The process should be within an established participatory system or at least be consistent and complimentary to any established participatory system.

  5. Clarity of the objectives: Establish clear expectations of results and limits to their scope. Fulfilment of the objectives.

  6. Planning and resources: Confirm a detailed plan including sensible objectives, stages, schedules, and sufficient provisioning of economic and personnel resources.

B. Participant-related criteria

  1. Number of participants: Percentage of participants in relation to the reference population. Percentage of organized players versus the reference total.

  2. Diversity: compare the demographics of those participating in the process, with the demographics of those in the reference society. Consideration for social groups that are usually under-represented (e.g. women, immigrants, disabled, low-income). Compare the profiles of the participating organizations (e.g. neighbourhood, corporate, union, cultural, sports, political) to the theme of process. Look for key stakeholders, players and organizations that were not included but should have been.

  3. Representativity of the participants (specific to organization representatives): Where organization representatives play a role, what is the democratic quality of how they recognize and present the views of their constituents. E.g. information flow to and from representatives and their constituents, use of elections, consistency between the organizations’ formal statements and that of the representative.

  4. Openness of the process: The public is invited to participate and take part in the decision-making process.

C. Criteria related to the reason for participation

  1. Relevance: The topic of the process is within the governments agenda and viewed as important and relevant to the citizens. Significance of the potential result on the government budget.

  2. Capacity of intervention by the [government] administration: The topic of the process is within the governments administrative scope.

D. Criteria related to the type of participation

  1. Participatory diagnosis: Start with a diagnosis that defines the main problems and matter for debate. Ideally this diagnosis would be participatory in nature.

  2. Capacity to make proposals: Allows citizens to make proposals.

  3. Level of participation: On an upward scale from Information to Communication, Query, Deliberation, and then Decision.

  4. Quality of the information: How effective the information channels were at distributing key information to prepare and support participants. Plurality, objectivity, clarity and utility of the information provided.

  5. Deliberation methods and techniques: techniques or mechanisms were used to avoid inequalities between participants in the deliberations. Each participant felt empowered to voice their opinion.

E. Criteria related to the consequences of the process:

  1. Substantive results: The results respond to the needs recognized in the initial planning of the process.

  2. Implementing the results: Verify the implementation of tangible results. The existence, or the study to create a body to follow-up on the outcomes.

  3. Result feedback: The participants know the results, validate them and consider the process as ended.

  4. Improvement of relationships among the players: Strengthen relationships among the participants, their organizations and the government administration.

  5. Training: Trained participants in the field of citizen participation.

  6. Building a political participatory culture: Participant satisfaction with the process and the willingness to participate once again.

Again, while you may find these criteria useful, I would recommend using the Rowe and Frewer criteria instead.


Evaluation Criteria for Public Participation Methods

In search of a solid frame of reference for comparing various international citizen assembly processes I am investigating, I discovered an excellent paper called “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer, two Ph.D.s working at the Institute of Food Research in Norwich, United Kingdom. Below I copied and pasted together all nine key criteria defined in the paper and their related suggestions. You can use these criteria and suggestions to help plan and evaluate any stakeholder engagement processes you are involved with, such as members surveys, focus groups, public hearings and citizen juries. I think it is very useful.

The original paper was published in 2000 in the Science, Technology, & Human Values journal and has since been the most read article on the journal’s web site. You can download a free copy of the PDF from or see the attached here. The full paper is 27 pages long with 75 bibliographic entries. It is definitely a key paper in the emerging canon of research on public participation mothods.

A Summary of the Criteria from “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer

Acceptance Criteria

  • Representativeness: The public participants should comprise a broadly representative sample of the population of the affected public.
    Suggestions: select a random stratified sample of the affected population; involve the use of questionnaires to determine the spread of attitudes with regard to a certain issue, using this as a basis for the proportionate selection of members.

  • Independence: The participation process should be conducted in an independent, unbiased way.
    Suggestions: steering committee or management team incorporates members from diverse bodies or neutral organizations, such as university academics; disclosure from participants of any relationship to the sponsoring body ; rhe use of a respected facilitator.

  • Early [public] involvement: The public should be involved as early as possible in the process as soon as value judgements become salient.
    Suggestions: Public debate should thus be allowed on underlying assumptions and agenda setting and not just on narrow, predefined problems

  • Influence: The output of the procedure should have a genuine impact on policy.
    Suggestions: ensure that there is a clear acceptance beforehand as to how the output will be used and how it might direct policy; use of the media to inform the general public about the specific ways in which the output has influenced policy.

  • Transparency: The process should be transparent so that the public can see what is going on and how decisions are being made.
    Suggestions: releasing information on aspects of the procedure, varying from the manner of the selection of the public participants to the way in which a decision is reached to the minutes of meeting; if any information needs to be withheld from the public, for reasons of sensitivity or security, … admit the nature of what is being withheld and why.

Process Criteria

  • Resource accessibility: Public participants should have access to the appropriate resources to enable them to successfully fulfil their brief.
    Suggestions: information resources (summaries of the pertinent facts); human resources (e.g., access to scientists, witnesses, decision analysts); material resources (e.g., overhead projectors/whiteboards); and time resources (participants should have sufficient time to make decisions).

  • Task definition: The nature and scope of the participation task should be clearly defined.
    Suggestions: [clearly define at the outset] the scope of a participation exercise, its expected output, and the mechanisms of the procedure.

  • Structured decision making: The participation exercise should use/provide appropriate mechanisms for structuring and displaying the decision-making process.
    Suggestions: A variety of decision-aiding tools might be incorporated into a participation procedure, such as decision analysis, decision trees, multiattribute utility theory, and the Delphi technique; structure the decision process in groups; important to structure the decision process in [small] groups; ian ndependent decision analyst could be usefully involved; use of an [experienced] group facilitator to employ rules for effective group decision making.

  • Cost-effectiveness: The procedure should in some sense be cost-effective.
    Suggestions: take account of the potential costs of the alternative methods, in both time and money, and to consider the extent to which they
    fulfill the other criteria.

If I find other evaluation frameworks I will post them to this blog.

Attachment Size
Rowe frewer public engagement.pdf 79.99 KB

Dotmocracy Demonstrations in My Last Weeks in Venezuela

(See my previous blog post to understand why I'm in Venezuela and what the are Communal Councils that I was investigating).

My last month in Venezuela was very fruitful. After sending out hundreds of emails to various government contacts and journalists, I lucked out with one response from a passionate political writer, Ramón E. Garcia S. (read his blog in Spanish) who's day time job is a computer network system administrator for the government tax office. He invited me to stay with his family in Bolivar City and to demonstrate my Dotmocracy group decision-making technique (Método “¿Que opinas?” in Spanish) to various government representatives and community organizers there.

Two women discuss an idea part of a 65 person Dotmocracy demonstration for
Two women discuss an idea part of a 65 person Dotmocracy demonstration for “Mision Cultura” in Bolivar City.

Over my two weeks in Bolivar City we had many disappointments when meetings were cancelled or there was low attendance. We also had some great successes including a demonstration for the enthusiastic Communal Council of Marhuanta, a well attended workshop at the new socialist Bolivarian University of Venezuela, two mornings of demonstrations in a public school as parents signed up their kids for the new year, a small community leader training workshop, a 65 person demonstration for local representatives of “Mision Cultura” and we even got a short article with photo published in a regional newspaper.

Marahuanta Communal Council uses Dotmocracy
Communal Council of Marhuanta shows of some of their 38 Dotmocracy agreements


After big hugs goodbye to Ramón, his lovely family and his friends who tirelessly supported my presentations and investigations, I flew to Caracas for my final five days at the end of this six month adventure. Thanks to some phone calls from Ramón, within two hours of landing I was able to walk into the Ministry of Popular Participation and Social Protection and present Dotmocracy to various coordinators that work with Communal Councils. The next day I pressed my luck and walked into the Fundacomun building in hope of meeting the director of education for Communal Councils, Iluska Salazar. After only a few minutes wait she was happy to see me. Apparently she had received some emails about my work and thus was not a total stranger to Dotmocracy. After a 20 minute chat she invited me to present my method in a workshop in three hours later. The workshop was for a group of facilitators who were part of a training-the-trainers pilot project in the La Vega area of Caracas. This was the pinnacle of my input into the Venezuelan revolution to date. If these people adopted Dotmocracy it could potentially be distributed throughout the country as part of the Fundacomun education activities. I also gave each contact a CD full of facilitation resources in Spanish.

Fundacomun Caracas learns Dotmocracy
Iluska Salazar, director of Popular Power Education for the Venezuelan government (far left) and several faclitation trainers after our Dotmocracy training workshop

Overall I was amazed how much of the real changes were pursued by local citizens who were now empowered by a political climate change towards participation and support for community initiatives. I met a miner who helped people get new houses, a home-maker who coordinated upgrades to the infrastructure of her neighbourhood, an air-conditioning repairman who help get new health clinics built and a corner store owner who organized the construction of a community daycare centre. These were the people that made the revolution real.

It's been about six weeks since I left Venezuela. I have heard from Ramón in Bolicar City that dotmocracy is being shared within the new United Socialist Party of Venezuela and was used for student consultations at the Bolivarian University. From my friends at Fundacomun in my original residence city of Cumaná, they say Dotmocracy is being shared there too. But really I have yet to receive any clear indication that Dotmocracy has been truly adopted by any groups. I keep my hopes high but my expectations realistic.

I don't know when I will return to Venezuela but I have been forever changed by the vision I have seen there of real participatory democracy and real positive political change being pushed at all levels, especially the local communities and the president's office. Now I'm back in Toronto where we are still using the same political system since 1792. But that could change this year. See

¡viva la revolucion!

P.S. See all my photos from my Venezuela trip on