| Video recording of the webinar [Koi Tū website] |
| Colin Megill’s presentation slides (Art O’Leary did not use slides) |
| Transcript of the webinar |
| Responses to some of the unanswered questions |
| Information about the webinar, topic and presenters |
This is the 30th country I’ve spoken about citizens’ assemblies to in the last 11 months. … there’s a hunger across the entire world … to get citizens involved in policy development, and also for citizens to become involved in issues as well, because they want to make a contribution.
Art O’Leary, Chief Executive, Irish Electoral Commission – An Coimisiún Toghcháin
I think we can say that there is a hunger for more democratic and effective forms of citizen participation here in Aotearoa New Zealand: high levels of anticipation, interest and excitement were palpable, even via Zoom, for this webinar! It was great that over 270 people registered with 10% from universities, 20% from central government agencies, 25% from local government and 10% from unions. Hopefully our representatives in Parliament will watch the video or read the transcript below in their own time: all were invited but none registered. Democratic innovation without them is possible but much harder.
Why might so many people be interested in democratic innovation here? Isn’t New Zealand ranked as one of the world’s most democratic countries?
At a recent event, our former Prime Minster Helen Clark identified complacency as one of the main threats to our democracy. We tell ourselves that we run A+ elections and yet 25% of those eligible to vote don’t (and let’s not mention voter turnout for local elections). Political leaders often crow that corruption amongst officials is rare and yet most of our major political parties have recently been charged with breaking electoral financing rules. And despite having the first absolute parliamentary majority since MMP, the government has struggled to develop, let alone implement, policy – just consider the ‘3 Waters’ reforms or efforts to regulate agricultural greenhouse gas emissions. Could a failure to develop better ways of doing democracy lie behind these problems? Has our ‘complacency’ stopped us innovating, stopped us taking inspiration from successful international innovations such as citizens’ assemblies or Polis, or from Aotearoa examples such as the process used to develop the Matike Mai report?
There is no doubt in anybody’s mind that citizens’ assemblies have transformed the way that Irish people and the Irish political system deals with policy development and … intractable political problems
Art O’Leary
Art O’Leary and Colin Megill could respectively be considered co-creators of the Irish Citizens’ Assembly and Polis, and you can hear them talk in this webinar about their genesis, goals, underlying principles and methods, and impacts by watching the video recording or reading the transcript below. I hope their words inspire more democratic innovation in Aotearoa New Zealand, the development of new knowledge and capacities, and strong public demands for improved democratic participation.
The goal [with Polis] is to leverage machine learning for deliberative democracy to make meaning of large groups of people so that we can produce a map that can be useful for all sorts of different processes, citizens’ assemblies included. But this is not necessarily replacing any other policy making process, but we would say it’s augmenting and potentially enabling new ones.
Colin Megill, President, The Computational Democracy Project
We should, however, be under no illusion: innovation, especially transformational innovation, is neither predictable nor straightforward. Transformations face an uphill struggle to overcome existing ways of doing and thinking by incumbent actors (e.g., government, political parties, the media, consultants, lobbyists, civil society organisations, etc). This is why we need to protect, support and champion lots of democratic experimentation, including the complex conversation work at Koi Tū and the Talanoa-Wānanga experiment in Porirua.
There’s more to a well-functioning democracy than free and fair elections, and a lack of corruption. A well-functioning democracy needs to include and empower people in public agenda setting, problem solving and decision making. Done well, this public participation improves decisions, and contributes to improved policy, its effective implementation and social cohesion. This is why we need more democratic innovation.
This reflection about democracratic innovation and the webinar was written by Simon Wright, who is the Chair of Trust Democracy and was one of the webinar organisers.
Transcript of the webinar
| Anne Bardsley’s introduction |
| Art O’Leary on the Irish Citizens’ Assembly |
| Colin Megill on Polis |
| Questions and Answers |
Introduction
Anne Bardsley:
Mōrena. Kia ora tātoa. Nau mai, haere mai. On behalf of Trust Democracy and Koi Tū: the Centre for Informed Futures, I welcome you to this conversation on democratic innovation: why we need it and how to do it.
My name is Anne Bardsley. I am the Deputy Director of Koi Tū. In the background we’ve got our friends from Trust Democracy, John Pennington and Simon Wright, who are going to help with curating the questions.Please use the chat function for questions.
We’ve got a lot of people registered and I’m sure a lot of interest in what our esteemed speakers have to say today. They’re each going to speak for 15 to 20 minutes and then we will turn over to a Q&A. And we do want this to be quite conversational. We suggest that you might want to use speaker mode because we don’t have the audience cameras on so you can turn directly to your speakers and who I will introduce.
Now, we’ve got Art O’Leary, who’s the Chief Executive at the Electoral Commission in Ireland. He’s overseeing electoral and wider democracy matters, but has been involved in in many citizens’ assemblies so far. And secondly …. so Art will speak first about his experience running citizens’ assemblies.
That will be followed by Colin Megill, who’s the co-founder of Polis and President of the Computational Democracy Project. Polis is an online tool to gather open-ended feedback from large groups. We’ve been experimenting with it here in New Zealand and want to see what else we can do with it in the near future.
So I will turn over first now to Art to give us some words on his experience.
The Irish Citizens’ Assembly
Art O’Leary:
Many thanks Anne, it’s a great pleasure to join you here from Dublin, Ireland, where for you guys, it’s last night. So greetings from Tuesday night in Dublin. I’m going to talk a little bit about citizens’ assemblies and the Irish experience.
You know, there is no doubt in anybody’s mind that citizens’ assemblies have transformed the way that Irish people and the Irish political system deals with policy development and dealing with intractable political problems.
But I’m going to start off at the very beginning where I’m sitting at home on my sofa watching reruns of The West Wing at 11:00 one night, and my phone rang, you know, and my heart sank because, I don’t know about you guys, but nobody rings me at 11:00 at night to say they’re having a good day.
So I nervously answered the phone. I was … at the time, I worked in the Irish Parliament. I was Director of Communications, among many other things, and I assumed it was the editor of a newspaper who was ringing me with a story that was going to make my life miserable for the next couple of weeks.
But it was our Prime Minister on the phone and he said, “Art, I have a job for you.”
And I said, “It’s going to be a short conversation and I have a job. Thank you very much.”
And he said, “No, no, we have this thing – it’s called the Constitutional Convention based on the principle of a citizens’ assembly. It’s never, ever going to work, but it’s in our programme for government. We said we’d do it, so we have to do it.”
And I said, “Well, I’ve never heard of it, tell me a little bit about it.”
And he said, “We’re going to put 100 people in a room. And we’re going to ask them a whole pile of questions and they’re going to give us some answers.”
And I said, “Random people?”
He said, “[unclear word or words] citizens who are going to be representative of Irish society.”
So I had a conversation with him at the end of which I said: “Let me see if I have you straight, Prime Minister, as long as I don’t kill anybody or spend any money, I can do whatever I like with this group of 100 people.”
And he said. “I knew you were the man for me.”
So I rang my boss and said, “Listen, I’ve to go and do a job for six months. I’ll be back.”
And nothing surprises me more that 11 years later I’m still talking about it, and perhaps even worse, I’m still doing it.
You know, this is – I just mentioned it to Anne and John earlier on – this is the 30th country I’ve spoken about citizens’ assemblies to in the last 11 months. And there is a hunger across the world – it’s not just a European issue or a Western civilisation issue – there’s a hunger across the entire world amongst the political system to get citizens involved in policy development, and also for citizens to become involved in issues as well, because they want to make a contribution.
So I’m going to talk to you just for the next ten or 11 minutes.
Just the answer to this is very simple, you know – this is not a replacement for the political system or the parliamentary system; it’s a support.
In Ireland, we have developed policy over the last hundred years that our state has existed using three mechanisms. You know, we have a government which is elected, they have a manifesto and they say: “here’s all the things we promised to do.” We have a civil service – guys in grey suits and comb-overs whose job it is to invent a whole lot of reasons why we don’t/can’t do things, but they offer some specialist assistance as well. And thirdly, we engage with stakeholders – people who are going to be affected by policy, the policy change or anything like that.
So all we do with the citizens’ assembly is add a fourth leg to this three-legged stool. This helps and enhances and enriches the decision making program – so it’s not a threat, so politicians don’t need to be afraid of citizens’ assemblies. They’re not a replacement, they’re simply a support.
The second big criticism is that, to use a pejorative term: “But this is just a focus group, isn’t it?”
And well, there’s one big difference between a focus group and a citizens’ assembly. With a focus group, a polling company normally puts a whole pile of people in the room and they ask them questions. You know, what’s your favourite breakfast cereal? Who does your life insurance? Who’s your favourite political party leader? What do you think of abortion or marriage equality or whatever it is? And they gather up all the opinions and write them all down. And at the end of it, they publish this as news: here’s what the people think. And everybody accepted, this is what the people in Ireland think.
With a citizens’ assembly, we gather the people in a room and we fill them with information. We make them experts, subject experts, in whatever the subject is … so abortion, marriage equality, our electoral system, a directly elected mayor for Dublin, whatever the thing is. And at the end of that process, then we asked them the question: “what do you think we should do?” So you get an informed and a reasoned opinion from the people in the room, not just a gut instinct based on whatever information people have to start with.
So there are two big criticisms, and we have in this country, we have dealt with those reasonably satisfactorily in the last 11 years because citizens’ assemblies are now accepted as a part of the political infrastructure and the decision making infrastructure as well.
So … interestingly, our last citizens’ assembly that we had – we’re always asking them questions, our citizens – so on their first day when they arrived, we asked all 100 of them why they got involved. And interestingly, a third of them said they were interested in the subject, one third said that they wanted to make a contribution to public service, and the final third, which was fantastic from my perspective, was that citizens’ assemblies are a way that Ireland has of getting hard jobs done.
That’s amazing, you know, in ten short years that it would become an indispensable part of the way that we develop policy.
And we have the unedifying spectacle in Ireland of fights within the political system over the order in which we’re going to deal with citizens’ assemblies. You know, there’s a row with: “no, no, no, I want my issue taken first before your issue”, because they know that this is a way of achieving outcomes.
So in the short time that I have left, I’ll just talk a little bit about the three main issues, from my perspective, having been doing this now for about 11 years.
There are three critical elements to a successful citizens’ assembly and one is … the first one, always above everything else, the representativeness of the room. The room has to be the same as your society. So, I mean, someone of this stuff is easy. We call it Ireland in one room. So we have 51 women and 49 men, because that’s the way the population breaks down in this country. We have 28 people from our capital city, Dublin, because that’s where 28% of the population lives. We have 11 people between the age of 18 and 24 because that’s the proportion that we have. But we also need to capture a wide variety of other factors. Our unemployment rate is 3% at the moment. We need three unemployed people in the room. We need four farmers. We need 11 people with a disability. We need 17 people who weren’t born in Ireland. And we work really hard. We deal with social class. We know their occupations. We know where they live. We can make a stab at social class as well. And slowly but surely, we build up this representative sample.
And probably the greatest example of the success of this was in the Abortion Citizens’ Assembly, where they looked at this very, very sensitive issue. Ireland has not been able to have a conversation, a civilised conversation about abortion for the last 60 years. And so it was kicked off to a citizen’s assembly without much hope of achieving success.
But anyway, we had a very respectful conversation five meetings, weekend meetings, over six months. And at the end of this process, where they heard from doctors and lawyers and the Catholic Church and medical personnel and women’s groups and all, a whole variety of stakeholders here, our 100 citizens voted to introduce, to recommend, a very, very liberal regime of abortion, which shocked the political system and shocked the media. And they said, well, citizens’ assemblies don’t work, the Irish people will never vote for that. So the citizens in the room voted 67 to 33 in favour of this regime. When the question was put to the Irish people in a referendum, the result was 66 – 34.
So that says two things. One is that the … that room was absolutely representative of the Irish people. But more interesting from my perspective is that the Irish people were way ahead in their thinking of the political system. And this happens, the behavioural economists among you will know this as the ‘wisdom of crowds’. In the recent pandemic, Irish people were wearing masks on public transport months before the government told them that they had to because they looked at the evidence and they said: “well, I’m going to protect my own health and protect my passengers and I’m going to wear a mask.” And then when the government came along six months later and said, you’re going to have to wear a mask on public transport, we laughed: we’ve been doing it for a long time. So sometimes the political system isn’t on top of these issues and in the way that they could or should be.
So representatives of the room, absolutely critical. Whatever your issue, whether it’s a small community or a national question, your room has to look like your community.
The second question or the second issue is the issue itself. Not every subject is suitable for a citizens’ assembly. Normally, we would need a big, meaty issue with a number of policy options, a variety of possible solutions to this challenge.
For instance, in the current climate, a country might look at whether you would use nuclear power as an energy source, and that would be legitimate. Would you ask a room full of randomness to design your nuclear power station? Probably not. So you have to be very careful about the issue that you select.
And our political system, because of the success of citizens’ assemblies, said, woah, citizens’ assemblies can solve everything. You know, so every single intractable problem we have, … it’s just we need to be very careful about the issue that you choose.
The third issue that you need just to bear in mind is what happens afterwards. I mean, in Ireland, we’re quite good at this in the way that some of the big failures all across the world happen because people make rash promises.
You know, the first French citizens’ assembly where President Macron had his gilets jaunes problem and he decided to create a citizens’ assembly to look at the issue of climate change. And he said, well, listen, you all go off and think of this issue and send me the recommendations and I will go and implement them. And then, when they completed their work – 150 citizens and 18 months – and they sent a report to Parliament and to President Macron, and he said, well, I can’t do that one; and well, this one is just impossible, I can’t do that either; and no, I’d never get away with doing that. And slowly but surely, he removed the authority of the assembly.
The Irish system is reasonable. We now have … Our first assembly had politicians on it, the other five since have just been purely citizens. And we’re very clear at the outset: Parliament establishes the Assembly and we say: you go independently and do your work and then you send your report back to government or back to Parliament, and Parliament is going to create a committee to consider the recommendations. Then the government is going to come into a special debate in the Parliament and we’re going to reply to every single one of the recommendations. We’re going to tell you which ones that we’re going to accept and when they’ll be implemented. And we’re going to tell you which ones we can’t accept and give you a reason why. So everybody very clear at the outset in relation to why this works and why that their recommendations … the government needs to have some kind of decision making role in this. We have no quibbles with that at all.
And, as I always encourage our citizens, you know, is that the more reasonable you are, the more thoughtful you are, the more work you do here, the better the chance that the government will accept your recommendations. Because if you say something mad or ludicrous or something that isn’t politically acceptable, then it’s not going to be accepted. And you have to accept that as well.
So I think that’s just 15 minutes, Anne, that’s as short as I can make it. So very happy to take questions about this.
It has been a great experience for Irish people and I hope that the people of New Zealand get the opportunity to try it out in the not too distant future.
Anne Bardsley:
Thank you, Art. We have so many questions about citizens’ assemblies. We did have a one recently in Auckland on our future source of water and it was an empowered assembly and the water authority did take up the recommendations, as you’ve just discussed. That’s our one example where that’s happened so far. And it was only last year and we’ve run it through Koi Tū. So there is a lot of interest in more of these.
And I have lots of questions and there’s already a lot of questions in the chat. But we’re going to move quickly just to get Colin into the conversation. He, as I said before, he developed the online tool, Polis, which is a mechanism to involve even more people into a conversation about a complex policy issue.
And we’ve been trialling Polis. We think it’s a great tool. We want to really try how to, see how we can get that representativeness in and understand representativeness in Polis but we’ll get to that question later. And I will now turn over to Colin and let Colin give a bit of a background on Polis. Thank you.
Polis
Colin Megill:
Thanks so much Anne, and thank you, Art, for a lovely, lovely presentation and really excited to hear that story.I’m just going to share [these presentation slides] briefly just to confirm that everybody can see this and just get some confirmation that everyone can.
Anne Bardsley:
Yes, we can see it, Colin, thank you.
Colin Megill:
Okay. So … great to be with you all today. My name is Colin Megill. I am co-founder of Polis and President of the Computational Democracy Project.
I will briefly say that because there is a technology, which is open source and on the Internet, that anybody can use, like the Government of Taiwan and the Government of Finland can have their own versions that they own and set up and maintain, the Government of the U.K., etc. And that is separate … and I’ll just note that, for naming at the beginning, that when we’re talking about Polis, we’re talking about a technology and a method, and when we’re talking about the Computational Democracy Project, that’s a US-registered 501c3 that is supporting and maintaining and advancing the technology. And so I’ll use those two throughout the presentation.
So I will briefly give a bit of background on the tool and the usage and trajectory, and then talk a little bit about the overall philosophy and of tool building in this space, and some future thoughts, as well as some references to new usage in New Zealand, which is really exciting.
So Polis is a tool for making meaning of perspectives at scale and it does use machine learning and artificial intelligence to do that. There are few ways to talk about it. We can say that it produces a kind of a map of an opinion space, so a bunch of people participating, a bunch of things that they have to say. And we also can think of it as kind of like a survey that is created by the people that are taking it is one way to think of it. And we will talk a little bit about what methods and techniques it implements over time.
First, before we do that, why? Why the technology? What’s the motivation and what’s the history?
Polis started development in 2012 – we started building. And for me, this question goes back to issues of scale in democracy. How do we coordinate large groups of people? We saw this on display with social media, Twitter, in the role that it played in Iran’s Green Movement and in Tahrir Square and in Occupy Wall Street, where people were being mobilised. But that didn’t necessarily mean that there was coherence or that the same platform that was getting people into the streets was able to bring them together or keep them together afterwards or was a tool for movements.
In this case, we can see that a movement and especially, let’s take just Occupy in the United States, people coming on camera and saying that they’re speaking for everybody in the movement. And this is really common, right? People go to a square, everyone has a general sense of why they’re there, but in fact, there’s lots of different interests and lots of positions.And so, you know, sometimes there’s a distillation of this in some movements, like in Hong Kong: 5 demands, not one less. And so this is a background and a backdrop of saying we have activity in a political space and we have lots of participation, what can we do with 100,000 or a million people? What can we do with even 10,000 or even 1000 people if they want to participate in a policymaking process?
And so the goal here is to leverage machine learning for deliberative democracy to make meaning of large groups of people so that we can produce a map that can be useful for all sorts of different processes, citizens’ assemblies included.
But this is not necessarily replacing any other policy making process, but we would say it’s augmenting and potentially enabling new ones.
Okay, Polis as technology is completely open source, all of the code, and there’s also a whole bunch of reference data that has been generated by exercises, including a whole bunch of open data from New Zealand, much of it, all of it generated by Simon Wright who is on the call and Trust Democracy.
Okay, so one of the larger and well known case studies at national scale of Polis’s technology, which some of you may or may not be familiar with, is the case of Taiwan. In Taiwan’s usage of Polis was specifically geared towards technology policy. And this is amenable because if we want to have lots of stakeholders and we want to enable new forms of policymaking and processes which operate at nation scale and must move faster.
If a venture-backed company like Uber or Airbnb is coming into a country, then they may change the housing system or the transportation system overnight. And so the idea of producing a map of many stakeholders and then creating new forms of the policymaking process that can leverage that map very quickly to make intelligent decisions quickly, bold, potentially bold trade-offs, and yet intelligent trade-offs quickly. This was seen as a good kind of testing ground and proving ground for new technology-enabled policymaking processes.
And so Uber was in Taiwan, in the case of Taiwan, regulated through a process called vTaiwan, and vTaiwan leveraged Polis to initially engage thousands of people across the spectrum of taxi drivers. In this case, a spectrum of stakeholders – taxi drivers, riders, uber drivers, the general public, policymakers, the companies themselves and the representatives – and the results of Polis in this case, and the consensus that was that was generated within the platform, did go through and become law. And so this became a very well-studied example since there was a kind of generative process by a lot of stakeholders that then, and I’ll also mention ‘wisdom of the crowds’ here is a good reference for that, did become, then through a consensus metric and mechanism which I’ll discuss in a second, did become law.
Polis has been used, after Taiwan, in a number of key cities, including New Zealand [and] recently used by the UNDP – United Nations Development Program – in Pakistan, Bhutan and East Timor. There is a in this case [referring to a video playing in the presentation slideshow], you see a number of politicians in tuk-tuks with iPads engaging youth going around the city, and they’ll have a conversation. New statements or new dimensions that come up go into the conversation. Then everybody on every iPad can participate. And then there’s a conversation that comes out the other side of that, which is where there’s meaning being made, which is greater than in any individual conversation, which has happened across all of them. And then that can also be shared out widely over social media channels for a conversation. And that’s language that we generally use to discuss an individual Polis conversation, which is usually on one topic with one target population.
Okay, so, the system itself. You can see a screenshot here [in the presentation slideshow]. In Polis, participants who come to participate in the conversation can share a share statement – that’s optional. Usually about one in ten people will share statements. And then everyone who participates – you participate by agreeing, disagreeing and passing on statements which have been produced by other people in the conversation. So what this does is … it a structured reply. It doesn’t allow anyone to reply to anybody else directly. All of the statements which are submitted go into a growing pool of statements and then they’re shown semi randomly to everyone in the conversation so that they can react. And that structured reaction is effectively what we are then doing statistical analysis on.
Okay, so, other use cases, beyond UNDP, within government, in this case, I’ll point at the UK policy lab, which has used it with a couple-thousand-plus civil servants inside of the government on designing a policy design.
And I can also point towards usage outside government. And in this case – there’s an interesting story here – Anne Applebaum in The Atlantic pointed towards Polis specifically that Twitter should attempt to use Polis and Polis-like methodologies to address misinformation. Twitter, after this article came out, did in fact reach out to us – this was before the acquisition, under Twitter’s previous leadership. They did launch a program called Community Notes – maybe some of you have seen Birdwatch, it was previously called Birdwatch, then called Community Notes. It is built on Polis’s methodology and algorithms. They have a paper, which cites Polis, which I’m happy to share with everybody here, and that was covered in Wired.
There was some discussion about that and so there’s also some opportunity here to talk about what does it mean to scale these systems up, these kinds of systems – Community Notes is a Polis-like system. It uses a slightly different algorithm that operates in continuous space, which is … you can just forget that. I must say, if there’s a question about what the differences are, I’m also happy to discuss the specific algorithmic differences between Twitter’s implementation of this kind of system and ours. And we responded with a with a paper about how to generalise Twitter’s system or systems like these, And I’m happy to discuss that as well, if anyone’s interested.
These kinds of systems are, were termed, bridging systems by researchers at Harvard and King’s College. The idea behind bridging algorithms in general – Twitter’s algorithm, Polis’s algorithm – is that they’re basically, you know, you’re looking at groups and clusters of people as they are in their own words, and then you’re looking to show them content and expose them to content and then find areas that may bridge between these clusters. So if there’s five groups, what brings groups A and E together, and what brings groups B and D together? And this is interesting in multi-stakeholder negotiation, because some groups may differ across many complex issues, as all facilitators know. And so we’re really producing tools for facilitators to then take that information about where groups might have breakthrough agreement and then bring that into subsequent stages of policymaking and discussion.
Okay, so Polis is built to produce coherent output from, scalably, many people. Our largest conversation that we’ve run is a 33,000-person conversation in a political party in Germany. And some of the principles at its heart are: there in non-violent communication and the idea that if you mirror back someone’s perspectives to them first, and let them know they are seen, they’re more willing to accept that other people think different ways, and what those ways might be, before you show them that actually you agree on these few issues. So we’re bringing the consensus to the fore and using it as a metric for how you prioritise content and show content, and then also how you make meaning.
Okay, so I kind of already showed this before, but in general, this is the interface. People can submit statements, they go to a pool, then there’s opportunity to agree, disagree and pass by each participant on each statement submitted by everyone else. And this produces an opinion matrix, produces basically what everybody has to say – and that’s maybe, again, one in ten people who participate are going to submit a comment – by what everybody thinks about what everybody had to say. And that’s pretty meaningless to humans – this is a giant matrix of like one, zero and negative one for agree, disagree and pass – but you can make meaning of that if you apply techniques of machine learning called unsupervised learning.
And generally in unsupervised learning you’re kind of going out into the data in search of meaning – you’re looking for patterns and then you’re clustering and this is based on similarity. So the idea in Polis is that those people who vote similarly across many statements are going to be clustered together and those who voted differently are going to be clustered apart. And then across many interactions, pattern will emerge in the data. And you do this automatically every time someone votes.
Again, the largest conversation that we ran – and you can kind of see these ratios at work [in the figures in the presentation slide] – 33,000 people submitted, 783 statements – so not as many statements – but a lot of agree, disagree and pass – a lot of voting, almost 2 million votes. So this is also an attempt to engage those who might lurk and not participate elsewhere where the primary mode of interaction is standing up and saying something or submitting, you know, statements in a forum – most people don’t tweet, most people don’t submit comments on Reddit but people will like and upvote and things like that.
Okay, so another principle here at work is distance and, you know, having in-person things versus online things. As we move to more online mediums of communication in more online forums, can we show that that there’s – in this case [referring to a map of Canada in presentation slides], in the national conversation on media and visual arts and careers in visual arts – over time, can we show that the geographic distribution is better off in reaching people in more distant areas, in this case in Canada, than we would otherwise if we asked people to show up in person?
Okay, once again, we kind of see way more people willing to click agree, disagree, pass than they are to write. The number of actions with writing is a heavy and high thought process, agreeing and disagreeing and passing is a kind of low effort process, and so you combine these things and that’s part of the strategy.
And comment boxes, in general, don’t get … they might get 577 statements, but they won’t get necessarily the numbers of people and the numbers of votes, especially, is what we’re really trying to differentiate against many existing government processes of just, you know, having a comment box or a submission form.
So to the degree to which this is a producing much more data and much more meaning, it’s meant to be, and intended to be, an input to deliberative processes. In this case, we can point at policymaking processes like in Canada or new forms of online participatory forums like in Taiwan.
Okay. Some limitations to a system like this. The number of comments in the system. You can have, for instance, a million people vote on ten statements but you cannot have ten people vote on a million statements. So the number of comments in the system is a limitation. And so that’s something that is an active area of research and development for us.
[Another limitation is] voting asynchronously, [which] means that as you proceed through the conversation over time, [say] it’s open for two weeks, some people that participate in the beginning might not come back at the end. And then also integrating and summarising results is also a specific challenge.
Okay, so what I want to end on here in this is that the history of computation is bound up with counting as statistics about populations. And [referring to a pictures in presentation slides] in this case is the Hollerith electric counting machine – this was for census data – and also UNIVAC was for census. The government was trying to do millions of operations continuously and invested in the foundations of computing before, during and after World War Two.
And so, you know, there’s an incredible book that came out recently by Jill Lepore on the history of data brokering and simulating and predicting elections, and the way in which that led to modern election voting and electioneering in the United States. This particular story, I think, is very timely because it gets at the foundations of something we kind of take for granted at this point, which is that there are giant opinion polling organisations. These giant opinion polling organisations are attempting to measure the population, predict the votes, predict which votes matter, and then go through a process of advertising to.
And so, you know, Jill Lepore’s book gets at some of the, you know, this early protean efforts to model the public and then who ended up owning those models. And if we contrast what’s being done in a census kind of operation, where we’re counting the population and counting their attributes and properties anonymously, we can see this is a really big contrast with what large corporate players and then large political campaigns are interested in doing with population data, which is to say, you know, they’re gathering all of this data and they actually do know what bridges society. The political campaigns are well aware that most Americans are in favour of certain gun regulations, but those are not useful for advertising. And so the point here is that the way that we gather opinion and get it into the public discourse, what stories we tell with it, really matter and that, currently, our method and means of doing that is wrapped up with elections, as they are, and consensus is discarded. And that if perhaps we had a different process, that with different tools and different intentions, we can build models of the public that are for the public, and public is excited to participate in them, and data is open, and then we can use them for very different kinds of processes and insert them into very different kinds of processes, which quite affects the downstream policy-making process.
And so the idea here is that there is an opportunity to return agency and agenda setting power to the public specifically in the case of making these models about themselves. And that’s something that we’re very passionate about and the effort behind getting tools to the public to do this kind of work is a primary preoccupation.
And so some of the things that we do in that direction: we maintain the code base and make it available, pol.is which is free and available to all public institutions and non-profit institutions to do, go and use, and we pay for the compute. We organise methods and training around that and we guide implementations around the world. And we do advanced research and development to advance this kind of work in the world and the application of machine learning to deliberative democracy in theory and also in practice.
I believe this is a generational opportunity to transform policymaking, because as data gets worse because people check out because the data is gathered for exploitative means for exploitative reasons, there are permissionless ways to begin to think about data differently and to think about the tooling for gathering data differently and the entire, effectively, lifecycle of political data in democracies. And so I think that, you know, this is an area where I’m also happy to talk or field questions about how we might look at both the policy-making processes and also the kind of data all the way through – you know, there’s a system level aspect there.
I really appreciate you having us today and I’ll pass back to Anne.
Question and Answer Session
Representiveness, strategies to achieve it and the risk of hijacking by organised interests
Anne Bardsley:
Well, thanks for that. That was a very intense introduction to Polis.
And I hope people, I mean, we have so many questions in the chat, I think we could have a whole additional hour on this, but we only have another 20 minutes, so we’ll try to get through what we can.
There are questions about, and we’re back to Art’s first point, first important point about citizens’ assemblies, and that’s about the representativeness and how we get it in the room. We can talk about that whole sortition process, but it’s also a question for Polis, where it’s open to thousands of people and can it be hijacked by interest groups, and on a small scale, we have seen that happen a little bit, that it can skew the conversation depending on who is in, who is participating, because it’s quite open as opposed to a citizen’s assembly, which Art can speak to, which uses a very strict sortition method to ensure that that participation is representative and in some cases will actually sort on, you know, whether they have an advocate for a certain position or not, you know, that that can be used as a criteria to establish your assembly.
And that’s the one question I have for Art, because you described it very precisely that this is Ireland in a room and is it the same every time or do you sort on different demographics that are more important to a question because we know it’s quite difficult sometimes to get every demographic bucket filled and which are the most important ones for a certain question? And also that question, so this is for both of you.
Colin you’re still on. So if you want to talk about that representative question for Polis, that would be great.
Colin Megill:
Yeah mine’s quick and I dropped a link in chat of Polis being used by Demos, the UK-based think tank, with a representative sample. Polis as a tool is sample agnostic so it really depends on who you invite.
Anne Bardsley:
Right, so have you seen the conversation sway in a certain direction and, you know, because you can actually look at the demographics of those opinion groups that form, how often? And what’s your take on how balanced those conversations are if it’s just open or not using a representative sample?
Colin Megill:
Great question. I would say that’s completely dependent on the sample and when we see it used with a representative sample, we would, you know, get what you would expect.
And if it’s with a, for instance, in the case of Uber, Uber invited all of its drivers and they blasted out messages. It’s important to remember though, in the case of Polis, I think, what you’re doing is you are attempting to map the stakeholders so anybody that gets invited is going to be mapped. And so we don’t necessarily … if there’s a smaller group or a smaller number of people with a distinct opinion, that’s something that will show up and be and be mapped.
It’s basically to say that when Uber did that, and when Airbnb did that, those people showed up as citizens as well. We have seen attempts by, in that case, a corporate actor to send a thousand people towards the tool. But when they showed up, they participated as citizens in a conversation and didn’t necessarily, they weren’t necessarily on a corporate message or agenda. I would say they were … it was useful, though, to have the population and to have it mapped.
So I think, you know, there’s two ways to use it, in general: a stakeholder way – it’s more of a snowball sample of trying to get as many stakeholders in as you can – versus a random sample, a representative sample. Those are the two most effective ways I’ve seen it used.
Anne Bardsley:
Okay, Art, do you want to answer the question as far as the Irish assemblies, are they always using the exact same sortition criteria or does it vary?
Art O’Leary:
Yeah, no, it’s exactly the same, you know, because I mean, when you’re looking at marriage equality, you don’t necessarily want more gay people in the room, because particularly these issues are going to be put to a referendum of the Irish people as a whole and for it to have any value at all, it needs to be representative of society. So every time we have a census, then obviously things change what … we use the same Sortition criteria all at the same time.
Can I just quickly go through … there was a number of questions in the chat about how we select our citizens, and I didn’t spend any time on this in my initial conversation. Our Prime Minister writes a personal letter of invitation to 20,000 randomly selected citizens. So we use our National Postal Service database to identify random households based exactly on the population in our country. So 20,000 people get a letter saying the government has a problem and will you come and help us solve it? And that the most recent one was drug use. We’ve done biodiversity loss, we’ve done climate change, abortion, marriage equality, all of these things. And the normal response rate internationally is something between 3 and 5%. So you would think that about a thousand people would reply from which we would select our hundred, and that’s how we do it. Our most recent citizens’ assembly on biodiversity loss, we had two and a half thousand people reply, you know, 13% of the population, who received this random invitation, put up their hand and said, please, I’d like to be involved, you know, which gave us an …
In the [Zoom] chat … yes, we do pay our citizens for their time and it’s very important. Yes, we do pay for childcare because we recognise that particularly women between the ages of 25 and 44 find it impossible to get away because they have childcare arrangements. So we allow them, bring their families to the hotel and bring their husband and a partner, whatever it is, or else if grandmother or a partner is at home minding kids, then we give them a few quid just to acknowledge that there is a value in having parents there as well.
It is very difficult. Some people, particularly in the hospitality sector, find it very difficult because weekends are their busiest time. So waiters, waitresses, people working in minimum wage jobs, you know, weekends are like Christmas Eve for them.
So we do our best as well. I’ve been called upon … I always treat the 100 citizens as these are my children, is my family now for as long as they are with me. So I have rung their bosses to say this person needs a weekend off – happy to do double shifts during the week, please go and arrange it. Your government thanks you for your service and I’ve arranged letters from our Prime Minister to thank them, and all sorts. We do whatever we can to get the room to be representative and it’s where most of our effort goes into.
Anne Bardsley:
Yeah, I know that that’s a really big part of getting the assembly to work and that personal touch too, I mean, I think the fact that your letters are coming from the Prime Minister probably carries a lot of weight but was your first citizens’ assembly … have you found that this has grown and it’s easier to get people to agree now that they understand the importance and what these assemblies do and mean, because we’re way at the starting place where people don’t really know what they are. So it’s much harder to get people to say, yeah, I’ll do that. I mean, although people are interested, it’s a harder task. And have you found anything easier?
Art O’Leary:
Yeah, it’s impossible. You know, when I was designing the original system back in 2012, I had hopes that this was going to be like Willy Wonka’s golden ticket, you know? People will be waiting at home for this invitation to arrive. Our first citizens’ assembly, we used a polling company and we got them to knock on doors, you know, and you turn up in the main street of Ballygobackwards and my job here is to get a 65-year-old man and a 23-year-old woman, and I can’t leave this town until I have these people. One in 40 people said ‘yes’ when we knocked on doors, because you can imagine, you know, out of the blue, someone knocks on the door: hello, I’ve got a fantastic opportunity for you. You can spend a year of your life for no money – we didn’t pay them for the first one – for no money in a room with 99 other people to answer some questions that the government wants you to answer. And I mean, I’ve got calls from our police service, you know, because some people thought that these were burglars casing the joint rather than a polling company, because the idea was so preposterous that – what is this thing, a citizens’ assembly?
But now? The most recent one where we had a response rate of two-and-a-half thousand out of 20,000, said that they wanted to give up six weekends in a single year. And in order to do this, we stay in a nice hotel, we feed them well, we treat them really well, and they feel they’re making a contribution. This is about making the experience as warm and as comforting for people – they have to want to come the next time.
Localising and scaling citizens’ assemblies
Anne Bardsley:
Yeah, I mean, there are different scales that you can do this in. The Irish ones are so commendable. To be able to span that over a year and so there are full weekends that you use. I guess that is not a small undertaking and we are looking at how … there’s a lot of interest in can you scale back these processes and what kind of power do they have if you can do them on a smaller scale. So we’re working on that. We’re also working on the question of what does it mean to localise this process and do you think that there’s anything specific about the Irish culture or that you would modify or think wouldn’t work elsewhere because we have specific issues here in Aotearoa with our Treaty obligations that make it a different conversation sometimes? So we’re trying to work through that, but any advice would be most welcome.
Art O’Leary:
Listen, I mean, it’s again the three issues: the representative of the room, of the community that that’s supposed to answer the question, the issue that you select, and how you deal with the outcome.
You know, in my experience, I’ve been to your country a couple of times and in my experience New Zealand is the place most like Ireland, outside of Ireland. If it works for us, it’ll work for you. I have no doubt about that as well. So, but listen, happy to answer any questions or to be of assistance as you’re doing this.
Interesting, I’ve spoken, in July of last year in New York, in upstate New York, about different assemblies to a room of American politicians. And, I mean, it was a profoundly depressing experience, you know, because their society is so divided on some of the big issues, you know, Roe versus Wade, you know, health care, gun control, racism, discrimination, all of these big issues which you would think might be suitable for a citizens’ assembly. But the real successes were happening at a community level, you know, at the small communities all gathering together and saying locally: we have a problem that we need to solve.
And my favourite one was three small towns in upstate New York, and each of them had a very small hospital, none of which was working very well. So the health authorities decided that they were going to knock down the three hospitals and build one hospital, and they created a citizens’ assembly to select the town who was going to have the hospital. You know, so every citizen wants, needs [it] to be in my town because health care, etc., etc. So they spent time in a room and they looked at the issue from all perspectives – from where doctors and nurses wanted to live, transport arrangements, the site, venue and access, and all of these things – and they came up and they made a recommendation and it worked fine. It was beautiful. So in a place where the big, big issues are really very difficult politically, perhaps the best way to start is at a grassroots community level where you can see the power of assemblies. And as I always say, nothing succeeds like success. If people see that this is something that can work to deal with difficult problems, then they’ll go for it.
Is the key difference between citizens’ assembly and Polis that one informs people and then asks questions and the other asks questions of uninformed people?
Anne Bardsley:
That’s great and I agree. You’ve kind of just talked about that whole bridging of polarised communities, which is also, as Colin said, what Polis is trying to do. And there’s a very good question in the chat just now about that, so I’ll get Colin to address quickly: “Is the key difference between citizens’ assembly and Polis that one informs people and then asks questions and the other asks questions of uninformed people?”
I wouldn’t say that’s exactly the difference because there are ways to help inform people through Polis as well but I’ll get Colin to comment on that.
Colin Megill:
I wouldn’t necessarily agree with the distinction in this case. I would say you can do as much participant education as you’d like. And in fact, Simon Wright and the group Trust Democracy, when they have run them in New Zealand, have done a huge amount of participant education beforehand.
And so I think that again, Polis is a tool for the interaction and meaning making and summarisation step, but it’s not necessarily going to select a population and it’s not going necessarily going to educate them. It really is just for that narrow that narrow role.
And it seeks to play a relatively narrow role where you want to put some thousands of people through something at a relatively minimum amount of time and make quite a lot of meaning out of it. And I think that I would contrast it mostly with the level of commitment.
I think it’s actually one meaningful way to differentiate them: in attempting to engage thousands or tens of thousands of people within a matter of 15 to 30 minutes, then there’s going to be necessary trade-offs to what the output is.
Combining Polis and Citizens’ Assemblies
Anne Bardsley:
Yeah, and I agree. We are thinking of actually experimenting with that in Polis – of how you, if you can take a group through a process of having a choicebook [a method of information provision] ahead of time – and whether that changes the conversation.
But I also think that Polis could be used piggybacking on to an assembly where people are observing a representative group making a decision. And then that can feed the framing of a Polis and carry the conversation on.
Colin Megill:
I’ll actually share from our side. I’ll share here in the chat … there a facilitator, our Chief of Operations and Head of Partnerships, Liz Barry, has written a memo – she’s a professional facilitator – has written a memo as she has helped people around the world, who are facilitators using Polis in the context of citizens’ assemblies. The memo is on how one might go about thinking about applying Polis to a specific set of methodologies that are seen as best practices and interleaving these technologies. And so I’ll leave that in the chat and then I’m happy to answer any questions around it as well.
Information provision, experts witnesses, mis- and dis-information
Anne Bardsley:
Yeah, there is a question about the questions because there are so many great questions here. And I think we will … post answers to these questions more fully later. But I actually see a question from Chris Donney that I’d like to ask, which is of both of you … about how scientific evidence and other forms of knowledge are provided to the citizens’ assembly participants, or those using Polis, I mean, I know how we’ve done it, but I’d like to hear … he’s asking specifically: how is uncertainty in scientific and other information relevant to the subject being discussed and actually conveyed and considered by the participants? Do you want to quickly answer that? We only have a couple of [minutes left].
Colin Megill:
I would say Art, go for it, please.
Art O’Leary:
Yeah, well, good, easy, I mean, it’s … we have to be very careful, I think, because some of, many of our citizens haven’t been in formal education for decades, so we have to appeal to all different learning styles. So, I mean, some people like to hear presenters, some people like to watch films, some people like to listen to podcasts, and we’ve started in recent years with the innovations of bringing people on field trips, you know. So with biodiversity loss, we brought them out to see habitat destruction and just to see biodiversity initiatives as well. So we have to make it very real for our citizens.
All the papers and all the presentations we look at beforehand to make sure that they’re accessible. And one of my favourite jobs as a civil servant is to send papers back to academics to say, nah, I don’t understand that, go and make it more understandable. So the whole idea is to make, without dumbing it down, to find that fine line between making complex and technical issues understandable by the ordinary citizen, because they will ask questions anyway.
We have a rule of thumb: for every 10 minutes they spend listening to somebody, they spend 20 minutes talking about what they’ve just heard. So during that 20 minute period, they generate questions, they deal with misinformation and disinformation, which we encourage, by the way, and we also get our experts, some of the greatest experts in the world, in the room to be available to our citizens to answer any questions that they have. So nobody leaves the room until everybody understands what it is that we’re talking about.
Anne Bardsley:
Yeah, I think that is one of the other hard things, aside from getting that recruitment right and getting that representation and all of the legwork that that takes, is ensuring that that information is unbiased and accessible to that broad representative public, and then getting the right people who actually speak the language and not be too academic to the room so that it’s meaningful and their questions are answered in a satisfactory way.
We only have a minute left. I don’t know if we’ve got, Colin, did you have any further comment on the information space and, may be, that mis- or dis-information question that might come into a Polis conversation?
Colin Megill:
I think that we are building tools for facilitators and so I think it’s really a question of how Polis is used. So the population and the framing are something that we enable but are in the hands of the facilitator.
Art O’Leary:
In the 30 seconds that that’s left, … we encourage misinformation, disinformation, any wild conspiracy theory that you have – put it on the table and let’s talk about it. We have the greatest experts in the world in the room; they’ll tell you whether it’s true or not. So it’s something that we need to deal with in a very open and respectful way.
Anne Bardsley:
That’s interesting to actually encourage, you know: what have you heard on this issue and what do you think? You’ve heard this, you know, because that is how the conversation is otherwise going on about social media. So that’s an interesting point that both of these things could be where you, especially in a citizens’ assembly where you have access to experts and lots of different experts …
Art O’Leary:
And I mean, yeah, ‘abortion gives you cancer’ – you know, actually, this is very simple. Let’s deal with this, you know, and it’s very straightforward.
Responses to some of the unanswered questions
The moderator, Anne Bardsley, was able to pose to Art O’Leary and Colin Megill many of the questions that were posted to the chat during the webinar but not all of them. The webinar organisers have drafted responses to some of the unanswered questions below.
What does the ‘representative room’ mean when working in partnership under Te Tiriti o Waitangi
What ‘representative’ means depends on context and should be worked out in collaboration with Māori and other stakeholders. New Zealand’s first citizens’ assembly, the Watercare Citizens’ Assembly on Auckland’s next source of water, provides one model. Other models (e.g., 3-house models) will be developed and tried as more of these processes are organised in Aotearoa New Zealand.
Note that what we might call deliberative processes today in fact have a very long history (c.6,000 years) and have been used in many places by many peoples to resolve issues of all sorts.
How can citizens assemblies activate the necessary political will to address issues such as climate change?
The experience to date has been that citizens’ assembly processes can provide the mandate for political action. The public seems to trust a group of peers to consider and discuss the evidence and develop recommendations. This is similar to criminal juries.
Does the order in which statements are presented [in Polis] matter, and how is it decided?
Statements are presented in a semi-random order. This ensures that early statements don’t get all the consideration. The algorithm give slight priority to newer statements to ensure they are considered. Priority is also given to metadata/demographic statements, which are needed to analyse the data.
How does Polis manage where there may be mixed language conversations?
Polis has built in Google Translate, which works well for Polis’s short, twitter-like statements. Polis translates statements based on the default language of the user’s browser.
Can Polis allow different groups to explore their differences, “learn” from each other and help ‘bridge’ divides?
‘Bridges’ are often built during polis discussions. This is supported by the way Polis maps opinion. By identifying areas of agreement and difference, people with different opinion may realise that there are areas of common ground. By highlighting areas of difference, Polis allows participants to submit new solutions for wider consideration, which can also help bridge differences.
Information about the webinar, topic and presenters
Date: Wednesday 14 June 2023
Time: 8am – 9am
Online: via Zoom (link will be sent to registered attendees)
The webinar will explore the potential of citizens’ assemblies and Polis to include and empower the public in agenda setting and problem solving.
Art O’Leary organised the first Irish citizens’ assembly on constitutional reform. He has been closely involved with the subsequent four citizen’s assemblies, which are transforming Irish politics.
Colin Megill, inspired by his time in Occupy Wall Street, went on to co-create Polis and the non-profit Computational Democracy Project. Polis is used extensively worldwide to facilitate large-scale online discussions on important issues.
Format
15-20 minute presentations from each speaker, followed by Q&A.
The Aotearoa NZ context
We have been slower than other countries in exploring new methods to involve the public in decision-making, but we’re now starting to make progress with some public entities showing interest.
For example, in 2022, Koi Tū organised a citizens’ assembly on behalf of Watercare to tackle the critical question of Auckland’s future water sources. The assembly gave Watercare confidence that open dialogue and deliberation with members of the public can produce constructive solutions.
Trust Democracy worked with Koi Tū to run a polis for the Te Manatū Waka Ministry of Transport to test the use of Polis for involving stakeholders in its project on the future of the land transport revenue system. The results of the polis fed into the next phase, with Koi Tū leading a series of representative deliberative events across New Zealand.
We’re at a critical juncture. Democracies around the world are under enormous pressure as a result of issues like climate change, poverty, housing and equity. Existing democratic processes are no longer sufficient to meet these challenges. Innovations that promote much greater public input and discussion will be required.
About the speakers
Art O’Leary is the Chief Executive at the Electoral Commission – An Coimisiún Toghcháin, which oversees electoral and wider democracy matters. He is also the Secretary to two national Citizens’ Assemblies on the crisis on Biodiversity Loss and local government structures in Dublin. He designed and led Ireland’s first Citizens’ Assembly in 2012, which resulted in a number of successful constitutional referendums. Art served as Secretary General to the President of Ireland from 2014 to 2021. He is an Adjunct Professor at the School of Politics and Government at University College Cork and holds an MBA from Henley Business School
Colin Megill is cofounder of pol.is and President of The Computational Democracy Project. Polis is an online tool used to gather open ended feedback from large groups of people. It is well suited to gathering organic, authentic feedback while retaining minority opinions. Polis has been widely covered in the popular press — including articles from The New York Times, The Guardian, The Economist and MIT Tech Review — as well as in documentaries from the BBC and books from Penguin.
The organisers
This webinar is brought to you by Trust Democracy and Koi Tū: The Centre for Informed Futures at Waipapa Taumata Rau University of Auckland
