Can you enforce transparency when applying algorithms? Should a human be in the loop in AI systems? How do you construct an ethical framework before you apply predictive modelling to welfare services?
The debate on the rise of algorithms and the impact they have on the decision making that affects New Zealand citizens has been the focus of the first session of the Privacy Forum in Wellington today.
Justice minister Justice Andrew Little opened the forum by telling the audience of around 320 attendees, that algorithms in public policy are being discussed at ministerial level. He says privacy law is constantly evolving and urged the audience to consider submissions to proposed changes to the legislation by the closing date of 24 May.
Associate Professor Colin Gavaghan from Otago University is researching how AI is being handled in a number of jurisdictions, notably Europe and the US. He says an area of debate is if it’s possible to design models in which the decisions that are made by algorithms can be explained. That is, how can you maintain transparency when the algorithm is straightforward at the start but once it has been applied to data, it ‘learns’ and changes and as a result its decision-making process may become opaque?
Transparency is also a key consideration for the Ministry of Social Development, according to Deputy Chief Executive Nick Blakeley, who says MSD is looking at how it can be more open about how it uses people’s data. “It’s not just putting an algorithm on the website, you have to explain how it is used, what is the business setting, is a human in the loop?”
Blakely admitted that the MSD hasn’t always got it right, but that it takes privacy and ethics extremely seriously and is building a privacy human rights ethics framework to be applied at the beginning of the process when designing systems.
Earlier in the week Privacy Commissioner John Edwards welcomed “a shift in MSD policy”. Noting the announcement by Prime Minister Jacinda Ardern and Social Development Minister Carmel Sepuloni to put plans in place “to reach a shared understanding with NGOs and service providers about how personal information collected by the Ministry of Social Development would be used, shared and protected.”
“The Government plans to invite over 1500 groups to participate in its consultation on how personal information can be used in investing in social wellbeing. The move comes after a previous policy required NGO service providers to provide information about individual clients in order to receive government funding. The information included clients' names, number of children and other social services they engaged with,” he says.
Today’s forum is part of Privacy Week, which the Privacy Commission kicked off by releasing its biennial survey which is carried out by UMR Research and involves a telephone survey of 600 and an online survey 1000. It showed that the three biggest issues for New Zealanders concerned about privacy are children putting information about themselves on internet, businesses sharing personal info with other businesses and the security of personal information. In the survey, 62% of New Zealanders said they trust the government with their personal information.