Scotland’s AI Strategy: Response to scoping document

Question 1: What do you think of the proposed definition of AI for the purposes of the strategy?

Artificial Intelligence is generally considered to be a field of computer science that incorporates a wide array of different sub-fields, including machine learning, neural networks, computer vision, natural language processing, speech processing. There is a great deal of cross-over from these fields, which makes giving a clear, and useful definition of “Artificial Intelligence”, somewhat difficult.

The scoping document even acknowledges the diversity of the sector. The question from Open Rights Group would be what purpose does such a definition serve? The key questions for the public are likely to be:

– What forms of AI are Scotland seeking to adopt?

– In which sectors are seeking to adopt them?

– What effect will the proposed adoption have on individuals, and the sectors?

Currently this definition gives an illustration of what might be expected to happen- an adoption of techniques that allow computer to do tasks normally reserved for humans- however, it doesn’t give much clarity of what techniques, where, and having what impact.

Further, this definition doesn’t appear to add to the existing definitions, already available. It is also unclear why robotics and automation have been pushed out of scope. These two areas may have significant roles to play in the future economy and if they fail to be included in the AI strategy this would be seen as a missed opportunity.

Question 2: Do you agree that the strategy should be people-centred and aligned with Scotland’s National Performance Framework?

Open Rights Group is supportive of the adoption of the National Performance Framework as a guiding approach. In particular we want to emphasise the following outcomes:

“We respect, protect and fulfil human rights and live free from discrimination.”

“We are creative and our vibrant and diverse cultures are expressed and enjoyed widely.”

“We are open, connected and make a positive contribution internationally.”

With this in mind, we make the following recommendations:

Recommendation 1 : Assess the legal framework that underpins the form of AI to be adopted, specifically sector-based regulation to ensure compatibility with fundamental human rights.

Recommendation 2 : Conduct assessment of individual rights framework in Scotland as it relates to AI, and seek to strengthen the rights-based system

Recommendation 3 : Review intellectual property status for public-private partnerships in the field of technical procurement, adjusting to seek to preserve public benefit and limit the risk of exploitation of public data for private benefit.

Legal framework and Safeguards

It is generally considered that existing regulation, does not provide a strong response to forms of Artificial Intelligence. For instance, the Royal United Services Institute’s research into police adoption of algorithms revealed a widespread concern from police due to the lack of national guidance for the use of algorithms in policing. Further, we are seeing companies seeking to adopt internal practices, suggesting a vacuum in clear rules under which their products must operate.

It should be noted that when the use in Scotland of one of the most prominent forms of artificial intelligence, facial recognition, was explored, a Parliamentary committee found no justification for its use, and cautioned against adopting it without proper safeguards. This pre-emptive investigation of a controversial technology is an encouraging form of governance and one that we encourage the strategy to view as a blueprint: when there is a risk of significant human rights interference, assess the legal framework of the sector prior to adoption or roll-out.

From individual rights, towards group rights for accountability.

Connected to a suitable legal framework is whether an individual is empowered to hold to account an AI system for a decision that has been made about them by the system. The Scottish Government should consider assessing the individual rights framework, its shortcomings and where it can be strengthened as part of its work.

For a discussion of individual rights, many would turn to the standards set in the General Data Protection Regulation, in particular two articles:

Article 22:

The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling,which produces legal effects concerning him or her or similarly signficantly affects him or her;

and

Article 15(1)(h):

 the existence of automated decision-making, including profiling, referred to in Article 22 and…meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject.

Michael Veale and Lilian Edwards explored the current individual rights frameworks suitability for combatingharms that stem for algorithms, specifically the General Data Protection Regulation. The study found that an interaction between Article 15(1)(h) and Article 22 is currently unlikely to present a complete remedy to algorithmic harms. In particular the question of whether a meaningful explanation could be given under the current rules, due to carve-outs for intellectual property (see more on this below), and the sheer complexity of a decision-making system were put forward. The conclusions from this work pointed out that individual rights may be the wrong approach to what are often group harms, and suggested exploring remedies aimed at empowering or protecting groups, such as a “supercomplaint” system to empower third party organisations to take action against harms caused by algorithms.

Open Rights Group recommends that the Scottish Government consider, as part of the work in this strategy, what can be done to strengthen the individual rights framework, including exploring group remedies. In effect checking proposals against the established respect, protect, fulfil framework established at the United Nations includes the right to an effective remedy. An exploration of this area would meet one of the key pieces of the National Performance Framework.

Intellectual Property and the value of open procurement

Another issue, that interacts present within the National Performance Framework and that interacts with individual rights, is the restriction on information to be provided to the public as a result of intellectual property. As a result of the nature of the development and operation of these systems, often by private parties, with techniques protected by Trade Secrets, transparency is limited regarding what is occurring within a system, and how. This limits individual’s ability to query the system, and reduces accountability.

As the scoping document points out, AI requires data to be useful. The data provided creates the insights or value that the AI is able to extract from its processing. Without one or the other the beneficial outcomes are not there.

So who is benefiting from the adoption of this technology? If private parties are in a position to use public data to train their systems, this may have some efficiency gains for the public sector, but what is really improving is the learning held within the particular AI system. Even if there is no public benefit such as efficiency, the system has learned something that can be retained and retooled for the next contract, whereas the public benefit never arrived.

For example, the example of DeepMind’s collaboration with the NHS and the Royal Free Hospital, according to contracts available online, DeepMind claims ownership of any “Developed IP” in their dealing with the NHS. What does this mean for new versions of technologies created on the basis of patient data and NHS resources? Does DeepMind retain all the value derived from that insight? If no openly available research paper is produced by DeepMind, the current form of providing insights, then has the public lost its benefit?

If you cannot separate what the model learns from the model itself, then that question needs to be answered first before opening up Scotland’s data.

Scotland should consider whether the public procurement, and the trade secrets restrictions available to private companies when operating in the public sector using AI techniques truly represents the creation of “an open, connected” society that makes a positive contribution internationally.

Question 3: How do you think AI could benefit Scotland’s people, and how do we ensure that the benefits are shared and no-one is left behind?

Recommendation 4: Conduct comprehensive sectoral impact assessments prior to the introduction of any AI initiative seeking to explore, among other things, what effect adoption will have on Scotland’s people, followed by impact evaluations.

Artificial Intelligence does not impact individuals or sectors equally. As a result society will react differently to different techniques and the areas they are deployed in.

This is well illustrated in the Ada Lovelace Institute report “Beyond Face Value” which explored public perception of facial recognition. The public’s acceptance of facial recognition changed based on who was deploying it, and for what purposes, and also which section of the public you asked. For example, a majority of people surveyed were supportive of the use of facial recognition by police in criminal investigations (63%), whereas a majority of people were uncomfortable with the use of facial recognition in schools (67%). BAME communities, for whom facial recognition technology is less accurate, are more likely to agree that they should be able to opt out of the use of facial recognition technology.

Open Rights Group recommends that during the course of the AI strategy Scottish Government explore how the public feel about other forms of Artificial Intelligence on a sector by sector approach.

More importantly, seeking to establish a consistent sectoral impact assessment for any adoption of artificial intelligence would be a significant outcome for Scotland amongst other countries. Another report from Ada Lovelace, Examining the Black Box, found that there was no definitive approach to assessing algorithmic systems. Importantly this is a growing area that has had developments[2] (for example: AI Now Institute, Alan Turning Institute, the Canadian Government, the UK’s Information Commissioner’s Office) . As a result seeking to establish a formula, contributing to the growing body of practice of both impact assessments, and impact evaluations could allow Scotland to be a standard bearer for the responsible adoption of AI.

Depending on the sector, and the technology, the effect on fundamental rights could be null, or profound. A company that employs artificial intelligence to identify where they can make energy savings is unlikely to create any implications for fundamental rights. But we are seeing in other sectors that the adoption of some technologies will push artificial intelligence into human rights violating territory.

For example, the case earlier in 2020 of the Syri system in the Netherlands, a benefit fraud risk calculation model was found to be in violation of human rights. The system, deployed in primarily low-income neighbourhoods was found to contain insufficient safeguards against privacy intrusions and suffered from a serious lack of transparency about how it worked. The District Court in the Hague concluded that in the absence of more information, the system may, in targeting neighbourhoods, amount to discrimination on the basis of socio-economic or migrant status.

This particular issue, known as the digital welfare state, has received a great deal of scrutiny from the UN Special Rapporteur on extreme poverty and human rights, and national press. It suggests that form some sectors, adoption comes at too high a price. This balancing exercise should be part of the wider discussion across Scotland, and will seek to show that balance is shared and that no-one is left behind.

We would also reiterate that the question of public benefit should also be included in the sectoral impact assessment. Additionally an exploration of a sector’s readiness for adoption of forms of artificial intelligence should include an assessment of the quality of the data within the sector. If a poor level of quality, accuracy, or coverage of data is available it would negatively affect the perceived benefits of adoption of technology.

Question 4: What do you think of the proposed overarching vision of the strategy, and the two strategic goals that are proposed to underpin this?

The scoping strategy makes proper reference to governance as vital component to a sustainable AI ecosystem. However, the use of AI may not achieve the goal of “respect, protect and fulfil human rights…” but it could be an area where, depending on the technology adopted, could face strain. With this in mind a slight change is proposed

The overarching vision of Scotland’s AI Strategy will be to use AI to benefit Scotland’s people and organisations, and help to achieve the transformational change envisioned in the National Outcomes, while respecting fundamental rights throughout.

As the respect of human rights is a key point, we recommend a slight change to the:

– the people of Scotland will flourish and their fundamental human rights will be upheld.

Question 8: Please comment on any other aspect of AI that you feel it is important for Scotland’s AI Strategy to address.

No comment.