………………………………………………………………………………………………………………………
……….
………..
.
.
.
CONCLUSIONS FROM THE
I INTERNATIONAL SEMINAR ON
ADMINISTRATIVE LAW AND ARTIFICIAL INTELLIGENCE*
University of Castilla-La Mancha.
Faculty of Legal and Social Sciences
Toledo, April 1st 2019
I.- GENERAL COMMENTS
1. Artificial intelligence (AI) refers to systems that display intelligent behaviour by analysing their environment and taking actions – with some degree of autonomy – to achieve specific goals. AI-based systems can be purely software-based, acting in the virtual world (e.g. voice assistants, image analysis software, search engines, speech and face recognition systems) or AI can be embedded in hardware devices (e.g. advanced robots, autonomous cars, drones or Internet of Things applications)[1].
Generic reference to AI includes the use of algorithm systems, machine learning and deep learning systems, neural networks, intelligent robotics and big data as the source for AI. It has been identified as one of the five emerging technologies that can transform our society in the upcoming decades and is the foundation of the fourth industrial revolution.
2. The public sector in Spain is already using AI in the execution of its operations and in the provision of public services. However, there are serious problems in identifying and controlling IA that is currently operating or is going to be implemented in the near future. Generally, he information available is only informative – journalistic or institutional – and very superficial. As such, active dissemination of these initiatives and projects is important for their categorization, analysis and evaluation. While solid AI public actions can already be identified with potential legal repercussions, so far the situation is basically experimental, primarily based on behaviour patterns and automatic classification systems, as well as image and spatial recognition. In any case, there is a striking lack of algorithmic transparency and the absence of an adequate perception by the Public Administrations of the need to approve a specific legal framework. There is only a certain amount of concern regarding compliance in the sphere of data protection, which is perceived as a limit.
According to an initial analysis of experiences in the public sector in Spain, AI is used, still in a very incipient way, to analyse data (fire risk, locations to be inspected), process natural language (review of requests made by citizens or detection of irregularities and fraud in public tenders), identifying images (counting of people in public spaces or detecting possible offenders), taking or facilitating decision-making (allocation of state subsidies, determining the streets for police patrolling, identifying schools that may have a higher rate of school abandonment or treatment for a particular disease) or customizing public services (provision of information services, counselling and attention to citizens).
3. Notwithstanding the benefits of the use of AI in the public sector administration, it is important to consider the risks, tensions and violations it may entail for the purposes of legal certainty and fundamental rights such as equality, privacy or the protection of personal data and the principles of the administrative procedure. This is the case, for example, with regard to the inviolability of the duty to motivate as a consequence of machine learning, the reduction in the effectiveness of the right to formulate claims if the operation of the personal data protection algorithm is unknown when Public Administrations use massive data or elaborate profiles or, where appropriate, if minority groups are discriminated.
4. The Public Administration of the future will not only be electronic but also intelligent and will require a very different profile of public servants. It is necessary to reflect on and ensure the training of current and future civil servants, as well as the suitability of the civil service selection systems.
5. This requires research on the impact that IA may have on Public Law and fundamental rights as well as the promotion of new mechanisms to guarantee these principles and rights by default and through design. Public law must, so to speak, be embedded in the source code. In addition, this is a key element in the development of AI in the European Union[2].
6. Law is responsible for the materialization of the so-called Governance and Ethics of IA and its essential principles: the protection of dignity and human rights; the five basic principles of beneficence and non-maleficence, justice, freedom, human autonomy versus artificial “autonomy” and justification and transparency. Regulatory compliance, respect for dignity and fundamental rights, privacy in the design, compliance with the requirements of competition law and other goods and ethical-legal values should be inserted in the code of the design and development of AI, especially in the public sector.
For these reasons, it is essential at this initial phase, to request the participation of legal experts in the design, implementation and use of AI systems.
7. We are aware of the great difficulties in generating a single regulatory framework, given that Law is slow in adapting to technological reality and also requires new regulatory techniques and procedures. The elaboration of a new common European framework and the adaptation of norms and rules of Spanish law are unavoidable. Both regulatory action and good practices are required at all levels, as well as the implementation of regulations and standards already applicable.
Law must enable and encourage innovation, create conditions, structures and institutes that allow technological development while avoiding undesirable risks. Law must also seek to redistribute benefits and costs equitably.
We must be aware that Law in these areas tends to act in a reactive way, not in a proactive way, preventing the stimulation of the development of AI. Likewise, peremptory Law may motivate non-compliance, failing to encourage initiative in society. Private Law may be best to address some of these challenges, as it takes into consideration self-interest and allows creativity, but this may negatively affect the public interest of society as a whole. The aim is to know the extent of what is socially tolerable and to promote a public Law that enables innovation in this area.
8. We also consider that it is necessary to positively assess and be open to the introduction of techniques for a biodegradable law, which is experimental in a continuous beta, while developing legal solutions that are already available, such as the so-called regulatory sandboxes or sunset clauses. In spite of the criticisms that may be directed at them, we consider necessary to move towards the innovation of regulation. It is also essential to make progress in the inclusion of computer scientists and legal specialists in the several regulatory forms of the technological sector. Similarly, we need to be alert and adopt safeguards against the dangers of the lack of legitimacy, capture and algorithmic manipulation of the regulator. We also believe that mechanisms and techniques relating to the principles of precaution and responsibility should be evaluated.
9. In any case, the fundamental principles of transparency and good governance as well as the legal obligations deriving from the citizens’ right to good administration must be reshaped in order to address the requirements of transparency, accountability and justification. A number of fundamental and constitutional rights already impose these requirements: the right of access to information, due process and judicial and administrative guarantees or the right to data protection, among others. Justification includes the access to algorithms and the systems into which they are integrated, the data and its quality, and the identification of the person accountable for the operation. In addition, it includes the reasons of a public decision that affects a specific person or a group, in order to avoid violating the constitutional principle of the interdiction of arbitrariness.
Moreover, it is necessary to have transparency and justification from the outset of the design that generates verifiable data: intelligent systems must be designed in such a way that they can be supervised and monitored either by humans or by other intelligent systems. Likewise, a guardian AI is needed. In short, Law needs intelligent tools to deal with intelligent tools.
10. The rights to equality and non-discrimination must be reinforced in order to avoid the damage that the intense use of big data and AI technologies can generate as they permit citizen control and monitoring systems with more or less spurious intentions. Voluntarily or involuntarily, the fact is that the handling of poor-quality data or a poor design can generate biases and discrimination, which must be minimized. At the same time, it is necessary to ensure that the intelligent system does not lose its effectiveness through minimisation. It is also essential to apply the techniques of anti-discriminatory law for the correction of possible biases.
11. The jurisprudential contributions and case-law techniques relating to the review of discretional powers of the Public Administration and its approach to the effectiveness of the right to good administration (with its corresponding obligation to provide motivated decisions) must be modulated and adapted to the use of predictive tools that support decision-making or replace human activity. This is necessary, since it may reduce the scope of discretion conferred on the administration and its human decision-makers. Nevertheless, the future possibilities of a properly exercised AI are many and the use of algorithms may result in greater objectivity in the decision-making process of the Public Administration.
In this sense, it might be convenient to apply a “piercing the veil” to the automated or algorithmic decision until reaching the human decision. Current regulations only establish rules and guarantees in relation to automated decisions with impact upon people, which excludes semi-automated or decision support systems from any protection. Therefore, we must be especially aware to the fact that humans habitually follow the decision proposed by the automated system.
It is also important to reflect on the need to preserve spaces and generate safeguards to provide human decisions in the areas of administrative and judicial discretion. In this sense, we must value the recognition of a right to a significantly human decision. Likewise, it is necessary to study the possibility of incorporating safeguards to ensure the need for humans to justify not following up the decision using algorithms. Equally, there must be contexts facilitating that humans do not automatically follow the decision of the algorithm.
12. We also consider that there is a need for structural conceptual changes: the determination of the nature of algorithmic systems, including their consideration as a possible source of Law; the reflection on what an administrative record is and what belongs to it, and the need to respond to informality and automated actions. It is also appropriate to assess the validity of the theories of the administrative body and the administrative decisions with respect to the IA activity of the public sector and the limits to the extension of these systems.
All of this also affects key elements of public Law, such as: the guarantees in the approval and verification of intelligent systems, relating to the responsibility in the adoption of the decision-making process; the distribution of roles from the perspective of the democratic State; the displacement of the decision and the burden of proof; the minimization of the human historical tendency towards the fascination and confidence in machines (which dates back to the Age of Enlightenment and before); the role of the collegiate human decisions in the context of the progress of AI; or the revision and monitoring of the systems.
13. We also consider it essential to adapt the public procurement legal framework in order to put the intense public-private collaboration in the development of algorithmic tools, AI and the creation and management of big data at the service of the general interest, while avoiding the capture of Public Administrations. Thus, public procurement must integrate the requirements of transparency and justification, as well as non-discrimination or compliance with privacy regulations. This, in addition, must allow for equality of bidders in contracting satisfying these principles in its design.
14. Special attention should be paid to the peculiarities of the implementation of AI in the context of smart cities, where an important area of experimentation is developing at the local level involving many specific legal questions and challenges.
15. It is essential to address the issue of the configuration of the ownership of data and AI systems and their possible treatment as “data as commons”. In this sense, it is necessary to assess assumptions in which it is necessary to demand that the system is not only a service provided to the public sector, but of public property.
16. It is also necessary to take into account the fight against a new gap: the one derived from the difficulties regarding the ability to read and exploit big data, since only those organizations with sufficient human and economic resources are able to benefit from all the advantages of these new tools.
In recent years, several research groups from different Spanish universities have begun to develop projects, which have addressed the opportunities and risks that can be generated using AI in the public sector from a variety of perspectives. Some answers have been proposed to facilitate the use of intelligence in Public Administrations in accordance with the principles inspiring Public Law. We understand that it is necessary to continue working in this direction, expanding academic collaborations between these groups and bringing in new ones in the future.
II.- WORK GUIDELINES FOR THE FUTURE
On the basis of the premises above, some important guidelines may be identified that could serve to give direction to the legal discussion and the practical application of these technologies in the public sector:
1. In general, it is essential to make a conceptual effort to help clarify the technological reality we are referring to when using expressions such as IA or algorithms. To this end, it would be advisable to promote inter/transdisciplinary working groups that help to specify and, where appropriate, prioritise the technical aspects that raise relevant legal implications, generating clear and precise documents that serve as a reference. It is also necessary to prioritize the technological issues that require an immediate legal response in the context of Public Administrations, distinguishing them from those that, on the contrary, require a medium-term analysis because they are less urgent.
2. Any attempt to tackle the analysis of the legal challenges posed by IA in the public sector should be based on an exhaustive catalogue of the experiences and initiatives already operating in reality, in which a series of minimum data standardised would be specified beforehand from the perspective of legal guarantees.
3. Beyond the great interest presented by the doctrinal analysis of their possible nature as a source of Law (regulation) and with regard to the legal nature of algorithms, priority should be given to the establishment of a due administrative procedure with adequate guarantees. In addition, the existing and applicable legal requirements (data protection, automatization of administrative action, etc.) and any others should be precisely defined.
4. In particular, special attention should be paid to the division of responsibility in the adoption of decisions, clearly distinguishing technical aspects of an executive nature from those which, on the contrary, may affect the definition of public interest. There are three requirements in this respect:
a) It would be very useful to specify what type of decisions and actions would allow a complete automation by means of AI techniques of those others that should be assumed by a person. In accordance with the current Spanish legal reservation to civil servants with a specific legal status for exercising authority, it should be explored whether there should be a correlative “human reservation” for certain decisions. In this sense, it should also be explored if the selection and future training of public employees could reinforce aspects related to AI, including the possible creation of specialized algorithmic intervention bodies.
b) On the other hand, control mechanisms should be established outside the decision-making processes, which should also perform periodical reviews. In this respect, it is worth considering the need to create public bodies specialised in the control of AI.
c) Finally, the role of collegiality should be further examined, taking into account the uniqueness of each of the levels mentioned above.
5. The influence of these technologies on discretionary powers and other similar concepts (undefined legal concepts, technical discretion, legal obligations for good administration, etc.) are particularly significant from the perspective of the control and motivation of administrative decisions, conferring a relevant role to the ethical perspective as an inexcusable legal requirement.
6.In the same way, it is essential to delve deeper into the scope of the requirements of data protection legislation in the public sector. In particular, it is a priority to clearly establish the legal consequences of non-compliance with such requirements in the context of the IA and, in particular, of:
a) privacy by design and by default,
b) principle of minimisation,
c) impact assessment and risk analysis,
d) adoption of measures to ensure adequate security in the processing of information.
7. The influence of the principle of transparency on the specific characteristics of this technology adopts a special significance, although with limits to be defined. It is essential to establish appropriate mechanisms to ensure its effectiveness and respect for the rest of the general interests in conflict (public security, control and inspection function, etc.), at least at a threefold level:
a) in the context of the approval procedure for the use of the corresponding algorithm,
b) in relation to the specific decisions to be taken,
c) in the control and review processes.
8. Public entities must take an active role in management of public procurement regarding the use of AI, by assessing beforehand what conditions should be respected by the private contracting entities and also taking into account the specificities of the public sector. In particular, consideration should be given to the transparency and access to the programming regime and the conditions under which ownership or the rights to use the algorithms could be made available.
9. Special attention should be paid to the implementation of IA in the ecosystems of smarts cities given the singularity of the information treatments undertaken in these areas and their impact on rights and freedoms, which can be particularly intense.
10. Legal prerequisites should be established to strengthen the control function of public entities using IA-based technologies[3].
In relation to each and every one of these ten guidelines for the future, it is worth underlining and promoting the role of specialists and academics in public and private AI research and projects.
In this sense and taking into account the above reflections and proposals, we consider the creation of the Administrative Law and Artificial Intelligence Network to be very timely and necessary. This network will allow the sharing of knowledge generated by the different research groups on the impact on Public Law as a result of the use of artificial intelligence by the public authorities. In addition, this will allow the creation of a permanent space for discussion in order to explore the legal responses that can be offered to the challenges of AI and the risks posed by its use in the public sector. The aim is also to facilitate the development of the use of artificial intelligence in the public sector in Spain in accordance with the requirements and parameters of a democratic state governed by the rule of law. This need is especially pressing in view of the complexity of AI and the multitude of challenges that it poses for Public Law, which requires an interdisciplinary response and the participation of large teams of researchers.
Universities have always been generators of knowledge. Its collaboration with the rest of the public sector and with the private sector must facilitate safe channels of innovation. An academic, multidisciplinary and integrating network can effectively contribute to facing some of the challenges that IA presents.
Toledo, April 17th 2019
* This is a translation of the original Spanish text (available for example at:
This English translation has been made by Ona Lorda, who holds a Law degree from the Barcelona University and is a collaborator at the Research Institute TransJus of the University of Barcelona. Professor Juli Ponce has also collaborated with a final review of the translation.
The elaboration of the original first Spanish version of this text corresponded to those who had entrusted the task of presenting the conclusions of the I International Seminar on Administrative Law and Artificial Intelligence (DAIA): Lorenzo Cotino, professor of Constitutional Law at the University of Valencia, and Julián Valero, professor of Administrative Law at the University of Murcia.
The seminar organizers and the speakers who participated in the Seminar also collaborated n the final version: Juli Ponce, Isaac Martín, Agustí Cerrillo, Luís Arroyo, Luciano Parejo, Ignacio Alamillo, Clara Velasco, Andrés Boix and David Restrepo-Amariles.
The two driving institutions in the creation of the network Administrative Law and Artificial Intelligence (the Center for European Studies “Luis Ortega Álvarez” of the University of Castilla-La Mancha and the Research Institute TransJus of the University of Barcelona), as well as the DerechoTics network and the Idertec research group, which also join the mentioned network, adhere to these conclusions.
[1] Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions, Artificial Intelligence for Europe {SWD(2018) 137 final}: https://eur-lex.europa.eu/legal-content/ES/TXT/?uri=CELEX:52018DC0237).
[2] It is worth recalling the European Commission’s “Coordinated Plan on Artificial Intelligence” of December 7th 2018 and its Annex on “Made in Europe” and “reliable” AI. To this end, the IA High Level Expert Group published on December 18th 2018 the “Draft Ethical Guidelines for Reliable AI”, which was submitted for public consultation. The final version has recently been approved: Ethics Guidelines for Trustworthy AI, available at
https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=58477 . The most distinctive feature of this Europa trademark is the “Ethics & Rule of law by design X-by design”. This includes compliance with basic ethical principles, the creation of ethics committees and delegates in all types of projects and corporations, the development of rules of professional conduct, information technology, users, etc.
[3] In that sense, it is necessary to underline the new automated system created by the recent Act 22/2018, November 6th, from the Valencian Parliament, regarding General Inspection of Services and the Alert System to Prevent Bad Practices in the Valencian Administration.
.
.
……..