AI and Law I - Lecture 9: European (proposal for) an AI Regulation I

Please also view the Lecture 1 Course for links to all the other lectures. Note: This lecture was used to finish the content of Lecture 7. The cards for that have been moved to Lecture 7.

35 flashcards · Law · by daniel-eder

1. AI Act - Current state 2023.12?

   2021: proposal by EU Commission 2022.12.06: Discussion in the Council 2023.06: Updated proposal 2023.12.12: Passed EU parliament

2. Subject Matter of AI Act?

   harmonized rules prohibition of certain practices specific requirements for high risk ai transparency rules for certain AI systems rules on market monitoring, market surveillance and governance measures in support of innovation

3. AI Act: When can prohibited AI Techniques be used?

   Only Biometric identification and only for law enforcement

4. AI Act: When is something a high risk AI system?

   When listed in Annex II AND required to undergo a third-party conformity assessment according to other (listed in Art 6) EU legislation When it is a safety component of a product required to undergo such conformity assessment When listed in Annex III unless purely accessory for the relevant action or decision to be taken and not likely to lead to a significant risk

5. AI Act: what is a provider?

   natural or legal person, public authority that develops an AI System or puts it on the market or puts it into service

6. AI Act: what is a user?

   any natural or legal person, public authority under whose authority the AI system is used -> Depending on the type of AI system the use may affect persons other than the user

7. What are the requirements for high risk ai systems?

   Art 8-15 Risk management system Data Governance Technical Documentation Record Keeping Transparency and Provision of Information to Users Human oversight Accuracy, Robustness, Cybersecurity

8. What is a "Quasi-Provider" of a High risk AI system?

   distributors, importers, operators or other third parties, if they: put their name or trademark make a substantial modification

9. Consequences of violating AI act?

   Administrative Fines: from 5.000.000 EUR / 1% of total annual worldwide turnover to 40.000.000 EUR / 7 % of total annual worldwide turnover multiple layers of maximum fines inbetween -> the concrete maximum fine depends on which articles have been violated

10. Obligations of Providers:

    Art 16-25 Quality Management System Documentation Keeping Conformity Assessment Automatically generated Logs Corrective Actions Duty of Information Cooperation with Authorities Authorized Representatives inside the Union for providers established outside the union

11. Who does the AI Regulation (AI Act) apply to?

    Providers placing on the market or putting into service in the EU Providers where the output of the system is used in the EU Users who are physically present or established in the EU Importers and distributors of AI systems product manufacturers who place on the market or put into service AI in their products under their own name or trademark in the EU Authorized representatives of providers in the EU

12. What is the AI act not applicable to?

    purposes outside the scope of EU LAW military purposes purposes relating to defence and national security

13. What is the prohibited AI technique "subliminal influence"?

    -> influence the subconscious of persons to change behavior e.g. hate speech being preferred on social network feeds

14. What is the prohibited AI technique "Exploitation of Vulnerabilities"?

    -> exploit vulnerabilities of specific group of persons due to age, disability, social/economic situation to change behaviour e.g. an advertising algorithm that is trained to present gambling ads to a socially challenged group

15. What is the prohibited AI technique "Social Scoring"?

    evaluation / classification of natural persons based on social behaviour or characteristics that lead to detrimental or unfavourable treatment unrelated to the context in which the data was collected or unjustified/disproportionate to their social behaviour e.g. social scoring algorithm; speeding tickets lead to legal and/or social disadvantages in unrelated areas, e.g. bank loans or travel restrictions

16. What is the prohibited AI technique "Biometric Identification"?

    e.g. face recognition only prohibited in publicly accessible spaces only prohibited by/for law enforcement authorities -> allowed for private institutions, communities, ...! allowed under certain strict limitations for limited purposes

17. Examples of (high-risk) systems listed in Annex III?

    Biometrics e.g. biometric ID, emotion recognition systems, ... Critical infrastructure e.g. road traffic, supply of water Employment, workers management, access to self-employment e.g. recruitment Law enforcement e.g. lie detection, profiling, evidence analysis, "robo judge"

18. What are the transparency obligations in the AI Act for chatbots?

    Natural persons must be informed that they are interacting with an AI system

19. What are the transparency obligations in the AI Act for biometric categorization?

    Natural persons must be informed that hat they are exposed to an AI system

20. What are the transparency obligations in the AI Act for emotion recognition?

    Natural persons must be informed that they are exposed to an AI system

21. What are the transparency obligations in the AI Act for deep fakes?

    Users of a "deep generator" must disclose that content has been AI generated / manipulated

22. How must transparency obligations be met?

    Information must be given in a clear and distinguishable manner at the latest at the time of the first interaction or exposure

23. What is the requirement for high risk ai systems: "Risk management system"?

    continuous iterative process throughout the entire lifecycle of system elimination or reduction of risks as far as possible through adequate design and development implementation of adequate mitigation and control measures in relation to risks that cannot be eliminated “residual risks” -> adequate information and training to users.

24. What is the requirement for high risk ai systems: "Data and Data governance"?

    if system uses training data: requires data governance and management practices for training, validation and testing data sets data shall be relevant, representative, free of errors and complete -> this gives a legal basis under the GDPR to process even special categories of data IF it is necessary to ensure the quality of high risk ai systems (e.g. removing bias)

25. What is the requirement for high risk ai systems: "Technical documentation"?

    tech doc needs to be prepared before the system is placed on the market and needs to be kept up to date must be provided to the national authorities if requested

26. What is the requirement for high risk ai systems: "Record keeping"?

    System must be able to record logs during the whole life cycle

27. What is the requirement for high risk ai systems: "Transparency and provision of information to users"?

    must be designed transparently to allow compliance with the other AI act requirements for users and providers must enable users to understand and use the system appropriately must be accompanied by usage instructions

28. What is the requirement for high risk ai systems: "Human oversight"?

    AI systems must be designed to be effectively overseen by natural persons

29. What is the requirement for high risk ai systems: "Accuracy, Robustness and Cybersecurity"?

    AI system must be designed with these in mind -> must be made resilient against attempts by unauthorised third parties to alter their use or performance by exploiting the system vulnerabilities.

30. What is the obligation of providers of high risk AI systems: "Quality Management System"?

    -> a system that ensures compliance with the AI Act contains strategies to ensure this compliance

31. What is the obligation of providers of high risk AI systems: "Automatically generated logs"?

    Must keep logs for 6 months

32. What is the obligation of providers of high risk AI systems: "Corrective actions"?

    if not compliant -> must bring system into compliance, or withdraw/deactivate/recall it

33. What is the obligation of providers of high risk AI systems: "duty of information"?

    where an AI system presents a risk and that risk is known to the provider -> provider must immediately inform the national competent authorities

34. What is the obligation of providers of high risk AI systems: "Cooperation with competent Authorities, ..."?

    Providers/Operators must provide a national competent authority with all information and documentation necessary to document compliance

35. What is the obligation of providers of high risk AI systems: "Authorized Representatives"?

    Providers established outside the EU must designate a representative in the EU before making their systems available in the EU