Security challenges in AI: detailed analysis by the EU Agency for Cybersecurity

The EU Cybersecurity Agency, ENISA, recently released a series of reports highlighting the challenges Artificial Intelligence (AI) faces in cybersecurity. These reports were released simultaneously with the AI Cybersecurity Conference, organized by ENISA, during which the cybersecurity implications for AI chatbots, research and innovation, as well as legal and industrial challenges were addressed. "To ensure the security of AI systems and ensure privacy, we need to look closely at how these systems work," said Juhan Lepassaar, Executive Director of the EU Cybersecurity Agency.

Privacy issues

ENISA has published further reports on the significant impact AI has on security and privacy, taking demand forecasting on electricity grids and the field of medical imaging diagnoses as case studies. One report points out, "Although security and privacy are not necessarily the same thing, they are closely related and just as important." ENISA suggests that "the entire context of cybersecurity and privacy (requirements, threats, vulnerabilities and controls) must be adapted to the context and reality of the individual organization".

Research gaps

Regarding the EU landscape on AI and cybersecurity in research and innovation, ENISA outlined the current state to identify potential gaps. The analysis points to six gaps in research and innovation, including the lack of adequate information and knowledge about the potential of AI solutions for cybersecurity, adequate documentation of implementation projects and demonstration activities. Furthermore, the issue is raised that only a minority of prototypes developed in the context of research and development (R&D) enter the market, there is a perception gap between the research and business communities, and there is limited capacity for such projects in solving existing and emerging problems. "The impact of AI on the overall risk landscape brings challenges and opportunities. Securing AI and AI-specific vulnerabilities are both organizational and R&D challenges," said Henrik Junklewitz of the European Commission's research department. ENISA plans to develop a roadmap and establish an observatory for cybersecurity research and development with a focus on AI.

Secure systems

During the conference, best practice for ensuring the security of AI systems was also discussed, using the example of the role of the German Federal Office for Information Security, BSI. The latter deals with modeling information security for digital technologies, including AI. Arndt von Twickel, from the Federal Office at the BSI, said: "We have to develop practical criteria, therefore, AI has to be considered in the usage system. This is what we have to consider when we think about securing AI systems." . Given the complexity of the lifecycle of an AI system, new vulnerabilities emerge. To successfully secure an AI system, all phases, covering planning, data, training, evaluation and operation must be considered. “We study the fundamental properties of AI in different domains. Our contribution will be 1) develop domain and use case specific documents and technical guidelines, 2) update the general model of AI, and use the results of the first two points to contribute to standardization, regulation and consultancy,” explained von Twickel.