DarkBERT: the AI weapon in the fight against cybercrime

The artificial intelligence (AI) sector has seen impressive growth in recent months, with large language models (LLMs) making their way onto the scene. The emblematic case is ChatGPT, an AI chatbot, capable of generating contents of various kinds - texts, images, audio - thanks to the analysis of an existing database and the learning of specific patterns. These tools have been used to generate content using generative AI, but now researchers have developed a first-of-its-kind LLM designed to assess and counter cybersecurity threats. Curiously, this model was only trained on information from the dark web.

DarKBERT: a new model to fight cybercrime

DarkBERT is an encryption model which is based on the RoBERTa architecture. Unlike the other models, DarkBERT was not trained on the web in a general sense, but on a vast database of dark web pages, integrating information from criminal internet sources such as hacker forums and scam sites. Its creators, in an article entitled "DarkBERT: a language model for the dark side of the internet" published on arxiv.org, state that DarkBERT can revolutionize the fight against cybercrime, identifying and analyzing the most hidden, undetectable areas of the network by search engines.

The DarkBERT creation process

The dark web is normally hidden and inaccessible to the general public, but researchers have used the Tor network to access and collect data from its pages. This data then underwent various processes, such as deduplication, category balancing, and pre-processing, to create a refined database of the dark web. This was eventually fed to RoBERTa, leading to the creation of DarkBERT over a 15 day period.

Potential applications of DarkBERT in cybersecurity

Trained on a database of dark web pages, DarkBERT can have a wide range of applications in the field of cybersecurity. It can monitor illicit activities and strengthen cyber security measures. According to the researchers, it can "counteract the extreme lexical and structural diversity of the Dark Web which can be detrimental to building an adequate representation of the domain." DarkBERT can automate the monitoring of dark web forums, where illegal information is usually shared, detect websites that disseminate sensitive or confidential data and that sell ransomware. It also uses the "fill-mask" feature of the BERT family language model to detect and filter expressions related to criminal activity, thus helping to identify and deal with new cyberthreats.