Imagine millions of lines of instructions. Then try and picture how one extremely tiny anomaly could be found in almost real-time and prevent a cyber security attack.
A trio of Virginia Tech computer scientists has tested their innovation, called a “program anomaly detection approach,” against many real-world attacks.
One type of attack is when an adversary is able to remotely access a computer, bypassing authentication such as a login screen. A second example of attack is called heap feng shui where attackers hijack the control of a browser by manipulating its memory layout. Another example of attack is called directory harvesting where spammers interact with vulnerable mail servers to steal valid email addresses.
The prototype developed by the Virginia Tech scientists proved to be effective and reliable at these types of attacks with a false positive rate as low as 0.01 percent.
Their findings were reported this week in a presentation at the 22nd Association of Computing Machinery (ACM) Conference on Computer and Communications Security, in Denver, Colorado.
“Our work, in collaboration with Naren Ramakrishnan, is titled, “Unearthing Stealthy Program Attacks Buried in Extremely Long Execution Paths,” said Danfeng (Daphne) Yao, associate professor of computer science at Virginia Tech. Xiaokui Shu a computer science doctoral student of Anqing, China, advised by Yao, was the first author.
“Stealthy attacks buried in long execution paths of a software program cannot be revealed by examining fragments of the path,” said Yao, who holds the title of the L-3 Communications Cyber Faculty Fellow of Computer Science.
“Modern exploits have manipulation tactics that hide them from existing detection tools. An example is an attacker who overwrites one of the variables before the actual authentication procedure,” Yao explained, “As a result, the attacker bypasses critical security control and logs in without authentication.”
Over time, these stealthy attacks on computer systems have just become more sophisticated.
The Virginia Tech computer scientists’ secret formula in finding a stealth attack is in their algorithms. With specific matrix-based pattern recognition, the three were able to analyze the execution path of a software program and discover correlations among events. “The idea is to profile the program’s behavior, determine how often some events are supposed to occur, and with which other events, and use this information to detect anomalous activity,” Ramakrishnan said.
“Because the approach works by analyzing the behavior of computer code, it can be used to study a variety of different attacks,” Yao added. Their anomaly detection algorithms were able to detect erratic program behaviors with very low false alarms even when there are complex and diverse execution patterns.