Machine learning is being used by hackers to find weaknesses in chips and systems, but it also is starting to be used to prevent breaches by pinpointing hardware and software design flaws.

To make this work, machine learning (ML) must be trained to identify vulnerabilities, both in hardware and software. With proper training, ML can detect cyber threats and prevent them from accessing critical data. As ML encounters additional cyberattack scenarios, it can learn and adapt, helping to build a more sophisticated defense system that includes hardware, software, and how they interface with larger systems. It also can automate many cyber defense tasks with minimum human intervention, which saves time, effort, and money.

ML is capable of sifting through large volumes of data much faster than humans. Potentially, it can reduce or remove human errors, lower costs, and boost cyber defense capability and overall efficiency. It also can perform such tasks as connection authentication, system design, vulnerability detection, and most important, threat detection through pattern and behavioral analysis.

“AI/ML is finding many roles protecting and enhancing security for digital devices and services,” said David Maidment, senior director of market development at Arm. “However, it is also being used as a tool for increasingly sophisticated attacks by threat actors. AI/ML is essentially a tool tuned for very advanced pattern recognition across vast data sets. Examples of how AI/ML can enhance security include network-based monitoring to spot rogue behaviors at scale, code analysis to look for vulnerabilities on new and legacy software, and automating the deployment of software to keep devices up-to-date and secure.”

This means that while AI/ML can be used as a force for good, inevitably bad actors will use it to increase the sophistication and scale of attacks. “Building devices and services based on security best practices, having a hardware-protected root of trust (RoT), and an industry-wide methodology to standardize and measure security are all essential,” Maidment said. “The focus on security, including the rapid growth of AI/ML, is certainly driving industry and government discussions as we work on solutions to maximize AI/ML’s benefits and minimize any potential harmful impact.”

Zero trust is a fundamental requirement when it comes to cybersecurity. Before a user or device is allowed to connect to the network or server, requests have to be authenticated to make sure they are legitimate and authorized. ML will enhance the authentication process, including password management, phishing prevention, and malware detection.

Areas that bad actors look to exploit are software design vulnerabilities and weak points in systems and networks. Once hackers uncover these vulnerabilities, they can be used as a point of entrance to the network or systems. ML can detect these vulnerabilities and alert administrators.

Taking a proactive approach by doing threat detection is essential in cyber defense. ML pattern and behavioral analysis strengths support this strategy. When ML detects unusual behavior in data traffic flow or patterns, it sends an alert about abnormal behavior to the administrator. This is similar to the banking industry’s practice of watching for credit card use that does not follow an established pattern. A large purchase overseas on a credit card with a pattern of U.S. use only for moderate amounts would trigger an alert, for example.

As hackers become more sophisticated with new attack vectors, whether it is new ransomware or distributed denial of service (DDoS) attacks, ML will do a much better job than humans in detecting these unknown threats.

Limitations of ML in cybersecurity
While ML provides many benefits, its value depends on the data used to train it. The more that can be used to train the ML model, the better it is at detecting fraud and cyber threats. But acquiring this data raises overall cybersecurity system design expenses. The model also needs constant maintenance and tuning to sustain peak performance and meet the specific needs of users. And while ML can do many of the tasks, it still requires some human involvement, so it’s essential to understand both cybersecurity and how well ML functions.

While ML is effective in fending off many of the cyberattacks, it is not a panacea. “The specific type of artificial intelligence typically referenced in this context is machine learning (ML), which is the development of algorithms that can ingest large volumes of training data, then generalize and make meaningful observations and decisions based on novel data,” said Scott Register, vice president of security solutions at Keysight Technologies. “With the right algorithms and training, AI/ML can be used to pinpoint cyberattacks which might otherwise be difficult to detect.”

However, no one — at least in the commercial space — has delivered a product that can detect very subtle cyberattacks with complete accuracy. “The algorithms are getting better all the time, so it’s highly probable that we’ll soon have commercial products that can detect and respond to attacks,” Register said. “We must keep in mind, however, that attackers don’t sit still, and they’re well-funded and patient. They employ ‘offensive AI,’ which means they use the same types of techniques and algorithms to generate attacks which are unlikely to be detected.”

ML implementation considerations
For any ML implementation, a strong cyber defense system is essential, but there’s no such thing as a completely secure design. Instead, security is a dynamic and ongoing process that requires constant fine-tuning and improvement against ever-changing cyberattacks. Implementing ML requires a clear security roadmap, which should define requirements. It also requires implementing a good cybersecurity process, which secures individual hardware and software components, as well as some type of system testing.

“One of the things we advise is to start with threat modeling to identify a set of critical design assets to protect from an adversary under confidentiality or integrity,” said Jason Oberg, CTO at Cycuity. “From there, you can define a set of very succinct, secure requirements for the assets. All of this work is typically done at the architecture level. We do provide education, training and guidance to our customers, because at that level, if you don’t have succinct security requirements defined, then it’s really hard to verify or check something in the design. What often happens is customers will say, ‘I want to have a secure chip.’ But it’s not as easy as just pressing a button and getting a green check mark that confirms the chip is now secure.”

To be successful, engineering teams must start at the architectural stages and define the security requirements. “Once that is done, they can start actually writing the RTL,” Oberg said. “There are tools available to provide assurances these security requirements are being met, and run within the existing simulation and emulation environments to help validate the security requirements, and help identify any unknown design weaknesses. Generally, this helps hardware and verification engineers increase their productivity and build confidence that the system is indeed meeting the security requirements.”

Fig. 1: A cybersecurity model includes multiple stages, progressing from the very basic to in-depth. It is important for organizations to know what stages their cyber defense system are. Source: Cycuity

Steve Garrison, senior vice president, marketing of Stellar Cyber, noted that if cyber threats were uncovered during the detection process, so many data files may be generated that they will be difficult for humans to sort through. Graphical displays can speed up the process and reduce the overall mean time to detection (MTTD) and mean time to response (MTTR).

Fig. 2: Using graphical displays  would reduce the overall meantime to detection (MTTD) and meantime to response (MTTR). Source: Stellar Cyber

Testing is essential
Another important stage in the design process is testing, whereby each system design requires a vigorous attack simulation tool to weed out the basic oversights to ensure it meets the predefined standard.

“First, if you want to understand how defensive systems will function in the real world, it’s important to test them under conditions, which are as realistic as possible,” Keysight’s Register said. “The network environment should have the same amount of traffic, mix of applications, speeds, behavioral characteristics, and timing as the real world. For example, the timing of a sudden uptick in email and social media traffic corresponds to the time when people open up their laptops at work. The attack traffic needs to be as realistic as possible as well – hackers try hard not to be noticed, often preferring ‘low and slow’ attacks, which may take hours or days to complete, making detection much more difficult. The same obfuscation techniques, encryption, and decoy traffic employed by threat actors needs to be simulated as accurately as possible.”

Further, due to mistaken assumptions during testing, defensive systems often perform great in the lab, yet fail spectacularly in production networks.  “Afterwards we hear, for example, ‘I didn’t think hackers would encrypt their malware,’ or ‘Internal e-mails weren’t checked for malicious attachments, only those from external senders,’” Register explained. “Also, in security testing, currency is key. Attacks and obfuscation techniques are constantly evolving. If a security system is tested against stale attacks, then the value of that testing is limited. The offensive tools should be kept as up to date as possible to ensure the most effective performance against the tools a system is likely to encounter in the wild.”

Semiconductor security
Almost all system designs depend on semiconductors, so it is important to ensure that any and all chips, firmware, FPGAs, and SoCs are secure – including those that perform ML functionality.

“Semiconductor security is a constantly evolving problem and requires an adaptable solution, said Jayson Bethurem, vice president marketing and business development at Flex Logix. “Fixed solutions with current cryptography that are implemented today will inevitably be challenged in the future. Hackers today have more time, resources, training, and motivation to disrupt technology. With technology increasing in every facet of our lives, defending against this presents a real challenge. We also have to consider upcoming threats, namely quantum computing.”

Many predict that quantum computing will be able to crack current cryptography solutions in the next few years. “Fortunately, semiconductor manufacturers have solutions that can enable cryptography agility, which can dynamically adapt to evolving threats,” Bethurem said. “This includes both updating hardware accelerated cryptography algorithms and obfuscating them, an approach that increases root of trust and protects valuable IP secrets. Advanced solutions like these also involve devices randomly creating their own encryption keys, making it harder for algorithms to crack encryption codes.”

Advances in AI/ML algorithms can adapt to new threats and reduce latency of algorithm updates from manufacturers. This is particularly useful with reconfigurable eFPGA IP, which can be implemented into any semiconductor device to thwart all current and future threats and optimized to run AI/ML-based cryptography solutions. The result is a combination of high-performance processing, scalability, and low-latency attack response.

Chips that support AI/ML algorithms need not only computing power, but also accelerators for those algorithms. In addition, all of this needs to happen without exceeding a tight power budget.

“More AI/ML systems run at tiny edges rather than at the core,” said Detlef Houdeau, senior director of design system architecture at Infineon Technologies. “AI/ML systems don’t need any bigger computer and/or cloud. For instance, a Raspberry Pi for a robot in production can have more than 3 AI/ML algorithms working in parallel. A smartphone has more than 10 AI/ML functions in the phone, and downloading new apps brings new AI/ML algorithms into the device. A pacemaker can have 2 AI/ML algorithms. Security chips, meanwhile, need a security architecture as well as accelerators for encryption. Combining an AI/ML accelerator with an encryption accelerator in the same chip could increase the performance in microcontroller units, and at the same time foster more security at the edge. The next generation of microelectronics could show this combination.”

After developers have gone through design reviews and the systems have run vigorous tests, it helps to have third-party certification and/or credentials to ensure the systems are indeed secure from a third-party independent viewpoint.

“As AI, and recently generative AI, continue to transform all markets, there will be new attack vectors to mitigate against,” said Arm’s Maidment. “We expect to see networks become smarter in the way they monitor traffic and behaviors. The use of AI/ML allows network-based monitoring at scale to allow potential unexpected or rogue behavior to be identified and isolated. Automating network monitoring based on AI/ML will allow an extra layer of defense as networks scale out and establish effectively a ‘zero trust’ approach. With this approach, analysis at scale can be tuned to look at particular threat vectors depending on the use case.”

With an increase in AI/ML adoption at the edge, a lot of this is taking place on the CPU. “Whether it is handling workloads in their entirety, or in combination with a co-processor like a GPU or NPU, how applications are deployed across the compute resources needs to be secure and managed centrally within the edge AI/ML device,” Maidment said. “Building edge AI/ML devices based on a hardware root of trust is essential. It is critical to have privileged access control of what code is allowed to run where using a trusted memory management architecture. Arm continually invests in security, and the Armv9 architecture offers a number of new security features. Alongside architecture improvements, we continue to work in partnership with the industry on our ecosystem security framework and certification scheme, PSA Certified, which is based on a certified hardware RoT. This hardware base helps to improve the security of systems and fulfill the consumer expectation that as devices scale, they remain secure.”

Outlook
It is important to understand that threat actors will continue to evolve attacks using AI/ML. Experts suggest that to counter such attacks, organizations, institutions, and government agencies will have to continually improve defense strategies and capabilities, including AI/ML deployment.

AI/ML can be used as weapon from an attacker for industrial espionage and/or industrial sabotage, and stopping incursions will require a broad range of cyberattack prevention and detection tools, including AI/ML functionality for anomaly detection. But in general, hackers are almost always one step ahead.

According to Register, “the recurring cycle is: 1) hackers come out with a new tool or technology that lets them attack systems or evade detection more effectively; 2) those attacks cause enough economic damage that the industry responds and develops effective countermeasures; 3) the no-longer-new hacker tools are still employed effectively, but against targets that haven’t bothered to update their defenses; 4) hackers develop new offensive tools that are effective against the defensive techniques of high-value targets, and the cycle starts anew.”

Related Reading
Securing Chip Manufacturing Against Growing Cyber Threats
Suppliers are the number one risk, but reducing attacks requires industry-wide collaboration.
Data Center Security Issues Widen
The number and breadth of hardware targets is increasing, but older attack vectors are not going away. Hackers are becoming more sophisticated, and they have a big advantage.

The post Using AI/ML To Combat Cyberattacks appeared first on Semiconductor Engineering.

Machine learning is being used by hackers to find weaknesses in chips and systems, but it also is starting to be used to prevent breaches by pinpointing hardware and software design flaws.

To make this work, machine learning (ML) must be trained to identify vulnerabilities, both in hardware and software. With proper training, ML can detect cyber threats and prevent them from accessing critical data. As ML encounters additional cyberattack scenarios, it can learn and adapt, helping to build a more sophisticated defense system that includes hardware, software, and how they interface with larger systems. It also can automate many cyber defense tasks with minimum human intervention, which saves time, effort, and money.

ML is capable of sifting through large volumes of data much faster than humans. Potentially, it can reduce or remove human errors, lower costs, and boost cyber defense capability and overall efficiency. It also can perform such tasks as connection authentication, system design, vulnerability detection, and most important, threat detection through pattern and behavioral analysis.

“AI/ML is finding many roles protecting and enhancing security for digital devices and services,” said David Maidment, senior director of market development at Arm. “However, it is also being used as a tool for increasingly sophisticated attacks by threat actors. AI/ML is essentially a tool tuned for very advanced pattern recognition across vast data sets. Examples of how AI/ML can enhance security include network-based monitoring to spot rogue behaviors at scale, code analysis to look for vulnerabilities on new and legacy software, and automating the deployment of software to keep devices up-to-date and secure.”

This means that while AI/ML can be used as a force for good, inevitably bad actors will use it to increase the sophistication and scale of attacks. “Building devices and services based on security best practices, having a hardware-protected root of trust (RoT), and an industry-wide methodology to standardize and measure security are all essential,” Maidment said. “The focus on security, including the rapid growth of AI/ML, is certainly driving industry and government discussions as we work on solutions to maximize AI/ML’s benefits and minimize any potential harmful impact.”

Zero trust is a fundamental requirement when it comes to cybersecurity. Before a user or device is allowed to connect to the network or server, requests have to be authenticated to make sure they are legitimate and authorized. ML will enhance the authentication process, including password management, phishing prevention, and malware detection.

Areas that bad actors look to exploit are software design vulnerabilities and weak points in systems and networks. Once hackers uncover these vulnerabilities, they can be used as a point of entrance to the network or systems. ML can detect these vulnerabilities and alert administrators.

Taking a proactive approach by doing threat detection is essential in cyber defense. ML pattern and behavioral analysis strengths support this strategy. When ML detects unusual behavior in data traffic flow or patterns, it sends an alert about abnormal behavior to the administrator. This is similar to the banking industry’s practice of watching for credit card use that does not follow an established pattern. A large purchase overseas on a credit card with a pattern of U.S. use only for moderate amounts would trigger an alert, for example.

As hackers become more sophisticated with new attack vectors, whether it is new ransomware or distributed denial of service (DDoS) attacks, ML will do a much better job than humans in detecting these unknown threats.

Limitations of ML in cybersecurity
While ML provides many benefits, its value depends on the data used to train it. The more that can be used to train the ML model, the better it is at detecting fraud and cyber threats. But acquiring this data raises overall cybersecurity system design expenses. The model also needs constant maintenance and tuning to sustain peak performance and meet the specific needs of users. And while ML can do many of the tasks, it still requires some human involvement, so it’s essential to understand both cybersecurity and how well ML functions.

While ML is effective in fending off many of the cyberattacks, it is not a panacea. “The specific type of artificial intelligence typically referenced in this context is machine learning (ML), which is the development of algorithms that can ingest large volumes of training data, then generalize and make meaningful observations and decisions based on novel data,” said Scott Register, vice president of security solutions at Keysight Technologies. “With the right algorithms and training, AI/ML can be used to pinpoint cyberattacks which might otherwise be difficult to detect.”

However, no one — at least in the commercial space — has delivered a product that can detect very subtle cyberattacks with complete accuracy. “The algorithms are getting better all the time, so it’s highly probable that we’ll soon have commercial products that can detect and respond to attacks,” Register said. “We must keep in mind, however, that attackers don’t sit still, and they’re well-funded and patient. They employ ‘offensive AI,’ which means they use the same types of techniques and algorithms to generate attacks which are unlikely to be detected.”

ML implementation considerations
For any ML implementation, a strong cyber defense system is essential, but there’s no such thing as a completely secure design. Instead, security is a dynamic and ongoing process that requires constant fine-tuning and improvement against ever-changing cyberattacks. Implementing ML requires a clear security roadmap, which should define requirements. It also requires implementing a good cybersecurity process, which secures individual hardware and software components, as well as some type of system testing.

“One of the things we advise is to start with threat modeling to identify a set of critical design assets to protect from an adversary under confidentiality or integrity,” said Jason Oberg, CTO at Cycuity. “From there, you can define a set of very succinct, secure requirements for the assets. All of this work is typically done at the architecture level. We do provide education, training and guidance to our customers, because at that level, if you don’t have succinct security requirements defined, then it’s really hard to verify or check something in the design. What often happens is customers will say, ‘I want to have a secure chip.’ But it’s not as easy as just pressing a button and getting a green check mark that confirms the chip is now secure.”

To be successful, engineering teams must start at the architectural stages and define the security requirements. “Once that is done, they can start actually writing the RTL,” Oberg said. “There are tools available to provide assurances these security requirements are being met, and run within the existing simulation and emulation environments to help validate the security requirements, and help identify any unknown design weaknesses. Generally, this helps hardware and verification engineers increase their productivity and build confidence that the system is indeed meeting the security requirements.”

Fig. 1: A cybersecurity model includes multiple stages, progressing from the very basic to in-depth. It is important for organizations to know what stages their cyber defense system are. Source: Cycuity

Steve Garrison, senior vice president, marketing of Stellar Cyber, noted that if cyber threats were uncovered during the detection process, so many data files may be generated that they will be difficult for humans to sort through. Graphical displays can speed up the process and reduce the overall mean time to detection (MTTD) and mean time to response (MTTR).

Fig. 2: Using graphical displays  would reduce the overall meantime to detection (MTTD) and meantime to response (MTTR). Source: Stellar Cyber

Testing is essential
Another important stage in the design process is testing, whereby each system design requires a vigorous attack simulation tool to weed out the basic oversights to ensure it meets the predefined standard.

“First, if you want to understand how defensive systems will function in the real world, it’s important to test them under conditions, which are as realistic as possible,” Keysight’s Register said. “The network environment should have the same amount of traffic, mix of applications, speeds, behavioral characteristics, and timing as the real world. For example, the timing of a sudden uptick in email and social media traffic corresponds to the time when people open up their laptops at work. The attack traffic needs to be as realistic as possible as well – hackers try hard not to be noticed, often preferring ‘low and slow’ attacks, which may take hours or days to complete, making detection much more difficult. The same obfuscation techniques, encryption, and decoy traffic employed by threat actors needs to be simulated as accurately as possible.”

Further, due to mistaken assumptions during testing, defensive systems often perform great in the lab, yet fail spectacularly in production networks.  “Afterwards we hear, for example, ‘I didn’t think hackers would encrypt their malware,’ or ‘Internal e-mails weren’t checked for malicious attachments, only those from external senders,’” Register explained. “Also, in security testing, currency is key. Attacks and obfuscation techniques are constantly evolving. If a security system is tested against stale attacks, then the value of that testing is limited. The offensive tools should be kept as up to date as possible to ensure the most effective performance against the tools a system is likely to encounter in the wild.”

Semiconductor security
Almost all system designs depend on semiconductors, so it is important to ensure that any and all chips, firmware, FPGAs, and SoCs are secure – including those that perform ML functionality.

“Semiconductor security is a constantly evolving problem and requires an adaptable solution, said Jayson Bethurem, vice president marketing and business development at Flex Logix. “Fixed solutions with current cryptography that are implemented today will inevitably be challenged in the future. Hackers today have more time, resources, training, and motivation to disrupt technology. With technology increasing in every facet of our lives, defending against this presents a real challenge. We also have to consider upcoming threats, namely quantum computing.”

Many predict that quantum computing will be able to crack current cryptography solutions in the next few years. “Fortunately, semiconductor manufacturers have solutions that can enable cryptography agility, which can dynamically adapt to evolving threats,” Bethurem said. “This includes both updating hardware accelerated cryptography algorithms and obfuscating them, an approach that increases root of trust and protects valuable IP secrets. Advanced solutions like these also involve devices randomly creating their own encryption keys, making it harder for algorithms to crack encryption codes.”

Advances in AI/ML algorithms can adapt to new threats and reduce latency of algorithm updates from manufacturers. This is particularly useful with reconfigurable eFPGA IP, which can be implemented into any semiconductor device to thwart all current and future threats and optimized to run AI/ML-based cryptography solutions. The result is a combination of high-performance processing, scalability, and low-latency attack response.

Chips that support AI/ML algorithms need not only computing power, but also accelerators for those algorithms. In addition, all of this needs to happen without exceeding a tight power budget.

“More AI/ML systems run at tiny edges rather than at the core,” said Detlef Houdeau, senior director of design system architecture at Infineon Technologies. “AI/ML systems don’t need any bigger computer and/or cloud. For instance, a Raspberry Pi for a robot in production can have more than 3 AI/ML algorithms working in parallel. A smartphone has more than 10 AI/ML functions in the phone, and downloading new apps brings new AI/ML algorithms into the device. A pacemaker can have 2 AI/ML algorithms. Security chips, meanwhile, need a security architecture as well as accelerators for encryption. Combining an AI/ML accelerator with an encryption accelerator in the same chip could increase the performance in microcontroller units, and at the same time foster more security at the edge. The next generation of microelectronics could show this combination.”

After developers have gone through design reviews and the systems have run vigorous tests, it helps to have third-party certification and/or credentials to ensure the systems are indeed secure from a third-party independent viewpoint.

“As AI, and recently generative AI, continue to transform all markets, there will be new attack vectors to mitigate against,” said Arm’s Maidment. “We expect to see networks become smarter in the way they monitor traffic and behaviors. The use of AI/ML allows network-based monitoring at scale to allow potential unexpected or rogue behavior to be identified and isolated. Automating network monitoring based on AI/ML will allow an extra layer of defense as networks scale out and establish effectively a ‘zero trust’ approach. With this approach, analysis at scale can be tuned to look at particular threat vectors depending on the use case.”

With an increase in AI/ML adoption at the edge, a lot of this is taking place on the CPU. “Whether it is handling workloads in their entirety, or in combination with a co-processor like a GPU or NPU, how applications are deployed across the compute resources needs to be secure and managed centrally within the edge AI/ML device,” Maidment said. “Building edge AI/ML devices based on a hardware root of trust is essential. It is critical to have privileged access control of what code is allowed to run where using a trusted memory management architecture. Arm continually invests in security, and the Armv9 architecture offers a number of new security features. Alongside architecture improvements, we continue to work in partnership with the industry on our ecosystem security framework and certification scheme, PSA Certified, which is based on a certified hardware RoT. This hardware base helps to improve the security of systems and fulfill the consumer expectation that as devices scale, they remain secure.”

Outlook
It is important to understand that threat actors will continue to evolve attacks using AI/ML. Experts suggest that to counter such attacks, organizations, institutions, and government agencies will have to continually improve defense strategies and capabilities, including AI/ML deployment.

AI/ML can be used as weapon from an attacker for industrial espionage and/or industrial sabotage, and stopping incursions will require a broad range of cyberattack prevention and detection tools, including AI/ML functionality for anomaly detection. But in general, hackers are almost always one step ahead.

According to Register, “the recurring cycle is: 1) hackers come out with a new tool or technology that lets them attack systems or evade detection more effectively; 2) those attacks cause enough economic damage that the industry responds and develops effective countermeasures; 3) the no-longer-new hacker tools are still employed effectively, but against targets that haven’t bothered to update their defenses; 4) hackers develop new offensive tools that are effective against the defensive techniques of high-value targets, and the cycle starts anew.”

Related Reading
Securing Chip Manufacturing Against Growing Cyber Threats
Suppliers are the number one risk, but reducing attacks requires industry-wide collaboration.
Data Center Security Issues Widen
The number and breadth of hardware targets is increasing, but older attack vectors are not going away. Hackers are becoming more sophisticated, and they have a big advantage.

The post Using AI/ML To Combat Cyberattacks appeared first on Semiconductor Engineering.

Given computer vision’s place as the cornerstone of an increasing number of applications from ADAS to medical diagnosis and robotics, it is critical that its weak points be mitigated, such as the ability to identify corner cases or if algorithms are trained on shallow datasets. While well-known bloopers are often the result of human decisions, there are also fundamental technical issues that require further research.

“Computer vision” and “machine vision” were once used nearly interchangeably, with machine vision most often referring to the hardware embodiment of vision, such as in robots. Computer vision (CV), which started as the academic amalgam of neuroscience and AI research, has now become the dominant idea and preferred term.

“In today’s world, even the robotics people now call it computer vision,” said Jay Pathak, director, software development at Ansys. “The classical computer vision that used to happen outside of deep learning has been completely superseded. In terms of the success of AI, computer vision has a proven track record. Anytime self-driving is involved, any kind of robot that is doing work — its ability to perceive and take action — that’s all driven by deep learning.”

The original intent of CV was to replicate the power and versatility of human vision. Because vision is such a basic sense, the problem seemed like it would be far easier than higher-order cognitive challenges, like playing chess. Indeed, in the canonical anecdote about the field’s initial naïve optimism, Marvin Minsky, co-founder of the MIT AI Lab, having forgotten to include a visual system in a robot, assigned the task to undergraduates. But instead of being quick to solve, the problem consumed a generation of researchers.

Both academic and industry researchers work on problems that roughly can be split into three categories:

• Image capture: The realm of digital cameras and sensors. It may use AI for refinements or it may rely on established software and hardware.
• Image classification/detection: A subset of AI/ML that uses image datasets as training material to build models for visual recognition.
• Image generation: The most recent work, which uses tools like LLMs to create novel images, and with the breakthrough demonstration of OpenAI’s Sora, even photorealistic videos.

Each one alone has spawned dozens of PhD dissertations and industry patents. Image classification/detection, the primary focus of this article, underlies ADAS, as well as many inspection applications.

The change from lab projects to everyday uses came as researchers switched from rules-based systems that simulated visual processing as a series of if/then statements (if red and round, then apple) to neural networks (NNs), in which computers learned to derive salient features by training on image datasets. NNs are basically layered graphs. The earliest model, 1943’s Perceptron, was a one-layer simulation of a biological neuron, which is one element in a vast network of interconnecting brain cells. Neurons have inputs (dendrites) and outputs (axons), driven by electrical and chemical signaling. The Perceptron and its descendant neural networks emulated the form but skipped the chemistry, instead focusing on electrical signals with algorithms that weighted input values. Over the decades, researchers refined different forms of neural nets with vastly increased inputs and layers, eventually becoming the deep learning networks that underlie the current advances in AI.

The most recent forms of these network models are convolutional neural networks (CNNs) and transformers. In highly simplified terms, the primary difference between them is that CNNs are very good at distinguishing local features, while transformers perceive a more globalized picture.

Thus, transformers are a natural evolution from CNNs and recurrent neural networks, as well as long short-term memory approaches (RNNs/LSTMs), according to Gordon Cooper, product marketing manager for Synopsys’ embedded vision processor family.

“You get more accuracy at the expense of more computations and parameters. More data movement, therefore more power,” said Cooper. “But there are cases where accuracy is the most important metric for a computer vision application. Pedestrian detection comes to mind. While some vision designs still will be well served with CNNs, some of our customers have determined they are moving completely to transformers. Ten years ago, some embedded vision applications that used DSPs moved to NNs, but there remains a need for both NNs and DSPs in a vision system. Developers still need a good handle on both technologies and are better served to find a vendor that can provide a combined solution.”

The emergence of CNN-based neural networks began supplanting traditional CV techniques for object detection and recognition.

“While first implemented using hardwired CNN accelerator hardware blocks, many of those CNN techniques then quickly migrated to programmable solutions on software-driven NPUs and GPNPUs,” said Aman Sikka, chief architect at Quadric.

Two parallel trends continue to reshape CV systems. “The first is that transformer networks for object detection and recognition, with greater accuracy and usability than their convolution-based predecessors, are beginning to leave the theoretical labs and enter production service in devices,” Sikka explained. “The second is that CV experts are reinventing the classical ISP functions with NN and transformer-based models that offer superior results. Thus, we’ve seen waves of ISP functionality migrating first from pure hardwired to C++ algorithmic form, and now into advanced ML network formats, with a modern design today in 2024 consisting of numerous machine-learning models working together.”

CV for inspection
While CV is well-known for its essential role in ADAS, another primary application is inspection. CV has helped detect everything from cancer tumors to manufacturing errors, or in the case of IBM’s productized research, critical flaws in the built environment. For example, a drone equipped with the IBM system could check if a bridge had cracks, a far safer and more precise way to perform visual inspection than having a human climb to dangerous heights.

By combining visual transformers with self-supervised learning, the annotation requirement is vastly reduced. In addition, the company has introduced a new process named “visual prompting,” where the AI can be taught to make the correct distinctions with limited supervision by using “in-context learning,” such as a scribble as a prompt. The optimal end result is that it should be able to respond to LLM-like prompts, such as “find all six-inch cracks.”

“Even if it makes mistakes and needs the help of human annotations, you’re doing far less labeling work than you would with traditional CNNs, where you’d have to do hundreds if not thousands of labels,” said Jayant Kalagnanam, director, AI applications at IBM Research.

Beware the humans
Ideally, domain-specific datasets should increase the accuracy of identification. They are often created by expanding on foundation models already trained on general datasets, such as ImageNet. Both types of datasets are subject to human and technical biases. Google’s infamous racial identification gaffes resulted from both technical issues and subsequent human overcorrections.

Meanwhile, IBM was working on infrastructure identification, and the company’s experience of getting its model to correctly identify cracks, including the problem of having too many images of one kind of defect, suggests a potential solution to the bias problem, which is to allow the inclusion of contradictory annotations.

“Everybody who is not a civil engineer can easily say what a crack is,” said Cristiano Malossi, IBM principal research scientist. “Surprisingly, when we discuss which crack has to be repaired with domain experts, the amount of disagreement is very high because they’re taking different considerations into account and, as a result, they come to different conclusions. For a model, this means if there’s ambiguity in the annotations, it may be because the annotations have been done by multiple people, which may actually have the advantage of introducing less bias.”

Fig. 1: IBM’s Self-supervised learning model. Source: IBM

Corner cases and other challenges to accuracy
The true image dataset is infinity, which in practical terms leaves most computer vision systems vulnerable to corner cases, potentially with fatal results, noted Alan Yuille, Bloomberg distinguished professor of cognitive science and computer science at Johns Hopkins University.

“So-called ‘corner cases’ are rare events that likely aren’t included in the dataset and may not even happen in everyday life,” said Yuille. “Unfortunately, all datasets have biases, and algorithms aren’t necessarily going to generalize to data that differs from the datasets they’re trained on. And one thing we have found with deep nets is if there is any bias in the dataset, the deep nets are wonderful at finding it and exploiting it.”

Thus, corner cases remain a problem to watch for. “A classic example is the idea of a baby in the road. If you’re training a car, you’re typically not going to have many examples of images with babies in the road, but you definitely want your car to stop if it sees a baby,” said Yuille. “If the companies are working in constrained domains, and they’re very careful about it, that’s not necessarily going to be a problem for them. But if the dataset is in any way biased, the algorithms may exploit the biases and corner cases, and may not be able to detect them, even if they may be of critical importance.”

This includes instances, such as real-world weather conditions, where an image may be partly occluded. “In academic cases, you could have algorithms that when evaluated on standard datasets like ImageNet are getting almost perfect results, but then you can give them an image which is occluded, for example, by a heavy rain,” he said. “In cases like that, the algorithms may fail to work, even if they work very well under normal weather conditions. A term for this is ‘out of domain.’ So you train in one domain and that may be cars in nice weather conditions, you test in out of domain, where there haven’t been many training images, and the algorithms would fail.”

The underlying reasons go back to the fundamental challenge of trying to replicate a human brain’s visual processing in a computer system.

“Objects are three-dimensional entities. Humans have this type of knowledge, and one reason for that is humans learn in a very different way than machine learning AI algorithms,” Yuille said. “Humans learn over a period of several years, where they don’t only see objects. They play with them, they touch them, they taste them, they throw them around.”

By contrast, current algorithms do not have that type of knowledge.

“They are trained as classifiers,” said Yuille. “They are trained to take images and output a class label — object one, object two, etc. They are not trained to estimate the 3D structure of objects. They have some sort of implicit knowledge of some aspects of 3D, but they don’t have it properly. That’s one reason why if you take some of those models, and you’ve contaminated the images in some way, the algorithms start degrading badly, because the vision community doesn’t have datasets of images with 3D ground truth. Only for humans, do we have datasets with 3D ground truth.”

Hardware implementation, challenges
The hardware side is becoming a bottleneck, as academics and industry work to resolve corner cases and create ever-more comprehensive and precise results. “The complexity of the operation behind the transformer is quadratic,“ said Malossi. “As a result, they don’t scale linearly with the size of the problem or the size of the model.“

While the situation might be improved with a more scalable iteration of transformers, for now progress has been stalled as the industry looks for more powerful hardware or any suitable hardware. “We’re at a point right now where progress in AI is actually being limited by the supply of silicon, which is why there’s so much demand, and tremendous growth in hardware companies delivering AI,” said Tony Chan Carusone, CTO of Alphawave Semi. “In the next year or two, you’re going to see more supply of these chips come online, which will fuel rapid progress, because that’s the only thing holding it back. The massive investments being made by hyperscalers is evidence about the backlogs in delivering silicon. People wouldn’t be lining up to write big checks unless there were very specific projects they had ready to run as soon as they get the silicon.”

As more AI silicon is developed, designers should think holistically about CV, since visual fidelity depends not only on sophisticated algorithms, but image capture by a chain of co-optimized hardware and software, according to Pulin Desai, group director of product marketing and management for Tensilica vision, radar, lidar, and communication DSPs at Cadence. “When you capture an image, you have to look at the full optical path. You may start with a camera, but you’ll likely also have radar and lidar, as well as different sensors. You have to ask questions like, ‘Do I have a good lens that can focus on the proper distance and capture the light? Can my sensor perform the DAC correctly? Will the light levels be accurate? Do I have enough dynamic range? Will noise cause the levels to shift?’ You have to have the right equipment and do a lot of pre-processing before you send what’s been captured to the AI. Remember, as you design, don’t think of it as a point solution. It’s an end-to-end solution. Every different system requires a different level of full path, starting from the lens to the sensor to the processing to the AI.”

One of the more important automotive CV applications is passenger monitoring, which can help reduce the tragedies of parents forgetting children who are strapped into child seats. But such systems depend on sensors, which can be challenged by noise to the point of being ineffective.

“You have to build a sensor so small it goes into your rearview mirror,” said Jayson Bethurem, vice president of marketing and business development at Flex Logix. “Then the issue becomes the conditions of your car. The car can have the sun shining right in your face, saturating everything, to the complete opposite, where it’s completely dark and the only light in the car is emitting off your dashboard. For that sensor to have that much dynamic range and the level of detail that it needs to have, that’s where noise creeps in, because you can’t build a sensor of that much dynamic range to be perfect. On the edges, or when it’s really dark or oversaturated bright, it’s losing quality. And those are sometimes the most dangerous times.”

Breaking into the black box
Finally, yet another serious concern for computer vision systems is the fact that they can’t be tested. Transformers, especially, are a notorious black box.

“We need to have algorithms that are more interpretable so that we can understand what’s going on inside them,” Yuille added. “AI will not be satisfactory till we move to a situation where we evaluate algorithms by being able to find the failure mode. In academia, and I hope companies are more careful, we test them on random samples. But if those random samples are biased in some way — and often they are — they may discount situations like the baby in the road, which don’t happen often. To find those issues, you’ve got to let your worst enemy test your algorithm and find the images that break it.”

Related Reading
Dealing With AI/ML Uncertainty
How neural network-based AI systems perform under the hood is currently unknown, but the industry is finding ways to live with a black box.

The post Fundamental Issues In Computer Vision Still Unresolved appeared first on Semiconductor Engineering.