The Quest for Bugs

Conference:	Verification Futures 2022 (click here to see full programme)
Speaker:	Bryan Dickman, Joe Convey
Presentation Title:	The Quest for Bugs
Abstract:	All verification engineers understand this. Searching for bugs in complex hardware designs is resource-limited “quest” to find as many bugs as possible before product release. This quest demands strategies and game-plans that will meet this verification challenge in the most cost-effective way and requires verification teams to wrestle with several verification dilemmas. For example, the “completeness dilemma”. How do teams deal with the inherent lack of completeness in all dimensions of verification from test planning to execution and sign-off? Understanding these dilemmas can help to reason about the effectiveness of the chosen methodologies and strategies. Also, do product development teams really understand the “cost of bugs”? By this we mean both the cost incurred to find bugs (the verification costs) and the cost impacts of not finding bugs. Keeping these 2 things in balance is important to ensure that your product meets your ROI objectives and allows you to reason about the investment costs of performing verification – which we all know can be substantial. How do you effectively find those difficult bugs that demand deep cycles of verification? What about security bugs or functional safety bugs, or bugs in performance or power? Do we understand where bugs come from and what strategies can be used to avoid them. Avoiding bugs can be more cost-effective than finding them, and let’s face it, you aren’t ever going to find all the bugs, so try to minimise them in the first place with good design practices. In the end, verification is all about bugs and we will talk about verification entirely from this angle of “bugs”. How to find them? What they look like? How to avoid them? What are the root causes? What does good practice look like? This talk will be partly philosophical in nature, and we will not be promoting specific technologies, but we will present this view from the perspective of several decades of experience in this challenging industry.
Speaker Bio:	Bryan Dickman: Over 22 years of leading engineering teams and engineering communities in the field of design verification for Arm Processors. In recent years, more focused on leading the effort to exploit Engineering’s vast data resources to drive insights and improvements to critical workflows. Currently working as an independent consultant and founder of Valytic Consulting Limited. Joe Convey: 14+ years in semiconductor and EDA. Close relationships with engineering management to understand needs and provide best-in-class engineering platform. Promoted commercial awareness in engineering teams, showing the value of Partnership with EDA vendors as a methodology to get optimised results from their tools and services. In-depth experience of negotiation with major EDA vendors to ensure delivery of a resilient platform within acceptable commercial limits.
Key Points:	Design Verification is a time and resource-bounded quest to find all hardware bugs prior to product release – some will inevitably be missed. Some of these bug escapes may be critical with far reaching impacts. The cost impact of bugs needs to be understood from many angles. It goes much further than the simple rework cost of fixing and re-verification. Understanding and modelling these impact costs helps product development teams to reason about product development engineering costs from an ROI point of view. There are 5 key dilemmas that need to be appreciated. Completeness, Complexity, Resources, Constrained-Random and Delivery. Understanding these verification dilemmas will help you to plan and execute effective verification campaigns.