Reinventing Hardware Design Assurance
This paper provides APSYS’s vision of the role that HW design assurance can play to support the development of electronic hardware, particularly for safety-critical systems. It discusses how top technical risks on such developments are evolving and the impact this has on objectives and priorities for HW design assurance activities. It also proposes guidance on how to fit HW design assurance in project and company organizations and processes to maximize its effectiveness and efficiency.
Historically, HW design assurance has often been limited to demonstrating compliance to process. This is relevant when new processes are being introduced and one of the top risks for developments is the misunderstanding of these processes’ objectives and methods of compliance. However, HW design assurance can be relied upon to manage any technical risk without relying exclusively on experts. Going forward, this ability to address any given project’s specific needs is likely to be a key to successful developments.
For a long time, civil aeronautics systems and equipment were sufficiently simple for complete verification (mainly test) to mitigate any significant risk of design error that could have a safety impact. The ability of an organization to efficiently develop a “good” product was mainly dependent on skills: experts were involved on any and every issue.
When the complexity of systems and components increased, it could no longer be considered that verification was sufficient to ensure that a system was safe. Control of development rigor became necessary. The processes developed by the industry to achieve this goal (such as DO-254) significantly reduced reliance on key individuals: experts were no longer singlehandedly responsible for avoiding design errors as requirements-based engineering provided a web of integrity that spanned all development activities. This allowed a significant increase of system complexity with no adverse impact on system safety.
The role of HW design assurance was essentially created to address the need to control development rigor. Its main objective was to ensure that all aspects of the process were understood and properly deployed. In many cases, design assurance and process assurance were the same thing.
Today, many suppliers and integrators in the civil aeronautics industry have organizations and internal processes that are mature and adequate to develop airborne electronics in compliance with regulatory requirements. There is therefore no longer a strong need to control adherence to process. However, the increase of system and organizational complexity and the introduction of new technologies have introduced technical risks that process alone cannot mitigate.
Furthermore, regulations and acceptable means of compliance are evolving towards objective-driven requirements. While this approach provides more flexibility for system design and compliance demonstration, it more heavily relies on skills-dependent engineering judgement rather than simple prescriptive compliance metrics. To adapt to this new reality, organizations must re-introduce key technical skills in areas where process assurance may have previously been sufficient. HW design assurance engineers typically have the skills to address this need.
Beyond civil aeronautics, all safety-critical electronic HW developments require the control of development rigor and technical risk mitigation commensurate with industry regulatory requirements and project needs. Through its historical role and its expansion to all forms of technical risk mitigation, HW design assurance is the most suitable role to reach these objectives.
What is Design Assurance?
The objective of any serious development program is to design a product that works. In a nutshell, design assurance could be generically defined as the set of activities deployed by companies to ensure that the development effort is adequate to meet this objective.
The most obvious way to demonstrate that a product works is to test it. Historically, this has been the primary method to demonstrate product integrity and remains a staple of any rigorous development process.
However, testing is often performed relatively late in the development of hardware. Any issue with the design found during testing could therefore have a significant impact on development cost and schedule. To mitigate this risk, successful companies rely on processes deployed by people with adequate skills to minimize the risk of design errors.
For a long time, development of safety-critical products relied on a simple approach: let organizations compete on development performance and efficiency and make sure that testing is sufficient to show that the end product is safe.
However, the advent of modern digital electronics introduced a new category of risk. The complexity of the products became such that testing alone did not allow the product to experience all possible operational states and conditions. Any error or weakness in the design that was not exposed by testing would therefore only be discovered once in the field with potentially significant safety consequences.
To address this risk, regulation and acceptable means of compliance (such as DO-254 for civil aircraft) were introduced to guarantee a minimum level of rigor in the development process. This impacted all aspects of HW development and some companies had to make significant changes in the way they worked to comply with these new rules. A new role was created with the objective to ensure a sufficient level of development rigor and gather evidence of compliance to regulations. The term design assurance has often been used to denote this role.
To this day, the scope of design assurance is often limited to specific regulatory needs which typically focus on complex electronic component development. This relies on adherence to processes written to address regulatory requirements. In many companies, design assurance and process assurance are essentially the same thing.
Figure 1 – Usual Scope of HW Design Assurance Today
Why Do We Need Design Assurance?
The regulations applicable to HW development for many safety-critical markets such as civil aeronautics have now been in place for decades and organizations have therefore fully adapted their internal processes to be compliant. This is visible in the evolution of top issues seen during developments and in the field: while the majority of major issues concerned certification when regulation was first introduced, the bulk of actual problems in the field now concern areas that are not strictly controlled by regulation.
This evolution has had an impact on the perceived relevance of design assurance. Since compliance to regulation is no longer a top technical risk, why should we spend on a role that specifically and exclusively focuses on this? This negative perception is further amplified by the fact that improved maturity has allowed design assurance to focus on details of process compliance, creating a strong contrast between its seemingly inconsequential findings and the real-life HW issues that it is not addressing.
This criticism is legitimate and should drive the evolution of a role that remains essential for three reasons.
The first is that system safety is a never-ending effort. Even companies that have been developing safety-critical systems for decades are prone to regression due to staff turnover and obsolete habits and assumptions. It would therefore be dangerous to remove any kind of control of development rigor.
Furthermore, there are still many companies that have limited experience in safety-critical products and must adapt their ways of working to show compliance with regulations. This also applies to industries where regulations are being introduced. For example, it is becoming common for military products to be required to show compliance with civil regulations. This being its historical role, design assurance has proven its efficiency to support such evolutions.
The second is that regulations are evolving. Industries such as civil aeronautics are shifting their regulatory requirements and acceptable means of compliance from prescriptive guidance to higher-level objectives. This provides more flexibility for system design but introduces the burden of developing custom methods of compliance with less guidance. Design assurance engineers have the skills and knowledge to take on this task.
Finally – and most importantly – design assurance is a great opportunity to better mitigate one of the top technical risks of today’s developments: segmentation. While modern organizations and processes were essential in making sure that increasingly complex systems remain safe, they have led to the appearance of highly specialized roles and teams with clear boundaries between them. Because of this, many of the issues seen in the field today can be traced to problems at functional, physical or organizational interfaces.
As an independent assessor of developments, design assurance is in a unique position to “think function” and address technical risks at all levels rather than focus on particular details of a product.
Design assurance therefore has the potential to be an effective means to identify and control all technical risks for any given project. If its scope and deployment can be tailored to achieve this goal, it will be instrumental in making sure that we continue developing good and safe products in the future.
What Should Design Assurance Be?
The design and development of any product are driven by a set of core needs. Regulatory needs have a particular importance because they are a product’s entrance ticket to an industry or a market. Failure to meet these needs eliminates any possibility for the product to be deployed in the field. While other needs such as performance, reliability, maintainability and cost are not as strictly controlled, failure to meet them will compromise the product’s commercial viability. All types of needs must therefore be addressed for products and the companies that market them to be successful.
Identifying means of compliance and gathering evidence of compliance to regulations are design assurance’s historical purpose. However, the evolution of regulations is providing more freedom for companies to propose means of compliance that better suit the particular needs and constraints of their products and organizations. Depending on how it is addressed, this change could either be a blessing or a curse: companies that truly understand safety risks controlled by regulation will be able to tailor activities to fully address these risks while limiting cost. On the other hand, companies that do not understand this are likely to overspend on compliance demonstration or even fail to obtain regulatory approval.
All other core needs may be addressed by the same methods as those addressing safety needs: sound design decisions and development rigor. It is therefore natural for design assurance to control all technical risks, regardless of the core need they address.
To help meet any given project’s objectives, design assurance should therefore be a technical risk assessment and mitigation activity that helps development teams focus on areas of particular risk. This is only achievable if the design activity and the product that it generates are at the center of the design assurance approach.
Figure 2 – Design as the Centre of Gravity of Design Assurance
How Should Design Assurance Work?
To be effective, design assurance activities must start as early as possible in the development cycle. This allows major milestones later in projects to simply confirm that all of what was planned was correctly deployed. This contrasts with many current projects where design assurance is first involved once technical activities are complete. Besides strict compliance demonstration, this cancels design assurance’s main benefits: avoiding issues before they have significant consequences and doing things right the first time.
Figure 3 – HW Design Assurance Workload
Any project should start with a thorough assessment of all technical risks with a specific focus on those that may have an impact on safety. Appropriate mitigation for each risk can then be found. This may be in the form of existing or new process, specific activities such as analyses, simulations or test, informal or formal reviews or workshops that allow independent assessment of the design and potential issues at interfaces, etc. This planning process should be supported by experts in all relevant fields.
Once this project-specific framework is set, design assurance can mainly focus on identifying any new risk or issue and reviewing that mitigation measures and corrective actions are appropriate and correctly deployed. Through this activity, the life-cycle data that provides evidence of compliance to regulation can be reviewed. This is most effective if it is performed regularly throughout the development rather than limited to formal milestone reviews only. Unless new complex risks or issues arise, it is usually not necessary for experts to be involved in these activities.
While it is important for design assurance engineers to be independent, they should be able to work in close collaboration with development teams. Furthermore, design assurance should be able to work across organizational boundaries to assess risks at all hierarchical levels. This allows design assurance to be both a supporting role for development teams and an independent means of identifying issues.
With this approach, all of design assurance’s activities are driven by project-specific risk. This allows any deviation to be assessed based on this risk, reducing the likelihood of focusing too much energy and resources on minor problems such as process deviations. It also provides the flexibility to adapt to any new technology, system architecture or regulation.
It is important to note that the risk discussed in this paper is eminently technical. It is necessary for a design assurance engineer to have the technical skills and knowledge to fully understand the product’s design and operation and deduce the system and safety effects that any given low-level HW issue may lead to.
Figure 4 – HW Design Assurance Engineer Required Skills and Knowledge
The vision of HW design assurance proposed in this paper is appropriate to address some of the most common issues seen in safety-critical product developments today. It is based on a simple concept: understand technical risk and tailor development and assurance activities to focus energy and resources on areas that deserve it. This provides the flexibility to effectively support developments regardless of the product, industry and regulatory framework.
For the past 35 years, APSYS has been helping industrial customers control technical, human and operational risks. Whether developing ever more reliable aircraft, making autonomous vehicles safe, anticipating obsolescence or protecting industrial assets against cyber-attacks, customers can rely on the expertise of APSYS consultants to define and help deploy tailor-made solutions that are right for them.
About the Authors:
Gabriel Godfrey is a Technical Advisor at APSYS for electronics and design assurance. He has been developing and integrating a variety of airborne electronics systems for 15 years. He has played an active role in the certification effort for 6 large aircraft programs and has led countless design reviews in the process.
Cédric Munoz is the Technical Officer for the Aerospace Business Unit at APSYS. He has been working on the development of critical embedded systems for almost 20 years in roles ranging from hardware designer to certification auditor (and everything in between). In the process, he has collaborated with several major aircraft integrators and certification authorities (EASA, FAA, DGA, etc.)
If you need more information please contact :
Gabriel GODFREY Gabriel.Godfrey@apsys-airbus.com
Cédric MUNOZ : Cedric.Munoz@apsys-airbus.com