This post was recently updated on July 6, 2020

Regulatory Agencies, like the FDA and Eudralex require companies to audit software vendors of their critical software and systems. Regulated companies include several industries – Pharmaceuticals, Biologics, Medical Devices, Clinical Studies, and Blood Products – and include organizations that sell in both the US and International markets.

Guidance from the FDA states that:

“…the manufacturer should consider auditing the vendor’s design and development methodologies … and should assess the development and validation documentation generated for the OTS software.”

Software vendors are typically audited on their product or service and processes. For software products, audits are usually performed at the vendor’s site. Auditors focus on the vendor’s conformance to standards, requirements or procedures for specific processes such as testing, change control and incident handling.

Why Should I Audit Software Vendors?

1. Regulatory agencies expect you to audit your software vendors

As mentioned in the beginning, regulatory agencies like the FDA, Eudralex and ICH require vendor assessments for your critical software and systems.

EudraLex states that supplier competence is key. The decision whether or not to audit should be based on a risk assessment and they expect risk assessments to be documented.

In EudraLex Annex 11, a supplier is defined as:

A third party who provides, installs, configures, integrates, validates, maintains, modifies, or retains a computerised system.

ICH Guidance states:

“The user should take all reasonable steps to ensure that it [software] has been produced in accordance with an appropriate system of Quality Assurance. The supplier of software should be qualified appropriately; this may include assessment and/or audit.”

2. It’s good business practice to safeguard your investments

When buying a house, you have and inspection done, right? You have an inspection done so that you know you are buying a reliable house. Does it have termites or mold? Are there cracks in the foundation or leaks in the plumbing? You want buy a good, solid house that will last as long as you plan to live in it. After the inspection, you can continue through negotiating and closing processes with a more educated opinion of the house and it’s overall condition.

Buying a software system is just like buying a house. You audit the software vendor to answer some key questions.

• Are you buying a reliable, high-quality system?
• Will the vendor continue to support the system when it’s older?
• Will the vendor be around to support the system?

3. You can utilize vendor documentation to save time and money

Think about our home-buying example. After the inspection, the inspector writes a report summarizing the condition of the house, specifying any critical or major faults. You then use the information in the inspection report to negotiate with the sellers and plan for any projects you will have down the road (like upgrading the kitchen). If the inspection report comes back squeaky clean, then you trust that you have a good house on your hands. Work after you move-in is minimal – paint the walls a favorite color and you’re ready to enjoy your new house.

You can utilize your software vendor’s documentation in a similar way, and the FDA says you can:

If the vendor can provide information about their system requirements, software requirements, validation process, and the results of their validation, the … manufacturer can use that information as a beginning point for their required validation documentation. The vendor’s life cycle documentation, such as testing protocols and results, source code, design specification, and requirements specification, can be useful in establishing that the software has been validated. — FDA Guidance: General Principles of Software Validation

This is proof from the FDA that when the vendor can provide things like requirements and validation results, these documents can be useful in your validation. In other words, you don’t have to recreate what the vendor has already done! Plus, leveraging their requirements and testing documentation can save your project time and money. It can also cut down on the amount of redundant work you would otherwise be doing.

When Should I Audit My Software Vendors?

There are multiple points at which you can and should audit software vendors, preferably early in the selection process.

Best Option – During the Vendor Selection Process

Just like inspecting a home before you buy it, the best time to audit your software vendor is during the vendor selection process of your project – prior to purchasing the system. Here you have the most leverage to get the vendor to remedy anything that’s not acceptable. The vendor will want to prove their system is the best choice for you and they’re more likely to fix issues to keep them alive in the selection process.

2^nd Choice – Prior to Validation

If you’ve already purchased the system, the next best time to do the audit is prior to validation. Do the audit early in the project planning phase to help determine the usability of vendor documentation. Auditing a vendor at this point can impact your project timeline either positively or negatively, so be aware of that.

Other Audit Timing

Outside of vendor selection and doing validation, there are a few other reasons you might choose to validate a vendor.

• Your company policy requires Periodic Vendor Review
• If the system is considered a ‘critical system’, you might choose to audit when accepting product upgrades
• The vendor had a critical issue in a previous audit that warranted significant risk mitigation activities
• The vendor has made quality system improvements and you would like to elevate their rating for your next project
• There has been serious deterioration in product quality or services

For in-depth training on the entire software vendor audit process, attend our free webinar.

Examples of FDA Warning Letters related to Software Vendors

Finally, to further emphasize the importance of auditing your software vendors, there have been several instances where the FDA has issued a warning letter for purchased software. Let’s take review a few of them, and you can download the warning letter for further reading.

Example 1 – Tecan US, 2004

Background: Company manufactures a laboratory workstation system that performs in-vitro diagnostic tests on patient samples. The FDA was investigating a problem where the workstation mismatched patient records when they found additional problems.

Observations: Company did not have complete documentation of the purchased software used in the workstation. They were missing complete requirements, complete design specs, and complete validation testing. Validation testing was incomplete and only for modification.

The warning letter cites:

Failure to have complete validation of the XXXXXX software program.

–Your firm did not have documentation of complete requirements specifications and software design specification for the entire XXXXXX software program.

–Documentation of the software program provided by the original developer of the software was limited and did not show a complete validation of the software program.

–Additionally, validation of the software upgrades was limited to the changes or additions made to the software program by the upgrades.

–Validation has not been conducted on the entire XXXXX software program. Test cases only tested the software modifications with no references or linkage to an overall software program validation.

Download the Tecan US Warning Letter

Example 2 – Omron Healthcare, 2008

Background: Firm is an importer and distributor of foreign manufactured diagnostic and therapeutic devices. There were underlying problems with products injuring patients and failure to report these problems to the FDA and manufacturers

Observations: Third party validation testing of the purchase system wasn’t reviewed and approved in house.

The warning letter cites:

Failure to adequately validate computer software used in an automated process for its intended use according to an established protocol.

–For example, no person from your firm reviewed or approved the third party approval test results for the original “XXXXXX Complaint System Validation” used in your firm’s quality system

Download the Omron Healthcare Warning Letter

Example 3 – Sunbeam Products, 2012

Background: Company manufactures heating pads. The FDA was investigating a problem where the auto-shut-off feature was not working correctly, and patients had been burned. There had even been one report of the device starting on fire and another report of the device burning the carpet.

Observations: Company was not ensuring the software suppliers were performing both functional and structural testing for medical device software.

The Warning Letter cites:

Firm’s SOP “Purchasing Control for Outside Services” does not assure that software suppliers complete both function and structural software testing.

Download the Sunbeam Products Warning Letter

Auditing software vendors is an important step in maintaining your regulatory compliance.

It’s just good business practice to know exactly what you’re purchasing. Ideally, it’s best to audit your software vendor prior to purchasing their product. But if you have already purchased a system, you should inspect your vendors to maintain confidence in their product and your relationship with them. In our examples, while the FDA was often investigating the companies for other reasons, warning letters included citations for issues related to 3rd party software.

  Read More Posts About: Compliance Audits