In IVD manufacturing, consistency remains a critical parameter across the product lifecycle. On the IVD front, consistency means maintaining reliable performance when measurements are repeated across different environments as well as when relevant variables such as instruments, operators, laboratories, or testing conditions change. Since IVD testing involves biological samples such as blood, urine, or tissue outside the human body, some degree of variance is inherent to the measurement process.
What exactly is variance?
Variance is the extent to which test results differ from one another when measuring the same samples under defined or standardized conditions. These differences can arise at various stages across the IVD lifecycle, with lab-to-lab and lot-to-lot variances representing two important sources of performance differences.
Lot-to-Lot Variance: Differences in IVD performance or reported results between successive manufacturing lots of the same product.
Lab-to-Lab Variance: Differences in IVD performance or reported results when the same IVD assay is used across different laboratories, where instruments, operators, workflows and testing conditions may vary.
These sources of variation can then be characterized using analytical performance measures such as repeatability, intermediate precision, and reproducibility, depending on the conditions and variables being assessed.
What Causes Lot-to-Lot and Lab-to-Lab Variance?
Both lot-to-lot and lab-to-lab variance can arise from different sources, but both ultimately affect how consistently an IVD performs across the conditions in which it is manufactured and used.
For medical device manufacturers, evaluating these sources is important because even relatively small shifts can significantly affect analytical performance, increasing the resources and efforts required to maintain consistency at scale.
Some common sources causing Lot-to-Lot Variance include:
- Raw material variability: Differences in the quality, purity, activity or stability of critical materials can affect assay performance.
- Reagent formulation: Changes in concentrations, mixing, dispensing or processing conditions can affect reaction kinetics and signal generation.
- Component tolerances: Variations in dimensions, material properties or surface characteristics can influence fluidics and sample handling.
- Manufacturing conditions: Changes in temperature, pressure, cycle time, assembly parameters, or other process conditions can introduce differences between lots.
- Calibration: Changes in calibrators or calibration procedures can create shifts in reported results
- Process drift: Small shifts that remain within individual specifications can accumulate across successive lots and become more significant over time
The potential impact is not merely theoretical. A recent study showed that kit lot changes in clinical labs lead to variations in patient results. The study observed variations during 60 reagent lot changes over the period of 1 year. Statistically significant differences between patient results and QC results were observed in 16.7% of the lot changes. It was also found that QC results varied by 3.3% across the lot-change events.
Although these results do not represent a universal rate of lot-to-lot variation across IVDs, it illustrates why manufacturers and laboratories need robust approaches to identify and evaluate changes between lots.
Lab-to-lab variance occurs when the same IVD produces different results across laboratories. It is primarily caused by the factors such as measurement environment, analyzer, workflow, and operator.
Lab-to-lab variance may be caused by:
- Analyzer differences: Variations in optical, thermal, fluidic, or electronic performance may directly influence measurements
- Sample handling: Collection, preparation, and storage practices can affect the final result
- Calibration: Differences in calibration materials, calibration status, or procedures can introduce systematic shifts
- Environmental conditions: Temperature and other laboratory conditions may influence IVD assay or analyzer performance
- Operator technique: Manual steps can introduce differences in timing, handling and execution
- Instrument maintenance: Differences in maintenance, component condition, and instrument performance can contribute to variability.
- Laboratory workflows: Differences in procedures, protocols and implementation can affect how the same IVD is used
- Sample concentration and Hook Effect: Very high analyte concentrations can cause an unexpectedly low IVD assay response in certain immunoassays. Differences in dilution practices can therefore contribute to variation in reported results.
For manufacturers, a change between different reagent lots may introduce a shift in accuracy or precision, while differences between laboratories, instruments, or operators can directly affect reproducibility. When these variations accumulate or interact, they may lead to inconsistent results, making it difficult for manufacturers to maintain a predictable IVD performance across production and real-world use.
How are IVD Manufacturers Tackling Variance?
IVD manufacturers tend to identify the variables that affect performance and control them throughout the product lifecycle. As such, they combine robust design, process control, automation, risk management, testing, and traceability across relevant sources of variation.
- Design for Manufacturability: DFM helps identify manufacturability risks early and optimize designs for production
- Component Standardisation: Standardising components can reduce unnecessary sources of variation while simplifying production and supply management
- Assembly Simplification: Simplifying and modularizing assembly processes can reduce process complexity and support more consistent production
- Risk Management: Embedding structured risk management and design controls throughout development can help identify and manage potential sources of product and process variation
- Automation: Standardized automation can make critical processes more controlled, repeatable, and programmable, reducing operator-dependent variation
- Testing and Verification: Automated and functional testing enables consistent, repeatable evaluation of product performance
- Traceability: Maintaining traceability across design, manufacturing and quality records can help identify and investigate sources of variation throughout the product lifecycle.
- Lifecycle Engineering: Managing design, manufacturing, and systems-level changes across product generations helps maintain consistent performance as an IVD evolves and scales.
A cohesive approach to analyzer development and assay testing helps identify interactions between the assay, analyzer, and manufacturing process earlier. By controlling variation from component selection and assembly through testing and lifecycle management, manufacturers can improve lot-to-lot consistency.
At the same time, controlling these variables can strengthen reproducibility across different instruments, operators and laboratory environments, helping ensure that changes in the testing context do not translate into unintended shifts in reported results. This becomes particularly important as IVDs scale across sites, where consistent system performance is essential for maintaining confidence in results and supporting broader clinical adoption.
What are Repeatability and Reproducibility in IVDs?
Controlling sources of variation is only one part of achieving consistency. IVD manufacturers must also assess how much variation remains when the same measurement is repeated under identical conditions or when expected variables change. Repeatability, intermediate precision, and reproducibility provide different levels of this assessment.
Repeatability: Consistency Under the Same Conditions
Repeatability assesses how consistently the same IVD measurement system produces results when the same sample is tested repeatedly under essentially identical conditions, such as the same instrument, operator, reagent lot, and testing environment. It helps identify short-term variation caused by factors such as pipetting, fluidics, optical performance, temperature, or reagent stability.
Intermediate Precision: Variation Within a Laboratory
Intermediate precision evaluates performance when expected variables within the same laboratory or measurement system change, such as testing days, runs, operators, or calibration events. It helps determine whether the IVD maintains consistent performance beyond a single tightly controlled test condition.
Reproducibility: Consistency Across Conditions
Reproducibility evaluates performance when broader sources of variation are introduced, such as different instruments, laboratories, or testing sites. Testing the same sample across these conditions can reveal the impact of analyzer characteristics, workflows, environmental conditions, calibration, and sample handling.
Together, repeatability, intermediate precision, and reproducibility help manufacturers determine whether variation originates within the measurement process, from expected changes within a laboratory, or from broader differences across instruments, laboratories, or sites. This provides a structured basis for evaluating consistency across the IVD measurement system.
What is the Business Impact of Uncontrolled Variance?
For IVD manufacturers, when the performance varies with lots, instruments or laboratories, the resulting variability can create additional operational effort, quality risk and challenges in scaling the product reliably.
Uncontrolled variance may have significant adverse effects, including:
- Higher investigation and troubleshooting costs: Unexpected shifts can require additional root-cause investigations, testing, and engineering resources to identify the source
- Increased retesting and rework: Variability that is beyond the defined acceptance criteria can result in additional testing, rework, rejected lots, or delays in product release.
- Complex validation and scale-up: As production moves from development to scale, uncontrolled sources of variation can make process validation and transfer more difficult.
- Quality and regulatory risk: Persistent or unexplained variation can affect analytical performance or consistency across intended-use environments, leading to additional scrutiny and corrective actions
- Delayed time-to-market: Additional investigations, design changes, process validation cycles, and regulatory activities can extend development timelines and delay commercialisation.
Ultimately, controlling variance helps manufacturers move from reactive troubleshooting to predictable product performance, improving the overall IVD performance across different lots and laboratory environments.
Controlling Variance Requires an Integrated Analyzer-Assay Approach
In an IVD, the assay and analyzer form the core of the measurement system. The assay and analyzer work together as a measurement system, with the assay generating the biological or chemical response, and the analyzer detecting, processing, and converting that response into a reported outcome. Factors such as reagent behavior, sample handling, fluidics, optics, thermal control, calibration, and signal processing can therefore affect the final measurement.
Maintaining consistent performance therefore requires manufacturers to assess how the complete system behaves under both controlled and changing conditions.
Evaluating this consistency requires more than a single measure of analytical performance. Controlling lab-to-lab and lot-to-lot variance can benefit from an integrated approach involving repeatability, intermediate precision, reproducibility, and proficiency testing:
- Repeatability: Evaluates how consistently the IVD produces results with the same method, instrument, operator, and testing conditions
- Intermediate Precision: Examines performance with the change in expected variables within a laboratory, such as days, operators, runs, or calibration events
- Reproducibility: Enables assessment of the same sample across broader variation sources, such as different instruments, laboratories, operators or testing sites
- Proficiency Testing: Compares laboratory results against an assigned value or peer-group performance, breaking down lab-to-lab variance
A cohesive approach that considers all four parameters can help identify sources of variation and assess their impact across different testing conditions. Together, these insights contribute to greater reproducibility across the IVD measurement system.
How Much IVD Variance Is Acceptable?
The tolerable variation levels in IVD testing depend on how the result is used clinically, intended device use, and the analytical performance requirements for the specific test. For instance, in quantitative assays, the total allowable total error can be established based on the clinical use of the measurement and the consequences of analytical error.
For IVD manufacturers, this means that variance should be assessed against predefined, scientifically justified acceptance criteria, rather than a fixed industry threshold. These criteria may consider factors such as:
- Measurand: The analyte or parameter being measured, along with the biological and clinical significance.
- Intended use: Whether the IVD is being used for screening, diagnosis, monitoring, or another clinical purpose
- Analytical performance: Requirements for precision, bias, accuracy, linearity, and measurement range
- Clinical consequences: The extent to which a shift in results could change clinical interpretation or patient management
- Method and measurement system: The characteristics of the assay, analyzer, calibration system, and associated components.
- Applicable standards and regulatory requirements: Relevant regulatory expectations and consensus standards may provide frameworks for evaluating performance criteria.
According to the US FDA’s recognized consensus standards for medical devices, CLSI EP46 provides guidance for establishing allowable total error goals and limits for quantitative medical laboratory measurement procedures, while CLSI EP21 addresses the evaluation of total analytical error against those predefined limits.
In the EU, Regulation (EU) 2017/746 (IVDR) requires manufacturers to address analytical performance characteristics including precision, repeatability and reproducibility as part of demonstrating conformity with the applicable performance requirements.
In India, the Medical Devices Rules, 2017 and CDSCO guidance similarly address repeatability, reproducibility and other analytical performance characteristics as part of IVD performance evaluation. Together, these frameworks highlight the importance of systematically evaluating measurement consistency across defined testing conditions and sources of variation.
A variance that is statistically detectable is not necessarily clinically significant, and a variance that falls within a predefined analytical limit is not automatically acceptable for every intended use. Thus, manufacturers should consider both the statistical behavior of the measurement system and the clinical context in which the result will be interpreted.
How an End-to-End IVD CDMO Can Improve Product Development
For IVD manufacturers, variance is not a single-point quality issue. It can originate in materials, reagents, components, manufacturing processes, analyzer performance, laboratory workflows, or the interaction between the assay and analyzer.
Thus, the focus should not be on eliminating every source of variation, but they evaluate and control the variation that can affect the intended performance of the IVD. This requires manufacturers to connect robust design, controlled manufacturing, verification, analytical performance assessment, and lifecycle management rather than addressing each source in isolation. This is precisely where integrated MedTech CDMOs can add value.
By bringing design, engineering, manufacturing, and testing capabilities together, manufacturers can address potential sources of variation earlier and manage them through scale-up and production. SJML’s IVD capabilities span design for manufacturing, microfluidics, optical detection, electronics, precision plastics, and scalable production, supporting a connected approach to engineering and manufacturing across the device lifecycle.
