PS01 — Paired difference experiment procedure
Category: Statistical methods
Bulletin: PS01
Document(s): SS01
Issue Date:
Effective Date:
Table of content
 1.0 Purpose
 2.0 Scope
 3.0 Reference
 4.0 Terminology
 5.0 Symbols
 6.0 Procedure
 Appendix A – Test results forms
 Appendix B – Analytical results form
1.0 Purpose
1.1 The purpose of this procedure is to provide instructions that may be used for performing an experiment to assess the effect of a change on a characteristic of a sample unit, using the sample unit as its own control.
1.2 This procedure^{Footnote 1} can be used as a reference by organizations to develop their own procedures for performing a replicated paired difference experiment.
1.3 This procedure assumes that the minimal conditions for applying ISO 3301:1975 have been met, but additional criteria can be used.
2.0 Scope
2.1 This procedure applies where:
 a replicated paired difference experiment may be conducted on retrofitted diaphragm gas meters in service with an automatic meter reading (AMR) device and/or register;
 the meters tested are similar in all respects except for the systematic difference which is being evaluated;
 the analysis of the test results is based on ISO 3301:1975; and
 a single operator is involved in effecting the change; or
 more than one operator is involved in effecting the change and a technical professional thoroughly examines the procedure and determines that there is no possibility for inter‑operator variability.
3.0 Reference
3.1 SS01 – Specifications for Random Sampling and Randomization
4.0 Terminology
For the purposes of this procedure, the following terms and definitions and those in the normative reference apply.
 Experiment (designed experiment)

Experimental plan selected so as to meet a specific objective.
 Replication

Repetition of an experiment more than once for a given group of variables used in regression to predict others variables.
 Operator

Person performing the operation of interest in the experiment.
 Grand mean

Mean of a set of means.
5.0 Symbols
For the purposes of this procedure, the symbols in the normative reference and the following symbols apply.
 d

the signed difference between paired observations
 m

the number of measurement replications
 n

the number of units in the sample
 x

the measured value of a characteristic under the initial set of conditions
 y

the measured value of a characteristic following the introduction of the change being evaluated

the value of x corresponding to the jth replicate of the ith sample unit

the mean value of x_{i} for the sample of n units

the estimate from the sample of the variance
 v

the number of degrees of freedom
 ∑

the sum of

the sum of all the x values when i takes integral values from 1 to n

the absolute value of an arbitrary variable g
6.0 Procedure
6.1 Planning
 Define the population or universe to which the results of the experiment will apply.
 Define the nature of the change that will be introduced to the population under study.
 Specify the characteristics of the population units which will be measured.
 For each sample unit, establish the size of the sample to be chosen from the population and the number of replicated measurements to be performed.
 Maintain a record of the data obtained in steps (a) to (d).
6.2 Sample selection and preparation
 Randomly select a representative sample from the defined population or universe in accordance with the reference cited in section 3.1.
 Condition the units of the sample to stabilize the characteristics to be measured.
6.3 Sample evaluation
 Measure the characteristics specified for each of the sample units, replicating the measurements the same number of times for each characteristic.
 Record all measurement results on the form supplied in Appendix A.
 Introduce the change to be evaluated to each of the sample units.
 Repeat steps (a) and (b).
6.4 Statistical calculations
The following calculations are to be applied when using Appendix A.
 For each characteristic of subject i under controlled conditions, calculate the mean () of each set of replicated measurements in accordance with the following equation:
 From the means obtained in step (a), calculate the grand mean in accordance with the following equation:
 For each characteristic of subject i under experimental conditions, calculate the mean () of each set of replicated measurements in accordance with the following equation:
 From the means obtained in step (c), calculate the grand mean in accordance with the following equation:
 Calculate the signed differences (d_{i}) between the means of the replicated measurements taken before and after the introduced change for each characteristic in accordance with the following equation:
 Calculate the mean () of the signed differences (d_{i}) for each characteristic in accordance with the following equation:
 Calculate the variance () and the associated degrees of freedom (v) of the signed differences (d_{i}) for each characteristic in accordance with the following equations:
 Calculate the ratio of Student's tdistribution for the sample size n in accordance with the following equation:
 Calculate the tolerance adjustment for the confidence interval and the variance of the sample for each characteristic in accordance with the following equation:
 Maintain a record of the results of all statistical calculations performed in steps (a) through (i).
6.5 Analysis of results
6.5.1 Statistically significant mean paired difference experiment
 Determine if a statistically significant mean paired difference is within upper and lower bias limits using the two one‑sided cases below:
Case 1
The hypothesis that the population mean of the differences for each characteristic is at least equal to the upper tolerance limit (d_{1}) (null hypothesis) is rejected if:
Case 2
The hypothesis that the population mean of the differences for each characteristic is at most equal to the lower tolerance limit (d_{2}) (null hypothesis) is rejected if:
 There is insufficient evidence that a mean paired difference outside a ±0.2% limit exists if the null hypothesis is rejected for Case 1 and Case 2 for each characteristic.
Appendix A – Test results forms
A.1 Test results form—Controlled conditions
Sample Unit 
Controlled conditions (before change introduced)  

xi1  xi2  xi3  xi4  xi5  i  
Sample units 1 to 30  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the lot number. 
Enter the grand mean of the means for the sample units. 
A.2 Test Results form—Experimental conditions
Sample Unit 
Experimental conditions (after change introduced)  

yi1  yi2  yi3  yi4  yi5  i  
Sample units 1 to 30  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the meter performance test results in percent error.  Enter the lot number. 
Enter the grand mean of the means for the sample units. 
Appendix B – Analytical results form
Description  Symbol  Characteristic 1  Characteristic 2  Critical value 

Sample size  n  Enter the number of units in the sample.  Enter the number of units in the sample.   
Replicates  m  Enter the number of measurement replications.  Enter the number of measurement replications.   
Degrees of freedom  Enter the degrees of freedom.  Enter the degrees of freedom.    
Controlled conditions  Enter the grand mean of the means for the sample units.  Enter the grand mean of the means for the sample units.    
Experimental conditions  Enter the grand mean of the means for the sample units.  Enter the grand mean of the means for the sample units.    
Confidence interval (%)        95 
Difference statistics  Enter the mean of the signed differences between paired observations.  Enter the mean of the signed differences between paired observations.    
Enter the variance of the signed differences between paired observations.  Enter the variance of the signed differences between paired observations.    
Enter the ratio of Student's tdistribution for the sample size.  Enter the ratio of Student's tdistribution for the sample size.    
A _{1}  Enter the tolerance adjustment for the confidence interval and the variance of the sample between paired observations.  Enter the tolerance adjustment for the confidence interval and the variance of the sample between paired observations.    
Analysis of dependent variables based on ISO 3301:1975  d _{1}      0.2 
d _{2}      0.2  
Case 1 (at most)  Inside/Outside  Inside/Outside    
Case 2 (at least)  Inside/Outside  Inside/Outside    
Conclusion (95%)  Inside/Outside  Inside/Outside  Inside/Outside tolerance 
Footnotes
 Footnote 1

The use of this procedure entails inherent risks and limitations with regard to the conclusions that may be drawn from it. Meter owners are therefore advised that, although conformity with the requirements of this procedure may qualify meter owners to seek accreditation for the installation of AMR devices on diaphragm meters in situ, relying solely on the use of this procedure will not provide meter owners with an assurance of compliance with the metering accuracy obligations prescribed under the Electricity and Gas Inspection Act. Contractors are ultimately responsible for ensuring the performance quality of the inservice meter lots which they own.
 Date modified: