On March 12, 1984, former Missouri State Land Surveyor, Robert E. Myers, issued a memorandum, addressed to "Users of the EDM Baselines," providing an "Explanation of the Calibration Report," in which he states,
"The rejection limit is based on three times the standard deviation determined by this calibration procedure. Some values will be outside the rejection limit."
Since the rejection limit values are being "determined by this calibration procedure," a look at the "calibration procedure" may provide an answer. The original DOS version of the processing application used by the MoDNR was written by John Paulsmeyer, using the Q-BASIC programming language. This program was the analysis tool used prior to the implementation of the online application (see link above). Examining the source code of this DOS version reveals the following in regard to the rejection limit:
DRL = 3# * 0.000005 * DH(X) + 3# * 0.005
where DRL = rejection limit and
DH(X) = the measured horizontal distance in meters for the segment.
Rearranging this equation produces the following result:
rejection limit = 3 * (0.005 m + (measured distance) * 0.000005), or
rejection limit = 3 * (5 mm + (measured distance) * 5 ppm)
With the equation in this form, it is apparent that the rejection limit in the processing program is based on a fixed system constant of 5 millimeters and a fixed system scale factor of 5 ppm. Referring again to Mr. Myers’ memo, the following statements are found:
"The system constant is composed of two components, the prism constant and the error of the phase difference determination. In the ideal situation, the prism constant will be known, so the error in phase determination will be the only unknown. For most EDM equipment the phase difference determination error will be plus or minus five millimeters."
and
"... all the error which would be shown by the system scale factor would be a result of the error in the modulation frequency. The standard error in frequency would be approximately 10 megaherts or approximately an error of 5 parts per million."
So, the rejection limit shown on the MoDNR report is based on EDM equipment having a manufacturer’s stated precision of ±(0.005 m + 5 ppm*D), which represents the equipment being used by land surveyors at the time that Mr. Myers composed his memo in 1984.
***************************
Note: In August 2013 the Missouri State Land Survey was transferred from the Missouri Department of Natural Resources to the Missouri Department of Agriculture.
----------------------------------------
Original composition by Steven E. Weible
