As the creator of PACKTRACK and an expert in the field I’m commonly asked about extending our K9 record keeping system to compute an accuracy percentage statistic during K9 detection training.
The requested method of calculating this statistic is simple. If the dog finds or “hits” the target odor then it’s marked as correct. When the dog doesn’t find or “misses” the target odor then it’s marked as incorrect. If the dog indicates that a target odor is present when it isn’t (sometimes referred to as a “false alert”) then it’s also marked as incorrect. The statistic is calculated based on the percentage of correct responses.
The request for this feature may come from a misunderstanding of the 2013 Supreme Court case of Florida v Harris or be based on advice received from a trainer. The US Supreme Court ruling in this case explicitly rejected an earlier Florida court ruling that favored “rigid rules” in favor of a “Totality of Circumstances” approach. The key problem with using an accuracy percentage calculation in deployments is that it’s impossible to accurately say whether the dog was incorrect. The Supreme Court ruling went even further by stating that probable cause cannot be and has never been based on a “bright-line test” such as a statistical calculation. Probable cause is based on a “totality of the circumstances” analysis. Based on this decision, the idea of using an accuracy statistic for training would not be beneficial in court.
Another problem with this idea is it presumes that the outcome of a training exercise provides confirmation that the training was done correctly. It seems logical to assume that hits are correct responses while misses and false alerts are incorrect responses. However, this simplification reads more into the data than is captured. The idea that we can equate locating a target odor (a hit) to a determination of whether the training was done correctly is absurd. It’s like presuming that since no tangible substance was found after a K9 indication during a deployment that this indication must have been a false alert or somehow incorrect.
Behavior modification in drug detection training is mainly done with positive reinforcement at the appropriate time. Handlers can inadvertently train canines to respond to a “cue”, a behavior, sound, or action that the canine learns as a signal to respond in a positive way that a target odor is present. The canine learns this behavior over a series of exercises where the cue (an action taken by the handler) was done and the canine was rewarded in a positive manner. This cueing is consistently reinforced by the handler during training exercises and may cause what some call a “false alert”. While this “false alert” will be correctly marked as an incorrect response by the dog, it is actually a red flag that there is a training issue. The real incorrect training exercises are those prior to the “false alert”, when the handler performed the cue where target odor was concealed, and the canine was rewarded. During those exercises the canine was taught the cue. These exercises would be marked as “hits” with the presumption that the exercises were conducted correctly when in fact they were not.
An accuracy statistic requires that we read too much into the outcome of an exercise. There is no way to provide this statistic and we shouldn’t try. Training time should be focused on finding and correcting the problems that we, as handlers and trainers, may have caused. Training is the time to push teams to their limits and beyond. Handlers should come to training feeling free to make and learn from their mistakes so they can prevent and correct them in future training. Exercises should condition handlers to perform proper technique while increasing their knowledge and perfecting the skill of dog handling.
In using the proposed statistic handlers are forced to only work to the limit of correct performance. They will only do exercises that they know they can complete properly in order to maximize their accuracy percentage. Who would want to push themselves to the point of error if a statistic, which provides no explanation, will be used to gauge their reliability?
My advice is to be careful if you quantify your hits, misses and false alerts as a percentage of accuracy. You are keeping records that can negatively impact the quality of your training and provide a false sense of proper training. And, if you’re challenged in court, it’s unlikely that this statistic will do much to help you win a reliability argument. There are better ways to train and document your records and we will talk about this in future posts. Stay tuned.
