Undestanding the P-Value
HOW DOES HEARLAB® KNOW IF A CORTICAL IS PRESENT OR NOT, AND WHAT IS THE P-VALUE?
HEARLab® ACA employs statistical techniques to determine whether a CAEP is actually present. The analysis result is expressed as a p-value. This value represents the probability that the measured response is caused by random noise only. For example, if a p-value is equal to p=0.005, it is only 1 chance out of 200 that the response is not caused by the auditory stimulus. A detection p-value < 0.05 is often taken to represent a significant response. The lower the p-value that is obtained, the greater is the probability that a CAEP is present. See section 8.6 of the HEARLab® manual for more information on statistical processing.
WHAT IS AN EPOCH?
An epoch is a part of the subject’s EEG, defined around the moment the sound stimulus has been presented. More specifically, from 200 milliseconds before to 600 milliseconds after the moment a stimulus is presented. Many epochs make up the averaged waveform.
WHY DO WE COLLECT SO MANY EPOCHS FOR EACH TEST RUN?
The greater the number of accepted epochs in a test run, the higher the quality of the resulting average waveform is likely to be. However, the number of epochs impacts on the test time and so it is a consideration, particularly if the tester wishes to conduct a large number of runs or has limited time to test.
WHAT STOPPING CRITERIA DO YOU SUGGEST?
Section 7.3.4 of the HEARLab® manual advises on use of detection statistics and the noise indicator and on deciding on the number of epochs to collect for each run. The following criteria has also have been adopted by Australian Hearing:
Predetermined number of epochs: 240 for very young infants, 200 for children older than 12 months.
- Stop when: predetermined number of epochs (i.e. 240 or 200) has been reached; then decide whether response is there or not based on p=0.05 value.
- Once all stimuli being tested reach p=0.001 (indicated in HEARLab® by p=0.001 or p=0.000). This criterion signifies there is less than 1/1000 chance the response is caused by random elements in the recording.
SHOULD 240 EPOCHS BE COLLECTED FOR BOTH CHILDREN AND ADULTS?
This predetermined number of epochs is recommended for young infants < 12 months. For older children, it doesn’t seem a problem for us to change to a lower value, like 200 epochs or even 100 epochs for adults. This likely will be done anyway when experience with HEARLab® increases. Changing the predetermined number of epochs has no influence on the results, as long as everybody just lets it run until the end if p<0.001 hasn’t been reached in the meantime.
WHAT IS THE MINIMUM CRITERION FOR EPOCHS?
30 accepted epochs – This condition is fulfilled automatically as HEARLab® doesn’t show p-values before 30 accepted epochs are collected. This minimum number has been chosen as large enough to allow the statistics to collect a sufficient number of epochs on which to base its p-value estimate.
THE P-VALUE TRACE KEEPS WRIGGLING AROUND DURING TESTING AND AT TIMES IT LOOKS PRESENT BUT THEN FALLS ABOVE 0.05 AT THE END OF TESTING– CAN WE SAY THAT THE SPEECH STIMULI HAS BEEN DETECTED?
No, you cannot say a person has detected that speech sound. You must allow a correct use of statistics to determine if the response is present or not. The reason why we instruct to only evaluate response presence when data collection has stopped is because of the increased chance of falsely detecting a speech sound when doing multiple tests. And that’s exactly what you’re doing if you follow the trace of the p-value during CAEP testing: each time the p-value trace creeps forward, you’re actually doing one extra statistical test, which (when all added together) will increase the chance the p-value will go smaller than 0.05 falsely. Compare it with crossing a highway in Sydney blindfolded. You have a 5% chance of being hit by a car, but if you do that ten times in row, chances will increase (definitely a false result!). So please only check the p-value when the system has finished data collection.
One exception: when you reach p = 0.001, chances are so small (1/1000) to make a false decision that you’re allowed to stop earlier.
CAN I EXPECT TO REACH SIGNIFICANT P-DETECTION VALUE IF I RUN THE EPOCH FOR LONGER?
Yes, it is possible to achieve a significant p-value under 0.05. But it is important to stop the recording at one point in time to allow a correct use of statistics (read not increasing the number of false detections which inevitably occur with a p-value of 0.05). Assuming you’ve accepted 240 epochs, the average will contain 240 epochs, so adding 20 more shouldn’t change it significantly. There may be occasions where you expect to see a response and you don’t, in this case you may wish to retest at a higher level to see if it is present.