Systems, Subjects, Sessions: To What Extent Do These Factors Influence EEG Data?

Lab-based electroencephalography (EEG) techniques have matured over decades of
research and can produce high-quality scientific data. It is often assumed that the
specific choice of EEG system has limited impact on the data and does not add
variance to the results. However, many low cost and mobile EEG systems are now
available, and there is some doubt as to the how EEG data vary across these newer
systems. We sought to determine how variance across systems compares to variance
across subjects or repeated sessions. We tested four EEG systems: two standard
research-grade systems, one system designed for mobile use with dry electrodes, and
an affordable mobile system with a lower channel count. We recorded four subjects
three times with each of the four EEG systems. This setup allowed us to assess the
influence of all three factors on the variance of data. Subjects performed a battery of six
short standard EEG paradigms based on event-related potentials (ERPs) and steadystate
visually evoked potential (SSVEP). Results demonstrated that subjects account for
32% of the variance, systems for 9% of the variance, and repeated sessions for each
subject-system combination for 1% of the variance. In most lab-based EEG research,
the number of subjects per study typically ranges from 10 to 20, and error of uncertainty
in estimates of the mean (like ERP) will improve by the square root of the number of
subjects. As a result, the variance due to EEG system (9%) is of the same order of
magnitude as variance due to subjects (32%/sqrt(16) = 8%) with a pool of 16 subjects.
The two standard research-grade EEG systems had no significantly different means from
each other across all paradigms. However, the two other EEG systems demonstrated
different mean values from one or both of the two standard research-grade EEG
systems in at least half of the paradigms. In addition to providing specific estimates of
the variability across EEG systems, subjects, and repeated sessions, we also propose
a benchmark to evaluate new mobile EEG systems by means of ERP responses.