Verification of Data Claims in the Paper This document explains and verifies every quantitative claim in the manuscript "Computational Operationalization Choices for Spatial Data Quality in Screen-Based Eye Tracking" (accuracy-precision.tex). All computations are collected in a single self-contained script. To run it, install the dependencies and execute from this directory: unzip data.zip pip install -r requirements.txt python verify_paper_claims.py The data files are distributed as data.zip. Unzip it first to create the data/ directory (including the data/microsaccade/ subdirectory). Data Sources All data files are in the data/ subdirectory: File Description Source data/etdqualitizer_sample.tsv ETDQualitizer sample validation data (SMI RED 500, 500 Hz) GitHub (Niehorster et al., 2025) data/etdqualitizer_sample_adjusted.tsv Shifted gaze for the illustrative example (Section 6) Derived from etdqualitizer_sample.tsv data/smi_data.tsv SMI RED raw data with per-sample 3D eye positions (EPOS) EMIP dataset (Bednarik et al., 2020, doi:10.1016/j.scico.2020.102520) data/microsaccade/*.tsv 6 validation recordings at >= 200 Hz for microsaccade analysis (Section 4) GitHub (Niehorster et al., 2025) The script loads the same gaze data referenced in the paper, reproduces each calculation, and compares the result with the number stated in the manuscript. Below we walk through every claim, explain what it means, and show the verification outcome. Section 5.2 — Small-Angle Approximation Paper (line 124): "relative errors for the small-angle approximation grow with eccentricity but stay in the single-digit percentages below 25°" What it means:Three small-angle approximations are commonly used to simplify angular-to-Cartesian conversion: sin(theta) ~ theta, tan(theta) ~ theta, and cos(theta) ~ 1. The paper claims all three stay below 10 % relative error for eccentricities below 25 degrees. Verification: Angle sin ~ theta tan ~ theta cos ~ 1 5 0.13 % 0.25 % 0.38 % 10 0.51 % 1.02 % 1.54 % 15 1.15 % 2.30 % 3.53 % 20 2.06 % 4.09 % 6.42 % 25 3.25 % 6.43 % 10.34 % Result: CONFIRMED. Below 25 degrees, all three approximations stay in the single-digit percentages. At 20 degrees the largest error is cos ~ 1 at 6.4 %. At exactly 25 degrees, cos ~ 1 reaches 10.3 %. Section 5.3 — Viewing-Distance Mismatch Paper (line 158): "a systematic shift in the low double digit percentages is conceivable" What it means:When the assumed viewing distance differs from the actual one, every angle computed from screen coordinates is systematically biased. A 60 cm assumption vs 66 cm actual maps to a ~10 % ratio. Verification: Offset from center Angle (60 cm) Angle (66 cm) Error 50 mm 4.76 4.33 +10.0 % 100 mm 9.46 8.62 +9.8 % 160 mm 14.93 13.63 +9.6 % Result: CONFIRMED. The error is in the low double-digit range (~10 %) and grows with larger viewing-distance mismatches (e.g. 60->70 cm gives ~16 %). Section 5.3 — Precision Doubling from Gaze-Origin Outliers Paper (line 158): "the SMI RED occasionally reports switched gaze origins for individual samples whose screen coordinates remain stable [...] Including these samples roughly doubled the computed precision value" What it means:The paper describes that per-sample 3D eye position (EPOS) data from the SMI RED can contain problematic values. When these are used to convert screen coordinates to angular values (instead of a fixed viewing distance), the outlier EPOS values inject additional variance into the angular data, inflating precision measures. Verification (from smi_data.tsv): The script identifies a concrete EPOS outlier (e.g., at index 705 in the recording, where L EPOS X jumps by +18.4 mm while the screen gaze POR X stays stable), then computes RMS-S2S precision for a 250-sample fixation interval containing that outlier, comparing fixed VD vs per-sample EPOS. Value RMS-S2S (fixed VD) 0.104° RMS-S2S (per-sample EPOS) 0.216° Ratio (EPOS / fixed VD) 2.07x The single switched EPOS sample causes two large consecutive angular jumps in the RMS-S2S computation, inflating precision by a factor of ~2x. Result: CONFIRMED. Precision ratio is 2.07x, consistent with "roughly doubled." Section 5.4 — Constant Conversion Factor Overestimation Paper (line 172): "The relative error is at least that of the cos(theta) ~ 1 approximation ... and can reach slightly more than double that value" What it means:A constant pixel-to-degree factor (arctan(1 mm / VD)) converts Euclidean distances on the screen surface into degrees. At eccentric positions, this overestimates the true angular error. The minimum overestimation (tangential displacement at eccentricity theta) equals the cos(theta) ~ 1 error; the maximum (radial displacement from center) can approach twice that. Verification: Eccentricity cos ~ 1 error Radial error (max) Tangential error (min) 5 0.38 % 0.78 % 0.38 % 10 1.54 % 3.14 % 1.54 % 15 3.53 % 7.22 % 3.53 % 20 6.42 % 13.30 % 6.42 % 25 10.34 % 21.81 % 10.34 % Result: CONFIRMED. The tangential error matches cos ~ 1 (the floor), and the radial error approaches ~2x that value (the ceiling). Section 5.5 — Coordinate-Wise vs Fréchet Median Paper (line 193): "In one specific case this yielded a trueness difference of 36 % (0.215 deg vs 0.158 deg)" What it means:When computing trueness, the centroid can be the coordinate-wise median (applied independently to azimuth and elevation) or the Fréchet median on the sphere (minimizing the sum of geodesic distances). These can give noticeably different results because the coordinate-wise median is not geometrically correct in angular space. Verification (SMI RED 500, left eye, target 2, VD = 60 cm): Paper Computed CW median trueness 0.215 0.215 Fréchet median trueness 0.158 0.158 Percentage difference 36 % 35.7 % Result: CONFIRMED. Absolute values match and the percentage (35.7 % ~ 36 %) is consistent. Section 5.6 — Fick vs Helmholtz Differences Paper (line 220): "the two parameterizations therefore yield different azimuth and elevation values ..., with the difference approximately following the cos(theta) ~ 1 curve" What it means:The same gaze direction, expressed in Fick vs Helmholtz angles, produces different numeric values for azimuth and elevation. The discrepancy grows with eccentricity, roughly tracking the cos(theta) ~ 1 error. Verification: Fick (theta, phi) Helmholtz (theta, phi) Delta theta Delta phi |Delta phi|/phi cos ~ 1 err (10, 5) (9.96, 5.08) +0.04 -0.08 1.5 % 1.5 % (15, 10) (14.77, 10.35) +0.23 -0.35 3.5 % 3.4 % (20, 10) (19.68, 10.63) +0.32 -0.63 6.3 % 6.0 % (25, 15) (24.09, 16.47) +0.91 -1.47 9.8 % 9.4 % The |Delta phi|/phi column shows the relative elevation difference, which closely tracks the theoretical cos(theta) ~ 1 error curve. Result: CONFIRMED. Section 5.6 — Coordinate-Axis Tilt Paper (line 220): "the coordinate-axis tilt is approximately phi * theta (e.g., approx 3.5 deg at 20 deg azimuth and 10 deg elevation)" What it means:In Fick coordinates, lines of constant elevation on the screen are not perfectly horizontal. They tilt by an amount proportional to the product of azimuth and elevation (both in radians). At (20 deg, 10 deg): phi * theta = 0.1745 rad * 0.3491 rad = 0.0609 rad = 3.49 deg Verification: Value Exact tilt 3.45 Approximation 3.49 Paper ~3.5 Result: CONFIRMED. Section 6 — Illustrative Example Paper (lines 255-261):Two pipelines process the same gaze data (485 samples, target at 16 cm right, 9 cm above screen center) with different operationalization choices. Pipeline A 60 cm assumed VD, coordinate-wise median in screen mm, Euclidean distance with a constant deg/mm factor, target-centered decomposition. Pipeline B 66 cm actual VD, Fréchet median on the sphere, geodesic distance, Fick decomposition (no recentering). Results Claim Paper Computed Status Pipeline A total trueness 0.217 0.217 MATCH Pipeline A elevation 0.098 0.098 MATCH Pipeline B total trueness 0.135 0.135 MATCH Pipeline B elevation 0.047 0.047 MATCH A vs B total difference ~61 % 60.7 % MATCH Elevation difference ~110 % 109 % MATCH Median accounts for > half > 50 % 63 % MATCH Factor Decomposition The 61 % overall difference can be decomposed multiplicatively: Factor Contribution Distance metric (Eucl+const vs geodesic) +6.6 % Central tendency (CW vs sphere median) +38.1 % Viewing distance (60 cm vs 66 cm) +9.1 % Product x1.61 The coordinate-wise median is the largest single factor (63 % of the total difference), confirming "more than half." Section 6 — ETDQualitizer Reference Values Paper (line 261): "ETDQualitizer reports a total trueness (what ETDQualitizer calls accuracy) of 0.185 deg and Delta phi = 0.083 deg at the actual viewing distance of 66 cm." What it means:This is a reference comparison showing what ETDQualitizer produces for the same data. Verification: Computation Total Elevation Paper claim 0.185 0.083 ETDQualitizer accuracy (nanmedian) 0.185 0.083 Result: MATCH. Using nanmedian as central tendency, ETDQualitizer reproduces the paper values. Why ETDQualitizer computes trueness (not accuracy in the ISO sense) Paper claims: ETDQualitizer reports trueness (Section 6), despite the function being named accuracy(). ETDQualitizer effectively computes a coordinate-wise central tendency (Section 5.5). Why "trueness":ETDQualitizer's accuracy() function (__init__.py) works as follows: Compute per-sample geodesic distance from each gaze point to the target (always positive). Compute per-sample direction from target to gaze point on the tangent screen. Decompose each offset into signed azimuth and elevation components: offset * [cos(direction), sin(direction)]. Apply the central tendency function (e.g. median) to the signed azimuth offsets and signed elevation offsets independently. Return hypot(ct(azi_offsets), ct(ele_offsets)). The key insight is that the offsets in step 3 are signed: gaze samples to the left of the target produce negative azimuth offsets, samples to the right produce positive ones. Taking the mean or median of signed offsets yields the centroid's displacement from the target — not the average absolute distance. By the ISO 5725-1 definitions used in the paper: Trueness = d(centr(g), t) — distance from centroid to target Accuracy = centr(d(g, t)) — central tendency of per-sample distances mean(signed_offsets) gives the centroid offset (trueness), whereas mean(|offsets|) would give the average distance (accuracy). ETDQualitizer computes the former, so it reports trueness. Why "coordinate-wise":The central tendency function is applied independently to each coordinate direction (step 4): median(azi_offsets) and median(ele_offsets). This is equivalent to computing the coordinate-wise median of the gaze positions on the tangent screen and then measuring the offset from that centroid to the target. It is not a Fréchet median on the sphere (which would minimize the sum of geodesic distances jointly). Section 4 — Microsaccade Analysis Paper (line 107): "microsaccades were infrequent during the selected time intervals and had negligible impact on accuracy and trueness" "their per-sample displacement was small relative to the tracker's noise floor" Method:Microsaccades are detected using the Engbert & Kliegl (2003) algorithm with parameters from Nystrom, Niehorster et al. (2021): Velocity threshold: lambda = 6 (median-based SD) Minimum duration: 5 ms Minimum inter-saccade gap: 10 ms Bartlett (triangular) lowpass filter: 20 ms window 5-point central difference velocity computation Data:Six data files with sampling rates >= 200 Hz from the ETDQualitizer example data (the 120 Hz Tobii file is excluded due to insufficient temporal resolution for reliable microsaccade detection): File Rate Eyes Source EyeLink_1000+_binocular_1000Hz.tsv 1000 Hz Both GitHub EyeLink_1000+_left_1000Hz.tsv 1000 Hz Left GitHub EyeLink_1000+_right_1000Hz.tsv 1000 Hz Right GitHub SMI_RED500_500Hz.tsv 500 Hz Both GitHub Tobii_Spectrum_600Hz.tsv 600 Hz Both GitHub Tobii_Spectrum_1200Hz.tsv 1200 Hz Both GitHub These are the same validation recordings used by ETDQualitizer's example scripts, recorded with a common screen configuration (528 x 297 mm, 1920 x 1080 px, viewing distance 650 mm) using 9 validation targets. Verification: Three sub-claims are verified: Claim 1 — "infrequent" Metric Value Mean microsaccade rate 3.2 Hz Median microsaccade rate 2.0 Hz Samples in microsaccades 3.0 % Only 3 % of all samples fall within detected microsaccade intervals. The rate of 2-3 Hz is consistent with typical microsaccade rates during fixation (1-3 Hz in the literature). Result: CONFIRMED. Microsaccades affect a small fraction of samples. Claim 2 — "negligible impact on accuracy and trueness" Both trueness (centroid offset from target, via ETDQualitizer.accuracy()) and ISO-accuracy (mean per-sample geodesic distance to target) are computed with all samples and again with microsaccade samples masked to NaN: Metric Mean change Median change Max change Trueness 0.6 % 0.1 % 11.8 % ISO-accuracy 0.4 % 0.2 % 2.8 % Result: CONFIRMED. Both mean changes are = cos ~ 1 (5.4) (math) (math) MATCH 5 CW vs Fréchet median 36 % (5.5) 36 % 35.7 % MATCH 6 CW vs Fréchet median values (5.5) 0.215/0.158 0.215/0.158 MATCH 7 Axis tilt ~ 3.5 deg (5.6) 3.5 3.45 MATCH 8 Pipeline A total (6) 0.217 0.217 MATCH 9 Pipeline A elevation (6) 0.098 0.098 MATCH 10 Pipeline B total (6) 0.135 0.135 MATCH 11 Pipeline B elevation (6) 0.047 0.047 MATCH 12 A vs B difference (6) ~61 % 60.7 % MATCH 13 Median > half (6) > 50 % 63 % MATCH 14 Elevation difference (6) ~110 % 109 % MATCH 15 ETDQualitizer total, median (6) 0.185 0.185 MATCH 16 ETDQualitizer elevation, median (6) 0.083 0.083 MATCH 17 Microsaccades infrequent (4) infrequent 3.0 % of samples CONFIRMED 18 Microsaccades negligible trueness impact (4) negligible 0.6 % change CONFIRMED 19 Microsaccades negligible accuracy impact (4) negligible 0.4 % change CONFIRMED 20 Microsaccade displacement vs noise, precision (4) small 0.84x RMS-S2S CONFIRMED