Physics-informed extreme learning machines (PIELMs) have emerged as optimisation-free surrogates to gradient-based physics-informed neural networks, yet their quantitative advantage over classical finite-element methods (FEM) has remained limited to regular-grid domains. This work delivers the first systematic, large-scale head-to-head study between a minimalist single-Rm PIELM and FEM on four elliptic benchmarks of escalating difficulty with random collocation points: (i) a 2-D Poisson verification case, (ii) a 2-D heat-conduction problem with an internal cavity, (iii) a 2-D hyperbolic PDE featuring steep gradients, and (iv) the 3-D extension of (ii). More than 4 × 103 random hyper-parameter realisations are executed, spanning seven decades in training loss and error. For the two smooth 2-D benchmarks the PIELM attains sub-10−8 root-mean-square (RMS) errors while being at least an order of magnitude faster than equally accurate FEM. On the hyperbolic test it retains its speed edge and lowers the FEM error by one order of magnitude. In 3-D the PIELM reaches RMS values within a factor 4 of the FEM optimum yet requires only a fraction of the wall-time, revealing dimensionality limits but preserving favourable accuracy-per-second scaling. Statistical analysis uncovers an almost linear log–log correlation between the collocation residual and the RMS error, enabling a practical a-posteriori stopping criterion. Sensitivity sweeps identify the hidden-layer width M and the weight scale Rm as the dominant hyper-parameters, whereas the interior-to-boundary collocation ratio primarily governs stability. Error visualisations expose complementary failure modes between PIELM (local spikes) and FEM (global diffusion), suggesting the need for more robust collocation-points selection strategies. Overall, the evidence positions the single-Rm PIELM as a compelling alternative for fast, high-accuracy solutions of low- to moderate-dimensional, smooth PDEs, and charts a roadmap for overcoming the remaining challenges in high-dimensional and highly non-smooth regimes.