Within the axiomatic framework of ``computational universe'' U_{comp} = (X,T,C,I), complexity geometry characterizes ``how much time/cost is needed to reach a configuration.'' However, complexity geometry alone is insufficient to describe ``what quality of information is gained for these costs.'' To address this, we construct a ``discrete information geometry'' theory compatible with computational universes within a fully discrete setting. We first introduce observation operator families O = \{O_j\}_{j\in J}, where each O_j maps configuration x\in X to a probability distribution p_x^{(j)} over some finite outcome set. Under fixed tasks or observation schemes, these distributions provide ``visible information states'' for each configuration x. We define task-aware relative entropy structures D_Q(x\Vert y) and derive information distances such as Jensen–Shannon distance d_{JS,Q}(x,y). These distances locally induce discrete Fisher structures: near a reference configuration x_0, the Hessian of second-order relative entropy D_Q(x\Vert x_0) yields a discrete information metric tensor around x_0. We prove that under natural regularity assumptions, discrete information structures converge in appropriate limits to a Riemannian information manifold (S_Q,g_Q) with Fisher-type metric g_Q. Correspondingly, ``information geometry on configuration space'' is realized through mapping \Phi_Q:X\toS_Q sending each configuration x to its visible information state. We further discuss volume growth of information balls B_R^{info}(x_0) and ``information dimension,'' providing general inequalities between information dimension and complexity dimension, characterizing ``limits of information resolution achievable under given complexity budgets.'' Finally, we construct a task-aware information–complexity joint action A_Q whose local Euler–Lagrange equations provide local descriptions of optimal computational trajectories ``maximizing information quality'' under finite time budgets, establishing discrete information geometry foundations for subsequent complete ``time–information–complexity variational principles.''