Irregular geographic spatio-temporal field data have been rapidly accumulating with the developing collection techniques; However, their data organization and operation are yet conducted in a segregated manner, leading to systematical drawbacks such as interface expansion difficulty and high coupling code in GIS implementation. The paper proposes a unified model for data organization and operation to fill such a gap. The proposed model has two main parts. The first part is called the conceptual model, where we introduce the concept of primitive elements, which are formally sets of data points, to serve as the smallest building blocks in data organization. Regarding three prevalent data irregularity types, we define their corresponding primitive elements to lay the foundation for unified data organization. The logical model adopts a layered architecture, separating data entities, computational methods, and visual presentation. The layered architecture segregates data operations from data types, allowing unification and consistency of data operations. For demonstrations, we conduct case studies, including sparse data interpolation, weak climate signal extraction, and local structure analysis to show the effectiveness of our model. In addition, we propose the ``plug & play'' template to allow convenient extensions to irregularity types and corresponding data operators.