To achieve behavior and structure optimization for a type of software program whose data exchange processes are represented by nonlinear polynomial systems, this paper establishes a novel formal description called a nonlinear polynomial transition system to represent the behavior and structure of the software program. Then, the notion of bisimulation for software programs is proposed based on the equivalence relation of corresponding nonlinear polynomial systems in their nonlinear polynomial transition systems. However, the exact equivalence is too strict in application. To enhance the flexibility of the relation among the different software systems, the notion of approximate bisimulation within a controllable error range and the calculation algorithm of approximate bisimulation based on symbolic-numeric computation are given. In this calculation, an approximate relation is represented as a MAX function that is resolved with the full filled method. At the same time, the actual error is calculable. An example on a multithreading program indicates that the approximate bisimulation relation is feasible and effective in behavior and structure optimization.