Several approaches for load balancing in distributed systems were introduced, however, most of them require prior knowledge of the environments operation conditions and/or constant monitoring of these conditions at run time. That allows the applications to adjust the load and redistribute the tasks when necessary. These techniques were designed with the assumption that there is no high communication delay in discovering dynamic load behaviors for the rescheduling purposes. This paper proposes a new delay-tolerant dynamic load balancing technique that can be used effectively for reducing the execution time of some distributed tasks while minimizing the control overhead. Such tasks include downloading large files from replicated FTP servers and executing parallel applications on multiple independent distributed servers. This technique we call DDOps (Dual Direction Operations) allows the parallel/distributed application to make use of available resources efficiently while not requiring any significant control overhead. In our approach, load balancing is automatically inherent from the technique. Since the tasks are handled from opposite directions, processing will continue until the workers meet at some point which indicates all tasks are done. Thus DDOps is most suitable for non-dedicated heterogeneous distributed environments where resources vary in specifications, locations, and operating conditions. The experimental results in file download and parallel computations all show how efficient DDOps is and how well it balances the load among the different tasks.