Modeling and closed loop control for resource-constrained load balancing with time delays in parallel computations

Load balancing for parallel computations is modeled as a deterministic dynamic nonlinear time-delay system. This model accounts for the trade-off between using processor time/network bandwidth and the advantage of distributing the load evenly between the nodes to reduce overall processing time. A distributed closed-loop controller is presented to balance load dynamically at each node by using not only the local estimate of the work load of other nodes, but also measurements of the amount of work load in transit. To handle the time varying delays arising in the closed-loop load balancing, a discrete event simulation based on OPNET Modeler is presented and compared with the experiments. Results indicate good agreement between the nonlinear time-delay model and the experiments on a parallel computer network. Moreover, both simulations and experiments show a dramatic increase in performance obtained using the proposed closed loop controller.