Supercomputers, memory hierarchy and dataflow computation model

Modern supercomputers are hierarchical, and the hierarchy depth tends to grow. Structure hierarchy (core–chip–node–card–cabinet–system) implies significant differences in communication time. Memory hierarchy also induces differences in access time: the larger is the size of the level, the slower is the access. Data movement overhead become the most significant factor of inefficiency, and thus the task of optimizing programs in this respect gets more and more difficult. We claim that these difficulties are largely a consequence of traditional programming paradigm that goes back to von Neumann. And although it has such a strong case as the acquirements and the legacy software, it is still useful at least in theory to understand the alternatives. We believe that the problem of the von Neumann programming model arises due to its exercising the so-called gather paradigm, as opposed to the scatter paradigm inherent to the proposed dataflow computation model, which provides more efficient and easier solution to the data movement optimization problem.