General parallel algorithms

Everything related to large computers and large tasks is accompanied by the characteristic word "parallel": parallel computers, parallel computing systems, parallel programming languages, parallel numerical methods, etc. This term came into widespread use almost immediately after the appearance of the first computers. More precisely, almost immediately after realizing the fact that the created computers are not able to solve many tasks in an acceptable time. The way out suggests itself. If one computer does not cope with solving the problem in the right time, then we will try to take two, three, ten computers and force them to work simultaneously on various parts of the overall task, hoping to get the appropriate acceleration. The idea turned out to be fruitful, and in scientific research, a specific number of connected computers quickly turned into an arbitrary and even arbitrarily large number. The integration of computers into a single system has led to many consequences. To provide individual computers with work, it is necessary to divide the initial task into fragments that can be performed independently of each other. This is how special numerical methods began to arise, allowing for the possibility of such a separation. To describe the possibility of simultaneous execution of different task fragments on different computers, special programming languages, special operating systems, etc. were required [1]. For more than half a century, parallel computing has attracted the attention of a wide variety of specialists. Three circumstances maintain a constant interest in them. Firstly, this is a very important area of activity. By doing parallel computing, the researcher realizes that he is doing something related to the biggest tasks, the biggest computers and, therefore, is at the forefront of science. At the very least, proximity to the cutting edge of science is inspiring. Secondly, it is a vast field of activity. It concerns the development of numerical methods, the study of structural properties of algorithms, the creation of new programming languages and much more related to the interface between the user and the computer itself. Parallel computing is closely related to the very process of computer engineering design. The structure of the algorithms suggests the need to make changes to computers that effectively support the implementation of structural features. Engineering innovations stimulate the development of new algorithms that effectively use these innovations. And finally, thirdly. From a formal point of view, the field of activity in question is easily accessible for research. It is enough to get more or less familiar with the subject at the level of popular literature and it is already possible to draw meaningful conclusions, perhaps even unpublished by anyone.