International Journal of High Speed Computing

In this paper we describe the experience gained working on a non conventional memory model for a LISP system. It seems to us that the results obtained may offers an useful strategy to design a new memory model exhibiting a sort of active behaviour in the computations.

We start describing the current LISP implementations of parallelism, pointing out some of the problems that have to be solved. Special attention is given to the problem of user overloading on parallelizing a program. It is also shown that there are problem exhibiting simultaneously different form of parallelism that are not captured from the current parallel languages extensions. Then we describe experimental testbed for an hashed LISP memory system, and the results obtained.

This paper serves a dual purpose. Firstly it is a specification of a distributed memory parallel reduction machine to evaluate lazy functional languages. Being a Mirandal script, it is also an executable specification. Secondly, it is an example of a large functional program (of the order of 2000 lines) which performs a nontrivial task, simulating a parallel machine.

The program has been written using the “inverse comment convention” [19], where all lines of program text start with a > symbol. In fact, the paper that you are leading is the program, and so you are assured that the text you are reading is the text we have executed.

iWarp is a joint project between Carnegie Mellon University and Intel Corporation to develop a system for high speed signal processing and scientific computing. An iWarp system consists of multiple iWarp cells and employs parallelism on multiple levels. The iWarp cell contains an iWarp component and some local memory; therefore each iWarp cell can execute a different program since programs and data are stored in the local memory of each cell. Furthermore, the iWarp component integrates a communication engine with a powerful computation engine, and these two engines operate independently. The iWarp computation engine is controlled by a wide instruction word, which allows the parallel operation of the major functional units, delivering a peak performance of 20 million integer operations/s plus 20 (10) million single (double) precision floating point operations/s. The iWarp communication engine supports various inter-processor communication styles, ranging from the well-known message-passing style to systolic communication.

This paper provides a brief overview of the software tools that deal with the parallelism in an iWarp system. Together they provide a powerful platform for the development of high-performance parallel programs for iWarp based systems.

The research was supported in part by Defense Advanced Research Projects Agency (DoD) monitored by the Space and Naval Warfare Systems Command under Contract N00039-87-C-0251. The views and conclusions contained in this paper are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency (DoD), Carnegie Mellon University, or Intel Corporation.

The purpose of this talk is twofold: to give an overview of the major parallel processor projects at IBM T.J. Watson research and then to discuss in some detail the VICTOR [1] message-passing parallel machine. Finally, concepts of a possible massively-parallel machine for very high performance are briefly discussed. In the first part of the talk the RP3 [2], the ACE [3], the GFII [4], the M/370, the Hybrid Dataflow [5], and the VLIW [6] projects will be discussed at a level sufficient to explain the key features of each of these projects, but without going into technical details. In summary, the RP3 is a 64-node shared memory machine, the ACE a 8-node shared bus workstation, the GF11 a switch-based SIMD machine with up to 576 nodes, the M/370 a project to study commercial applications of message-passing machines. The Hybrid Data Flow and the VLIW projects intend to explore very fine grained parallelism.

The VICTOR project is a family of transputer based machines which were designed and build at T.J. Watson. It includes V32 and V256 with 32 and 256 nodes, respectively, and a set of 16-node workstations. All of these are presently operational. All machines except V32 are T800 based, with 4 MB of memory/node. V256 incorporates a distributed 16-node fileserver with 10 GB capacity. Special hardware supports multiple-users via spatial partitioning and non-intrusive processor monitoring. The operating environment is based on a mixture of vendor compiler and loader technology and a runtime system developed in the VICTOR group. In particular, this includes a set of routing routines and support for the filesystem. Languages used are OCCAM, C, Pascal and Fortran 77. Presently, the Trollius [7] operating system which has been developed at Cornell University is being ported onto VICTOR.

Applications programs which have been written for VICTOR include presently fractals, ray-tracing, a nuclear physics Monte Carlo code, a computer pipeline model and a neural network code. Under development presently are two codes for VLSI design, dealing with parallel fault simulation and with circuit simulation. The latter one uses a parallel version of waveform relaxation techniques and is expected to be able to handle circuits with up to 10⁶ transistors. Other applications being developed are graphical process monitors, database research, 3D-seismic code and multi-robot simulation. Work in the Mathematical Sciences department is focussed on developing a parallel language with emphasis on dynamic process scheduling.

In the last part of the talk issues of scaling message-passing architectures into the Teraflop performance range are discussed.

Parallel computer architectures and hardware have evolved impressively in the last few years from the architecture and the hardware point of view. Progress on the software side can be best characterized as moderate. The lack of methodologies and software to support parallel programming is profound even on the most advanced parallel machines. Parallel programming is a complex task and the performance of a parallel program can be influenced by many different factors such as coding of parallel constructs and/or restructuring, scheduling schemes and scheduling overhead, synchronization and/or communication cost, program and data partitioning and memory allocation. In this paper we discuss the major aspects of parallel programming. Parallel programming environments are considered in three fundamental phases: parallelism specification, parallelism exploita tion and supporting environments and tools. A parallel programming environment built at the University of Illinois is discussed as a case study. Finally, we address the influence of parallel programming on multiprocessor operating systems and discuss future research directions.

Most AI researchers would, I believe, agree that truly intelligent machines (i.e., machines on a par with humans) will require at least four orders of magnitude more power and memory than are available on any machine today [20, 32]. There is now widespread agreement in the supercomputing community that by the year 2000 all supercomputers (defined as the most powerful machines available at a given time) will be massively parallel [10]. Yet relatively little thought has been given in AI as to how to utilize such machines. With few exceptions, AI’s attention has been limited to workstations, minicomputers and PCs. Today’s massively parallel machines present AI with a golden opportunity to make an impact, especially in the world of commercial applications. The most striking near-term opportunity is in the marriage of research on very large databases with case-based and memory-based AI. Moreover, such applications are steps on a path that can lead eventually to a class of truly intelligent systems.