Network Working Group A. Bogdanov Request for Comments: 3018 NKO "ORS" Category: Experimental December 2000 Unified Memory Space Protocol Specification Status of this Memo This memo defines an Experimental Protocol for the Internet community. It does not specify an Internet standard of any kind. Discussion and suggestions for improvement are requested. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (2000). All Rights Reserved. Abstract This document specifies Unified Memory Space Protocol (UMSP), which gives a capability of immediate access to memory of the remote nodes. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. The following syntax specification uses the augmented Backus-Naur Form (ABNF) as described in RFC-2234 [3]. Table of Contents 1. Introduction...................................................4 2. The UMSP Model.................................................5 2.1 128-bit Address Space.......................................5 2.2 Computing Model.............................................7 2.3 System Architecture.........................................9 3. Instruction Format............................................11 3.1 Instruction Header.........................................12 3.2 Extension Headers..........................................15 3.3 Instruction Operands.......................................17 3.4 Address Formats............................................17 4. Response of the Instructions..................................19 4.1 RSP, RSP_P.................................................20 4.2 SND_CANCEL.................................................20 5. Jobs Management...............................................21 Bogdanov Experimental [Page 1] RFC 3018 Unified Memory Space Protocol December 2000 5.1 Job Initiate...............................................23 5.1.1 CONTROL_REQ............................................24 5.1.2 CONTROL_CONFIRM........................................25 5.1.3 CONTROL_REJECT.........................................26 5.2 Task Initiate..............................................26 5.2.1 TASK_REG...............................................26 5.2.2 TASK_CONFIRM...........................................27 5.2.3 TASK_REJECT............................................28 5.2.4 TASK_CHK...............................................28 5.3 Establishment of session connection........................29 5.3.1 SESSION_OPEN...........................................29 5.3.2 SESSION_ACCEPT.........................................31 5.3.3 SESSION_REJECT.........................................31 5.3.4 Connection Profile.....................................32 5.4 Session Closing............................................33 5.4.1 SESSION_CLOSE..........................................34 5.4.2 SESSION_ABEND..........................................35 5.5 Task Termination...........................................35 5.5.1 TASK_TERMINATE.........................................36 5.5.2 TASK_TERMINATE_INFO....................................36 5.6 Job Completion.............................................37 5.6.1 JOB_COMPLETED..........................................37 5.6.2 JOB_COMPLETED_INFO.....................................38 5.7 Activity Control of Nodes..................................38 5.7.1 _INACTION_TIME.........................................39 5.7.2 STATE_REQ..............................................40 5.7.3 TASK_STATE.............................................41 5.7.4 NODE_RELOAD............................................42 5.8 Work without session connection............................42 6. Instructions of Exchange between VM...........................44 6.1 Data Reading/Writing Instructions..........................45 6.1.1 REQ_DATA...............................................45 6.1.2 DATA...................................................46 6.1.3 WRITE..................................................46 6.1.4 WRITE_EXT..............................................47 6.2 Comparison Instructions....................................47 6.2.1 CMP....................................................47 6.2.2 CMP_EXT................................................48 6.2.3 Response to Comparison Instructions....................48 6.3 Control Transfer Instructions..............................48 6.3.1 JUMP, CALL.............................................48 6.3.2 RETURN.................................................49 6.4 Memory Control Instructions................................50 6.4.1 MEM_ALLOC..............................................50 6.4.2 MVCODE.................................................50 6.4.3 ADDRESS................................................51 6.4.4 FREE...................................................51 6.4.5 MVRUN..................................................51 Bogdanov Experimental [Page 2] RFC 3018 Unified Memory Space Protocol December 2000 6.5 Other Instructions.........................................52 6.5.1 SYN....................................................52 6.5.2 NOP....................................................53 6.6 Work with Objects..........................................53 6.6.1 Reading/Writing of the Objects Data....................54 6.6.1.1 OBJ_REQ_DATA.......................................54 6.6.1.2 OBJ_WRITE..........................................55 6.6.1.3 OBJ_WRITE_EXT......................................56 6.6.2 Comparison Instructions of the Objects Data............56 6.6.2.1 OBJ_DATA_CMP.......................................56 6.6.2.2 OBJ_DATA_CMP_EXT...................................57 6.6.3 Execution of the Objects Procedures....................57 6.6.3.1 CALL_BNUM..........................................57 6.6.3.2 CALL_BNAME.........................................58 6.6.3.3 GET_NUM_PROC.......................................59 6.6.3.4 PROC_NUM...........................................59 6.6.4 The Objects Creation...................................59 6.6.4.1 NEW, SYS_NEW.......................................60 6.6.4.2 OBJECT.............................................61 6.6.4.3 DELETE.............................................61 6.6.5 The Objects Identification.............................61 6.6.5.1 OBJ_SEEK...........................................62 6.6.5.2 OBJ_GET_NAME.......................................62 7. Chains........................................................62 7.1 Sequence...................................................63 7.2 Transaction................................................64 7.2.1 _BEGIN_TR..............................................64 7.2.2 EXEC_TR................................................65 7.2.3 CANCEL_TR..............................................66 7.3 Fragmented instruction.....................................66 7.4 Buffering..................................................67 7.5 Acknowledgement of chains..................................69 7.6 Base-displacement Addressing...............................70 8. Extension Headers.............................................71 8.1 _ALIGNMENT.................................................71 8.2 _MSG.......................................................71 8.3 _NAME......................................................72 8.4 _DATA......................................................72 8.5 _LIFE_TIME.................................................72 9. Search of resources...........................................73 9.1 VM_REQ.....................................................75 9.2 VM_NOTIF...................................................75 10. Security Consideration.......................................77 11. Used Abbreviations...........................................78 12. References...................................................79 13. Author's Address.............................................80 14. Full Copyright Statement.....................................81 Bogdanov Experimental [Page 3] RFC 3018 Unified Memory Space Protocol December 2000 1 Introduction UMSP is the network connection-oriented protocol. It corresponds to session and presentation layers of model OSI. The protocol is designed for implementation in a wide class of systems, from simple devices based on the dedicated processors, up to universal computers and clusters. For the data exchange, the protocol uses transport layer service with reliable delivery. It is possible to use not providing reliable delivery protocol for the transmission of not requiring acknowledgement data. This document describes use TCP and UDP. The creation of network environment for the organization 128-bit address space of memory distributed between Internet nodes is the basic purpose of the protocol UMSP. The protocol defines algorithm of the connections management and format of network primitives. It doesn't control local memory on the node. As against the traditional network protocols, the user applications on different nodes interact not by the network primitives exchanging or working with the dataflows, but by immediate data reading/write or control transfers to the code in virtual memory of the remote node. The user's application can know nothing about existence of the protocol and network, and simply use the instructions with 128-bit addresses. Firstly, it is supposed to use UMSP in systems based on the virtual machines (VM), executing the pseudo-code. However, the protocol may be used in systems executing a processor code, for example, in clusters or in universal operational systems, for the organization of the distributed virtual address space. Besides, the minimal profile of the protocol may be used in simple devices, which do not have the operational system. The protocol gives various means for set the connection parameters and allows building systems with a high protection level without restriction applications functionalities. UMSP can essentially simplify the distributed systems development process. It gives an opportunity to unite not only information, but also calculating resources of the large number of polytypic computers without significant expenses for the programs standardization and development. Bogdanov Experimental [Page 4] RFC 3018 Unified Memory Space Protocol December 2000 2 The UMSP Model 2.1 128-bit Address Space UMSP is based on the 128-bit distributed address memory space model. The 128-bit address contains the information about the network type, network node address and local memory address. It has the following format: Octets 0 1 16 +------+--------------+--------------------+----------------+ |Header| FREE | NODE_ADDR | MEM_ADDR | +------+--------------+--------------------+----------------+ Complete address length is fixed and is equal to 16 octets. Header 1 octet. Address header field completely defines the address format. The header has the following format: Bits 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | ADDR_LENGTH | NET_TYPE | ADDR_CODE | +-----+-----+-----+-----+-----+-----+-----+-----+ ADDR_LENGTH 4 bits. The length of the network address. This field contains the number of octets in the NODE_ADDR field. The value 0 is not allowed. NET_TYPE 2 bits. The network type. This field specifies a type of network, in which the node is. ADDR_CODE 2 bits. The length code of the local memory address. The value of this field specifies the length of the local memory address. The following values of the field and appropriated to them length of the field MEM_ADDR are defined: Bogdanov Experimental [Page 5] RFC 3018 Unified Memory Space Protocol December 2000 %b00 - 16 bit %b01 - 24 bit %b10 - 32 bit %b11 - 64 bit The values combination of the three fields of heading is named address format number. These fields unequivocally define a network, in which the node is located. Format number writes as follows: N - - For example, N 4-0-2 defines the address with length of the node network address 4 octets and memory address with the length 32 bits. The network type 0 for such address format is defined for the network IPv4 in the presented document. If the network type is equal to zero, it may be missed during the writing of the address format number. For example, format N 4-0-2 and 4-2 are equivalent. If both fields NET_TYPE and ADDR_CODE are set to zero, they may be omitted. Thus, a format number writes as one figure. One or several address format numbers must be assigned for each global network, included in unified system. FREE 0 - 12 octets. This field is unused by the protocol. It may contain any additional information, which is necessary for the control system of the node memory. If this field is not used, the zero value must be set in all octets. Using of this field results that the network instructions must contain only complete 16 - octet address and the short address of local memory cannot be used. NODE_ADDR 1 - 13 octets. The node address. The format of this field is defined separately for each address format number. The field of the node address should not necessary precisely correspond to the real network address. If the real network address is longer than this field, it is necessary to organize in the network a subset of supporting the protocol UMSP addresses. Bogdanov Experimental [Page 6] RFC 3018 Unified Memory Space Protocol December 2000 MEM_ADDR 16/24/32/64 bits. The address of local memory. This field is the memory address in system, which is set by a field NODE_ADDR. The node completely responds for its memory control. The protocol does not define the order of using and format of this field. 128-bit address for the user applications is one field. The user code cannot know about a physical arrangement of addressed memory. The 128-bit memory address may be transmits between nodes, as the data, for example, in the buffer of function parameters, or in the instruction of copying the data. Therefore, it must identify the given node from any other nodes unequivocal. Any certain algorithm, connecting real network and 128-bit address, does not exist. All used address formats must be known beforehand. As UMSP has its own address space, it can unite several global networks. The nodes can have internal local networks or subordinated addressable devices connected with the node by the not-network communications. Any node by address format number must have an opportunity to define the gateway respond for routing of this address. 2.2 Computing Model Computing model is three-layer: (1) Job (2) Task (3) Thread of control The job corresponds to the user application. The job is distributed and can simultaneously be executed on many nodes. The job control is carried out centralize, from the node named as Job Control Point (JCP). One JCP can control the some jobs. JCP can be located on the same node, on which the job is created, or on any other addressed net point. The task is the job presentation on the separate node. The task includes one or several computing threads of control. The job has only one task on each node. The job is finished, when the appropriate user application is finished. At the end of the job all tasks of this job on all nodes are finished. Bogdanov Experimental [Page 7] RFC 3018 Unified Memory Space Protocol December 2000 The job has its isolated 128-bit address space. The address space is segmented. A segment is the local memory of one node. Besides, the protocol allows working with objects. The objects are separate associative memory of the node. The task thread represents the concrete control thread, which are executed by VM in the certain node. The thread can read and write to any address of 128-bit address space of the job. The control transfer to the address from other (remote) node, results to the creation of the new thread on the remote node. The continuous code segment cannot be distributed on several nodes. In addition, it is impossible to receive continuous memory area distributed on several nodes. The protocol does not demand to support the different tasks of not- crossed memory space from the separate VM node. The supporting of multi-thread is not also the obligatory requirement. The 128-bit Global Job Identifier (GJID) is defined by protocol. It is assigned on JCP, which will control the job. All active GJID have the unique values in the unified system at each moment of time. The job can contain VM code of different types. Different types VM can be situated on one or different nodes. The mechanism of association of different VM types in groups on one node is stipulated, so to the non-uniform code can be executed on one node in a context of one job. The groups are described in details in section 9. VM, incorporated in groups, must work in common memory space (to have a common subsystem of memory control). Bogdanov Experimental [Page 8] RFC 3018 Unified Memory Space Protocol December 2000 2.3 System Architecture System structure, based on using Virtual Machines, is given in the following figure: Node 1 Node 2 -------- -------- +--------------------+ +--------------------+ | User Application 1 | | User Application 1 | +-----------------------+ +-----------------------+ | User Application N | | User Application N | +--------------------+ +--------------------+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ | VM1 | | VM2 | . . . | VMn | | VM1 | | VM2 | . . . | VMn | +--+--+ +--+--+ +--+--+ +--+--+ +--+--+ +--+--+ | | | | | | +--------------------------+ +--------------------------+ | | | | | +-----+ U M S P | | U M S P | | | JCP | | | | | +-----+ | +-------------+------------+ +-------------+------------+ | | +-----+-----+ +-----+-----+ | TCP | | TCP | +-----+-----+ +-----+-----+ | | | +-----------------/ | /------------------+ / | +-----+-----+ Node N | TCP | -------- +-----+-----+ | +------------+------------+ | +-----+ | | | JCP | U M S P | | +-----+ | +-------------------------+ Figure 1. Structure of the system based on use VM. Bogdanov Experimental [Page 9] RFC 3018 Unified Memory Space Protocol December 2000 One or several VM are working on upper level for UMSP. The VM layer is not network level. Last network level is UMSP. Therefore, VM layer has no its own network primitives and uses together with UMSP the same field of operation code. The end services user of the protocol is the user code, which is executed by the virtual machine. It has the instructions with the 128-bit address. VM translates these instructions to network commands, which are transmitted through the UMSP protocol for the executing by the remote machine. Internal organization VM, command system and API can be anyone. The protocol defines only format of primitives, which the virtual machines exchange through a network. The protocol does not control the jobs memory. Control of memory should realize VM. If a few VM works on one node, they may have the common memory space or may be completely isolated. UMSP uses the transport layer with reliable delivery for the data exchange. This document defines of using TCP. For the transfer of not requiring acknowledgement data may be used UDP. Thus, the connection through TCP is obligatory. Use of multiple connections TCP with multiplexing is supposed. The control of transport connections is not the part of the UMSP protocol. The UMSP instructions do not contain network addresses of the receiver and sender. The protocol requires that one address UMSP must correspond to the one transport layer address. Accordingly, it is necessary to define unequivocal the node address on transport layer by the 128-bit address of memory. Except the TCP, it is possible to use other transport protocols or not network communications. The following requirements are showed to them: o Reliable delivery. The transport layer must inform about delivery or its impossibility; o The violation of a sequence of transmitted segments is allowed; o The duplication of segments is not allowed; o At emergency reload of nodes it is necessary to guarantee identification of segments concerning session connections, assigned up to reload; o Use connectionless-mode is possible. VM is the independent program and the interaction with the protocol is necessary for it only when it executes the instructions with the 128-bit address, concerning to other node. VM can execute several Bogdanov Experimental [Page 10] RFC 3018 Unified Memory Space Protocol December 2000 user tasks. Each task can contain several threads of control. VM must be able to interpret the application instructions with the 128- bit address to one or several instructions of the UMSP protocol. The session connection opens between nodes for the data exchange. One connection is relational only with one job. There may be several session connections for the different jobs simultaneously between two nodes. Besides, the protocol provides the connectionless data exchange. The exchange between UMSP nodes can include the instructions of the following type: o Immediate reading/write in memory; o Requests of allocation/free memory; o Comparison instructions; o Call-subroutine and unconditional jump instructions; o Synchronization instructions; o Work with objects instructions - reading / writing in memory of objects and execution of objects procedures. UMSP does not trace the user control threads. VM must provide itself the necessary order of performance of the instructions. The length of UMSP instructions does not depend on segment length of the transport layer. The segmentation is provided for transfer of the long instructions. The packing of the short instructions in one segment with a possibility of compression of headings is used for its transfer. The minimal size of necessary for work segment is 6 octets. For realization of all functions, it is necessary 54 octets. 3 Instruction Format The UMSP instruction includes the basic header, extension headers and operands. All fields have variable length. +----------------+----------------------+------------------------+ | Header | Extension headers | Operands | +----------------+----------------------+------------------------+ The header contains operation code and the information necessary for the instruction interpretation. The optional extension headers contain the additional information, not defined in basic header. The operands contain instructions data. Bogdanov Experimental [Page 11] RFC 3018 Unified Memory Space Protocol December 2000 The instruction format allows calculating common instruction length, without knowing definition of separate operation code. The instructions headers provide for the short and extended format for maintenance of the effective protocol work in wide range of network speeds. Besides, there is a simple algorithm of the headers compression. The all instructions and extension headers the identifiers are given which enter the name by upper case symbols. The identifiers of the instructions begin with the letter. The identifiers of the extension headers begin with underlining symbol. 3.1 Instruction Header The header has the following format: Octets: +0 +1 +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 0: | OPCODE |ASK| PCK |CHN|EXT| OPR_LENGTH| +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 2: | OPR_LENGTH_EXT | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 4: | CHAIN_NUMBER | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 6: | INSTR_NUMBER | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 8: | | + SESSION_ID + | | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 12:| | + REQ_ID + | | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ OPCODE 1 octet. The operation code. Value of this field is identified by the instruction. Values of operation codes are divided into the following intervals: 1 - 112 management instructions 113 - 127 reserved 128 - 223 instructions of exchange between VM 0, 224, 255 reserved Bogdanov Experimental [Page 12] RFC 3018 Unified Memory Space Protocol December 2000 ASK 1 bit. The flag of response necessity. This flag defines presence of field REQ_ID in header. If ASK = 1, there is field REQ_ID in the instruction. If EXT = 0, the field REQ_ID in the instruction are absent. PCK 2 bits. The Header compression attribute. These bits are used for packing instructions headers transmitted on one connection TCP or for sending of the several instructions in one package UDP. Use of these bits is based on the assumption that two following in succession instructions concern to one session connection, or one chain, with a high probability. The PCK bits have one of the following values: %b00 - The instruction does not belong to the definite session. The fields CHAIN_NUMBER, INSTR_NUMBER and SESSION_ID are absent in header of such instruction. %b01 - The given instruction concerns to the same session connection, as previous. The field SESSION_ID in the instruction header is absent. %b10 - The given instruction belongs to the same connection and same chain, as previous. The fields CHAIN_NUMBER, INSTR_NUMBER and SESSION_ID in header of such instruction are absent. The INSTR_NUMBER value of the current instruction calculates by addition of one to INSTR_NUMBER value of the previous instruction. %b11 - The given instruction may does not concern to the same session, as previous. The field SESSION_ID is present at it. The presence of fields CHAIN_NUMBER and INSTR_NUMBER is defined by CHN flag. CHN 1 bit. The flag of chain. Transmitted on one session connection and concerning one job instructions, may be unified in a chain. Chains are considered in details by section 7. If SEQ = 1, the instruction is connected with chain and there are fields CHAIN_NUMBER and INSTR_NUMBER (if PCK is not set to %b10) at it. If bit CHN = 0, the instruction is not connected with chains and there are no fields CHAIN_NUMBER and INSTR_NUMBER in it. Bogdanov Experimental [Page 13] RFC 3018 Unified Memory Space Protocol December 2000 EXT 1 bit. The flag of extension headers presence in the instruction. If EXT = 1, there is one or more extension headers in the instruction. If EXT = 0, the extension headers in the instruction are absent. OPR_LENGTH 3 bits. The number of 32 bit words in the operands field. The value 0 defines absence of operands field. The value %b111 specifies use of the extended header format. In the extended format, the length of operands is defined by the field OPR_LENGTH_EXT, and the field OPR_LENGTH is not used. OPR_LENGTH_EXT 2 octets. The number of 32 bit words in the operands field. The field OPR_LENGTH_EXT is present in header, only if OPR_LENGTH = %b111. If OPR_LENGTH < > %b111, the field OPR_LENGTH_EXT is absent. If OPR_LENGTH_EXT = 0, the field of operands is absent. There are following reasons, on which it is necessary to use field OPR_LENGTH_EXT instead of OPR_LENGTH: (1) If operands length must be more than 24 octets (2) If making the fields alignment of 4 octets is more effective, than compression of header of 2 octets. CHAIN_NUMBER 2 octets. The number of chain. This field contains number of chain, to which the given instruction concerns. The values %x0000 and %xFFFF are reserved. INSTR_NUMBER 2 octets. The instruction number. This field contains the serial number of instruction in a chain. The numbering begins with zero. Value %xFFFF is reserved. SESSION_ID 4 octets. It is the identifier of the session connection assigned by the instruction receiver. During the session connection opening, each side sets its own identifier to connection and informs it to other side. The zero value of this field specifies that the instruction does not concern to the definite session. The value %xFFFFFFFF is reserved. Bogdanov Experimental [Page 14] RFC 3018 Unified Memory Space Protocol December 2000 REQ_ID 4 octets. The request identifier. It is uses for establishment of correspondence between requests and responds to it. Further, the identifier OPR_LENGTH is used at the description of the instructions format. It means using of OPR_LENGTH_EXT field, if OPR_LENGTH = %b111. The instruction with length of operands, which are not exceeding 24 octets, may be transmitted with header in the short format (OPR_LENGTH < > %b111) or in the extended format (OPR_LENGTH = %b111). Both forms are equivalent. Minimal header length in the short format is 2 octets, in the extended format - 4 octets. Maximal header length is 16 octets. 3.2 Extension Headers If the EXT flag in the instruction header set to 1, the instruction contains from one up to thirty extension headers. The extension headers are used for the following purposes: o For sending of the service information which were not provided in the basic header. o For sending of the data of length more than 262240 octets in one instruction. The extension headers have the following common format: Octets: +0 +1 +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 0: |HXT| HEAD_LENGTH | HEAD_LENGTH_EXT | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 2: | continued HEAD_LENGTH_EXT | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 4: |HSL|HOB|HRZ| HEAD_CODE | HEAD_CODE_EXT | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 6: | RESERVED | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 8: | | / DATA / / / | | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ Bogdanov Experimental [Page 15] RFC 3018 Unified Memory Space Protocol December 2000 HXT 1 bit. Specify length of the field of data length. If HXT = 0, length of the extension header is defined by a field HEAD_LENGTH. The field HEAD_LENGTH_EXT in this case is absent. If HXT = 1, length of header is defined by unification of fields HEAD_LENGTH and HEAD_LENGTH_EXT. HEAD_LENGTH 7 bit. The number of 16 bit words in DATA field. If HXT = 0, this is independent field. If HXT = 1, it is the senior bits of complete length field. HEAD_LENGTH_EXT 3 octets. The number of 16 bit words in DATA field. If HXT = 0, this field is absent. If HXT = 1, it is the younger bits of complete length field. HSL 1 bit. The flag of last header. It is set to 1 for last extension header in the instruction. In other extension headers, this flag is set to 0. HOB 1 bit. The flag of obligatory processing. It defines the order of the instruction processing, if the receiving node does not know purpose of the extension header or cannot process it by any reason. If HOB = 1, instruction must not be carried out. If HOB = 0, it does not influence on the instruction processing. The protocol must process all extension headers, irrespective of errors presence. HRZ 1 bit. The field is reserved for the future expansions. This field must not be analyzed by the protocol on receiving. It must be set to 0 at sending. HEAD_CODE 5 bits. If HXT = 0, the field contains the extension header code. If HXT = 1, this field joins the field HEAD_CODE_EXT. It is the senior bits of the header code. Bogdanov Experimental [Page 16] RFC 3018 Unified Memory Space Protocol December 2000 HEAD_CODE_EXT 1 octet. If HXT = 0, this field is absent. If HXT = 1, it is the younger bits of the header code. RESERVED 2 octets. If HXT = 0, this field is absent. If HXT = 1, this field is reserved for further use. The field RESERVED must not be analyzed by the protocol during the receiving in the current realization of the protocol. It must be set to 0 at sending. DATA The data field of the extension header. If HXT = 0, the length of field is 0 - 254 octets, if HXT = 1, the length is 0 - 4 * 10^9 octets. The format of this field is defined separately for each value of the header code. On the receiving side, the extension headers must be processed in that order, in what they follow in the instruction. If the instruction contains more than 30 extension headers, it is considered erroneous. It is necessary to break off the session connection, on which it was transmitted, after the reception of such instruction. The identifiers HEAD_LENGTH and HEAD_CODE are used further in the text at the description of the extended headers format. It assumes using of fields HEAD_LENGTH + HEAD_LENGTH_EXT and HEAD_CODE + HEAD_CODE_EXT, if HXT = 1. The headers with the code 0 - 30 can be sent in short (HXT = 0) and in extended (HXT = 1) format. 3.3 Instruction Operands The operands field contains the instruction data. The length of operands field is showed in OPR_LENGTH or OPR_LENGTH_EXT and it is multiple to four octets. If necessary, 1 - 3 zero-value octets are padded in the end of a field. Maximal length of operands is 262140 octets. The extension headers are used, if the instruction must contain longer data. The format of the operands field is defined separately for each instruction. 3.4 Address Formats The following address format numbers are definite for nodes, immediately connected to the global IPv4 network: Bogdanov Experimental [Page 17] RFC 3018 Unified Memory Space Protocol December 2000 N 4-0-0 (4) N 4-0-1 (4-1) N 4-0-2 (4-2) The appropriate formats of 128-bit addresses: Octets: +0 +1 +2 +3 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0: |0 1 0 0|0 0|0 0| Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4: | Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 8: | Free | IP address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 12:| IP address | Local memory address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0: |0 1 0 0|0 0|0 1| Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4: | Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 8: | Free | IP address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 12:| IP address | Local memory address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0: |0 1 0 0|0 0|1 0| Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4: | Free | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 8: | IP address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 12:| Local memory address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Free It is not used by the protocol. IP address It sets the node address in the global IPv4 network. Bogdanov Experimental [Page 18] RFC 3018 Unified Memory Space Protocol December 2000 Local memory address It is described in section 2.1. IP-address defines the nodes of the given type unequivocally. The TCP is used for the interaction with such nodes. For sending of not requiring response instructions, using UDP is allowed. IANA has assigned ports TCP and UDP 2110. This port must be open for the listening (receiving). TCP node, initialing the connection opening, or the UDP node, carrying out the package sending, can use any port. Using several TCP connections with multiplexing is supposed. 4 Response of the Instructions The protocol instructions are divided into two types: (1) The management instructions transmitted on UMSP layer (OPCODE = 1 - 112). (2) The instructions of the exchange between VM (OPCODE = 128 - 223). The processing of two types of the instructions differs as follows: o The field of the identifier of request REQ_ID is formed by the protocol in the instructions of the first type, and it is formed by VM for the instructions of the second type. o The protocol must analyze the field REQ_ID and compare it with the instructions, transmitted earlier, after receiving of the response instruction of the first type. o The protocol must not analyze the field REQ_ID after receiving of the response instruction of the second type. This instruction is simply sent to VM. The response instructions have the field ASK equal to 1. It means, that the header have the field REQ_ID. The value taken from the confirmed instruction is written into the field REQ_ID. The response instruction does not require response. A few VM can be connected to the protocol on the node. Everyone VM can work in its own address space. The identifiers of requests for different VM can coincide. Therefore, instruction is identified by two fields: o The session identifier SESSION_ID, which is connected with definite VM. o The request identifier REQ_ID. Bogdanov Experimental [Page 19] RFC 3018 Unified Memory Space Protocol December 2000 4.1 RSP, RSP_P "Response" (RSP) and "Response of the protocol" (RSP_P) instructions have the identical format. The difference is only in the operation code: OPCODE = 129/1 ; correspondingly to RSP/RSP_P ASK = 1 PCK = %b01/11 EXT = 0/1 CHN = 0 OPR_LENGTH = 0/1 SESSION_ID and REQ_ID - The values is taken from the confirmed instruction. Operands: 2 octets: The basic return code. 2 octets: The additional return code. The optional extension header: _MSG - contains the arbitrary error description. The instruction without operands is used for the positive response. It is equivalent to zero values of the field of the basic and additional return codes. The zero basic return code is used for positive response. The additional return code may have non-zero value. The instruction with non-zero basic return code is used for negative response. The basic return code defines the error category. The additional return code identifies an error. The instruction RSP is formed upon the VM request. The return codes must be received from VM. If the protocol cannot deliver the requiring response instruction to VM, it forms negative response RSP independently. The instruction RSP_P is always formed at the UMSP layer. If the protocol cannot define on what instruction the RSP_P is transmitted, nothing actions is executed. 4.2 SND_CANCEL There can be a necessity to cancel sending after the part of the data have been already transmitted and have occupied the buffer on the reception side, by sending of the long fragmented instructions or transactions. The protocol provides the instruction "The sending is canceled" (SND_CANCEL) for this purpose. This instruction has the following fields value: Bogdanov Experimental [Page 20] RFC 3018 Unified Memory Space Protocol December 2000 OPCODE = 2 ASK = 0 PCK = %b01/10/11 EXT = 0/1 CHN = 1 OPR_LENGTH = 1 SESSION_ID - The value is taken from the cancelled chain. CHAIN_NUMBER - Number of the chain, which sending is cancelled. INSTR_NUMBER - Always has zero-value. Operands: 2 octets: The basic return code. 2 octets: The additional return code. The optional extension header: _MSG - contains the arbitrary error description. The instruction SND_CANCEL is used for the cancel of the partially transmitted transaction or fragmented instruction. At the receiving the SND_CANCEL instruction, all the earlier received data in the chain are rejected. 5 Jobs Management The jobs management includes the following functions: o Initiation and completion of jobs; o Initiation and completion of tasks; o Opening and closing of session connections; o Activity control of nodes. The instructions with OPCODE = 1 - 112 are used for jobs management. These instructions must be sent through TCP. Use UDP is not allowed, even if the instructions do not demand response. UMSP bases on model with the centralized control of the separate job. The reason is that the pointers control is not obviously possible in the decentralized system. Any task can be finished at any moment or the node can be reloaded. There is no way guaranteeing the notification about in the decentralized system all other nodes, on which the job works. As the job continues to exist - the task concerning the job can be initiated on the same node again. This task can allocate new dynamic resources. The addresses for the again allocated resources can be crossed with addresses of resources, which existed on the node before the task restart. The old pointers can be kept on other nodes. It may be the formally correct pointers, but they will actually specify other objects. The uncontrollable work of the application can be consequence of such situation. Bogdanov Experimental [Page 21] RFC 3018 Unified Memory Space Protocol December 2000 UMSP solves this task as follows: o It allows defining the node, on which the task was completed, precisely. o If the task on the node is finished before end of the job, all nodes, on which the job is executed, are notified of it. o The repeated task initialization on the node is allowed, while all nodes will receive the message about the first task end. The protocol does not control the pointers. VM supervises the pointers correctness. VM must have architecture, in which 128 - bit pointers are stored in special memory areas, for this purpose. The protocol informs VM about the nodes, on which task have finished the work. VM must make all pointers concerning such tasks, invalid. It results in exclusive situations at the access under these pointers. If the application provides processing exceptions, it keeps the capacity for work, or it is finished emergency. Such decision allows excluding unguided applications working. For the decision of the specified questions at UMSP level, the control job node is defined for each job. It names Job Control Point (JCP). It may be the same node, on which the job is initiated, or it can be another dedicated node. The basic JCP function is to trace the initialization and the end of the job tasks. Besides, the dedicated JCP node may be used for the centralized users identification and the attack protection. The following identifiers are definite for the jobs and tasks control: o Locally Task Identifier (LTID) is assigned to each active task on the node. LTID length is equal to the length of local memory address defined for the node. All LTID on the node must give unique values at each moment of time. It is allowed to establish LTID, used earlier in the already completed tasks, for the again initiated tasks. o JCP assigned the Control Task Identifier (CTID) to each task of the job. Its length is equal to length of the local address memory on the node JCP. All CTID on the JCP must give unique values at each moment of time. As against LTID, the CTID value is chosen with some restrictions. o Globally Task Identifier (GTID) is assigned to each task. GTID has the same format, as the 128 - bit address of node memory has. The address of local memory is replaced on LTID in it. o Globally Job Identifier (GJID) is assigned to the each job. GJID is defined on the JCP node. It has the same format, as the 128 - bit address of node JCP memory has. The address of local memory Bogdanov Experimental [Page 22] RFC 3018 Unified Memory Space Protocol December 2000 is replaced on CTID of the first (initial) task of the job in it. GJID is used in the procedure of session connection opening for the definition JCP, which controls the job. LTID and CTID are written at the instructions in the field of length 2/4/8 octets. If the allocated for identifier field in the instruction is longer than identifier, LTID (CTID) writes in the last octets. In the initial octets, the value 0 must be written. If received LTID (CTID) is shorter than the local memory address, it is necessary to pad it with the zero octets in the beginning. GTID and GJID are written at the instructions in the field of length 4-16 octets. The field FREE is not present at these identifiers (see section 2.1). It is considered, that it contains the zero-value octets. Length of the identifier is defined in header of the address. By sending of instructions CONTROL_REQ, TASK_REG and SESSION_OPEN, the protocol uses timeout. The value of timeout is assigned by node and must be more than three intervals of the maximal time of delivery at the transport layer. The timeout is not influenced the waiting period in queue to the transport layer. 5.1 Job Initiate The job concerns to the user application executed on VM. The UMSP job initialization can be made simultaneously with the application user start or during its working. The task, appropriated to its job, is initialized on the node together with the job. LTID is binding to this task. If the node, on which the user application was loaded, is chosen for JCP, the question of the job initialization lays beyond the scope of the network protocol. Other node can be chosen as JCP for the following reasons: o The job initialization node is connected to network by slow-speed or overloaded channel. It is undesirable to send the managing traffic. o The node has no computing possibilities for conducting the managing tables. o The authentication on the detailed node is necessary. If the other node is chosen for JCP, the node, that initiates the job, must register the job at JCP. Bogdanov Experimental [Page 23] RFC 3018 Unified Memory Space Protocol December 2000 5.1.1 CONTROL_REQ The instruction "To request a control" (CONTROL_REQ) is sending from the node, initial the job, to JCP of other node. The instruction has the following values of fields: OPCODE = 3 PCK = %b00 CHN = 0 ASK = 1 EXT = 0/1 OPR_LENGTH = 2/3 ; Depends on LTID length. REQ_ID - The value is assigned by the sender node protocol and then will be sent in the response. Operands: 4 octets: The control parameters profile. This field has the following format: bits 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | | + JOB_LIFE_TIME + | | +-----+-----+-----+-----+-----+-----+-----+-----+ | CMT | Reserved | VERSION | +-----+-----+-----+-----+-----+-----+-----+-----+ | Reserved | +-----+-----+-----+-----+-----+-----+-----+-----+ JOB_LIFE_TIME 2 octets. The job lifetime in seconds. The zero-value signifies that the restriction of the job lifetime is unused. CMT 1 bit. The flag of several JCP using. This field is reserved for the future expansion of the protocol. VERSION 1 octet. The number of the UMSP version. It must contain the value 1. Bogdanov Experimental [Page 24] RFC 3018 Unified Memory Space Protocol December 2000 Reserved 3 + 8 bits. All bits must be set to 0. 4/8 octet: LTID of task of the job, assigned on the node, which initiate the job (by the sender of this instruction). The optional extension headers: _JOB_NAME - This header contains the name of the Job. Is assigned once and must not change further. _INACT_TIME - This header contains the inaction time (see section 5.7). At reception of the CONTROL_REQ instruction JCP checks the LTID value from the received instruction and makes the following: (1) If the node, which has sent CONTROL_REQ, already has registered on JCP the active job with such LTID, the notification about abnormality end of the registered job is sent, as is described in section 5.5.2 (it is considered, that the node was reloaded). After that, the sanction to an initiation of the new job is sent. (2) If the node has no registered job with received LTID, it allows the new job initiation at once. If JCP confirms the control, it will send the instruction CONTROL_CONFIRM, or else CONTROL_REJECT. 5.1.2 CONTROL_CONFIRM The instruction "To confirm the control" (CONTROL_CONFIRM) is sent from JCP as the positive response to CONTROL_REQ instruction. CONTROL_CONFIRM has the following values of fields: OPCODE = 4 PCK = %b00 CHN = 0 ASK = 1 ; The instruction does not need to be responded. This flag specifies presence of the REQ_ID field. EXT = 0/1 OPR_LENGTH = 1-4 ; Depends of length of the GJID. REQ_ID - The value is taken from the instruction CONTROL_REQ Operands: 4-16 octets: The GJID assigned to the job on the JCP. The sending of the instruction CONTROL_REQ means request of control and request of task initiation. Assigned to the task CTID is part GJID (field of the local memory address). Bogdanov Experimental [Page 25] RFC 3018 Unified Memory Space Protocol December 2000 5.1.3 CONTROL_REJECT The instruction "To reject the control" (CONTROL_REJECT) is sent from JCP as the negative response to CONTROL_REQ instruction. CONTROL_REJECT has the following values of fields: OPCODE = 4 PCK = %b00 CHN = 0 ASK = 1. The instruction does not need to be responded. This flag specifies presence of the REQ_ID field. EXT = 0/1 OPR_LENGTH = 1/2 ; Depends on presence of the control parameters profile field. REQ_ID - The value is taken from the instruction CONTROL_REQ Operands: 2 octets: The basic error code. The zero-value is not available. 2 octets: The additional error code. 4 octets: The control parameters profile (see section 5.1.1), that is allowed by JCP. This is optional field. The optional extension headers: _INACT_TIME - This header contains the inaction time (see section 5.7). _MSG - contains the arbitrary error description. 5.2 Task Initiate The job is executed on several nodes simultaneously. The task, appropriate to it, must be initialized on each node. There is corresponding only one task to one job on the node. Each task must be connected only with one job. The task is initiated together with the job on the node, which had created the job. On the other nodes, the task is initiated during the receiving of the first request on the opening of the session connection, which is appropriate to the job. The request about openings of session connection contains GJID. GJID contains the JCP address. It is necessary to receive the sanction from JCP for the task start. If the request about the opening of session has been received from JCP node, it is not necessary to request the sanction. 5.2.1 TASK_REG The instruction "To register a task" (TASK_REG) is sent from the node, which initials the task, to JCP of the remote node. The instruction has the following values of fields: Bogdanov Experimental [Page 26] RFC 3018 Unified Memory Space Protocol December 2000 OPCODE = 6/7/8 ; For length CTID of 2/4/8 octets. PCK = %b00 CHN = 0 ASK = 1 EXT = 0/1 OPR_LENGTH = 2-8 ; Depends on length of the GTID and LTID. REQ_ID - The value is assigned by the sender node protocol and then will be sent in the response. Operands: 2/4/8 octets: CTID of the task initiated the job. It CTID is a part GJID from the instruction SESSION_OPEN. 4-16 octets: GTID, assigned on the node, initialed session connection. GTID is formed of sender addresses (at transport layer) and field LTID of the instruction SESSION_OPEN. 2/4/8 octets: LTID, assigned on the node, initialed the task (by the sender of this instruction). The optional extension headers: _INACT_TIME - This header contains the inaction time (see section 5.7). The instruction TASK_REG must be sent only if the task with given GJID was not initiated on the node. JCP confirms initiation of a task at observance of the following conditions: (1) Task with received GTID already has registered on JCP. (2) Task with LTID for the node requesting for initiation has not registered. In all other cases, JCP will not confirm a task. If JCP confirms the task, it will send the instruction TASK_CONFIRM, differently TASK_REJECT. 5.2.2 TASK_CONFIRM The instruction "To confirm the task" (TASK_CONFIRM) is sent from JCP as the positive response to TASK_REG. TASK_CONFIRM has the following values of fields: OPCODE = 9 PCK = %b00 CHN = 0 ASK = 1. The instruction does not need to be responded. This flag specifies the field REQ_ID presence. EXT = 0/1 Bogdanov Experimental [Page 27] RFC 3018 Unified Memory Space Protocol December 2000 OPR_LENGTH = 1/2 ; Depends on length of the CTID. REQ_ID - The value is taken from the instruction TASK_REG. Operands: 4/8 octets: The CTID assigned to the task on the JCP. The optional extension headers: _JOB_NAME - This header contains the name of the Job. 5.2.3 TASK_REJECT The instruction "To reject the task" (TASK_REJECT) is sent from JCP as the negative response to TASK_REG instruction. TASK_REJECT has the following values of fields: OPCODE = 10 PCK = %b00 CHN = 0 ASK = 1. The instruction does not need to be responded. This flag specifies presence of the REQ_ID field. EXT = 0/1 OPR_LENGTH = 1 REQ_ID - The value is taken from the instruction CONTROL_REQ Operands: 2 octets: The basic error code. The zero-value is not available. 2 octets: The additional error code. The optional extension headers: _INACT_TIME - This header contains the inaction time (see section 5.7). _MSG - contains the arbitrary error description. 5.2.4 TASK_CHK With the purposes of a safety the node, which have received request about the opening of session connection, may check up at JCP the node, which has initialed connection, even if the task was already initiated. The instruction "To check up the task" (TASK_CHK) is sent from the node, which has received the instruction of the establishment of session connection SESSION_OPEN, to JCP. The task with given GJID, must have existed on the node already. The instruction TASK_CHK format coincides with TASK_REG. OPCODE = 11. The response to the instruction TASK_CHK JCP forms instructions TASK_REG similarly. JCP confirms the instruction TASK_CHK if a task with received GTID and LTID already has registered on JCP. The sending of the TASK_CHK is optional. Bogdanov Experimental [Page 28] RFC 3018 Unified Memory Space Protocol December 2000 5.3 Establishment of session connection The session connection is established between two tasks of one job. The connection is established under the VM initiative and it is used for the exchange of the instructions between VM. One session connection must be connected only with one task on the node. The task may have several connections with different nodes. Between two nodes must be only one session connection with one GJID. The request about the establishment of session connection contains the global identifier of the job GJID. If the node receives the request about the establishment of connection with GJID, which is not presented on the given node, VM must create a new task. If the task has been already initialized, the new task is not created. The session connection needs to be established over TCP. After the connection is established, the sending of the instructions, whi