💾 Archived View for spam.works › users › emery › AgtDoc.gmi captured on 2023-07-10 at 19:09:20. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-06-16)
-=-=-=-=-=-=-
"[We] use bad software and bad machines for the wrong things." -- R.W. Hamming ___ ___ ___ ___ ___ |_ ___ '___| | ` | | '___| | | | |___| |___| | |___| |___| | | |__ |___. .___| version 1.1b (C) 2000, 2001 Michal Zalewski <lcamtuf@tpi.pl> (C) 2000, 2001 Argante Development Team <argante@cgs.pl> Argante Development Team: Michal Zalewski <lcamtuf@tpi.pl> Maurycy Prodeus <z33d@eth-security.net> Bulba <bulba@intelcom.pl> Marcin Dawcewicz <marcel@linux.com.pl> Artur Skura <arturs@people.pl> ArtGabi team [http://www.artgabi.com.pl] bikappa <bikappa@itapac.net> Adam Podstawczyński <adam@english.w3.pl> scrippie <ronald@grafix.nl> Lukasz Jachowicz <honey@linuxnews.pl> eru <eru@ibbrain.ibb.waw.pl> Jaroslaw Pyszny <arghil@bigfoot.com> James Kehl <ecks@optusnet.com.au> Mariusz Woloszyn <woloszyn@ipartners.pl> Our website and mailing list: Homepage: http://agt.buka.org Souce code: http://lcamtuf.na.export.pl/arg.tgz Mirrors: see Documentation/MIRRORS Mailing list: mail -s 'subscribe' argante-request@cgs.pl </dev/null WWW archive: http://argante.buka.org ======== CONTENTS ========
"Argante" is a virtual operating system, created in a big part during a few days. In the present stage of its implementation I take care of most things and monitor code developement, but I hope many other people will join the project :) You're currently reading the (almost) complete documentation for Argante OS release 1. This is quite early version of this system, so many things are still scheduled for futher development. Our goal is to show this solution to the open-source community, to discuss our ideas and concepts, and to decide if we should continue our work on this system. The rationale for its creation and what makes it different from other systems will be explained in section 2 of this text. In the meantime I would like to thank all those, who even minimally contributed to the present form of the system (I have deliberately omitted co-authors of the code and who have been mentioned earlier): Maja :) - for her being and that's it... Sławomir Krawczyk - for usual sarcastic remarks ;) Agnieszka Słota - for her interest in the idea and care Filip Niedorezo - for the First Independent Program ;> Marek Białogłowy - for "you are scaring me" Wojciech Purczyński - for nice polemics Jarek Sygitowicz - for plans of world domination negativ - good ideas :) eloy - mailing list maxiu - ideas, ideas concerning optimalisation This list is rather short, I hope it will change :) If you feel you have been omitted, write to me. Write also if you think tsi project is interesting or if you have some critical points to make, some ideas etc. Every idea is very precious. But first of all it is essential to acqaint oneself with the whole document and try to find answers for questions coming to your mind there. Programmers accustomed to Unices, classic assembler and trditional constructions, may find many things here "don't make sense", but I assure you almost every element of Argante is justified by something; with some good will you will find this justification here :) We'd especially like to thank the ArtGabi team - you can visit their webpage at http://www.artgabi.com.pl. They created Argante logo and designed our website for free. Essential literature: Steven Muchnick, "Advanced Compiler Design and Implementation" Andrew S. Tanenbaum, "Distributed Operating Systems" Doreen L. Galli, "Distributed Operating Systems - concepts & practice" Andrew S. Tanenbaum, "Modern Operating Systems" Eric S. Raymond, "The Cathedral & the Bazaar" Illiad, "Evil Geniuses in a Nutshell" (I don't have more at hand ;) This documentation is split into two parts. First part, which you're currently reading, described concepts and implementation of Argante OS. Second part describes practical approach to Argante and programming and development guidelines.
Argante is a fully virtual environment for running applications on Unix systems. This makes many people think about Java and its sandbox for example, although the technical reasons Argante is based on were totally different. For one thing, Argante is a complete operating system. It has its own implementation of processes, inter-process communication, filesystem, access control... All built on the top of basic real OS low-level implementation, but with own control mechanisms, own semantics and so on. Why all this? I will try to explain: The standard architecture of operating systems and hardware (e.g processors) falls flat when it comes to security and stability of the software. To be short: it lacks low lewel support for general access control, error handling (primitive techniques existing in, say, the 80386 series are not enough), and the architecture of stack or data segment usage is based on some mistaken assumptions. Trying to fix these errors at a higher level is generally risky and unsuccessful. The authors of Java have created a miserably slow and, as a matter of fact, not always secure / portable solution with very limited application range; moreover, they were unable to force software authors to use safe, verified architecture models, e.g. OSI, limited trust and interaction architecture which presumes that only the two closest data processing layers work together and the code itself is divided into functional segments. Programs written in C using the model "listener -> fork() -> client handling" are still easier to implement and less prone to failure. In this way the list of these and other remarks concerning the popular hardware and software architecture model came into being. Its essence can be best summed up in the motto at the beginning of this document: "[We] use bad software and bad machines for the wrong things." -- R.W. Hamming Except complaints I had many ideas which in my opinion should be, and, with minimal cost could be, taken into account in implementations at both these levels: hardware and software. At a certain point I had a difficult decision to make: I could modify existing implementations, trying to patch them with temporary solutions, risking that most of these ideas will never be carried out, and being aware that the project will become a series of compromises, among which the sense of its realisation will be lost :) I could as well do another thing: sit and rewrite everything from scratch. Forgetting about compatibility, conventions, trying to create a solution which will defend itself, or one nobody will notice :) In this way the idea of Argante started, having at its basis the four principal proposals: - security and stability - functionality - efficiency - simplicity Argante is supposed to be a system with no compromises. That is why always when in the traditional system we would face choice "security or functionality", instead of choosing one variant we concluded the choice itself is bad and created its outline from scratch or changed the model in order to reconcile our requirements with expectations. Why, then, is it an "embedded" system? There are many reasons for that. For one thing, an embedded implementation does not enforce OS change, makes first attempts and projects easy, providing integration with existing solutions on native Unix platforms. In this way Argante introduces an additional abstraction and protection layer, acting as a completely independent hardware architecture, and without enforcing serious changes. Its being written in C assures efficiency and portability. Moreover, an implementation which can use existing system drivers, devices, system functions, becomes a much simpler task and permits programmers to concentrate on the substance instead of implementation details (bootloader, drivers etc.). Naturally, when speaking of stability and security of an embedded system I mean its implementing access control systems independent from the native platform, its own multi-process model: all these solutions are safe and independent of the real system. That's why Argante will be a safe solution on almost any Unix (or maybe even Windows?;) provided that elementary security of the native platform will be ensured; in the simplest variant, all network services should be removed (hybrid solutions, described with rIPC and network, constitue a separate case). In order to satisfy the four enumerated proposals, I have created general guidelines for the system. They were as follows: - the core of the system will be a microkernel providing base functionality; all input/output operations will be performed using loadable modules, easy to implement by the user and added/removed while the system is running; the modules can also contain other, necessary functions, for example providing advanced operations on text strings and similar procedures, - the system will provide _any_ functionality permitting software creation, starting from a database server to a graphics application without any need to change system code, and at the same time ensuring the highest security level, - the system will have its own, low-level, hardware platform independent virtual machine language; this language will be simple and efficient enough to ensure speed and effectivity, and at the same time it will ensure full separation from the real system and will not allow native code execution, - system management will be fully separated from processes run in the virtual system; user-space and kernel-space will also be fully separated, without any possibility of interference into kernel-space from the level of user-space, - every process run in the system will dispose of its own, private address space, separate stack segment which will not be directly addressed (used only by jump/return functions); the same applies to the code segment, which will not be directly addressable (note: we are providing user-space metastack features for reentrant routines and/or local variables). Only the code segment will be executable, - a process will be allowed to allocate memory blocks, separately mapped to its own addressing space (with the possibility of write protection); the system will control all attempts of going beyond the allocated block (buffer), - the system will support low-level exception handling and will allow the program to handle them (LLX - low-level exceptions), - the system will have its own, secure and resource-saving implementation of multitasking and its own, static process model (SMTA) with assigned fixed privilege lists; multi-user applications will also be supported by the possibility of defining a subgroup identifier in a given privilege domain, - a new philosophy of privilege granting and dropping, without risks inherent in the Unix implementation, - from its very beginning the system will support secure solutions (e.g. unbounded strings instead of null-terminated ones, etc.), - the system will provide hierarchical, centralized and universal implementation of Hierarchical Access Control (HAC), permitting defining privileges with arbitrary detail level; additionally, the system will enforce the "switch" architecture, forcing the programmer to define which privileges are necessary in order to perform a given task without permitting having any others, - the system will strongly support the OSI architecture, including distributed architecture, providing advanced mechanisms of inter-process communication IPC (a specific solution, different from the one existing in Unix) and rIPC (remote IPC session distribution among equivalent processes, communication between tasks on different computers transparent for user-space); rIPC will also support transparent cluster architecture - the system will have its own implementation of a virtual filesystem, accessible from the level of a real filesystem, and at thse same time permitting establishing arbitrary inner structure and full access control compatible with HAC - changing any functionality will be possible without stopping the system - to avoid "state of art" coding, Argante can be easily mixed with real system code at any moment, communicating with system daemons, services, being able to set up and modify real system if necessary (and *only* if programmer wants to do it). Argante favours creating hybrid solutions, for example applications of the real systems coordinated / protected by Argante code. This will enable one to transparently create reduntant, heterogenic clusters with morphing possibilities, self-assigning new objects in existing hierarchy and full redundance as well as load balancing without _any_ programming costs. It doesn't matter whether the system will work on one machine or a hundred, with redundance and load balancing - the rIPC philosophy solves distributed systems problems in a way transparent for applications. What else? Well, Argante could act not only as a cluster development platform, but, in fact, it makes complex development really easy and clean. For example, to design distributed, fault-tolerant virtual router, you could use only several thousands lines of readable and elegant code, which can be maintained for years with no risk. Well, but that's not all. To prove AOS isn't only the "distributed networking software", we decided to develop svgalib connectivity module to demonstrate how fast and effective - especially when compared eg to Java - Argante can be. Enjoy. I know it sounds like a wish list, but I'm writing these words having implemented most of the system's code and, to my surprise, I can (no-so-modestly) say that I have suceeded in attaining these aims. What have I got? - security and stability: - practically speaking, impossibility of taking control over an application in the system (stack, data segment and buffer control, the approach of passing parameters to syscalls without depending on C conventions, like null-term); because of a quite limited number of RSIS opcodes, privilege control is a trivial matter, - even if it were possible, no possibility of getting privileges enabling one to breach the security of the rest of the virtual system (separation of management from the virtual system, from kernel-space), - even if it were possible, lack of any possibility of influencing real system (separate implementation of multitasking, not using the implementation of the real system), - faciliating programming compatible with the secure OSI architecture, it is simply intuitive in this system, - enforcing control of code execution correctness by raising exceptions, - full access control to any resources (HAC), the above mentioned new philosophy of privileges, a new approach to linking privileges with the pricess and a new process model, etc... - destabilisation of the native filesystem is practically impossible, - redundance and request distribution support - functionality and simplicity: - the system is universal by providing commode modules and centralized control as well as an effective virtual processor architecture with limited but efficient command set - the possibility of creating distributed systems without having to modify the code; the possibility of request propagation without any need to modify the code (of an application) - exceptions make exception handling easier - introducing even serious system changes may happen on the fly by module exchange - efficiency: - load balancing, creating clusters, distributing the solution among machines can be done without modifying the source code of its elements - by using a low level virtual code, instead of -- as in the case of Java -- a high level code, efficiency reduction is not so striking, nor does it limit the abilities of the code. Loops of the "idle" kind (i.e. a repeated jump) is a few times slower than in a compiled C program running on a given hardware platform, which is a very good result. In case of more complicated operations (e.g. I/O), efficiency reduction is much lower, oscillating around 15-30%, - the kind of multitasking implemented is far more stable and much more memory-saving than on the native system; it results in part from the fact that a virtual Argante processor needs less information to maintain a process than Unix does, and also from imperfection of many systems. We wanted to combine QNX, HURD and all our "loose" ideas to create a really secure and effective solution :) Later, Pawel Krawczyk pointed out that Inferno embeded system, developed by Lucent, contains several solutions quite similar to Argante. Of course, there are also major differences (Argante is all-purpose environment for secure applications that doesn't enforce any high-level solutions and focus on the low-level security). We believe we avoided such strange half-solutions - like moving high-level functionality to low-level layer with no good reason (and thus decreasing freedom of design and making overgrown code); we decided for such step only in specific, well-documented and explained cases, where we're sure it will offer some real good for the programmer without enforcing static, complex solutions where they are not necessary. Details on Inferno can be found at: http://www.vitanuova.com/inferno/papers/bltj.html. And another thing: you can view a simple but joyful tutorial starting three programs by typing "./build test". What's still unfinished in Argante? I believe some new modules should be developed to give Argante the access to appliances where it could be usable. While making AOSr1, we focused on the things that are absolutely necessary to make it interesting and innovative, but also, we had to delay some developments (mainly because we do not have enough people). Here's our list of things to be done in AOSr2 (more recent version can be found in Documentation/TODO file): 1) Endian block translation (in advmem). Assigned person: lcamtuf 2) Solaris/BSD portability for low-level networking (packet.c); well, in fact, someone should write #ifdef code for every platform when it comes to packet sniffing. BPF support would be nice. Assigned person: bikappa (as long as he will found some free time to write AND TEST his code on a few platforms ;) 3) Solaris, BSD, IRIX: ripcd portability; some minor fixes are required to make it portable accross these platforms. Assigned person: (???) 4) Mainstream Argante: HP/UX, AIX ports. Assigned person: (???) 5) Bytecode interpreter JITs (jumps in table) instead of multiple ifs. Almost done by Mariusz Woloszyn. 6) X Window: GTK-based GUI (agtses, agtback functionality; basic console commands as menu items / buttons; vcpucons / agtexe in xterm window). Assigned person: (???) 7) AHLL should be rewritten using some flex/bison-alike stuff. Assigned person: Bulba or lcamtuf (???) 9) Math module should be optimized (floating point arrays -> fixed point or int arrays, etc). Assigned person: z33d 10) Some examples - SMTP, POP3 functionality should be good. Assigned person: (???) - after AHLL rewriting! 11) PDF/sgml/html documentation, completely revised and clean. Assigned person: lcamtuf 12) Modula-3 compiler. Assigned person: Marcin Zukowski Release date: one day in the future ;) We are seriously considering separating the bytecode interpreter from I/O / debugging functionality in future releases of Argante. What do we mean? Well, our bytecode interpreter, which is actually pretty easy to implement, will be ported to several platforms: - "software solution" it is right now - where time has to be shared between real system, I/O operations and currently executed code, - cheap microcontrollers (eg. Motorola 68376) on the PCI/ISA cards - in this case, bytecode interpreter is stored in EEPROM, and controller is executing it, calling Argante I/O modules (running in real system) only if there's such need (on syscalls); controller will have its own memory, and all transfers will be done using DMA. This will really speed-up whole solution, decrease the usage of real system, and made it even more secure - RSIS code won't be executed by main processor. Also, it will become fault-tolerant - even if real system crashes, VCPUs might survive, waiting for real system to resume I/O services :) We're going to design such RSIS-interpreter card in near future, as it isn't really complicated or expensive (M68376 costs $25). - cheap external "processors" (eg. spare i386 box); in this case, bytecode interpreter will be launched at the boot time, with no OS layer; the problem is to provide fast enough half-duplex link with almost no latency between two boxes with no additional, expensive hardware; dedicated ethernet _might_ be the answer, - one day, maybe dedicated RSIS hardware solutions - eg. chips implementing RSIS functionality as a native language?:) Well, the last option is S-F for now ;) What are the consequences? Well, one I/O mid-end might connect several solutions - software boxes, dedicated hardware, and so on - providing unified input and output for the project, with no risk one box might affect work of other boxes. Also, even if mid-end crashes, properly written AOS software will survive it and resume its work after rebooting the mid-end. Ok - we sure this list isn't closed - and so, your comments, ideas and suggestions will be more than welcome :) PLEASE NOTE: this documentation is still evolving; our source code is evolving as well. This might cause some minor differences between this documentation and source code. Sometimes, one look at the source code might be more explainative than 10 pages of documentation. Oh, and feel free to report any mistakes and/or corrections to us :)
The concept of processes in Argante might seem shocking to many people, especially to those accustomed to the Unix scheme: one client, one process. In Argante processes are static objects -- they are started from the management console or scripts. According to standard Argante semantics (which, naturally, can be changed by adding a new syscall), processes cannot multiply, create offspring or execute other programs in their place. Instead, Argante supports the OSI-alike model, where a process is assigned not an object (as a connecting client, for example), but a certain function (e.g. database or connection service) - depending on programmer's will. Although it seems like additional burden, I am sure when you finish reading this document you won't think it is something bad. The process is read into virtual processor (VCPU) space and exists there until it finishes its job or a critical error arises (like an unhandled exception). Most process parameters, e.g priority or a "domain" set (which are an object similar to supplement groups in Unix), are assigned to the binary image of a given executable during compilation. Below is an example structure of an ftp daemon fulfilling OSI requirements, easy to implement in Argante (maybe simpler than in C), and at the same time much more secure and... efficient: TCP/IP database user files | | | "reality" --|-------------------------------------------|-----|-------------------- (net) (ipc)---(ipc)-(ipc)-(ipc) (ipc)-(ipc) (fs) (fs) kernel space --|-----|-------|-----|-----|-----|-----|-----|-----|-------------------- | | | | | | | | | user space <A>----+ <B> <B> <B>----+ <C>----+----<D> A - proces handling network connections: accepts a connection, connects to one of the B processes with IPC and transfers commands to them B - processes serving clients (any number, automatic request propagation); they handle commands, communicate with the authorizing process with IPC; thanks to ease of use of IPC and context support, handling of many sessions in one process is not a problem. C - the process realizing authorization; using the fd verifies entries in the local database available in SVFS D - after authorization, every request is passed from B to D (and back). When describing the HAC system we will explain how inter-process communication and group changing work. For now it should be said that processes will never be able to, say, operate using the net and ipc module at the same time (the "switch", mentioned earlier), and the process A will never be able to communicate with C via IPC. Firstly, security. Secondly, level of code complication at the level not extending the same one written in C. Thirdly, much higher efficiency when compared to the model using fork(). How does multitasking work? Generally speaking, it is rather just ;) Every process in a given cycle of process handling is assigned as many machine cycles of its virtual processor, as is the value of the "priority" of the process. Consequently, a process with the priority of 10000 will be given 10000 cycles, and a process with the priority of 1 - one cycle. Obviously, it is advised to give processes reasonable priority values, ranging from 100 to 10000. Processes might be in the state STATE_SLEEPFOR, in which their execution is suspended for a determined number of cycles of process handling; the state STATE_SLEEPTILL is also possible: it is a condition where the process is waiting for a given number of microseconds; or the state STATE_IOWAIT, where the process is waiting until, let's say, it has been granted the right to write to a file another process is writing to, or for receiving data from a socket (of course only if it is made to enter in this state, because it can also execute the function with the NONBLOCK option. For details on proper project development, please see Part II, where you can find practical guidelines and precautions...
Technically, VCPU is a virtual machine with a limited but easier to use, when compared to traditional machine code, instruction set; three register blocks (for operating on 32-bit unsigned and signed integers and floats, respectively) with 16 registers each; stack space (used only for function return addresses, data storage is done otherwise); an identification number in inter-process communication model; executable program space; space used for allocating memory blocks with an implementation of access control and dynamic reallocation as well as a few other, less significant variables. Allocated memory blocks which can be used for data storage and processing are distinct and overwriting one block when going beyond the limit of another is impossible. As has been mentioned, stack and program code modifications are impossible, either. Allocated memory addressing, in contrast to 8-bits used in traditional systems, is done with dwords, 32-bit jumps. It entails improved efficiency in most applications and, at the same time, safer data access. Code space addressing is similar. Every instruction is coded using 12 bytes. Instructions without arguments, as NOP etc., have only the first byte set, referring to the opcode. In other cases, two subsequent bytes mean the parameter type; one byte is a padding, the two another contain parameters. In fact we could do with 10 bytes, but efficiency would decrease (at present we have three times 32-bit dword). Waste of space? Not really, if you read futher specifications :) This solution allows for safer movement within the code segment (including jumps) as well as lessening opcode number and extremal gain as far as the number of parameters needed to perform a given operation is concerned, which is made up for by the size of a single instruction: 1 2 3 4 5 6 7 8 9 10 11 12 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ xxxxx xxxxx xxxxx RSRVD xxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxx | | | | | | | | +-----------------------+- two 32-bit parameters | | +------------------------------------- second parameter type | +------------------------------------------- first parameter type +------------------------------------------------- opcode, e.g. MOV Parameter types: IMMEDIATE - 32-bit number UREG - unsigned register number SREG - signed register number FREG - float register number IMMPTR - a number pointer to a 32-bit numeric value UPTR - an unsigned register number containing a pointer to a 32-bit numeric value Note: in case of a jump instruction, passing an IMMEDIATE or UREG parameter type refers simply to the address. The case of the MOV instruction is different: if we want to refer to the address of a given memory location, we should use IMMPTR or UPTR types. It is a convention making using commands more effective. The following registers are accessible: u0 .. u15 - unsigned register types (0..15) s0 .. s15 - signed register types (100..115) f0 .. f15 - float register types (200..215) The systems provides type conversions during register operations, however it is time consuming and shouldn't be used too often. Values taken from the memory aren't converted (so when writing the value of the register f0 = 0.123 to the address 1234 and then reading the value from this address to the register u0, we will probably get an unpredictable result; the solution consists in reading the value again to the register f0 and using mov u0,f0). The process run on a VCPU can only use the native instruction set called RSIS without the possibility of directly executing the machine code of the real processor.
A coarse description of the machine commands of the system: Mnemonic: NOP Parameters: - Opcode: 0 Description: do nothing Result: - Exceptions: - Mnemonic: JMP <addr> Paramaters: <addr> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 1 Description: conditionless jump to an absolute address Result: IP change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: IFEQ <x> <y> Paramaters: <x> = any, <y> = any Opcode: 2 Description: execution of the next statement if <x> = <y> Result: conditioned IP change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: IFNEQ <x> <y> Paramaters: <x> = any, <y> = any Opcode: 3 Description: execution of the next statement if <x> != <y> Result: conditioned IP change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: IFABO <x> <y> Paramaters: <x> = any, <y> = any Opcode: 4 Description: execution of the next statement if <x> > <y> Result: conditioned IP change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: IFBEL <x> <y> Paramaters: <x> = any, <y> = any Opcode: 5 Description: execution of the next statement if <x> < <y> Result: conditioned IP change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: CALL <addr> Paramaters: <addr> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 6 Description: conditionless jump to an absolute address with address push Result: IP change, pushing the addres on stack Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT, STACK_OVER Mnemonic: RET <cnt> Paramaters: <cnt> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 7 Description: return to the address <cnt> popped from the stack Result: IP change, popped from the stack Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT, STACK_UNDER Mnemonic: HALT Paramaters: - Opcode: 8 Description: termination of VCPU work; also in respawn mode Result: - Exceptions: - Mnemonic: SYSCALL <nr> Paramaters: <nr> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 9 Description: syscall execution Result: dependent on syscall Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT, NOMODULE + syscall dependent Mnemonic: ADD <x> <y> - opcode 10 SUB <x> <y> - opcode 11 MUL <x> <y> - opcode 12 DIV <x> <y> - opcode 13 MOV <x> <y> - opcode 19 Paramaters: <x> = UREG, FREG, SREG, IMMPTR, UPTR <y> = IMMEDIATE, UREG, FREG, SREG, IMMPTR, UPTR Description: arithmetic operations (+, -, *, /, assignment) Result: first argument value change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: LDB <dst addr>, <src addr> Opcode: (fixme!!!) Parameters: <src addr> - address of source memory address / reg <dst addr> - address of destination memory address / reg s0 - source byte offset Description: single byte accessibility (endian-independent) Result: dst will contain src[s0] (single bytes) negative indexing is possible in memory-based addressing; register-based addressing requires s0 to be in range 0 to 3 Mnemonic: STOB <dst addr>, <src addr> Opcode: (fixme!!!) Parameters: <src addr> - address of source memory address / reg <dst addr> - address of destination memory address / reg s0 - destination byte offset Description: single byte accessibility (endian-independent) Result: dst[s0] will contain src youngest byte (single bytes) negative indexing is possible in memory-based addressing; register-based addressing requires s0 to be in range 0 to 3 Mnemonic: MOD <x> <y> - opcode 14 XOR <x> <y> - opcode 15 REV <x> <y> - opcode 16 (unimplemented for now) AND <x> <y> - opcode 17 OR <x> <y> - opcode 18 Paramaters: <x> = UREG, SREG, IMMPTR, UPTR <y> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Description: binary operations Result: first parameter value change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: CWAIT <x> Paramaters: <x> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Opcode: 20 Description: puts the process to sleep for <x> SMTA ticks Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: TWAIT <x> Paramaters: <x> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Opcode: 21 Description: puts the process to sleep for [at least] <x> microseconds Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: ALLOC <size> <prot> Paramaters: <size> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR <size> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Opcode: 22 Description: allocates a memory block with size <size> and access flags <prot> Result: u0 - id block number, u1 - map address Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT, NOMEM Mnemonic: REALLOC <nr> <size> Paramaters: <nr> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR <size> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Opcode: 23 Description: reallocates a memory block with number <nr> so that it has size <size>. NOTE: if <size> is '0', u0 is examined to modify memory block permissions. Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT, NOMEM Mnemonic: DEALLOC <nr> Paramaters: <nr> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR Opcode: 24 Description: deallocates a memory block with number <nr> Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: CMPCNT <addr1> <addr2> Paramaters: <addr1> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR <addr2> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR s0 - dwords count Opcode: 25 Description: compares <addr1> and <addr2> within s0 bytes Result: u0 - 0 = comparison succeeded, !0 - negative Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: CPCNT <addr1> <addr2> Paramaters: <addr1> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR <addr2> = IMMEDIATE, UREG, SREG, IMMPTR, UPTR s0 - dwords count Opcode: 26 Description: copies <addr2> to <addr1> within s0 bytes Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: ONFAIL <addr> Paramaters: <addr> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 27 Description: a jump to an absolute address on exception; discarded after RET under current execution level Result: - Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: NOFAIL Paramaters: - Opcode: 28 Description: remove ONFAIL on current execution level Result: - Exceptions: - Mnemonic: LOOP <addr> Paramaters: <addr> = IMMEDIATE, UREG, IMMPTR, UPTR s0 - loop counter Opcode: 29 Description: jump to an absolute address if s0 is greater than zero, s0 is increased by one Result: IP change, s0 change Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT Mnemonic: RAISE <nr> Paramaters: <nr> = IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 30 Description: raise exception <nr> Result: exception raised Exceptions: OUTSIDE_REG, BAD_PARAM, OUTSIDE_MEM, PROTFAULT These commands were introduced to provide userspace meta-stack implementation. To create re-entrant subroutines that might be called recursively, you might use the following combination (I assume u15 register is reserved in your implementation for stacking purposes): // Somewhere in the memory, we have writable playfield we can // use for stack. I assume its address can be found in u1 // and its size in dwords (!) is stored in u2: mov u0, 0 SETSTACK u1, u2 :Reentrant_Routine PUSHS u15 ALLOC <space required for local objects>, <protection flags> MOV u15, u1 // Now, we can use local stack space by accessing address stored in u15. // You can call Reentrant_Routine again, and u15 or your local, // private space won't be damaged. ... // On exit from Reentrant_Routine: DEALLOC <block num derived from u15> POPS u15 RET 1 More advanced implementations might catch user stack exceptions to resize stack. So, metastack structure looks this way: +--<---< this is a metastack | | +--- <local buffers of function nested()> | | +--------- <local buffers of function called_from_nested()> | | +-------------- <local buffers of function...> WARNING: There's no automatic cleanup or shrinking of user stack on exceptions! You have to perform eventual cleanup on your own (PUSHing magic value when declaring exception handler might be useful). Mnemonic: SETSTACK <addr>,<size> Parameters: both IMMEDIATE, UREG, IMMPTR, UPTR u0 -- initial stack ptr Opcode: 31 Description: set user-stack pointer (size=0 - disable stack) Result: modified VCPU internals Exceptions: - Mnemonic: PUSHS <dword> Parameters: IMMEDIATE, UREG, IMMPTR, UPTR Opcode: 32 Description: set user-stack pointer (size=0 - disable stack) Result: modified VCPU internals Exceptions: NOUSTACK, USTACK_OVER Mnemonic: POPS <dword> Parameters: UREG, IMMPTR, UPTR Opcode: 33 Description: set user-stack pointer (size=0 - disable stack) Result: modified VCPU internals and Exceptions: NOUSTACK, USTACK_UNDER As you have probably noticed, memory management was not placed in a separate module and is an integral part of the system. It's an exception aimed at improving efficiency and functionality,although it is perfectly possible to create a sophisticated memory management system using syscalls. Another important remark: some people complained thet the limited RSIS instruction set doesn't allow, say, for effective graphics operations or memory operations in general. In RSIS (as well as in C, for example), the basic language offers only the simplest operations and constructions allowing one to control execution flow etc. All advanced functions, however, as for example memfrob() and similar, don't belong to the language but to libraries. E.g. in Argante you can write the module advgraph.c, responsible for complex operations on graphical objects and communication with the card, but you cannot expect it from RSIS (so there will be no Argante MMX release ;). Although the module for graphics services is not currently planned, many functions useful for working with memory blocks and textual data will be placed in the advmem.c module. For the same reasons, we're not following Inferno authors, and not putting for example garbage collection, array addressing and so on at the low-level, and not using specific string-related conventions. We'd like to give the developers freedom in implementing things in the way they want to.
Exceptions are one of the things we decided to move from high-level to low-level. Why? Because this operation doesn't mean more complex RSIS language, nor does it enforce any conventions, except enforcing programmer to handle errors, instead of checking for them occasionally (or not checking at all). Exception stack and shrinking at the same time call stack does. You can declare exception handler for current execution level (or cancel it). It will be cancelled when RET is called, and will be inherited during any calls (but can be temporarily shadowed with ONFAIL handler declared in subroutine). If there's no exception handler at current execution level, the stack will be shrunk to the nearest handler. For example, if you declared the handler in procedure DoSomething, and then you called DrawABox, which does not have its own exception handler, but caused an exception, execution will return immediately to DoSomething, and the exception handler will be called from the place DrawABox was called. If there's no exception handler, the task will be terminated. Handlers are not specific. It means, any exception causes handler to be called. u0 register is saved (it will be restored after RET to code which caused first exception), then it's overwritten with exception code (these codes are specific for different modules, and are described below). Exception handlers are not cancelled at the time they're called, but, if an exception will appear in exception handler code, lower-level handler will be called instead of calling current handler again. Handler can decide, if it wants to cover current exception (and then, for example, return to code using RET), or to pass the exception to low-level handler (it can be done using RAISE u0 if exception code was not matched). The accepted policy consists in informing the program (using the non_fatal() function) using raised exceptions about untypical/alarming situations. Thus the syscall checking whether a file exists should not raise an exception if it doesn't exist. On the other hand, the syscall for opening files should raise an exception in these circumstances. So, exceptions raised by commands: #define ERROR_STACK_OVER 0x1 Stack overflow. Might happen on CALL or when exception handler is called at current execution level, but there's no stack space available. #define ERROR_STACK_UNDER 0x2 Stack underflow. Might happen on excessive RET attempt. Can be handled only at the lowest-level exception handler, of course, because stack is at "ground" level at the time this exception is raised. #define ERROR_OUTSIDE_CODE 0x3 Instruction pointer outside addressable process space (eg. no HALT at the end of code or exceessive JMP / CALL / ONFAIL). #define ERROR_OUTSIDE_REG 0x4 Excessive register number. Shouldn't happen, because these numbers are validated by compiler, unless you're messing with configuration options. #define ERROR_BAD_PARAM 0x5 Bad opcode parameter - for example MOV with immediate value as first param. In some cases - eg. REALLOC / FREE - it might mean incorrect memory block number specified. #define ERROR_BAD_INSTR 0x6 Illegal instruction (compiler brain damage or serious incompatibility). #define ERROR_OUTSIDE_MEM 0x7 Attempt to access non-allocated memory address. #define ERROR_PROTFAULT 0x8 Syscalls - buffer passed as a parameter is not suitable for reading or writing. #define ERROR_TOOBIG 0x9 ALLOC / REALLOC attempt with size larger than per-block limit. #define ERROR_NOMODULE 0xa No handler for specific syscall number. #define ERROR_BAD_SYS_PARAM 0xb (obsolete) #define ERROR_ACL_PROBLEM 0xc HAC subsystem cannot be initialised - missing configuration file. It will cause all HAC-based syscalls to fail. #define ERROR_NOPERM 0xd Access permission denied on HAC level #define ERROR_NOMEM 0xe No more free memory / memblock slots (ALLOC/REALLOC). #define ERROR_DEADLOCK 0xf Deadlock - cannnot access required system resource (eg. entropy pool). #define ERROR_NOOBJECT 0x10 Filesystem: object can't be located within SVFS mapping hierarchy. #define ERROR_FSERROR 0x11 General filesystem fault - for example, requested task cannot be completed due to low-level real system syscall error. #define ERROR_FS_BAD_PATH 0x12 Filesystem: path is incorrect; either whole path or one of its elements is too long, or you used relative path but have not current working directory set. It might also indicate path contained illegal non-printable characters. #define ERROR_FS_OPEN_ERROR 0x13 Cannot open file - for example, it does not exist (or disappeared during blocking open call). #define ERROR_FS_BAD_OPEN_MODE 0x14 Requested access mode is invalid. #define ERROR_FS_CREATE_ERROR 0x15 Cannot create file - for example, it exists already. #define ERROR_FS_BAD_VFD 0x16 Incorrect VFD number passed to fs syscall. #define ERROR_FS_NOSEEK 0x17 File can be accessed in append-only mode, while effective seek operation was requested. #define ERROR_FS_EXISTS 0x18 Rename: destination object exists. #define ERROR_FS_NOFILE 0x19 Object does not exist or object type mismatch (dir instead of file / file instead of dir). #define ERROR_FS_NODIRENT 0x1a LIST_DIR: requested offset is invalid. #define ERROR_RESULT_TOOLONG 0x1b Result will be longer than buffer passed to store it. Take a look at Examples/RSIS/error2.agt if you're still not sure how exceptions work.
A unified privilege management mechanism (HAC) has been created. A sample entry in the file access.set, the configuration file of the subsystem: 12345:00000 fs/ftp/users fs/fops/new/dir allow | | | | | +-------------+ | entry description| | | +-----------------------------------+ - allow or deny | | | | | +-- hierarchical identfier of acces type: acces space is 'fs'; a branch | | for file operations (fops), operation type: object creation | | (new), object type: directory (dir). This convention is recommended, | | although as I said earlier the kernel is not responsible for | | authorisation - it is done by modules, passing the data to the | | function is_permitted(). | | | +---- hierarchical resource identifier; in this case object space, | filesystem space (ftp file) and a concrete catalog are described. | +------ group membership; the value of '0' means the rule is of "generic" type and refers to all grups; the value after ':' refers to a the supgroup. In the case of rules specyfying a non-zero group, the value must be integer. The sequence of entries in the configuration file decides of their priority. Consequently, more specific entries (e.g. containing denial of access to resources for a given subgroup) should be given before more general ones. NOTE: If operation identifier in in the configuration file is, say, 'fs/fops', it means someone fulfilling other criteria and requesting access to 'fs/fops/new/file/text' will be granted access. Obviously, it doesn't work both ways and the entry 'fs/fops/new/file/text' doesn't imply access to the whole hierarchy 'fs/fops'. Using '/' as separators is necessary: for example the entry 'fs_ops' doesn't mean access to the object 'fs_ops_new_file'. HAC requires detailing operation rules, making one precise their object type which is undergoing operation. I n this way an entry of the type given below is _ALWAYS_ correct: +- wdops ---- cwd | +------- pwd | +- setup ---- ... | fs -+- fops --+-- create -- file ----------+-- binary | +----- directory | | +-- text +-- delete -- file | +----- directory | +-- read ---- directory +------ file On the other hand, entries like fs/fops/file/delete, fs/fops/file/create, etc. are INCORRECT. Although it may seem illogical at a glance, but this second entry would actually make rules generalisation impossible (e.g. granting privileges to create objects in a given part of the filesystem means entering fs/fops/create, whereas if we used the other notation, it would require many entries). As a means of protection against attempts of filing misinformation to modules managing the filesystem, the authorising system refuses access to objects containing the sequence "/..". The module should take care (and it does) of eliminating them. As for subsystems where it's impossible to define resources, or their defining would be doubling access type (e.g. the module displying text on the virtual console - defining operation type is enough in this case), the resource should be 'none'. Rule testing before their ctualisation can be done with the included program 'actest' (in the 'tools' directory), which provides decent diagnostics HAC. Rules actualisation is done with '^' within the management console (see below). If you know nothing about modules yet, you can come back here later. I will explain the HAC interface for modules below: >From the point of view of the author of a module, the most comfortable interface to access control is the VALIDATE() macro contained in the file include/acman.h. The macro accepts three parameters: the processor number, resource identifier as well as access type identifier. For example: VALIDATE(c,"net/tcp/destination/10.0.0.1/1234","net/connect"); In case where access is possible, macro will have no effect. If access is denied, the macrow will raise exception NOPERM with situation description and will exit the function syscall_handler() it should be called from. To handle it in a more refined way, we can use the function is_permitted(), accepting parameters corresponding to the parameters of the VALIDATE() macro, but returning the value of 0 (refusal) or 1 (access granted). There is no place for function return, nor exception is raised. To be precise, the VALIDATE() macro is constructed as a wrapper for the function is_permitted() in the following way: #define VALIDATE(cp,res,act) { \ char errbuf[512]; \ if (!is_permitted(cp,res,act)) { \ if (!cpu[cp].fail_safe) \ snprintf(errbuf,200,"permision denied [%d:%d] act='%s' obj='%s'", \ cpu[cp].current_domain,cpu[(cp)].domain_uid,act,res); \ non_fatal(ERROR_NOPERM,errbuf,(cp)); \ return; \ } \ } The module should also send as detailed requests as it is possible, specyfying complete data needed for access verification. The module responsible for graphics should not ask for 'graph', but about 'graph/control/setmode' and the resource 'graph/res/640/480/16bpp'. Similarly, the rules in the configuration file access.hac should be as precise as possible. NOTE: for now, HAC supports wildcards in object path. These wildcards are supported in two ways. First of all, you can use them in access.hac to specify general rules. For example: 1 0 fs/ftp/users/*/mail fs/fops/list/directory allow In this case, HAC request matching domain, uid and operation will be approved if, at the time of HAC call, object path was either: fs/ftp/users/mike/mail, fs/ftp/users/david/mail/archive or so. NOTE: wildcards are not working accros path segments. This mean access attempt to fs/ftp/users/mike/private_files/mail will FAIL, because single '*' can substitute single path element only. /* ...there's no second way at the moment, sorry ;) ... */ Please note: take a look on fs module if you are going to include user-supplied data within object path in your own module. It is very important to parse it properly, elliminating unwanted wildcards etc! But, in fact, it is more important to avoid user/process-supplied strings in HAC calls, except for filesystem module.
The filesystem is defined in the file conf/fsconv.dat. It contains virtual filesystem mapping to the real filesystem, separated with spaces. The rules for inclusion are similar as in HAC: fs/ftp/test1 /Argante/fs/another directory fs/ftp /Argante/fs/ftp_server HAC controls access at the elvel of virtual directories. The above entries mean that the fs/ftp/test1 is mapped to another place than the directory fs/ftp. If a process has a HAC entry permitting operations like fs/create/directory on the object fs/ftp, it will have acces to both directories (according to the principles of HAC, provided that this has not been excluded earlier). When creating the directory fs/ftp/nope, the real entry will be created in /Argante/fs/tests/nope. On the other hand, the same operation for fs/ftp/test1/nope, will result in the file /Argante/fs/inny_katalog/nope. However, an attempt to access the object fs/ftp/../nope will fail - the filesystem module will recognise it as access to the object fs/nope, whereas such an entry doesn't exist in the SVFS hierarchy. The filesystem architecture in Argante presupposes resource access control and real filesystem protection, and at the same time the possibility of integrating the SVFS filesystem with objects of the real filesystem. The SVFS system is well simplified but fully functional subset of operations on the filesystem. In the original version it doesn't include support for symlinks andd hardlinks, however, it supports the ones existing at the real filesystem level. Including essential resources / system directories directly in the SVFS hierarchy is possible (e.g. making the /etc directory accessible), but discouraged. For more details on proper SVFS mapping, see Part II.
Applications developed under control of Argante Operating System are forced to use limited interaction and trust architecture. There is no way of forking, executing another binary image, or passing parameters directly to other processes. Also, enforced OSI model allows only two closest data processing layers to work together at the same time. It may look that this makes application development harder or at least less efficient, but it's not true. Whole interprocess communication is provided by IPC module that allows to write code wether or not each part of application is working on the same phisical system. This approach helps alot when developing distributed and fault-tolerant programs. Once written application, provided that it uses IPC, can be run on many system creating cluster-like structure, with request distribution and redundancy. IPC module allows processes to send short messages containing two 32bit words, create stream connections or use block devices. All this is based on limited trust architecture, so targets of each IPC request get all data about requestor and then decide to accept or deny it. On the other side, requestor gets full information about the process that accepted his request. Additionaly by using HAC access control, one can make application to work in OSI style, allowing only communication between nearest data processing layers. Request destination is specified by structure containing target vcpu number, virtual system number and ipc registered id. This allows to send requests that get to one process, many processes, one of group of processes, or even each process on one or each system connected to rIPC network. For example, we know that authentication processes had registered ipc id 100. By sending request to ipc id 100, and marking other address structure fields as unimportant we can be sure that this request gets to at least one authentication process. Which one will respond first, depends on system load, amount of authentication tasks on local system and in rIPC network. You may assume that the least busy process on the least busy system should answer request before the others do. Almost every IPC module syscall can be called in blocking or nonblocking maner, thus allowing to make server applications that must be connected to many other processes and switch context between them without any unnecesary delay. ArganteOS takes care about request queueing requests, creating accepted connections, and data exchange once request has been allowed. In nonblocking mode, process just has to check the status of sent request. In blocking mode process goes asleep until request is accepted by one or more processes or dropped by all targets. Possible applications of IPC system are ranging from simple process synchronization to distributed cluster-like web servers with load balancing.
Whenever I write "management scripts" or "operator" I don't mean a special, privileged superuser account, but a management console, controlled from the level of kernel-space. When the system boots up, starting scripts are executed (which can be used, among other things, to load modules and start rIPC session connections, as well as load processes). Remember: the console, in fact, is an unified "boot script interpreter" and debugger. It is not intended to be process working console. See section 17 for more details on process consoles. Management of the virtual system's operation is not performed from the level of tasks executed inside the system (at least, it isn't by default; you can always insert the connectivity layer between your VS programs and the console using unix socket daemon). Managing the work of the virtual system is not performed from the level of tasks executed inside the system Argante console offers quite a simple command set, used mostly for starting processes and library management. These commands are listed below: ? - help ! - system statistics $fn - load binary image from file fn and run it on the first VCPU available %fn - as above, loads a task in RESPAWN mode (it will be run again if the process will be terminated by any command different than HALT) NOTE: this mode is used for executing programs which should work all the time; in general, however, one should focus on proper functioning of the process and exception handling in all situations, and on creating redundant processes in the IPC hierarchy; this option should be an auxiliary solution. As a means of protection against abuse of this mode, there exists a variable MIN_CYCLES_TO_RESPAWN defined in the file config.h, which defines the minimal work cycle number before a situation leading to the death of a given process (32). If an error is encountered before, the program will not be restarted. >fn - load library from file fn to a free slot <id - remove library in slot 'id' # - list libraries with statistics (supported syscalls, number of calls) @fn - run a console script -nn - kill a process on VCPU number nn =nn - display statistics for a process on VCPU number nn . - system halt
A sample source file displaying numbers from 10 to 0 and making a bit of noise: -- !SIGNATURE "lcamtuf's test program" .DATA :Enter "\n" :Tekst " Hello world\n" :Die "Aghrrr... I die.\n\n" .CODE mov u0,:Enter mov u1,^Enter syscall $IO_PUTSTRING mov s0,0xa :Again mov u0,s0 syscall $IO_PUTINT mov u0,:Tekst mov u1,^Tekst syscall $IO_PUTSTRING twait 500000 loop :Again mov u0,:Die mov u1,^Die syscall $IO_PUTSTRING halt .END -- You can find different examples (*.agt files) in the subdirectory compiler/examples: apart from a similar "hello world", there is also an example of exception handling (error.agt) and filesystem management (fs.agt). The syntax of the language itself is as follows: .DATA, .CODE - definitions of subsequent segments (.data is optional). Thanks to bulba, you can switch between segments whenever you want :) .END - ends code segment :xxx - in the code segment, as well as in data segment: it refers to a symbolic name used to point to an object in the next line; it must occur in a separate line, in the data segment all objects must be named. Data may have the following format: "xxxx" - a sequence of characters 123 - an integer (32 bit) 123.0 - a float (32 bit) 0x123 - a hexadecimal value (32 bit) NN repeat 123 - a block of 123 repeats of the NN value (float or integer) block 100 - next 100 lines will contain values (dwords) to be entered into structures References to symbols passed as parameters must have the following form: ':Symbol'. Another possibilities are: - '^Symbol' - returns object length in bytes, useful for text strings - '%Symbol' - returns object length in dwords. !xxx - compilation directive, defines process parameters. Accepted values: !DOMAINS x x x - list of execution groups !PRIORITY x - program priority !IPCREG x - starting IPC identifier !INITDOMAIN x - starting execution group !SIGNATURE x - code signature (author, description) !INITUID x - initial subgroup identifier Defining syscalls with their symbols is acceptable, provided it is known to the compiler. The list of syscalls can be found in syscall.h in the modules/ directory. The syscall name has to be preceded by the $ sign, e.g.: 'syscall $io_putstring' (note: we omit the syscall_ suffix). In the same way you can refer to exception numbers: their names are in include/exception.h, and we omit error_. Oh, and priority '1' is the default value, although it is not reasonable. I suggest priorities ranging from 10 to 10000, as in that case in each cycle more machine operations are executed, and parsing them at once is more effective than subsequent jumps. Registers should be used in the format of "xNN", where NN is register number and x is one of the following: 'u' (ureg), 's' (sreg), 'f' (freg). For example, 'u0' refers to ureg[0]. If a numeric value, symbol or an 'u' register is preceded by '*', it refers to the value located at that address. For example: mov u0,*:Test will write to the register u0 the value from the address Test, whereas mov u0,:Test will write to the address pointed to the identifier 'Test' to the register u0. The compiler, at least in the current version, doesn't support arithmetics at compilation level. The system doesn't support many separate memory blocks assigned when the binary file is loaded. Compiler is run by typing "compiler/agtc plik.agt". As a result, you will receive a binary file plik.img, which can be loaded with the $ command from the management console.
AHLL translator has been written in a hurry. Our goal was to introduce a good high-level language for effective programming before releasing Argante 1.0. Unfortunately, writing good and usable HLL compiler, and, what's probably the most important, implementing a good language, is a complex task. I spent several sleepless nights working on it myself, and results are not shocking. Yes, you're able to write high-level programs in AHLL, but current implementation is far, far away from the thing I wanted to achieve. It will change in future AOS releases, but for now we cannot delay the first release just because AHLL is not perfect. So, the first thing you should ABSOLUTELY understand: AHLL is *NOT* AN INTEGRAL PART OF ARGANTE OS. What do I mean? Well, it's an example on how it can be done, and a useful tool, but nothing more. You can implement any other language (or, better, its reasonable subset) -- my favourite replacement for AHLL is a well-chosen subset of Ada. If you're interested in it, let me know. OK. AHLL code is really dirty, obfuscated and ugly. It's also buggy as hell, full of buffer overflows and so on - just deal with it. It should be and will be completely rewritten - now all I could do is to make it work. I cannot guarantee it will produce usable executables in all cases, but I hope so :))) Code generated by AHLL is highly ineffective, and there are several restrictions, like: - Recurrent procedure calls are deadly - YOU SHOULD NOT DO THAT FOR NOW; current version of AHLL is broken (and should be redesigned), so if you enter procedure A once, and then, without leaving it, call A again, and finally, this second call will be finished, you'll notice A parameters / locals were modified by this second call; that's because AHLL does NOT support dynamically allocated call parameters / local functions stack. If you really need to do that, use parameterless (or called in constant way) procedures, and implement simple dynamic allocation. In any other case, you should not use A's local parameters / variables after calling A within A ;> - When accessing structures and arrays, only following conventions are available: table[simple_variable], str_table[simple_variable].field, structure.field. So you cannot nest: table[table[table[n]].field], and you cannot directly access arrays inside structures (eg. str.field[nn]). If you need such access, you should use pointer assignments, eg: pointer_to_array_copy := str.field; ...and now you can access pointer_to_array_copy[nn]; - There's no complex arithmetics! Only one operator per expression. Also, there's no assign-when-calling-function-when-comparing-to... but I'm in doubt if such C conventions are good at all ;> - Floating point arithmetics is untouched. - There are no "helper" statements like for - you have while and loop instead, which are equivalent. I know it sucks, but I have no time to work on it right now. Please help us creating better HLL environment! Now, let's talk about Another-Hard-to-Learn-Language and its the precompiler... If you're familiar with C, you'll have no problem with understanding code constructions - but, in AHLL, you don't have, for example, C pointer arithmetics :> Following description is only a rough draft, but should be enough to catch the idea. AHLL is case insensitive. 1) Precompiler ~~~~~~~~~~~~~~ #include "filename" - this directive will include file at current position; there are some standard include files in hll/include directory. #define SYMBOL value - SYMBOL will be replaced with value - no macros are supported, unlike in C #compiler ... - following statement will be passed as-is to the AGTC RSIS compiler, useful for ! directives. #cstring name "value" - specific construction to make string initialization easier; it has been introduced due to weak AHLL implementation in AOS 1.0; it will be described later. 2) Type declarations ~~~~~~~~~~~~~~~~~~~~ Predefined types: unsigned, signed, float. You can declare a new type or create a subtype. While subtype is "usable" with other similar subtypes, types cannot be mixed without implicit conversion. This mechanism is more or less similar to Ada, except in AHLL shipped with AOS 1.0 it isn't really accurate ;P Type declaration: [sub]type new_type_name is base_type; (subtype can be applied only to simple types, not to arrays, structures and so on) Arrays: type new_type_name is array start .. stop of base_type; Bytechunks: type new_type_name is bytechunk start .. stop; Bytechunk is a packed array of bytes of specified length. For now, its fields cannot be accessed directly. Structures: type new_type_name is structure { field_name : [modifiers] type; field_name : [modifiers] type; } Possible modifiers: 'pointer to', 'addressable'. "Pointing" variables might be used in the same way as normal variables (there's no difference in calling method or so), except they're "mirroring" base objects, not having their own memory allocated. Only "addressable" variable can be assigned to pointer. For pointer examples, see Examples/AHLL/ptrs.ahl - pretty good. Complex type declarations are not allowed. For example, you cannot use: type new_type is array 1 .. 20 of structure { ... }; You have to split such declaration into two typedefs. Types can be declared only at a high-level (no local type declarations are allowed). 3) Variable declarations and initializers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Variable declarations may appear in global or local {} scope. In both cases, the format is as follows: var_name : [modifier] type_name [ := initializer ]; For modifiers, see paragraph 2. An initializer may be an immediate value, a string, or a complex initializer. Initializers are not allowed for pointers. Complex initializer contains list of array / structure fields. No nested initializers are allowed (so you cannot initialize array of structures at once): var_name : type_name := { value_for_field1, value_for_field2, // List MUST end with ','. } To skip complex types / pointers in structure initializers, use keyword "none". 4) Procedures ~~~~~~~~~~~~~ There are no functions in Argante. A procedure can accept any number of parameters, and then modify these marked as "writable" - which are used as output data: procedure ProcedureName ( [writable] param1 : param1_type, [writable] param2 : param2_type ... ) { local { // Local declarations } // Local code exception { // Exception code } } There's special, parameterless procedure, called Main, which is executed at the beginning (entry point). It has to be present in every program. Procedure calling within local code can be done as follows: ProcedureName ( [modifier] param1, [mod] param2, [mod] param3 ); Allowed modifiers: convert - implicit conversion of types address - address of specific variable 5) Exception handling ~~~~~~~~~~~~~~~~~~~~~ exception {} block is called if exception occurs within guard {} block in local code: guard { // Some commands... } // Other commands... exception { // Handler } Only an exception in guarded code will cause execution of exception {} block. In this block, you have to use "case" commands to handle specific exception numbers (see switch {} block). The differences between exception {} code and normal code: ignore - this command will return to the point where exception happened; not smart. 'return' should be used to return to calling function, 'raise NN' should be used to pass the exception to high-level handlers (declared before calling this procedure). Exception will be passed after reaching the end of exception {} block as well. AHLL generates some exception code while doing range-checking and pointer validation. 6) Conditional statements, loops, gotos ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ if [not] one_value { ... } - executed if non-zero/zero if one_item = other_item { ... } - complex comparison if one_val op other_val { ... } - simple op ( <, >, =, <>, &, <=, >=). while CONDITION { ... } - where CONDITION can be the same as in "if" - repeat code while... loop CONDITION { ... } - like "while", but check is done at the end of every pass. continue; - jump to the check condition of the loop break; - exit from current loop switch simple_value { - well, you should know; there's no need case val; to 'break' before every next case. // Code case other_val; // Code case default; // Code } label: - local jumps goto label; 7) Assignments and arithmetics ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ x := y; - copy (can be used on complex types) x [op]= y; - arithmetics (+, -, |, &, %, *, ^, /, ~) create x; - assign new variable to pointer x := bind y; - bind pointer to addressable variable destroy x; - destroy dynamically created variable unbind x; - unbind pointer x := {address|convert} y - see functions. 8) Syscalls ~~~~~~~~~~~ To do syscall, you should use following notation syscall ( SYSCALL_NAME, - should be #defined u0 := some_value, u1 := other_value, // ... - parameters to put in registers something_writable := s0, // ... - return values ); You can find some AHLL examples in Examples/AHLL subdirectory. To compile, you should change your working directory to hll/, and then, use ./acc or ./ahlt (ahlt will produce .agt file from .ahl file - so in fact it's the main translator; ./acc will produce .img from .ahl file - so full compilation will be done). On systems with /bin/bash, you can run ./acc with -e option (eg: ./acc -e examples/dir.ahl) to perform dead-code elimination. Another example you can find there is Mini-HTTP server. It's trivial, but effective tool :)
Here is the list of syscalls supported by currently available modules. These syscalls have corresponding AHLL library calls, so this knowledge isn't mandatory, but might explain some troubleshooting problems. Also, as there's no AHLL library calls documentation available at the moment, it might be helpful in understanding AHLL procedures. Module display.c --------------- Status: done Purpose: displaying basic data on the console from user within a process; debugging etc. NOTE: the module shouldn't be used for user interaction, as this will be solved otherwise. For the time being, it is advised to use the network module. Syscall: IO_PUTSTRING Parameters: u0 - character string address u1 - number of characters Result: displaying character strings Exceptions: BAD_PROTFAULT - attempt to display wrong memory fragment HAC: operation=display/output/text object=none Syscall: IO_PUTINT Parameters: u0 - value to be displayed Result: displays numeric values Exceptions: - HAC: operation=display/output/integer object=none Syscall: IO_PUTFLOAT Parameters: f0 - value to be displayed Result: displays numeric values Exceptions: - HAC: operation=display/output/float object=none Syscall: IO_PUTCHAR Parameters: u0 (lowest 8 bits) - character to be displayed Result: displays an ascii character Exceptions: - HAC: operation=display/output/character object=none Module access.c -------------- Status: done Purpose: privilege management; active domain (group) and subgroup identifier change. HAC system support. Access control: none Syscall: ACCESS_SETDOMAIN Parameters: u0 - group number Result: active group change Condition: group belongs to the !domains set, defined during compile time Exceptions: NOPERM - if the group doesn't belong to the set mentioned above. HAC: unsupported Syscall: ACCESS_SETUID Parameters: u0 - subgroup identifier number Result: changes active subgroup Condition: none Exceptions: none HAC: unsupported Module fs.c ---------- Status: done Purpose: access to SVFS. Access control: HAC + existing objects in the SVFS hierarchy. Exceptions: standard + FSERROR - SVFS resources access error. Syscall: FS_OPEN_FILE Parameters: u0 - filename address, u1 - filename length u2 - flags: FS_FLAG_READ, FS_FLAG_WRITE, FS_FLAG_APPEND, FS_FLAG_NONBLOCK Result: opens given file in the given mode: s0 - VFD (virtual file descriptor); -1 = locked file Note: if the flag NONBLOCK is not given and an attempt to open the file for writing and at the same time another process is writing data to the file, process enters the state IOWAIT until it receives access; NONBLOCK causes immediate return of -1. HAC: fs/fops/open/file/{read|write|append} Syscall: FS_CREATE_FILE Parameters: u0 - filename address, u1 - filename length u2 - flags: FS_FLAG_WRITE, FS_FLAG_APPEND Result: creates a given file in appropriate mode returns s0 - VFD (virtual file descriptor) HAC: fs/fops/create/file/{write|append} Syscall: FS_CLOSE_FILE Parameters: u0 - VFD number Result: closes VFD , ends working with the file. If file is open for writing, it is truncated at current offset. HAC: none Syscall: FS_WRITE_FILE Parameters: u0 - VFD number, u1 - pointer, u2 - length (bytes!) Result: writing data to the file if access rules to VFD and memory permit it HAC: none Syscall: FS_READ_FILE Parameters: u0 - VFD number, u1 - pointer, u2 - length (bytes!) Result: reads data from file to memory, if access rules to VFD and memory permit it HAC: none Syscall: FS_SEEK_FILE Parameters: u0 - number VFD, u1 - position, u2 - typ Result: syntax analogous to lseek() in libc. With files in the append() mode, only u1=0, u2=1 (current) is accepted (returns the current position). s0 - position. HAC: none Syscall: FS_MAKE_DIR Parameters: u0 - name, u1 - name length Result: directory creation ;) HAC: fs/fops/create/directory Syscall: FS_DELETE Parameters: u0 - name, u1 - name length Result: removes file or directory HAC: fs/fops/delete/{directory|file} Syscall: FS_RENAME Parameters: u0 - name, u1 - name length u2 - new name, u3 - new name length Result: filename or directory name change HAC: fs/fops/delete/{directory|file} for old name fs/fops/create/{directory|file} form new name Syscall: FS_PWD Parameters: u0 - buffer, u1 - buffer size Result: writes current directory to buffer, writes to s0 actual name length HAC: bnone Syscall: FS_CWD Parameters: u0 - buffer, u1 - buffer name Result: current working directory change (doesn't verify whether the directory exists!) HAC: none Syscall: FS_END_DIR Parameters: - Result: directory cache allocated for FS_LIST_DIR is freed. HAC: none Syscall: FS_LIST_DIR Parameters: u0 - start new session (0 - no, 1 - yes) New session parameters: u1 - address to directory name, u2 - len New session return: s0 - number of directory entries Existing session params: u3 - directory entry number, u1 - buffer, u2 - buffer size Existing session return: u1 - entry name, u2 - entry len, s0 - entries left HAC: object=directory, oper=fs/fops/list/directory Note: this syscall is designed to operate on frozen image of requested directory, to avoid dir-scanning races (eg. hiding files or so). It's good to free allocated cache after finishing. It will be automatically freed if new session starts Syscall: FS_STAT Parameters: u0 - resource name, u1 - name length Result: u0 - last modification time u1 - 0 = no access to resource 1 = resource is a file 2 = resource is a directory u2 - file size HAC: fs/fops/stat locallib.c module ---------------- Status: being implemented Purpose: system resource access Access control: HAC Exceptions: standard Syscall: LOCAL_GETTIME Result: u0 - seconds, u1 - microseconds HAC: local/sys/real/time/get Syscall: LOCAL_TIMETOSTR Parameters: u0 - returned by GETTIME, u1 - buffer address, u2 - buffer size Result: writes string to buffer, s0 returns number of characters HAC: none Syscall: LOCAL_GETHOSTNAME Parameters: u0 - buffer address, u1 - buffer size Result: writes local computer name to buffer, s0 - number of characters HAC: local/sys/real/hostname/get Syscall: SYSCALL_GETRANDOM Result: u0 - random dword; function gets dword from a local entropy source (/dev/urandom) HAC: local/sys/random/get Syscall: SYSCALL_LOCAL_VS_STAT Result: u0 - number of active VCPUs; u1 - number of idle cycles from start, u2 - number of work cycles from start, u3 - number of syscalls, u4 - number of wrong syscalls, u5 - fatal errors... HAC: local/sys/virtual/stat Syscall: SYSCALL_LOCAL_RS_STAT Result: u0 - uptime in seconds, u1 - load average (1 min), u2 - RAM size in kB, u3 - free RAM in kB, u4 - swap size in kB, u5 - free swap in kB, u6 - number of RS processes HAC: local/sys/real/stat Here's documentation of network module syscalls written by Marcin: I: input parameters O: return values Note: All parameters to network syscalls must be given in host byte order. LLX section describes only exceptions specific to network module. NET_CONNECT params: I: u0 - destination address ($IP) u1 - destination port ($PORT) i u2 - source address (0 - default) ($IP) u3 - source port (0 - ephemeral) ($PORT) u4 - time limit (usecs, 0 - native OS dependent) u5 - TCP/UDP switch (0/1) O: s0 - new descriptor (connected socket) effect: connects to inet socket (TCP/UDP) LLX : NET_PORT_OOR - destination/source port > 65535 NET_SOCK - can't create new native socket NET_NO_FREE_SD - no free socket descriptors NET_NONBLOCK - can't make native socket non-blocking NET_BIND - can't bind native socket NET_TIMEO - time limit exceeded NETERROR - some kind of other internal error ... HAC : oper: net/sock/connect obj : net/address/{tcp,udp}/{source,dest},{$IP,default}/$PORT notes : If timeout is set then u3 is overwritten by syscall. NET_SUN_CONNECT params: I: u0 - destination process number ($PID) u1 - destination socket ID ($SID) u4 - time limit u5 - stream/datagram switch (0/1) O: s0 - new descriptor (connected socket) effect: connects to other process via a Unix socket LLX : the same as for NET_CONNECT except NET_PORT_OOR, NET_BIND HAC : oper: net/sock/connect obj : net/address/dest/unix/{$PID,external}/$SID notes : If u0 is excessive (>65535), external program is assumed. NET_LISTEN params: I: u0 - local address (0 - all) ($IP) u1 - local port (0 - ephemeral) ($PORT) u2 - backlog (TCP only) u5 - TCP/UDP (0/1) O: s0 - new descriptor (listening socket) effect: creates listening socket LLX : NET_PORT_OOR - local port >65535 NET_BAD_BLOG - backlog too high (default: >5) NET_SOCK - can't create new native socket NET_NO_FREE_SD - no free socket descriptors NET_NONBLOCK - can't make native socket non-blocking NETERROR - internal error HAC : oper: net/sock/listen obj : net/address/source/{tcp,udp}/{$IP,all}/$PORT notes : NET_SUN_LISTEN params: I: u1 - socket ID ($SID) u2 - backlog (stream only) u5 - stream/datagram (0/1) O: s0 - new descriptor (listening socket) effect: creates a listening Unix socket LLX : the same as for NET_LISTEN except NET_PORT_OOR HAC : oper: net/sock/listen obj : net/address/source/unix/{dgram,stream}/self/$SID notes : NET_ACCEPT params: I: u0 - descriptor (listening socket) u4 - blocking/non-blocking (1/0) O: s0 - new descriptor (connected socket) s1 - return code (1 - accepted, 0 - would block) effect: accepts next client on listening socket LLX : NET_BAD_SD - invalid (unused ? too high ?) descriptor NET_SOCK_NON_LISTEN - descriptor points to non-listening socket NET_NO_FREE_SD - no free socket descriptors NET_NONBLOCK - can't make native socket non-blocking NETERROR - internal error HAC : notes : Return code is used only when syscall is called non-blocking. NET_RECV params: I: u0 - descriptor (connected socket) u1 - data buffer address u2 - buffer length (bytes) u4 - blocking/non-blocking (1/0) O: s0 - bytes received s1 - return code (1 - received, 0 - would block) effect: Receives data through connection. LLX : NET_BAD_SD - invalid (unused ? too high ?) descriptor NET_EPIPE - broken pipe NET_EOF - remote party disconnected (TCP/stream only) NET_SOCK_NOT_CONN - descriptor points to not connected socket PROTFAULT - can't access buffer for writing NETERROR - internal error HAC : notes : Return code is used only when syscall is called non-blocking. NET_SEND params: I: u0 - descriptor (connected socket) u1 - data buffer address u2 - buffer length (bytes) u4 - blocking/non-blocking (1/0) O: s0 - bytes sent s1 - return code (1 - sent, 0 - would block) s2 - if s1 == 0, bytes still waiting to be sent effect: Sends data through connection. LLX : The same as for NET_RECV except that PROTFAULT means no perms for reading from buffer. HAC : notes : Return code is used only when syscall is called non-blocking. This syscall, unlike in most Unices, returns s1 == 1 when working in non-blocking mode ONLY if whole data has been send. Otherwise, amount of bytes waiting to be sent is put in s2. NET_SHUTDOWN params: I: u0 - descriptor u1 - how effect: closes opened connection LLX : NET_BAD_SD - invalid (unused ? too high ?) descriptor NET_BAD_HOW - invalid 'how' parameter NETERROR - internal error HAC : notes : NET_ISWAITING params: I: u0 - descriptor O: s0 - result (1 - client, 0 - no clients) effect: checks if there are pending connections to listening socket LLX : NET_BAD_SD - invalid (unused ? too high ?) descriptor NET_SOCK_NON_LISTEN - descriptor points to non-listening socket NETERROR - internal error HAC : notes : this function is non-blocking of course ;P => And here's documentation of advmem module by z33d Module advmem.c -------------- Status: almost done Purpose: Advanced memory operations Access control: none Exceptions: standard + ERROR_MEM_FORMAT - when given string isn't convertible ERROR_MEM_OFFSET - when offset < 0 Note: Offset may be bigger than 0..3, then base address (of dwords) is suitably increased. Syscall: SYSCALL_MEM_STRCPY u0 - destination address u1 - offset of destination (0..3 in addressed dword) u2 - source address u3 - offset of source u4 - size in bytes Syscall: SYSCALL_MEM_MEMSET u0 - address u1 - offset u2 - character u3 - size in bytes Syscall: SYSCALL_MEM_BZERO u0 - address u1 - offset u2 - size in bytes Syscall: SYSCALL_MEM_ENDIAN u0 - address u1 - bytelength u2 - current format u3 - expected format (0 - big endian, 1 - little endian, 2 - native endian) Syscall: SYSCALL_MEM_STRCHR u0 - address u1 - offset u2 - character u3 - size in bytes RETURN: u0 - address of matched dword u1 - offset in this dword u2 - 0 if NOT succeed Syscall: SYSCALL_MEM_STRRCHR u0 - address u1 - offset u2 - character u3 - size in bytes RETURN: u0 - address of matched dword u1 - offset in this dword u2 - 0 if NOT succeed Syscall: SYSCALL_MEM_STRCMP u0 - address of 1st string u1 - offset of 1st string u2 - address of 2nd string u3 - offset of 2nd string u4 - size in bytes RETURN: u0 - like strcmp from libc Syscall: SYSCALL_MEM_STRCASECMP u0 - address of 1st string u1 - offset of 1st string u2 - address of 2nd string u3 - offset of 2nd string u4 - size in bytes RETURN: u0 - like strcmp from libc Syscall: SYSCALL_MEM_STRSTR u0 - address u1 - offset u2 - size of 1st string u3 - addres u4 - offset u5 - size of 2nd string RETURN: u0 - address u1 - offset u2 - 0 if NOT succeed Syscall: SYSCALL_MEM_STRRSTR u0 - address u1 - offset u2 - size of 1st string u3 - addres u4 - offset u5 - size of 2nd string RETURN: u0 - address u1 - offset u2 - 0 if NOT succeed Syscall: SYSCALL_MEM_TOUPPER u0 - address u1 - offset u2 - size Syscall: SYSCALL_MEM_TOLOWER u0 - address u1 - offset u2 - size Syscall: SYSCALL_MEM_STRTOINT (converts string to integer) u0 - address u1 - offset u2 - size in bytes RETURN: u0 - integer or exception (ERROR_MEM_FORMAT) Syscall: SYSCALL_MEM_STRTOHEX (converts string to unsigned int) u0 - address u1 - offset u2 - size in bytes RETURN: s0 - value or exception (ERROR_MEM_FORMAT) Syscall: SYSCALL_MEM_STRTOFLOAT (converts string to float) u0 - address u1 - offset u2 - size in bytes RETURN: f0 - float or exception (ERROR_MEM_FORMAT) Syscall: SYSCALL_MEM_STRHEXINT (string may begin with '0x' or ...) u0 - address u1 - offset u2 - size in bytes RETURN: s0 - unsigned integer or exception (ERROR_MEM_FORMAT) Syscall: SYSCALL_MEM_HEXTOSTR (like sprintf("%x" ... in libc) u0 - address u1 - offset u2 - size of buffor s0 - value to convert RETURN: s0 - number of written bytes Syscall: SYSCALL_MEM_INTTOSTR (like sprintf("%d" ... in libc) u0 - address u1 - offset u2 - size of buffor u3 - value to convert RETURN: s0 - number of written bytes Syscall: SYSCALL_MEM_FLOATTOSTR (like sprintf("%f" ... in libc) u0 - address u1 - offset u2 - size of buffor f0 - value to convert RETURN: s0 - number of written bytes Module math.c -------------- Status: under development Purpose: Mathematical routines Access control: none Exceptions: standard + ERROR_MATH_RANGE - arc-function range checking (-1 .. 1) ERROR_MATH_DIV - math_table_div: division by zero ERROR_MEM_FORMAT - math_table_*: unsigned char conversion isn't implemented Note: When function uses cache returned values may be inaccurate. Syscall: SYSCALL_MATH_SIN u0 - type (0 - noncached, 1 - cached value) f0 - value in radians RETURN: f0 - sine of given value Syscall: SYSCALL_MATH_COS u0 - type (0 - noncached, 1 - cached value) f0 - value in radians RETURN: f0 - cosine of given value Syscall: SYSCALL_MATH_TAN u0 - type (0 - noncached, 1 - cached value) f0 - value in radians RETURN: f0 - tangent of given value Syscall: SYSCALL_MATH_ASIN u0 - type (0 - noncached, 1 - cached value) f0 - value (-1..1) RETURN: f0 - arcsine of given value or ERROR_MATH_RANGE exception Syscall: SYSCALL_MATH_ACOS u0 - type (0 - noncached, 1 - cached value) f0 - value (-1..1) RETURN: f0 - arccosine of given value or ERROR_MATH_RANGE exception Syscall: SYSCALL_MATH_ATAN u0 - type (0 - noncached, 1 - cached value) f0 - value (-1..1) - strange ;> RETURN: f0 - arctangent of given value or ERROR_MATH_RANGE exception Syscall: SYSCALL_MATH_FILLSIN u0 - address of buffer s0 - count of sines to write (in dwords or in bytes when u2 == 2) f0 - first value f1 - 'step' u1 - type (0-noncached, 1-cached) u2 - type of results (0-int, 1-float, 2-unsigned char) u3 - value to multiply with results (0 is like 1) RETURN: Table of sine values Syscall: SYSCALL_MATH_FILLCOS u0 - address of buffer s0 - count of cosines to write (in dwords or in bytes when u2 == 2) f0 - first value f1 - 'step' u1 - type (0-noncached, 1-cached) u2 - type of results (0-int, 1-float, 2-unsigned char) u3 - value to multiply with results (0 is like 1) RETURN: Table of cosine values Syscall: SYSCALL_MATH_FILLTAN u0 - address of buffer s0 - count of tangents to write (in dwords or in bytes when u2 == 2) f0 - first value f1 - 'step' u1 - type (0-noncached, 1-cached) u2 - type of results (0-int, 1-float, 2-unsigned char) u3 - value to multiply with results (0 is like 1) RETURN: Table of tangent values Syscall: SYSCALL_MATH_TABLE_MUL u0 - address of table with values to multiply u1 - size of this table (dword - float and int, byte - unsigned char) u2 - type of values in first table (0 - int, 1 - float, 2 - unsigned char) u3 - address of second table u4 - size u5 - type (0 - int, 1 - float, 2 - unsigned char) u6 - type of results (0 - int, 1 - float, 2 - unsigned char) u7 - value to multiply with results (it's used during float to int conversion, fast operations ... only without float) Ofcourse 0 is like 1. Syscall: SYSCALL_MATH_TABLE_DIV u0 - address of table with values to division u1 - size of this table (dword - float and int, byte - unsigned char) u2 - type of values in first table (0 - int, 1 - float, 2 - unsigned char) u3 - address of second table u4 - size u5 - type (0 - int, 1 - float, 2 - unsigned char) u6 - type of results (0 - int, 1 - float, 2 - unsigned char) u7 - value to multiply with results (it's used during float to int conversion, fast operations ... only without float) Division by zero will call ERROR_MATH_DIV exception Syscall: SYSCALL_MATH_TABLE_ADD u0 - address of table with values to addition u1 - size of this table (dword - float and int, byte - unsigned char) u2 - type of values in first table (0 - int, 1 - float, 2 - unsigned char) u3 - address of second table u4 - size u5 - type (0 - int, 1 - float, 2 - unsigned char) u6 - type of results (0 - int, 1 - float, 2 - unsigned char) u7 - value to multiply with results (it's used during float to int conversion, fast operations ... only without float) Syscall: SYSCALL_MATH_TABLE_SUB u0 - address of table with values to subtract u1 - size of this table (dword - float and int, byte - unsigned char) u2 - type of values in first table (0 - int, 1 - float, 2 - unsigned char) u3 - address of second table u4 - size u5 - type (0 - int, 1 - float, 2 - unsigned char) u6 - type of results (0 - int, 1 - float, 2 - unsigned char) u7 - value to multiply with results (it's used during float to int conversion, fast operations ... only without float) -- END OF DESCRIPTION -- Module: packet.c (beta test version by bikappa) ----------------------------------------------- This module can be used for all low-level networking purposes, including: sniffing / packet analysis, packet sending, packet forwarding / firewalling etc. Syscall: SYSCALL_LOW_NET_INITDEV Initializes RAW listener (sniffer) socket Parameters: u0 / u1 - interface name address / len Returns: s0 - socket number HAC: operation=net/raw/open/listener, object=net/dev/phys/IFACE_NAME Syscall: SYSCALL_LOW_NET_RAW; Initializes RAW sender socket Parameters: none Returns: s0 - socket number HAC: operation=net/raw/open/sender Syscall: SYSCALL_LOW_NET_RECV; Reads RAW packet thru listener socket Parameters: u0 - socket number, u1 / u2 - packet buffer address / len Returns: s1 == 1 - success, data received (s0 - packet length) s1 == 0 - failure, no data present No blocking low_net_send for now Syscall: SYSCALL_LOW_NET_SEND; Sends RAW packet thru sender socket Parameters: u0 - socket number, u1 / u2 - packet data address / len Returns: s1 == 1 - success, all data sent s1 == 0 - failure, s2 - data left (not sent) No blocking low_net_send for now Syscall: SYSCALL_LOW_NET_CLOSE; Closes listener or sender socket Parameters: u0 - socket number No return, no HAC. Syscall: SYSCALL_LOW_NET_GETHWADDR Parameters: u0, u1 - interface name Return: u0:u1:u2:u3:u4:u5 - hardware address HAC: net/raw/hwaddr/get on net/dev/phys/<iface> Exceptions: standard memory access, HAC, internal error (if unable to create temp socket), ERROR_BAD_SYS_PARAM (unknown interface) -- --- This is a gfx.so module documentation - ask honey for more details, that's all I have: GFX MODULE MANUAL version 0.000000000000000[...]01 (it's one "zero" less than in 1st version of manual) manual (and, by the way, GFX module) was written by Lukasz Jachowicz <honey@linux.net.pl> Hi, I've just finished writing the Very First^H^H^H^H^HSecond version of a graph module for Argante OS. I know it's not ideal but I already have some nice ideas and I'll code them ASAP. At the moment you can use some functions described below. SYSCALL_GFX_MODE Inits svgalib in case it wasn't inited before. Then sets current videomode to u0. Returns nonzero value in u1 in case of problems, so your software can react in a way you want it to react. But - remember - you won't be able to set the mode that is DENIED or unavaiable on your graphics hardware - the program will stop with an error message ("I can't use this mode")... You can find list of avaiable modes (and their numbers) in Argante/hll/include/gfx.hll. SYSCALL_GFX_CHECKMODE Send the mode you want to check to u0 and then call this function. u0 will tell you if the mode is (nonzero) or is not (u0=0) supported by your hardware and allowed by Argante's HAC SYSCALL_GFX_CLEARSCREEN If you want to clear the mess on your screen - this function is for you. SYSCALL_GFX_MEMCOPY The most important function in this library. Copies some amount (set it in u1) of data from *u0 and sends directly to your graphic card memory... So if you were a asm-coder on demoscene, you're at home... SYSCALL_GFX_SETPALETTESNGL Let's assume you want to change the background color from black to white. Let's assume your background is filled with color nr "0". What do you do? You just put the color's value to u0, and the new rgb to (u1,u2,u3) and... done :) Oh, don't forget to call this #$#@$ function ;) SYSCALL_GFX_SETPALETTE Using the function presented above for every color avaiable could be a little boring. So prepare the table with (r,g,b,r,g,b,r,g,b,...) values for some colors, decide, which color's number is the 1st one to be changed (put its value to u1), put number of colors in the table to u2, put the pointer to this magic place in memory to u0 and call this function. Done. SYSCALL_GFX_SETCLUT8 I don't know why, but svga lib uses just 4 bits/pixel when you change a value in color palette. Call this function to change it to 8 bits/pixel... SYSCALL_GFX_VC If you want to stop people from moving to another virtual console, just insert zero to u0 and syscall $GFX_VC. To allow 'em changing - put any nonzero value to u0 and call this function. -- EOF -- NOTE: every module using HAC may return, besides standard exceptions, values: ACL_PROBLEM, NOPERM. For rIPC API documentation, see section 9.
In the current implementation modules are dynamically linked programs written in C or ADA. Requirements are as follows: - there must be syscall_load(int* x); this function is called when the module is loaded; its duty is to fill the table of x values with syscall numbers it will support; the values of these syscalls are to be found in the syscall.h file (of course, new functions should have new ones, added to syscall.h). The list cannot exceed MAX_SERVE from the file config.h and must end with a negative value. - another function required is syscall_handler(int c,int sysnum) - it will be called if VCPU with the number 'c' calls syscall with the number found in the list registered for this module (the actual number is given in sysnum). The value 'c' permits referring to the structure vcpu_struct declared in task.h (see this file for details). - optionally, there could be syscall_unload, executed when syscall terminates - optionally, there could be syscall_task_cleanup, executed whenever any task terminates (removing open descriptors etc.). I could go on with describing module construction, so I will just paste a sample one, supporting primitive console output: -- void syscall_load(int* x) { *x=SYSCALL_IO_PUTSTRING; *(++x)=SYSCALL_ENDLIST; printk("<< Welcome to I/O module >>\n"); } void syscall_handler(int c,int num) { int cnt; int from; char* start; if (num==1) { from=cpu[c].uregs[0]; cnt=cpu[c].uregs[1]; start=verify_access(c,from,(cnt+3)/4,MEM_FLAG_READ); if (!start) { non_fatal(ERROR_PROTFAULT,"Can't print non-accessible memory",c); return; } write(2,start,cnt); } } Function non_fatal is used for reporting exceptions. Library exchange consists in loading a new one into any free slot, and the unloading the old one from its slot. Syscall management will be uninterrupted. And no, syscalls CANNOT block the system, exactly as it is e.g. in Linux. Therefore, when it is necessary to wait for an operation (like recv()), it is recommended to set process state (cpu[nn].state) adding the flag VCPU_STATE_IOWAIT and at the same time setting cpu[nn].iohandler so that it points to the function accepting a single parameter (the number of the VCPU): int handler(int cpu_num). Additionally, the field cpu[nn].iowait_id could be used to define the identifier of the resource the process is waiting for.You could use it, but don't have to. >From this point on, the process won't work (the situation is analogous to STATE_SLEEP). Instead, in every cycle of serving tasks, the function iohandler(numer_cpu) will be called. The function should check the number of the resource the task is waiting for. If it is not accessible, it should return 0. If it is accessible, the function should manage the results appropriately and return a non-zero value (e.g. 1) to automatically leave the state IOWAIT. A given module should itself take care of storing information concerning where to write return information for a given task, etc. To enter the IOWAIT state, the safest way is to use the macro: ENTER_IOWAIT(cpu_number,resource_number,iohandler) You should remember not to pass nor take from the process "raw" objects from the real system, like file descriptor numbers, nor to leave access control to the system (e.g. attempting writing to a file and then checking for success). Argante ensures full control on its own side in a unified way, whereas all "real" objects are stored in tables separate for every processor, giving the process at most an identifier within these tables. The best example of a correct module construction is the fs module. This is a short description of string management philosophy at a low level (which will probably be of no interest to an AHLL programmer, but is essential when creating modules), which I wrote for Artur: [...] Oh, but gethostbyname is a rather good example. In general we do it in this way: - user passes us buffer address and its size (in, say, registers u0 i u1). - we check whether the user is authorised to perform a given operation - in your case it is sufficient to use the macro VALIDATE(c,"none", "local/sys/real/uname/get"); the macro will "return" itself if the user isn't authorised to access the object. - You have to check whether the address given by the user is writable at all its length: if not, naturally we cannot process its syscall and report an exception: if (!(sth=verify_access(c,cpu[c].uregs[0],(cpu[c].uregs[1]+3)/4, MEM_FLAG_WRITE))) { non_fatal(ERROR_PROTFAULT,"gethostname: Attempt to access protected" " memory",c); failure=1; return; } verify_access accepts the following parameters: VCPU number, address, size (but note it is in dwords, so we have to recalculate the size given in bytes; as the operator '/' in c on ints is simply idiv ignoring the modulo, we make sure we catch a case like: the user says we can write one byte, 1/4 according idiv = 0, so we check 0 bytes ;-), or as well as access type (READ or WRITE). The function returns either a pointer (already in the real system, normal void*) or NULL: it means it is not authorised to access the block and we should raise an exception, set failure (a convention, for my own comfort, it was justified somehow ;) and stop any further work. - OK, success, let's assume we have the pointer already, so we take what we need, write max.uregs[1] bytes to the address returned by verify_access, and the we return (say in s0) the number of characters taken. NOTE: we don't copy nor count the NULL-terminator which is a normal sign in Argante. Therefore we don't do things like: strncpy(sth,some_buffer,cpu[c].uregs[1]); but instead: memcpy(sth,some_buffer,strlen(some_buffer)) and we return strlen(some_buffer) w s0. Oh, but we have to check earlier whether strlen(some_buffer)>cpu[c].uregs[1] (i.e. whether we wan to write more than is needed) and possibly we should report an exception. That's enough about strings from the point of view of kernel-space. No, there are no plans for strings hard-linked to byte/word/dword referring to their length: it will be a matter of taste and implementation in HLL, but the information is passed to the kernel loosely :P More trouble with strings has only poor z33d who has to introduce a new value for certain operations ;) i.e. either to return the offset in bytes or the address argante + 0..3 of the offset ;> But it's not a big problem, either. [ z33d was writing the module advmem, responsible for, among others, concatenating / searching texts, etc]
Executable file header format is described below: unsigned int magic1; COnstant file signature, has the value of 0xdefaced char domains[MAX_EXEC_DOMAINS]; List of domains the program belongs to, ends with 0. unsigned int flags; Starting process flags. Currently no flags are supported. unsigned int priority; Priority defines how long a timeslice is assigned to a process in every processing cycle. Priority of 1 means that every time the process can execute one instruction. unsigned int ipc_reg; Starting IPC identifier IPC. If the value is greater than 0, it will be rewritten to VCPU. unsigned int init_IP; Starting instruction pointer, usually it is enough to assign it 0. int current_domain; int domain_uid; Current execution domain and UID. Honoured only when greater than 0. unsigned int bytesize; Code image size. unsigned int memflags; Memory flags (READ|WRITE, etc)... unsigned int datasize; Data image size. char signature[64]; Author's signature / short description of the program (optional). unsigned int magic2; Constant signature 0xdeadbeef What follows next is the code image block (size: 12*bytesize) and data block (optional, size = 4*datasize). Both blocks are mapped from the address 0 respectively onto the space of code and data. Bulba's program (tools/binedit.c) is used to modify the content of existing program headers. ---------------------- 16. Built-in debugger ---------------------- Author: z33d No compromises -- OK, I won't use Polish diacritic marks [impossible to render in English, anyway -- translator]. I have implemented a limited but fully functional interactive debugging system. Something like gdb. In order to initialise process debugging, it should be loaded with the 'd' command. As a result, the VCPU_FLAG_DEBUG flag, indispensable for debugging purposes, will be set. An excerpt from help: ('?') dfn - load and run binary in debug mode rnn - show nn vCPU registers xnn addr c - show c bytes of memory on nn vCPU nnn - step exactly one instruction of nn vCPU cnn - continue process on nn vCPU snn - continue process on nn vCPU to next syscall fnn - continue process on nn vCPU to next ret lnn - list breakpoints on nn vCPU bnn zz - add breakpoint on nn vCPU at zz IP unn zz - delete zz breakpoint on nn vCPU inn IP c - disassemble c instructions at IP on VCPU nn tnn - show stack trace on nn vCPU These commands should be clear; moreover, every exception raising (even intercepted) causes process execution to stop (VCPU_STATE_STOPPED). Besides the debugger, z33d has also written a disassembler, to be found in the tools/ directory. Although unfinished, it does well what it is supposed to do.
Argante has very poor display.so module. Well, in fact, it is provided for console-based debugging purposes only. Argante, while running in background, provides no access to console. But this does NOT mean your process cannot use fully-featured terminals, like local console, screen window, telnet terminal, xterm terminal etc. It can. Console support is provided by vcpucons utility, which can be found in tools/ subdirectory. It can be used in really simple manner. Let's consider some examples: - you want to access process console by hand - then all you have to do is to run vcpucons; using 'exec vcpucons' you can eg. replace current screen window with process console etc. - you want to access process console permanently, replacing one of the local consoles - then you have to launch vcpucons from /etc/inittab instead of mingetty session. - you want to launch process console instead of login shell for remote user (after local authorization) - well, all you have to do is to use vcpucons instead of login shell in /etc/passwd, - you want to launch process console instead of authorization eg. from in.telnetd - use in.telnetd -L /path/to/vcpucons in inetd.conf. - etc - no limitations at all. Ok, but how the VCPU is supposed to handle such requests? How can you write something on the console? It's simple! Consoles, from the programmer's point of view, are working extactly the same way as unix sockets are. Process should create listening, stream mode unix socket and wait for the connection. When vcpucons is invoked with specific parameters, it tries to connect to given unix socket and enters proxy mode. In this mode, all data read from the terminal is transferred to the process (and can be received as if it comes from network) and all data sent by process is put on the terminal. This link works in char-by-char mode, preserving all terminal control codes, until: a) process will close the connection endpoint b) process will exit / die c) vcpucons will catch fatal signal (SIGHUP, SIGINT etc) Please note that vcpucons can be used multiple times on the same socket and at the same time. It's process decision whether it wants to accept next connections and how to handle them. Usage: vcpucons [ -l ] path/to/sockets/VCPUid-socketid | | | +- This path should point to external VCPU unix socket | which is supposed to listen for vcpucons to connect. | For example, you can use fs/unix-sock/2-123. VCPU 2 | should wait for the connections - eg. using the following: | Listen_Unix(123,nn,NET_SOCK_STREAM,sock) | [ nn = connection backlog ] | +-------- One session vs loop. Without this option, console will quit after disconnect or on connect failure; wit it, it will keep reconnecting forever. By handling numerous connections to single unix socket, numerous vcpucons sessions can be handled differently. You can find nice, very simple example in hll/examples/console.ahl - just run it, connect to it using vcpucons fs/unix-sock/0-123... And that's it :)
You can find answers to >>the most frequently<< asked questions. Most of those answers include information you can find above, but - as you probably know - it's very easy to miss something. Anyway, we are very often asked about following things: 1) What is it all for? For pleasure. We're creating Argante not because we want to write another Linux - we'd rather like to find out whether it is possible to create a system that would connect security with functionality, performance, universality and that, at the same time, would break most conventions used in other systems. The second thing - we'd like to see if we can do it :-) Another matter is that some of Argante's solutions can potentially be interesting - for example, the management (plug and play) and the creation of a communication layer inside a cluster system, independent of the distribution of systems, and in a transparent (for a programmer, of course) way. We don't want this OS to be a product, so we've decided to start distributing it under terms and conditions of the LGPL license. The software, support and some solutions can be a product, the OS itself - shouldn't be. 2) Where will Argante be useful? Any distributed servers, where security and efficiency is vital, in cluster systems (as mentioned above) and in many more... No, we don't expect that Argante will be a desk-end product - we neither want to challenge Microsoft nor to duplicate the success of Linux. Argante is also the perfect solution for distributing network requests - including distributed scanning tasks and fault-tolerant networks. Read Part II of documentation to find out more. 3) Will Argante be a separate system? I've mentioned it, but it all depends on how it will develop. An embedded system has its advantages - among others, the possibility of very precise integration (as a part of described hybrid solutions) with a real system, and the lack of necessity to port all software at one time. Of course, we're interested in making Argante more independent one day. Or even implementing RSIS at hardware level, who knows? We already thought about building a PC card with simple chip and RSIS interpreter in EEPROM, plus some memory on the board. Only SYSCALLs, exceptions, and possibly debugging traps will be reported to real system using IRQ (so real system is responsible for I/O, while code execution is done at the card level). Building such card at the cost below $200 shouldn't be a problem for someone skilled with electronics and chip programming. 4) What about portability of Unix applications? There will be nothing like it, because Argante has a completely different base. We can talk about portability between DOS and Unix - on both systems you can run a "Hello, world" software, but more advanced software, because of very serious differences, won't be portable. That's why we didn't even try to port the C language - of course, if anybody wants to do it, he can, even if it's not an extremely secure language... 5) Why does Argante has its own programming language? In fact, Argante would operate on a subset of Ada's commands - on the other hand, we use many conventions used in C to control the code and we think it's not so bad :) That's why AHLL is a mixture of good parts of both languages and is very easy to learn. 6) Can I change system settings? Maximal number of VCPUs, the maximal stack size and most of environmental settings can be modified via config.h file; but remember that changing some vital parameters (for example, the number of registers) can cause incompatibility or mistakes in software. 7) How does Argante use the power of CPU? When all processes are "dead", they wait for something or are in IOWAIT, the VS clock slows down, giving most of its power to the real system. In case when at least one process is WORKING, the whole processor power available to Argante is divided between them. It can be controlled via the "nice" value and the scheduling scheme in the real system. It is possible to run more than one Argante at the same time, but you should mind the efficiency; you can modify multitasking settings in the real system and Argante's priority. If you create a hybrid system, in which Argante cooperates with some elements of the real OS, we suggest to set priority for Argante and the rest of processes in a correct way, so it could use a CPU's power in a efficient way. Running well-designed Argante's applications shouldn't load the system too much. 9) Portability At the moment, biniaries aren't portable between systems with different endians. We plan to put an automatic translator in a loader module, but at the moment the only thing portable is source code and - between systems with the same endian - biniaries. The source code should be portable without any problems. 10) Problems with compilation? In case of "memory exhausted" or "segmentation fault" errors during compilation, comment out everything after -Wall in the CFLAGF= line in the Makefile for a given OS (you can find it in a sysdep/ directory). It can decrease efficiency of Argante, but it will speed up compilation and will decrease resources needed to compile it. 11) Where will Argante work? Linux - native platform (with readline support) FreeBSD - tested NetBSD - not tested, should work OpenBSD - tested Solaris - tested AIX - ??? <if you have access, let us know> HP/UX - beta version present IRIX - tested ...other systems? 12) I want to use readline library. Am I able to do it? The readline library can be used only when you: - use Linux system - have a new version of libc6 (glibc 2.1.x). Otherwise, the readline support won't be compiled in your Argante. This library isn't ideal and we're too busy writing the REALLY IMPORTANT CODE, so write your own version of readline or... wait for it :) 13) I'd like to write something - where the h... is the CVS? At the moment - nowhere. I don't think we will run it before the first stable version is ready. Until then - I (lcamtuf@ids.pl) am a CVS and please send all ideas, propositions and diffs (diff -urN) to me. Don't send your own snapshots or diffs created with different options - it's hard to update the code manually.
Report any bugs, problems and suggestions to argante@linuxpl.org :) I strongly believe AOS in early testing phase and isn't widely used (for now ;), so probably you don't have to report your findings to BUGTRAQ or so :) Thanks in advance.
Specific installation is not required. First of all, copy argante, tools/agtses and tools/agtback to your PATH - eg. to /usr/local/bin. You can also put modules/*.so in eg /usr/lib/argante, if you want. This can be done automatically using './build install'. To set up basic project worskspace, you can use "agtproj" utility. To launch the system in background, with no interactive console, use: agtback argante SCRIPTNAME ROOTDIR You could always access console of such session using agtses command. To run foreground session (debugging purposes, for example), you could use: argante SCRIPTNAME ROOTDIR SCRIPTNAME - absolute path to boot script ROOTDIR - project root directory (argante working directory) For details on project root directory, see below sections.
This appendix assumes you have default paths defined in your include/config.h, pointing some guidelines for proper project design. Please read it carefully. Complete Argante project means, in fact, one virtual system instance. Project directory should be a subdirectory in real system containing all necessary configuration files, binary images and, preferably, SVFS mapping points for all private data. So, a sample structure looks like this: /MyProject | +- boot.scr - bootup script; take a look on conf/scripts in | Argante sources. It should load all necessary | modules and executable images in proper order, | eventually launching real-system daemons as | well. | +- /conf/access.hac - HAC control file; see below | +- conf/fsconv.dat - SVFS mapping file; see below, map points should | be within fs/ subdirectories. | +- /source - suggested location for AHLL / RSIS sources. | +- /images - suggested location for binary images (executables) | +- /modules - eventually, if you don't have modules installed | globally, or want specific versions for this | instance, you could place your modules here | +- /fs - suggested "top of the filesystem" | +- /sock-unix - location of Unix sockets for local inter-process | | communication (should exists if you're planning to | | use Unix sockets; generally, it's better to use | | IPC or rIPC, but, on the other hand, Unix sockets | | are not so bad ;). | | | +- /external - location for Unix sockets for external software | communication; see section 7. | +- /... - mapped SVFS directories, in general; you should use hierarchy that can be easily assigned to specific task and type of resources, for example: /fs/ftp_server/storage/users/userXXX/
As it's been told, implementation philosophy in Argante is somewhat different than eg. in Unix. Instead of treating your project as one big box, try to separate functional blocks, drawing connections between them, for example: FTP SERVER PROJECT: | network listener --- [ network layer ] | command processor -- [ log file ] | authorizator ------- [ access control database ] | filesystem access -- [ user directories ] Don't split your project into 1000 parts - but try to keep every kind of I/O interaction in a different module, avoiding dangerous solutions, like giving command processor direct access to filesystem - put authorizator between, and filter every filesystem request using authorization data. In this example, network listener will pass connections to command processor(s). These processors can communicate only with network through listener (so no abusive operations are allowed), and authorizator (no direct filesystem or password information access). Authorizator, on every request passed to filesystem, should verify user information, and pass it only if it's correct. The most sensitive layer - command processing - is now safely separated from sensitive information. That's quite simple and deadly effective. You should use rIPC for communication between modules, so you can: - split your project into for example 4 different machines with no code changes, - launch any number of authorizators, filesystem access processes or command processors in cluster, creating redundant structure with automatic load-balancing ("choose fastest responding" algorithm). - add / remove / modify layers with no code changes.
SVFS hierarchy should be designed carefully. For example, putting vital system configuration files directly in SVFS is just stupid. If you have to modify local system, you should read details on real system interaction below, and create interface between your processes and real-system tools. Symlinks are allowed (and treated just like regular files, as long as they are not dangling ;), but should be used carefully. The same applies to hardlinks. Generally, you should design your project in the way that does not need any kind of links in SVFS. Below, you'll find some precautions for mapping NFS objects or objects shared between different AOS instances.
Assign every type of operation performed in your project specific domain number. For example: 1 - user files access, 2 - network listening, 3 - making data connections back to user, 4 - accessing password files, 5 - communication with command parser, 6 - communication with user files module (sorry, only numeric domain names are supported for now). You cannot use numbers below 100. For every functional module, assign list of domains it have to access (using #compiler !domains a b c d...). Before any operation (syscall) accessing specific kind of resources / operations, set domain number respectively. Drop these privileges after finishing specific kind of operations. If you want to "act" as some user or subobject within specific domain, you can set domain UID as well. For example, before accessing files owned by user 1234, you can set your privileges to domain=1, domain_uid=1234. By using domain_uid, you'll be able to restrict access to specific resource within group. Then, when your HAC hierarchy is completed, and you can do list like: domain 2 should be able to bind to all IPs to port 21 EXCEPT specific IP domain 1 uid 1234 should be able to do anything within /fs/ftp_server/users/mark ...and so on, you can build conf/access.hac file. Please refer HAC documentation for specific modules. In above example, HAC access file should look this way: 2:0 net/address/source/tcp/SPECIFIC_IP/21 net/sock/listen deny 2:0 net/address/source/tcp/all/21 net/sock/listen allow 1:1234 fs/ftp_server/users/mark fs allow [...]
NOTE: Yes, there are no precautions for launching several instances of Argante on one physical system, with completely different projects. One of its purposes might be testing of distributed/cluster solutions. But you should respect one rule - only one VS instance should be able to write specific real system resource. For example, if you're deciding to put FooBar file, physically located in /TestMe directory, in SVFS space for two different argante instances on the same system at the same time, you should give write access only to VCPU(s) on one virtual machine ("file access manager(s)"), and arrange write-request passing using rIPC. The same applies to NFS shared resources - be careful. It's just there's no accurate and portable way to determine if file is locked for writing in real system by other instance, so file damage might occour.
Preferred way to interact with real system space is to use Unix sockets. For this purpose, you have fs/unix-sock/external/ directory. You can request both CONNECT or LISTEN operation for datagram or stream sockets for given numerical ID. This socket is mapped into real filesystem entry: /Project_Directory/fs/unix-sock/external/nnn, where nnn is choosen ID. So, if you want to modify machine IP address from Argante program, you should choose specific ID, let's say 1234, and then attach small helper utility to a chosen Unix socket in listen mode. Helper should validate received command (datagram mode is good in most cases), and then, execute requested action. You can reverse this scheme, and listen at the AOS side, if this is necessary. Most interesting use of such interaction is to communicate with locally running daemons / services in real systems, while requests are propagated using rIPC cluster - so, in this case, Argante become load balancing cluster management software. Argante, both using local system components and network functionality, can modify current network structure (eg. reconfigure managable switching devices, routers etc), or intercepting functions of machine that crashed. So, you can create cluster of five machines, where Argante is receiving requests on one IP, and distributing it within cluster (maybe to real-system software), causing equal load of all machines. If one machine crashes, fastest box intercepts its network functionality (changing IP number). In this solution, you have no "weak point", and don't have to modify server software nor to implement separated load balancing. Consider this example, quite classical: [ UPLINKS ] | | +--+--+ | | / \ / \ primary and secondary LoadBalancer ~~~ ~~~ | | +-+-+-+ | | | | Servers In this case, loads are not always equal, and only two load balancers are the weak point. "Heartbeat" solutions are better, but quite often they're lacking several functionality - ability to do good load balancing, for example ;) Unlimited request propagation capabilities, unlimited dispersion, ability to split functional parts of one program between several machines, implementing redundancy, ability to auto-configure - all lacking. Argante can be used to detect new "plugged" device with pre-installed Argante cluster software, and then, to easily measure load of specific services, automatically configure services on this box and so on; that's not all - it can automatically configure active network devices to rearrange vLANs and place box in proper location in network structure (if you have for example one line of web servers, then, database layer, and then maybe other layers). And all without need for very sophisticated, non-portable software tools - hybrid, almost-perfect cluster of Solaris, Linux and BSD boxes can be arranged in plug-and-play manner with really simple Argante rIPC code. At the beginning, you should just know what you want to serve and implement basic morphing features :) Such project for Argante will be developed and included in futher releases, but I guess reading rIPC description shows how simple it is. NOTE: avoid sending strings through Unix sockets! Send numerical IP or perform strong validation of input before passing received data to any other programs! We're not allowing direct real-system calls from Argante because we want you to think about doing input validation before executing program. Also, DO NOT write helpers calling eg system(AOS_supplied_data). Think twice before doing anything. YOU'RE INTERACTING WITH REAL SYSTEM! BY MAKING A MISTAKE, IT MIGHT BECOME VULNERABLE. NOTE: you should make sure other users are not able to interact with your helper. Best way to do it is to restrict access to fs/unix-sock/external in specific project to specific group.
Making redundant solutions using rIPC is really simple. You can connect your servers, even if they're in different countries, using redundant structure - so, your cluster connections might like this way: New York | | | +------------------------------ Chicago | | | | Warsaw ---------+ +-------------+ | | +--- London Even so simple structure will be fault-tolerant - crash of single link won't cause cluster to stop working - only rIPC communication will be routed using other way. But, of course, your cluster might be connected much better, eg by adding link between New York and London, rIPC routing can be improved. By launching the same modules in different part of the world, you can put authentication database in London, listener in Warsaw and command processors all around the globe, and it will work just fine, even if most of processes / machines will be overloaded. Moreover, adding new object to the hierarchy can be done without human intervention and without need for sophisticated code. It requires pre-installed Argante, knowledge of IP address of only one HUB (listener) point in the rIPC network (well, ANY location where specific authorization key is valid can be used for initialization), and - obviously - valid initial authorization key ;) You don't have to plan whole rIPC network when starting work on your distributed application - you can start with two arbitrary boxes, and add new ones instantly. Futher configuration can be done automatically, by downloading current rIPC network hierarchy configuration (so redundant / fault-tolerant links are set up automatically, if programmer wishes so, eventually relocating new box to the most desirable location in the logical and physical structure). For details on rIPC daemon and setting up rIPC circuits, please refer rIPC documentation (in part I of this README). Local cluster morphing can be implemented easily by arranging local communication with agtses to load / unload specific programs, or, if you're using hybrid systems, to start / stop specific services on local machines.
This section is unfinished as for AOSr1 - please take a look at Examples/AHLL directory for guidelines on designing .ahh files and .ahl code. Except the limitations I have described before, there are some language bugs we have no time to solve in this release: - parameterless functions are not working, - you have to finish all structural / array initializers with ',', - /* */ comments when #defining something _might_ be harmful, - generally, lack of separators _might_ be harmful, so write x := 1 rather than x:=1. Please take a look on Examples/AHLL - you will find numerous small AHLL examples there: dir.ahl - this cute program will dump the listing of SVFS directory contents. gfx.ahl - simple but juicy example of SVGAlib connectivity (on Linux, requires modules/gfx.so to be loaded) hello.ahl - "nn green bottles standing on the wall"... ptrs.ahl - some pointer manipulation examples http/ - sources for Mini-HTTP server. I guess both http/httpd.ahl and, let's say, gfx.ahl, are pretty good examples of AHLL style. Programs are perfectly readable and self-commenting.
In some cases, it is important to invoke Argante programs via unix command-line, from servlets, as a CGI scripts or from SSI. In this case, you can use tools/agtexe utility. This program is able to connect to running, background AOS session, load given program, catch any errors (returning apropriate error message or return code). It will continue running until program will be terminated (if it happened due to exception, it might display message or return specific code), eventually arranging I/O session between process and its console. Argante VCPU will be automatically terminated if any signal will be caught by agtexe. Usage: agtexe program_name f[cwm] pid | ||| | | ||| +------- AOS session pid | ||| | ||+---------- do not display messages on errors | |+----------- do not wait for process to terminate | +------------ do not arrange I/O session | +---- you have to use absolute path! all relative paths are relative to AOS cwd, not agtexe CWD! Examples: (execute work/test.img, default settings, find Argante) agtexec $PWD/work/test.img f `ps x|grep ':.. argante'|grep -v grep|cut -b-6|head -1` (execute /test/test.img, without console, be silent, given pid) agtexec /test/test.img fcm 12345 Exit codes: 0 - successful execution (process terminated by HALT or 'w' option) 1 - agtexe caught signal, process terminated 2 - execution failure (bad binary image) 3 - unhandled exception during execution 4 - couldn't attach to Argante session