This is the new checkpointing scheme which utilizes a direct dump and
restore of data and stack segments to and from a file - no core
dumps or rewriting of executables.  

Jan 21, 1994
-- mike

This version of the checkpoint code saves and restores not only the
process's image, but also the state of its open file descriptors.
This information is stored in a structure called the Open File
Table.  The Open File Table contains one record for each file
the user process has open - indexed by file descriptor number.
This table is also used for a couple of other purposes related
to remote execution.  While this version of checkpointing does not
yet support remote execution, the Open File Table does implement
the mechanisms it will eventually need for that.

Remember that when a process is running on some execution machine, it
should have a view of the file system on its initiating machine.  When
a user process is running on an execution machine its access to files
may come through two different mechanisms.  If the file is available on
the executing machine via NFS, the file access will be through the
kernel on the executing machine.  Otherwise the file will be
accessed via remote system calls to the shadow.  It is therefore
necessary to keep track of how the file should be accessed for each
user file descriptor.  Note also that both the shadow and the local
process each have a full set of file descriptor numbers to use.  This
means that two different files may have the same real fd - one from the
shadow, and one from the local kernel.  To keep the client code from
seeing the same fd for two different files we must maintain a mapping
between user fd and real fd.  The information about mapping and how
the file should be accessed are both kept in the Open File Table.

Since the condor user's program should only know about the file system
on its initiating machine, and should not be aware of any of this
remote access, mapping, etc. it cannot do system calls directly.
Instead the Condor library provides a "stub" for each system call which
has a potential need for remote execution.  For each system call the
stub will will check to see if the system call needs to be done locally
or remotely.  In the local case it will use the syscall(3) routine to
invoke the desired system call on the local machine, and in the remote
case it will use condor's REMOTE_syscall() routine to invoke the system
call on the remote machine.  Additionally, those stubs which use file
descriptors (e.g. read and write) will need to do the mapping between user
file descriptors and real file descriptors.  Finally, certain system
call stubs which directly affect the content of the Open Files Table
(e.g. open and close) will need to manipulate that data structure.

This leads to 3 classes of system calls - those which aren't concerned
about the Open File Table, those which read the Open File Table for
mapping and access methods, and those which must make changes to the table.
You will find the stubs for the routines that change the table
hand coded in "file_state.C".  The other two categories are generated
automatically from a template in "stubs.tmpl".  For those stubs
which don't use the Open File Table at all, the template is simply
an ANSI style prototype for the function.  For the stubs which
must map the user's file descriptor, the function "map" is placed
around the parameter which must be mapped.  For example the
system call stat() which has no file descriptor to map uses the
template:

	int stat( char *path, struct stat *buf);

but the system call fstat() which does need to map a file descriptor
uses the template:

	int fstat( map(int fd), struct stat *buf );

All the automatically generated stubs go in the file "syscall_stubs.c"
which is produced from the file "stubs.tmpl" by the program
"stub_gen".  The stub_gen program is a simple lex/yacc prarser
which does the necessary translation, (macros don't work well for
this kind of thing since the system calls have different numbers of
arguments).  All the source for stub_gen is in this directory,
the lex part in "stub_gen_lex.l", and the yacc part in "stub_gen_yacc.y".
The "Imakefile" in this directory has rules which will ensure
that both the stub_gen and the stubs are built before linking
any program which needs the stubs.

Feb 21, 1994
-- mike
