gc3libs.backends.shellcmd¶
Run applications as local processes.
-
class
gc3libs.backends.shellcmd.
ShellcmdLrms
(name, architecture, max_cores, max_cores_per_job, max_memory_per_core, max_walltime, auth=None, frontend='localhost', transport='local', time_cmd=None, override='False', keyfile=None, ignore_ssh_host_keys=False, spooldir=None, resourcedir=None, **extra_args)¶ Execute an
Application
instance as a local process.Construction of an instance of ShellcmdLrms takes the following optional parameters (in addition to any parameters taken by the base class
LRMS
):Parameters: - time_cmd (str) –
Path to the GNU
time
command. Default is/usr/bin/time
which is correct on all known Linux distributions.This backend uses many of the extended features of GNU
time
, so the shell-builtins or the BSDtime
will not work. - spooldir (str) – Path to a filesystem location where to create
temporary working directories for processes executed through
this backend. The default value None means to use
$TMPDIR
or/tmp
(see tempfile.mkftemp for details). - resourcedir (str) – Path to a filesystem location where to create a temporary
directory that will contain information on the jobs running on
the machine. The default value None means to use
$HOME/.gc3/shellcmd.d
. - transport (str) – Transport to use to connecet to the resource. Valid values are ssh or local.
- frontend (str) – If transport is ssh, then frontend is the hostname of the remote machine where the jobs will be executed.
- ignore_ssh_host_key (bool) – When connecting to a remote resource using ssh the server ssh public key is usually checked against a database of known hosts, and if the key is found but it does not match with the one saved in the database the connection will fail. Setting ignore_ssh_host_key to True will disable this check, thus introducing a potential security issue, but allowing connection even though the database contain old/invalid keys (the use case is when connecting to VM on a cloud, since the IP is usually reused and therefore the ssh key is recreated).
- override (bool) – ShellcmdLrms by default will try to gather information on the machine the resource is running on, including the number of cores and the available memory. These values may be different from the values stored in the configuration file. If override is True, then the values automatically discovered will be used instead of the ones in the configuration file. If override is False, instead, the values in the configuration file will be used.
-
cancel_job
(app)¶ Cancel a running job. If app is associated to a queued or running remote job, tell the execution middleware to cancel it.
-
close
()¶ Implement gracefully close on LRMS dependent resources e.g. transport
-
free
(app)¶ Delete the temporary directory where a child process has run. The temporary directory is removed with all its content, recursively.
If the deletion is successful, the lrms_execdir attribute in app.execution is reset to None; subsequent invocations of this method on the same applications do nothing.
-
free_slots
¶ Returns the number of cores free
-
get_resource_status
()¶ Update the status of the resource associated with this LRMS instance in-place. Return updated Resource object.
-
get_results
(app, download_dir, overwrite=False)¶ Retrieve job output files into local directory download_dir (which must already exists). Will not overwrite existing files, unless the optional argument overwrite is True.
-
peek
(app, remote_filename, local_file, offset=0, size=None)¶ Download size bytes (at offset offset from the start) from remote file remote_filename and write them into local_file. If size is None (default), then snarf contents of remote file from offset unto the end.
Argument local_file is either a local path name (string), or a file-like object supporting a .write() method. If local_file is a path name, it is created if not existent, otherwise overwritten.
Argument remote_filename is the name of a file in the remote job “sandbox”.
Any exception raised by operations will be passed through.
-
submit_job
(app)¶ Run an Application instance as a local process.
See: LRMS.submit_job
-
update_job_state
(app)¶ Query the running status of the local process whose PID is stored into app.execution.lrms_jobid, and map the POSIX process status to GC3Libs Run.State.
-
validate_data
(data_file_list=[])¶ Return False if any of the URLs in data_file_list cannot be handled by this backend.
The
shellcmd
backend can only handlefile
URLs.
- time_cmd (str) –