arc.job.ssh

A module for SSHing into servers. Used for giving commands, uploading, and downloading files.

class arc.job.ssh.SSHClient(server: str = '')

This is a class for communicating with remote servers via SSH.

Parameters:

server (str) – The server name as specified in ARCs’s settings file under servers as a key.

server

The server name as specified in ARCs’s settings file under servers as a key.

Type:

str

address

The server’s address.

Type:

str

un

The username to use on the server.

Type:

str

key

A path to a file containing the RSA SSH private key to the server.

Type:

str

_ssh

A high-level representation of a session with an SSH server.

Type:

paramiko.SSHClient

_sftp

SFTP client used to perform remote file operations.

Type:

paramiko.sftp_client.SFTPClient

change_mode(mode: str, file_name: str, recursive: bool = False, remote_path: str = '') → None

Change the mode of a file or a directory.

Parameters:

• mode (str) – The mode change to be applied, can be either octal or symbolic.

• file_name (str) – The path to the file or the directory to be changed.

• recursive (bool, optional) – Whether to recursively change the mode to all files under a directory.``True`` for recursively change.

• remote_path (str, optional) – The directory path at which the command will be executed.

check_job_status(job_id: int) → str

Check job’s status.

Parameters:

job_id (int) – The job’s ID.

Returns: str

Possible statuses: before_submission, running, errored on node xx, done, and errored: …

check_running_jobs_ids() → list

Check all jobs submitted by the user on a server.

Returns: list

A list of job IDs.

close() → None

Close the connection to paramiko SSHClient and SFTPClient

connect() → None

A modulator function for _connect(). Connect to the server.

Raises:

ServerError – Cannot connect to the server with maximum times to try

delete_job(job_id: Union[int, str]) → None

Deletes a running job.

Parameters:

job_id (Union[int, str]) – The job’s ID.

delete_jobs(jobs: Optional[List[Union[str, int]]] = None) → None

Delete all of the jobs on a specific server.

Parameters:

jobs (List[Union[str, int]], optional) – Specific ARC job IDs to delete.

download_file(remote_file_path: str, local_file_path: str) → None

Download a file from the server.

Parameters:

• remote_file_path (str) – The remote path to be downloaded from.

• local_file_path (str) – The local path to be downloaded to.

Raises:

ServerError – If the file cannot be downloaded with maximum times to try

find_package(package_name: str) → list

Find the path to the package.

Parameters:

package_name (str) – The name of the package to search for.

list_available_nodes() → list

List available nodes on the server.

Returns:

lines of the node hostnames.

Return type:

list

list_dir(remote_path: str = '') → list

List directory contents.

Parameters:

remote_path (str, optional) – The directory path at which the command will be executed.

submit_job(remote_path: str, recursion: bool = False) → Tuple[Optional[str], Optional[str]]

Submit a job to the server.

Parameters:

• remote_path (str) – The remote path contains the input file and the submission script.

• recursion (bool, optional) – Whether this call is within a recursion.

Returns: Tuple[str, int]

• A string indicate the status of job submission. Either errored or submitted.

• The job ID of the submitted job.

upload_file(remote_file_path: str, local_file_path: str = '', file_string: str = '') → None

Upload a local file or contents from a string to the remote server.

Parameters:

• remote_file_path (str) – The path to write into on the remote server.

• local_file_path (Optional[str]) – The local file path to be copied to the remote location.

• file_string (Optional[str]) – The file content to be copied and saved as the remote file.

Raises:

• InputError – If both local_file_path or file_string are invalid, or local_file_path does not exist.

• ServerError – If the file cannot be uploaded with maximum times to try

arc.job.ssh.check_connections(function: Callable[[...], Any]) → Callable[[...], Any]

A decorator designned for SSHClient``to check SSH connections before calling a method. It first checks if ``self._ssh is available in a SSHClient instance and then checks if you can send ls and get response to make sure your connection still alive. If connection is bad, this decorator will reconnect the SSH channel, to avoid connection related error when executing the method.

arc.job.ssh.check_job_status_in_stdout(job_id: int, stdout: Union[list, str], server: str) → str

A helper function for checking job status.

Parameters:

• job_id (int) – the job ID recognized by the server.

• stdout (Union[list, str]) – The output of a queue status check.

• server (str) – The server name.

Returns:

The job status on the server (‘running’, ‘done’, or ‘errored’).

Return type:

str

arc.job.ssh.delete_all_arc_jobs(server_list: list, jobs: Optional[List[str]] = None) → None

Delete all ARC-spawned jobs (with job name starting with a and a digit) from :list:servers (servers could also be a string of one server name) Make sure you know what you’re doing, so unrelated jobs won’t be deleted… Useful when terminating ARC while some (ghost) jobs are still running.

Parameters:

• server_list (list) – List of servers to delete ARC jobs from.

• jobs (Optional[List[str]]) – Specific ARC job IDs to delete.