Automating human intervention tasks with Expect

I administrate several servers and SSH password management were a big issue until I changed all my servers logins to use private keys. Things got really nicer when I discovered SSH aliases.

Some of my clients are hosted on shared hostings and some of them doesn’t allow password-less authentication “in behalf of my security”. Well, I can’t really afford opening my password manager, locating it and then copying and pasting back to my terminal. Too much time spent.

I have heard about expect long ago but never quite stopped to look at it since I never really had that need but in this case it came quite handy. Hop in aboard the new less typing world!

Expect is a Unix automation and testing tool, written by Don Libes [from NIST] as an extension to the Tcl scripting language, for interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, ssh, and others. It uses Unix pseudo terminals to wrap up subprocesses transparently, allowing the automation of arbitrary applications that are accessed over a terminal. With Tk, interactive applications can be wrapped in X11 GUIs.

Source: Wikipedia

Expect Overview

Expect runs a very simple kind of script file in its own language. This script contains the logic to perform the automation and it’s called as first argument to the expect command. For a argument-less script named fake_ssh_host-x.exp the call would be.

expect fake_ssh_host-x.exp

If it took any argument it would just be appended to the call:

expect fake_ssh_host-x.exp somefile.txt

Quick Expect anatomy

Expect directives are line-separated and not terminated. Expect even has some object-notation for grouping expect events.

Variables

Variables are assigned in expect with the set directive.

set pass "mypass"

Variables are further referenced prefixed with a dollar-sign $.

Command-line arguments

Command-line arguments are get through the $argv variable and the lindex call. Now that we have learned how to assign variables, let’s assign one with a command-line argument.

set filename [lindex $argv 0]

Note that indexes start at 0, as the standard.
You can have multiple command-line arguments:

set filename [lindex $argv 0]
set user [lindex $argv 1]
set host [lindex $argv 2]

Spawning the tampered process

We use Expect in order to tamper output from a process and thus automate the inputs. We need to tell our Expect script which process it must spawn. This is done though the spawn directive that prefixes a normal command-line statement for that call.

spawn ssh user@host

Timeout

Timeout determines the number of seconds that the process has to respond before being considered jammed (or b0rked) and is set through the timeout directive, much like as a variable is.

# one minute timeout
set timeout 60

timeout is disabled if set to -1

Observing data

The expect directive tells the script which text to expect from standard output.

expect "String"

Posting data

Posting data to standard input is done through the send directive. This statement when precede by an expect statement will execute only when given string matched.

send $pass

Grouping expectations and posts

This kinda object-notation is used to couple expect/send statements.

expect {
	"password:" {
		send "$pass\r"
	}
} 
# or shortened
expect {
	"password:" { send "$pass\r" }
}

If you need to append more commands on an expect block just append the line within the expected element closure (in expanded form) or append to the line in a semi-colon delimited form.

expect {
	"password:" {
		send "$pass\r"
		set timeout -1
	}
} 
# or shortened
expect {
	"password:" { send "$pass\r" ; set timeout -1 }
}

Coupling multiple expectations is also easy:

expect {
	"yes/no):" send "yes"
	"password:" { send "$pass\r" ; set timeout -1 }
	eof exit
}

Printing messages on screen

If you want to send messages to the standard output rather than the process’ input, you do exactly like the send statement described earlier in this article but the command is send_user.

send_user "Connected to host"

Outputting to a file

Sure, you can always pipe the output natively from the shell with the output redirectors > but we can also do that within Expect by setting the file variable coupled with the open directive.

# filename in example is set to expect.out
# 'w' opens file for writing, 'a' to append
set file [open expect.out w]

More documentation can be found on Expect’s website or in its man page.

man expect

Faking a private key authentication AKA Automating a password authentication

For both examples we will need variables and the expect, send and spawn directives.

SSH Session

We will expect the message asking us if we want to accept the key and send yes to accept it. Then we will expect for the password: string and send our password stored in a variable.

#!/usr/bin/expect -f
set timeout -1
spawn ssh username@host
set pass "mypass"
expect {
        password: { send "$pass\r"  }
	} "yes/no)?" {
		send "yes\r"
		set timeout -1
        } eof {
                exit
        }
}

SCP Session

We will follow the same logic from the above example but we’ll add some command-line arguments to catch the filename and path to be transfered.

#!/usr/bin/expect -f
set timeout -1
set filename [lindex $argv 0]
set dst_path [lindex $argv 1]
spawn scp $filename username@host:$dst_path
set pass "mypass"
expect {
        password: { send "$pass\r"  }
	} "yes/no)?" {
		send "yes\r"
		set timeout -1
        } eof {
                exit
        }
}

And you’re done. No more typing and unnecessary time spending.