At the very beginning of the user authentication phase of SSH, the client launches the plugin subprocess, if one is configured. It immediately sends the PLUGIN_INIT message, telling the plugin some initial information about where the SSH connection is to.
The plugin responds with PLUGIN_INIT_RESPONSE, which may optionally tell the SSH client what username to use.
The client begins trying to authenticate with the SSH server in the usual way, using the username provided by the plugin (if any) or alternatively one obtained via its normal (non-plugin) policy.
The client follows its normal policy for selecting authentication methods to attempt. If it chooses a method that this protocol does not cover, then the client will perform that method in its own way without consulting the plugin.
However, if the client and server decide to attempt a method that this protocol does cover, then the client sends PLUGIN_PROTOCOL specifying the SSH protocol id for the authentication method being used. The plugin responds with PLUGIN_PROTOCOL_ACCEPT if it's willing to assist with this auth method, or PLUGIN_PROTOCOL_REJECT if it isn't.
If the plugin sends PLUGIN_PROTOCOL_REJECT, then the client will proceed as if the plugin were not present. Later, if another auth method is negotiated (either because this one failed, or because it succeeded but the server wants multiple auth methods), the client may send a further PLUGIN_PROTOCOL and try again.
If the plugin sends PLUGIN_PROTOCOL_ACCEPT, then a protocol segment begins that is specific to that auth method, terminating in either PLUGIN_AUTH_SUCCESS or PLUGIN_AUTH_FAILURE. After that, again, the client may send a further PLUGIN_PROTOCOL.
Currently the only supported method is ‘keyboard-interactive’, defined in [RFC4256]. Once the client has announced this to the server, the followup protocol is as follows:
Each time the server sends an SSH_MSG_USERAUTH_INFO_REQUEST message requesting authentication responses from the user, the SSH client translates the message into PLUGIN_KI_SERVER_REQUEST and passes it on to the plugin.
At this point, the plugin may optionally send back PLUGIN_KI_USER_REQUEST containing prompts to be presented to the actual user. The client will reply with a matching PLUGIN_KI_USER_RESPONSE after asking the user to reply to the question(s) in the request message. The plugin can repeat this cycle multiple times.
Once the plugin has all the information it needs to respond to the server's authentication prompts, it sends PLUGIN_KI_SERVER_RESPONSE back to the client, which translates it into SSH_MSG_USERAUTH_INFO_RESPONSE to send on to the server.
After that, as described in [RFC4256], the server is free to accept authentication, reject it, or send another SSH_MSG_USERAUTH_INFO_REQUEST. Each SSH_MSG_USERAUTH_INFO_REQUEST is dealt with in the same way as above.
If the server terminates keyboard-interactive authentication with SSH_MSG_USERAUTH_SUCCESS or SSH_MSG_USERAUTH_FAILURE, the client informs the plugin by sending either PLUGIN_AUTH_SUCCESS or PLUGIN_AUTH_FAILURE. PLUGIN_AUTH_SUCCESS is sent when that particular authentication method was successful, regardless of whether the SSH server chooses to request further authentication afterwards: in particular, SSH_MSG_USERAUTH_FAILURE with the ‘partial success’ flag (see [RFC4252] section 5.1) translates into PLUGIN_AUTH_SUCCESS.
The plugin's standard input will close when the client no longer requires the plugin's services, for any reason. This could be because authentication is complete (with overall success or overall failure), or because the user has manually aborted the session in mid-authentication, or because the client crashed.