Deploying using ansible tower doesn't work

[sandbergja]

Expected behavior

We can deploy from ansible tower, or from the command line

Actual behavior

Ansible tower fails with:

#<Thread:0x00007f5227233f68 /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
/usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as deploy@lib-solr-staging4d: fingerprint SHA256:[REDACTED] does not match for "lib-solr-staging4d,128.112.204.144" (SSHKit::Runner::ExecuteError)
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/verifiers/always.rb:53:in `process_cache_miss': fingerprint SHA256:[REDACTED] does not match for "lib-solr-staging4d,128.112.204.144" (Net::SSH::HostKeyMismatch)
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/verifiers/always.rb:35:in `verify'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/verifiers/accept_new.rb:17:in `verify'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/verifiers/accept_new_or_local_tunnel.rb:17:in `verify'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/kex/abstract.rb:82:in `verify_server_key'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/kex/abstract.rb:49:in `exchange_keys'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/algorithms.rb:448:in `exchange_keys'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/algorithms.rb:248:in `proceed!'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/algorithms.rb:187:in `accept_kexinit'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:210:in `block in poll_message'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:190:in `loop'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:190:in `poll_message'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:227:in `block in wait'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:224:in `loop'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:224:in `wait'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh/transport/session.rb:89:in `initialize'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh.rb:255:in `new'
	from /usr/local/lib/ruby/gems/3.2.0/gems/net-ssh-7.1.0/lib/net/ssh.rb:255:in `start'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/connection_pool.rb:63:in `call'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/connection_pool.rb:63:in `with'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/netssh.rb:177:in `with_ssh'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/netssh.rb:130:in `execute_command'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
	from <internal:kernel>:90:in `tap'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/abstract.rb:80:in `execute'
	from /usr/local/lib/ruby/gems/3.2.0/gems/capistrano-3.17.2/lib/capistrano/scm/tasks/git.rake:8:in `block (3 levels) in eval_rakefile'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/backends/abstract.rb:31:in `run'
	from /usr/local/lib/ruby/gems/3.2.0/gems/sshkit-1.21.4/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as deploy@lib-solr-staging4d:

Steps to replicate

1. In ansible tower, run the "Deploy with Capistrano" template with pul_solr
2. When the task finishes, you should see the error workflow

Impact of this bug

Not too bad, since we can deploy from the command line