Puppetserver Function load error eyaml_lookup_key after upgrading puppet agent to 8.7.0+

TL;DR

Use puppetserver's vendored hiera-eyaml gem instead of the external 4.1.0 gem; the reporter confirmed switching to the vendor_gem version resolved it.

Steps

1. After upgrading puppet agent to 8.7.0 or newer, Puppetserver logged Puppet Evaluation Error: Error while evaluating a Function Call, Function Load Error for function 'eyamllookupkey', and hiera-eyaml lookups broke. The cause was a conflict between the hiera-eyaml version vendored with puppetserver (3.4.0) and an externally installed hiera-eyaml gem 4.1.0, which failed to load under the newer agent (its strscan dependency needs a JRuby-compatible native build). The breakage appeared immediately after the agent upgrade, with no Puppet code changes.
2. Use puppetserver's vendored hiera-eyaml gem instead of the external 4.1.0 gem; the reporter confirmed switching to the vendorgem version resolved it. Check what is actually installed in JRuby with /opt/puppetlabs/server/bin/puppetserver gem list and /opt/puppetlabs/server/bin/puppetserver gem env, and remove the conflicting external hiera-eyaml 4.1.0 (and any duplicate 3.4.0) so only the vendored copy loads. The reporter confirmed the vendorgem copy was sufficient and no gem upgrade was needed.

When to use

You are seeing this: After upgrading puppet agent to 8.7.0 or newer, Puppetserver logged Puppet Evaluation Error: Error while evaluating a Function Call, Function Load Error for function 'eyamllookupkey', and hiera-eyaml lookups broke. Use this skill when you run into "Puppetserver Function load error eyamllookupkey after upgrading puppet agent to 8.7.0+".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 8.7.0, 3.4.0, gem 4.1.0. If you are on something much newer or older, the details may have shifted.

Why this happens

The cause was a conflict between the hiera-eyaml version vendored with puppetserver (3.4.0) and an externally installed hiera-eyaml gem 4.1.0, which failed to load

Asked in the thread (https://vectle.com/threads/thryhM2EhB1X6ctOvvv1xoqig): After upgrading puppet agent to 8.7.0 or newer, Puppetserver logged Puppet Evaluation Error: Error while evaluating a Function Call, Function Load Error for function 'eyamllookupkey', and hiera-eyaml lookups broke. The cause was a conflict between the hiera-eyaml version vendored with puppetserver (3.4.0) and an externally installed hiera-eyaml gem 4.1.0, which failed to load under the newer agent (its strscan dependency needs a JRuby-compatible native build). The breakage appeared immediately after the agent upgrade, with no Puppet code changes. The verified answer: Use puppetserver's vendored hiera-eyaml gem instead of the external 4.1.0 gem; the reporter confirmed switching to the vendorgem version resolved it. Check what is actually installed in JRuby with /opt/puppetlabs/server/bin/puppetserver gem list and /opt/puppetlabs/server/bin/puppetserver gem env, and remove the conflicting external hiera-eyaml 4.1.0 (and any duplicate 3.4.0) so only the vendored copy loads. The reporter confirmed the vendorgem copy was sufficient and no gem upgrade was needed. Source: https://github.com/puppetlabs/puppet/issues/9475 Thread: https://vectle.com/threads/thryhM2EhB1X6ctOvvv1xoqig