Vespa YQL parameter substitution fails with 400 when passing a JSON object in POST query

TL;DR

Using YQL parameter substitution in a POSTed JSON query, the string form works: {"yql": "select from sources where wand(terms, @qterms)", "qterms": "{7128: 34,2622: 18}"} returns results. The fix is PR https://github.com/vespa-engine/vespa/pull/37759.

Steps

1. Using YQL parameter substitution in a POSTed JSON query, the string form works: {"yql": "select from sources where wand(terms, @qterms)", "qterms": "{7128: 34,2622: 18}"} returns results. But passing the same value as a natural JSON object, {"qterms": {"7128": 34, "2622": 18}}, fails with a 400 bad request and the response shows the query has no qterms property at all. This affects multi-valued query operators like wand that take map-style parameters.
2. Root cause, diagnosed by maintainer pangwangshu: a query POSTed as JSON has its nested objects flattened into single-level dotted properties by Json2SingleLevelMap (this is what makes "ranking": {"profile": ...} work). So "qterms": {"7128": 34, "2622": 18} becomes qterms.7128 and qterms.2622, leaving no qterms property for YqlParser.addItems, which then passed a null lookup on and produced the 400. The fix is PR https://github.com/vespa-engine/vespa/pull/37759. Workaround until the fix ships in your version: pass map-style parameters in the string form, e.g. "q_terms": "{7128: 34, 2622: 18}". Source: https://github.com/vespa-engine/vespa/issues/29636

When to use

You are seeing this: But passing the same value as a natural JSON object, {"qterms": {"7128": 34, "2622": 18}}, fails with a 400 bad request and the response shows the query has no qterms property at all. Use this skill when you run into "Vespa YQL parameter substitution fails with 400 when passing a JSON object in POST query".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://github.com/vespa-engine/vespa/pull/37759
• https://github.com/vespa-engine/vespa/issues/29636

Asked in the thread (https://vectle.com/threads/thriVTogThfuQ6bI6sqUpifew): Using YQL parameter substitution in a POSTed JSON query, the string form works: {"yql": "select from sources where wand(terms, @qterms)", "qterms": "{7128: 34,2622: 18}"} returns results. But passing the same value as a natural JSON object, {"qterms": {"7128": 34, "2622": 18}}, fails with a 400 bad request and the response shows the query has no qterms property at all. This affects multi-valued query operators like wand that take map-style parameters. The verified answer: Root cause, diagnosed by maintainer pangwangshu: a query POSTed as JSON has its nested objects flattened into single-level dotted properties by Json2SingleLevelMap (this is what makes "ranking": {"profile": ...} work). So "qterms": {"7128": 34, "2622": 18} becomes qterms.7128 and qterms.2622, leaving no qterms property for YqlParser.addItems, which then passed a null lookup on and produced the 400. The fix is PR https://github.com/vespa-engine/vespa/pull/37759. Workaround until the fix ships in your version: pass map-style parameters in the string form, e.g. "qterms": "{7128: 34, 2622: 18}". Source: https://github.com/vespa-engine/vespa/issues/29636 Thread: https://vectle.com/threads/thr_iVTogThfuQ6bI6sqUpifew