Immutable solution version
meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with...
Complete solution
Covers this Meltano problem: meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with.... If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem.
The problem
Issue meltano/meltano#7736 (closed, 18 comments): ### Meltano Version
2.19.0
Python Version
3.10
Bug scope
CLI (options, error messages, logging, etc.)
Operating System
MacOS 13.4 (22F66) Ventura
Description
I have added target-snowflake (transferwise variant) and configured it correctly. It works when invoked alone using meltano invoke target-snowflake.
Invoking a tap and piping into the target also works: meltano invoke tap-slack | meltano invoke target-snowflake
However, using meltano run results in a crash: meltano run tap-slack target-snowflake
[image]
There is no useful information given...
What fixed it
If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem. The maintainer's recommendation is to switch to the meltanolabs variant of target-snowflake, which is the default and better maintained.
Evidence
In gh:meltano/meltano#7736 (Meltano), the issue reported was: Issue meltano/meltano#7736 (closed, 18 comments): ### Meltano Version
2.19.0
Python Version
3.10
Bug scope
CLI (options, error messages, logging, etc.)
Operating System
MacOS 13.4 (22F66) Ventura
Description
I have added target-snowflake (transferwise variant) and configured it correctly. It works when invoked alone using meltano invoke target-snowflake.
Invoking a tap and piping into the target also works: meltano invoke tap-slack | meltano invoke target-snowflake
However, using meltano run results in a crash: meltano run tap-slack target-snowflake
[image]
There is no useful information given... The verified resolution from the thread: If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem. The maintainer's recommendation is to switch to the meltanolabs variant of target-snowflake, which is the default and better maintained. Source: mount the host path https at container path //github.com/meltano/meltano/issues/7736
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_6vmGJnQ3-GT9pDTwEF-xiA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with... <span>— Vectle</span></a>