Skip to content
← Back to activity

Immutable solution version

meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with...

Version 1Sign in to save

Complete solution

Covers this Meltano problem: meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with.... If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem.

The problem

Issue meltano/meltano#7736 (closed, 18 comments): ### Meltano Version

2.19.0

Python Version

3.10

Bug scope

CLI (options, error messages, logging, etc.)

Operating System

MacOS 13.4 (22F66) Ventura

Description

I have added target-snowflake (transferwise variant) and configured it correctly. It works when invoked alone using meltano invoke target-snowflake.

Invoking a tap and piping into the target also works: meltano invoke tap-slack | meltano invoke target-snowflake

However, using meltano run results in a crash: meltano run tap-slack target-snowflake

[image]

There is no useful information given...

What fixed it

If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem. The maintainer's recommendation is to switch to the meltanolabs variant of target-snowflake, which is the default and better maintained.

Evidence

In gh:meltano/meltano#7736 (Meltano), the issue reported was: Issue meltano/meltano#7736 (closed, 18 comments): ### Meltano Version

2.19.0

Python Version

3.10

Bug scope

CLI (options, error messages, logging, etc.)

Operating System

MacOS 13.4 (22F66) Ventura

Description

I have added target-snowflake (transferwise variant) and configured it correctly. It works when invoked alone using meltano invoke target-snowflake.

Invoking a tap and piping into the target also works: meltano invoke tap-slack | meltano invoke target-snowflake

However, using meltano run results in a crash: meltano run tap-slack target-snowflake

[image]

There is no useful information given... The verified resolution from the thread: If meltano run crashes at the end with the transferwise variant of target-snowflake while meltano invoke works fine, the culprit is the old pyarrow pin pulled in transitively via snowflake-connector-python; the reporter confirmed that was the problem. The maintainer's recommendation is to switch to the meltanolabs variant of target-snowflake, which is the default and better maintained. Source: mount the host path https at container path //github.com/meltano/meltano/issues/7736

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_6vmGJnQ3-GT9pDTwEF-xiA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">meltano run with any tap and `target-snowflake` fails at the end of the run, even though it works when piped with... <span>— Vectle</span></a>