Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeprecatedOutputFormatWrapper does not work well with PartitionTap #432

Open
wonlay opened this issue Mar 13, 2015 · 0 comments
Open

DeprecatedOutputFormatWrapper does not work well with PartitionTap #432

wonlay opened this issue Mar 13, 2015 · 0 comments

Comments

@wonlay
Copy link

wonlay commented Mar 13, 2015

When using DeprecatedOutputFormatWrapper with PartitionTap, the output won't be committed to the final location but discarded in the temporary directory.

There are two issues here:

  1. TapOutputCollector used by PartitionTag is testing if the output format is a FileOutputFormat or not, after wrapped, it's not a FileOutputFormat anymore, the filenames won't be created.
  2. The commit never happen when used in this case, the test for commit will always return false, since the real output format is writing to a different temp dir, and no intermedium commit to bridge them up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant