-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spring Batch issue with MultiResourceItemWriter and ClassifierCompositeItemWriter as its returning the invalid count into the output file #4660
Comments
@fmbenhassine - Were you able to re-produce this issue? |
No, I did not start working on this yet. I would love to help if you can package all code snippets with a pom.xml in a zip or a repository that I can clone/run. There is a project template that you can use as a starting point here: https://github.com/spring-projects/spring-batch/blob/main/ISSUE_REPORTING.md |
@fmbenhassine - I've uploaded code here: https://github.com/javaHelper/bug-4660/tree/main/bug-4660. Could you please have a look? |
Great! Thank you for the sample. I will take a look and get back to you asap. |
Thank you for providing a minimal example, well done! I am able to reproduce the issue and I think this is a bug. In fact, the An interesting finding I noticed as well is that when I set the chunk size to 10, everything is written in the same file (ie the I will plan the fix in a future patch release, because as of now we are working on the upcoming 5.2. If someone manages to fix the issue, then a PR is welcome. |
I think the problem isn't related to the The effect of |
@fmbenhassine - Thanks. Agree its a bug and its reproducible. When I tested in my project with the huge volume keeping 1/2 millions records as |
I am using Spring Boot + Batch v2.7.1 in my project and looks like there is a bug when Reading from FlatFileItemReader using ClassifierCompositeItemWriter and MultiResourceItemWriter as itemCountLimitPerResource value doesn't works well and gives wrong responses.
I am reading csv file and splitting into multiple files having max records in every file should be 5 only, but the code which I developed giving me 7 values.
Code Uploaded here: https://github.com/javaHelper/bug-4660/tree/main/bug-4660
employee.csv
Output: javaDeveloper-employee.csv-1
javaDeveloper-employee.csv-2
pythonDeveloper-employee.csv-1
pythonDeveloper-employee.csv-1
When I used chunk(0) - its resulting into correct count in to the file, but when it comes to huge data, the performance is very very flow. Any suggestion
The text was updated successfully, but these errors were encountered: