[NOT TICKET] FileSource_fixes

[Ioannis]

No description provided.

[benno]

It's hard to parse this PR because it seems to be doing multiple things. Can we split apart the FileSource bug fixes from the refactoring of duplicateFilterEntityData? (If we're going to do that refactoring, I'd like to address the overlap with TableMetaTrait and maybe a third similar implementation rather than introduce a new trait that does basically the same thing.)

[Ioannis]

It's hard to parse this PR because it seems to be doing multiple things. Can we split apart the FileSource bug fixes from the refactoring of duplicateFilterEntityData? (If we're going to do that refactoring, I'd like to address the overlap with TableMetaTrait and maybe a third similar implementation rather than introduce a new trait that does basically the same thing.)

@benno I have already addressed that. The first commit contains only the fixes, while the second commit includes the refactoring

(If we're going to do that refactoring, I'd like to address the overlap with TableMetaTrait and maybe a third similar implementation rather than introduce a new trait that does basically the same thing.)

Thank you for bringing this to my attention. Refactoring this area is next on my list of tasks, and the initial changes I made are intended as a foundation for the broader improvements planned.

However, I would prefer to address the refactoring of in a separate pull request, since it involves functional changes. This will help keep the changes focused and as minimal as possible, specifically targeting the merge between and . duplicateFilterEntityData``duplicateFilterEntityData``filterMetadataFields

With the second commit, my goal is to break the code into smaller, more manageable pieces. The PipelineTable.php file is quite large, and I am not altering its underlying functionality at this stage. Rather, I am working to compartmentalize the code to improve readability and maintainability.

[Ioannis]

@benno removed the refactoring. This pull request contains only bug fixes.

[Ioannis]

@benno Currently this is used by the Email Verifier plugin. Since it is very generic, i think it makes sense to be in the core code.

[benno]

This should be in its own issue since FileSource doesn't seem to be using it. Once that's done we can merge this PR.

[Ioannis]

Done