Picard's mark duplicates always removes duplicates instead of only marking them

[Karimelakoum]

The goal is to mark duplicates instead of removing them, however, when setting "remove-duplicates: true with duplicates-tool: "picard"", duplicates get removed instead of getting marked but when remove-duplicates is set to false, picard does even run and therefore doesn't mark the duplicates. Is there a way where i can run picard's markduplicates to mark duplicates only? Thanks in advance.

[lczech]

Hi @Karimelakoum,

yes, the removal is activated by default because in a typical run of grenepipe, as far as I can tell, duplicates are not used downstream anyway, so it made sense to just remove then to reduce intermediate file sizes.

This is done via the flag --REMOVE_DUPLICATES true, which is set in the config file here. Simply remove that flag from there in your run's config file, and duplicates should be kept then.

Do you have a specific use case where you want to just have them marked instead? Or do you think it should be changed by default to retain them instead? Might be that my reasoning is wrong or that I misunderstood the use of duplicates in downstream tools there.

Hope that helps, and so long
Lucas

[Karimelakoum]

Hi Lucas,
For example, as recommended by GATK best practices for variant calling, the official website (https://gatk.broadinstitute.org/hc/en-us/articles/360037052812-MarkDuplicates-Picard) states to mark duplicates instead of removing them as they are automatically ignored by GATK HaplotypeCaller but are still internally used by the algorithm to increase accuracy of snp calling. Additionally, a lot of papers* that call variants mark duplicates instead of removing for convenience I would say.
Thank you for your quick response!
Karim

*https://doi.org/10.1002/ece3.70434
https://doi.org/10.1038/s41467-021-24581-z
https://doi.org/10.1534/g3.117.039008
https://doi.org/10.1073/pnas.2217276120

[lczech]

Hi Karim,

thanks for the links! Hm, do you think it would make sense to change the default in grenepipe then? I am however not sure about the implications for the other variant callers that are implemented here (bcftools and freebayes) - do you happen to know how they deal with reads that have been marked rather then removed?

Right now, I'm not having the time to actively look into this, but let me know if you have any suggestions. If it's just a matter of changing the default here, or adding a few sentences of caveat to the documentation, I'm happy to do that!

Lucas