Skip to content

[Draft] Opencl lz4#3900

Open
kif wants to merge 22 commits intosilx-kit:mainfrom
kif:opencl_lz4
Open

[Draft] Opencl lz4#3900
kif wants to merge 22 commits intosilx-kit:mainfrom
kif:opencl_lz4

Conversation

@kif
Copy link
Member

@kif kif commented Jul 6, 2023

Changelog: LZ4 compressor on GPU

  • OpenCL code targeted to run on GPU
    • Find position of interest
    • Create segment descriptor (position in input, length of litterals and matches)
    • Concatenate several descriptor (between passes, in the case there are only litterals)
    • Write compressed stream, including tokens, offsets, overflows in litteral and matches
    • Manage the beginning of stream (4 bytes with uncompressed buffer size)
    • Manage end of stream (cannot end with matches, has to finish with litterals
    • Concatenate output stream to make it contiguous (1 or 2 kernel options ...)
  • Python code to call the OpenCL code
  • Non regression test
  • Profiling and performance assessment

@kif kif requested a review from pierrepaleo July 6, 2023 15:19
@t20100
Copy link
Member

t20100 commented Dec 11, 2023

I've added a commit to fix the conflict introduced by #3991.
A rebase would be best in order to sparse the merge of main in this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants