Activity

1. ik020 commented on Sep 5, 2026

   

   ik020

   on Sep 5, 2026

   More actions

   Hey,

   I am Sayak from the Kernels team at Hugging Face. I noticed that this project uses Flash Attention which includes a long build time. We ship pre-built binaries (which provide bit-exact outputs as the upstream) and thereby, we make it easy to use.

   Using FA3 on a supported machine is as easy as:

   make sure kernels is installed: pip install -U kernels

   from kernels import get_kernel

   kernel_module = get_kernel("kernels-community/flash-attn3")
   flash_attn_func = kernel_module.flash_attn_func

   flash_attn_func(...)
   Let us know if you'd be interested in this and and we'd be happy to provide a draft of how it would look in your repo.

   Hi @sayakpaul, thanks for the suggestion! I'd like to work on integrating this. I'll test it with kernels-community/flash-attn2 on a T4 GPU(if possible), verify the outputs match the current implementation, and benchmark the build/import time difference. I'll open a PR once I have results, let me know if there's a specific model file/module you'd point me to first, or if I should start with the main attention implementation in the multi-modality module.