Issues

• xlnet_preprocess_kernels: blockAttnMask primary template is written as an illegal function-template partial specialization

  NVIDIA/FasterTransformer#799 · Filter by author diskdog opened on Jun 25, 2026

• Vps

  NVIDIA/FasterTransformer#796 · Filter by author ramyalhars53-alt opened on Aug 23, 2025

• Can FasterTransformer support SAM2 ( Meta segment anything model 2)?

  NVIDIA/FasterTransformer#795 · Filter by author jackwei86 opened on Sep 5, 2024

• How to implement a gemm with FP16 and INT4 using kernel in FasterTransformer/src/fastertransformer/kernels/cutlass_kernels/fpA_intB_gemm

  NVIDIA/FasterTransformer#794 · Filter by author AkatsukiChiri opened on Jul 26, 2024

• An error occurred for the specific cuda version

  bug
  NVIDIA/FasterTransformer#793 · Filter by author CSLiuPeng opened on May 8, 2024

• can be used in diffusion models,like sd and sdxl? how?where is the demos?tks

  bug
  NVIDIA/FasterTransformer#792 · Filter by author henbucuoshanghai opened on Apr 24, 2024

• bug: memory of position_encoding_table is not malloced correctly.

  bug
  NVIDIA/FasterTransformer#790 · Filter by author johnson-magic opened on Mar 27, 2024

• How to know the correspondence between versions vcr.io/nvidia/pytorch:xx.xx-py3 and pytorch?

  NVIDIA/FasterTransformer#788 · Filter by author johnson-magic opened on Feb 29, 2024

• what is the mean of EFF-FT?

  NVIDIA/FasterTransformer#787 · Filter by author johnson-magic opened on Feb 29, 2024

• Are fuseQKV masked attention and Flash Attention the same?

  NVIDIA/FasterTransformer#786 · Filter by author likejazz opened on Feb 4, 2024

• on H800 can not exec nvidia/pytorch:23.09-py3 container success

  NVIDIA/FasterTransformer#784 · Filter by author chenglinjun opened on Dec 29, 2023

• Confidence is not returned in the decoding example？

  bug
  NVIDIA/FasterTransformer#783 · Filter by author liuzhuang1024 opened on Dec 26, 2023

• Does FasterTransformer support multi-stream pipeline parallelism ?

  NVIDIA/FasterTransformer#781 · Filter by author FlyingPotatoZ opened on Nov 21, 2023

• error You need C++17 to compile PyTorch

  bug
  NVIDIA/FasterTransformer#779 · Filter by author ranggihwang opened on Nov 7, 2023

• can support decoder only bart? such as MBartForCausalLM

  NVIDIA/FasterTransformer#778 · Filter by author sjtu-cz opened on Nov 3, 2023

• Sparsity support

  bug
  NVIDIA/FasterTransformer#775 · Filter by author zhang662817 opened on Oct 29, 2023

• How to get started?

  NVIDIA/FasterTransformer#774 · Filter by author ghost opened on Oct 28, 2023

• Is llama2 70b supported? Do you know minimal configuration?

  NVIDIA/FasterTransformer#771 · Filter by author ChristineSeven opened on Oct 23, 2023
  1

• Supporting for expert parallelism in MoE inference

  NVIDIA/FasterTransformer#769 · Filter by author iteratorlee opened on Oct 19, 2023

• Whether fastertransformer supports gpt-2 classification model, such as GPT2ForSequenceClassification？

  NVIDIA/FasterTransformer#768 · Filter by author cabbagetalk opened on Oct 19, 2023

• cuSPARSELt is slower?

  NVIDIA/FasterTransformer#767 · Filter by author BDHU opened on Oct 16, 2023
  1

• Incorrect inline ptx device assembly code usage

  bug
  NVIDIA/FasterTransformer#766 · Filter by author zhiweij1 opened on Oct 13, 2023

• CUDA code compile error with clang: function template partial specialization is not allowed

  bug
  NVIDIA/FasterTransformer#765 · Filter by author zhiweij1 opened on Oct 12, 2023

• How to calculate local batch size?

  NVIDIA/FasterTransformer#764 · Filter by author fotstrt opened on Oct 11, 2023

• src/fastertransformer/kernels/decoder_masked_multihead_attention /decoder_masked_multihead_attention_template.hpp:36 open this macro definition, it'll find a build error

  bug
  NVIDIA/FasterTransformer#763 · Filter by author pengl opened on Oct 11, 2023