Skip to content

fix(checkpoint): preserve keyword arguments with DeepSpeed - #1699

Open
iMountTai wants to merge 4 commits into
modelscope:mainfrom
HYGON-AI:fix/deepspeed-checkpoint-kwargs
Open

iMountTai wants to merge 4 commits into
modelscope:mainfrom
HYGON-AI:fix/deepspeed-checkpoint-kwargs

Conversation

@iMountTai

Copy link
Copy Markdown

The DeepSpeed checkpoint path flattens args and kwargs.values(), then invokes the model with all values as positional arguments. This loses keyword names: a call such as forward(x, *, scale, offset) fails, and keyword arguments passed in a different order can bind to the wrong parameters.

Capture the positional-argument count and ordered keyword names in the checkpoint closure. Reconstruct the original positional and keyword arguments for both the initial forward and backward recomputation. The existing no-gradient bypass is retained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant