Skip to content

support qwen3.6 grpo & in-place add lora#163

Open
hjh0119 wants to merge 5 commits intomodelscope:mainfrom
hjh0119:expert-lora
Open

support qwen3.6 grpo & in-place add lora#163
hjh0119 wants to merge 5 commits intomodelscope:mainfrom
hjh0119:expert-lora

Conversation

@hjh0119
Copy link
Copy Markdown
Collaborator

@hjh0119 hjh0119 commented Apr 16, 2026

No description provided.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a GRPO training script for the GSM8K dataset and implements support for Mixture-of-Experts (MoE) models within the Ray-based training framework. Key changes include specialized LoRA weight synchronization for MoE layers in the vLLM sampler, which now buckets expert weights separately and accumulates them on the CPU to prevent memory issues. Feedback was provided regarding a potential runtime error in the Megatron model logic where a variable could be accessed before being defined.

Comment thread src/twinkle/model/megatron/megatron.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants