enhancementhelp wanted
Repository metrics
- Stars
- (1,513 stars)
- PR merge metrics
- (Avg merge 9d 17h) (200 merged PRs in 30d)
Description
Once in a while, we do need to add some code to Megatron, e.g. to support new architectures. One potential way to verify its correctness is to run it as an inference engine, with an openai compatible endpoint, and use eval tools like Nemo Skills to check benchmark accuracy. Also, it would be great if this could reuse Megatron as much as possible, e.g. Megatron existing codebase, Megatron command line args, etc, to reduce maintenance overhead and also be a bit extra confident about we are testing the correct thing.
Last time I checked Megatron codebase, there seems to already be some code, but not checked whether it is ready to be used like this or still need some development.