designhelp wanted
Repository metrics
- Stars
- (12,006 stars)
- PR merge metrics
- (PR metrics pending)
Description
LLM is currently designed mainly around chat prompts. But there are other forms of LLM interaction, in particular "fill-in-the-middle" as seen in GitHub Copilot.
https://huggingface.co/smallcloudai/Refact-1_6B-fim is an example of a model that supports this:
prompt = '''<fim_prefix>def print_hello_world():
"""<fim_suffix>
print("Hello world!")<fim_middle>'''
Output from that:
<fim_prefix>def print_hello_world():
"""<fim_suffix>
print("Hello world!")<fim_middle>Prints 'Hello world!'"""<|endoftext|>
Which, if you figure out how to construct it again, ends up like this:
def print_hello_world():
"""Prints 'Hello world!'"""
print("Hello world!")
How could LLM support this? Would I need to design a new variant of the llm prompt command and model.prompt() method for this case?