Inference | Unsloth Documentation

unslotCopy from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(model_name = lora_model, # YOUR MODELUnsloth is a lhtweht tool that optimizes LLM modeling code and reduces memory usage and training time. It supports Llama and Mistral architectures and is compatible with 🤗 PEFT and TRL.