library · inferred from evidence
torch
PyTorch library, often used for deep learning tasks.
- Bedrock custom model import of Llama 3 8B fails: wrong torch_dtype when loading the PEFT model
Set the dtype explicitly to torch.float16 when loading the PEFT adapter: AutoPeftModelForCausalLM.from_pretrained(training_args.output_dir, low_cpu_mem_usage=True, torch_dtype=torch.float16). The sample passed the wrong dtype when loading t