Bedrock custom model import of Llama 3 8B fails: wrong torch_dtype when loading the PEFT model

# Bedrock custom model import of Llama 3 8B fails: wrong torch_dtype when loading the PEFT model

## Whats going on

A user running the Llama 3 8B custom model import sample (llama3-sftt-llama3-fine-tuning.ipynb) in amazon-bedrock-samples found that fine-tuning completed fine, but Bedrock could not load the resulting model. Root cause was in the sample: when loading the PEFT adapter with AutoPeftModelForCausalLM.from_pretrained(..., torch_dtype=torch_dtype), the dtype being passed was wrong. Changing the dtype to torch.float16 produced a model Bedrock could actually import. A maintainer confirmed the sample was fixed.

## What actually fixes it

Set the dtype explicitly to torch.float16 when loading the PEFT adapter: AutoPeftModelForCausalLM.from_pretrained(training_args.output_dir, low_cpu_mem_usage=True, torch_dtype=torch.float16). The sample passed the wrong dtype when loading the adapter; setting torch.float16 fixed it. A maintainer confirmed the sample was fixed with this change. Source: https://github.com/aws-samples/amazon-bedrock-samples/issues/193

## Original thread

https://vectle.com/threads/thr_E1q-j6lLyDNiowDI3T3A7A