Llama2-13B Fine-tuning on Custom Data

(All References are attached at the end)

model_id = "EleutherAI/gpt-neox-20b"

bnb_config = BitsAndBytesConfig(

load_in_4bit=True,

bnb_4bit_use_double_quant=True,

bnb_4bit_quant_type="nf4",

bnb_4bit_compute_dtype=torch.bfloat16

)