Finetune issue with Llama2-7b

[anasegor]

Hello, I encountered the following error when running finetuning Llama2:

Some indexes in the input tensor for the embedding layer are outside the vocabulary dimension. I guess the problem occurs when adding a special tokens to the HF Tokenizer vocabulary in tokenizer.py:

For myself, I solved this by changing the line:

Is this the right way?