Skip to content

Forward allowed_special in Llama3Tokenizer.encode - #1121

Open
MohammadHijjawi97 wants to merge 1 commit into
rasbt:mainfrom
MohammadHijjawi97:fix-llama3-tokenizer-allowed-special
Open

MohammadHijjawi97 wants to merge 1 commit into
rasbt:mainfrom
MohammadHijjawi97:fix-llama3-tokenizer-allowed-special