Folder Path

/backend/venv/lib/python3.9/site-packages/tokenizers/

9 directories 3 files 9.0 MiB total

	Name	Size	Modified
	Up
	__pycache__/	—	07/22/2025 10:09:35 PM +00:00
	decoders/	—	07/22/2025 10:09:35 PM +00:00
	implementations/	—	07/22/2025 10:09:35 PM +00:00
	models/	—	07/22/2025 10:09:35 PM +00:00
	normalizers/	—	07/22/2025 10:09:35 PM +00:00
	pre_tokenizers/	—	07/22/2025 10:09:35 PM +00:00
	processors/	—	07/22/2025 10:09:35 PM +00:00
	tools/	—	07/22/2025 10:09:35 PM +00:00
	trainers/	—	07/22/2025 10:09:35 PM +00:00
	__init__.py	2.6 KiB	07/22/2025 10:09:35 PM +00:00
	__init__.pyi	39 KiB	07/22/2025 10:09:35 PM +00:00
	tokenizers.abi3.so	8.9 MiB	07/22/2025 10:09:35 PM +00:00