flâneur

API Reference — TensorRT LLM

nvidia.github.io · 69,795 words · saved by 1 readers

model (Union[str, pathlib.Path]) – stable The path to the model checkpoint or the model name from the Hugging Face Hub. tokenizer (Union[str, pathlib.Path, transformers.tokenization_utils_base.PreTrainedTokenizerBase, tensorrt_llm.llmapi.tokenizer.TokenizerBase, NoneType]) – stable The path to the tokenizer checkpoint or the tokenizer name from the Hugging Face Hub. Defaults to None. tokenizer_mode (Literal['auto', 'slow']) – stable The mode to initialize the tokenizer. Defaults to auto. skip_tokenizer_init (bool) – stable Whether to skip the tokenizer initialization. Defaults to False. trust_remote_code (bool) – stable Whether to trust the remote code. Defaults to False. tensor_parallel_size (int) – stable The tensor parallel size. Defaults to 1. dtype (str) – stable The data type to use for the model. Defaults to auto. revision (Optional[str]) – stable The revision to use for the model. Defaults to None. tokenizer_revision (Optional[str]) – stable The revision to use for the tokenize

API Reference — TensorRT LLM Skip to main content Back to top Ctrl + K TensorRT LLM Choose version System Settings Light Dark API Reference # Note Since version 1.0, we have attached a status label to LLM , LlmArgs and TorchLlmArgs Classes. stable - The item is stable and will keep consistent. prototype - The item is a prototype and is subject to change. beta - The item is in beta and approaching stability. deprecated - The item is deprecated and will be removed in a future release. class tensorrt_llm.llmapi. LLM ( model : str | Path , tokenizer : str | Path | TokenizerBase | PreTrainedT

saved by

related reading