LLM.Model¶
| Name | Mandatory | Description | Default | Type |
|---|---|---|---|---|
⬅️ Input |
The input of the shard, if any | String[String] |
||
Output ➡️ |
The resulting output of the shard | LLM.Model |
||
UseMmap |
No | Use mmap to load the model | true |
BoolVar(Bool) |
GPULayers |
No | Number of GPU layers to use | 0 |
IntVar(Int) |