| 1B parameters |
4 GB |
NVIDIA |
RTX 4060 (8GB)
RTX 3060 (12GB)
RTX 2060 Super (8GB)
GTX 1660 Ti (6GB)
|
| 1B parameters |
4 GB |
AMD |
Radeon RX 6600 XT (8GB)
RX 5700 (8GB)
RX 6700 (10GB)
|
| 3B parameters |
8 GB |
NVIDIA |
RTX 4070 (12GB)
RTX 3070 (8GB)
RTX 2080 Ti (11GB)
RTX 3060 Ti (8GB)
|
| 3B parameters |
8 GB |
AMD |
Radeon RX 6700 XT (12GB)
RX 6800 (16GB)
RX 5700 XT (8GB)
|
| 7B parameters |
16 GB |
NVIDIA |
RTX 4080 (16GB)
RTX 3080 (10GB with optimizations)
RTX 3090 (24GB)
RTX A5000 (24GB)
|
| 7B parameters |
16 GB |
AMD |
Radeon RX 6800 XT (16GB)
RX 6900 XT (16GB)
RX 6950 XT (16GB)
|
| 13B parameters |
32 GB |
NVIDIA |
RTX 4090 (24GB, with optimizations)
RTX A6000 (48GB)
A40 (48GB)
Tesla A100 (40GB/80GB)
|
| 13B parameters |
32 GB |
AMD |
Radeon PRO W6800 (32GB)
Multiple RX 6900 XT (16GB each)
|
| 30B parameters |
48 GB |
NVIDIA |
RTX A6000 (48GB)
A40 (48GB)
Tesla A100 (80GB)
H100 (80GB)
|
| 30B parameters |
48 GB |
AMD |
Multiple Radeon PRO W6800 (32GB each)
MI210 (64GB)
|
| 1B parameters |
16 GB |
NVIDIA |
RTX 4080 (16GB)
RTX 3080 (10GB with optimizations)
RTX 3090 (24GB)
RTX A5000 (24GB)
|
| 1B parameters |
16 GB |
AMD |
Radeon RX 6800 XT (16GB)
RX 6900 XT (16GB)
RX 6950 XT (16GB)
|
| 3B parameters |
32 GB |
NVIDIA |
RTX 4090 (24GB, with optimizations)
RTX A6000 (48GB)
A40 (48GB)
Tesla A100 (40GB/80GB)
|
| 3B parameters |
32 GB |
AMD |
Radeon PRO W6800 (32GB)
Multiple RX 6900 XT (16GB each)
|
| 7B parameters |
64 GB |
NVIDIA |
Tesla A100 (80GB)
H100 (80GB)
Multiple RTX A6000 (48GB each)
|
| 7B parameters |
64 GB |
AMD |
MI210 (64GB)
Multiple Radeon PRO W6800 (32GB each)
MI250 (128GB)
|
| 13B parameters |
128 GB |
NVIDIA |
Multiple Tesla A100 (80GB each)
H100 (80GB)
Multiple A40 (48GB each)
|
| 13B parameters |
128 GB |
AMD |
MI250 (128GB)
MI250X (128GB)
|
| 30B parameters |
256 GB |
NVIDIA |
Multiple H100 (80GB each)
Multiple A100 (80GB each)
DGX H100
DGX A100
|
| 30B parameters |
256 GB |
AMD |
Multiple MI250X (128GB each)
Custom HPC solutions
|