Hugging Face lists which popular models run unmodified on AWS Trainium
Hugging Face has published a lookup page that shows whether popular models on its Hub run unmodified on AWS Trainium chips using TorchNeuron, the PyTorch backend for Trainium. For each tested model it shows the result, how many Trainium cores it was split across, the run time including compilation, peak memory, and the exact script used, so you can repeat the run. For now it only covers Transformers text-generation and image-text-to-text models that fit on four Trainium2 devices, and a pass only means the model generated and checked 32 tokens, not that it is fast or good at long outputs. The results are also a public dataset you can query.
Why it matters: If your team hosts open models on AWS and is weighing Trainium to cut GPU costs, check your model here before you plan a migration, then run your own speed tests.
Hugging Face ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments