Resources
Groq builds the world’s fastest AI inference technology.
FAQts
What is Groq?
Groq is the AI infrastructure company that builds the world’s fastest AI inference technology.
Groq® LPU™ AI inference technology is a hardware and software platform that delivers exceptional compute speed, quality, and energy efficiency.
Groq, headquartered in Silicon Valley, provides cloud and on-prem solutions at scale for AI applications. The LPU and related systems are designed and manufactured in North America.
What is Groq® LPU™ AI inference technology?
Groq LPU AI inference technology, with LPU standing for Language Processing Unit, is the hardware and software platform by Groq that delivers exceptional compute speed, quality, and energy efficiency. This new type of end-to-end processing unit system provides the fastest inference for computationally intensive applications with sequential components, such as AI language applications like Large Language Models (LLMs).
Why is it so much faster than GPUs for LLMs and GenAI?
The LPU is designed to overcome the two LLM bottlenecks: compute density and memory bandwidth. An LPU has greater compute capacity than a GPU and CPU in regards to LLMs. This reduces the amount of time per word calculated, allowing sequences of text to be generated much faster. Additionally, eliminating external memory bottlenecks enables the LPU Inference Engine to deliver orders of magnitude better performance on LLMs compared to GPUs.
For a more technical read about our architecture, download our ISCA-awarded 2020 and 2022 papers.
Does Groq run standard machine learning frameworks?
Groq supports standard machine learning (ML) frameworks such as PyTorch, TensorFlow, and ONNX for inference. Groq does not currently support ML training with LPU AI inference technology.
For custom development, the GroqWare™ suite, including Groq Compiler, offers a push-button experience to get models up and running quickly. For optimizing workloads, we offer the ability to hand code to the Groq architecture and fine-grained control of any GroqChip™ processor, enabling customers the ability to develop custom applications and maximize their performance.
How do I get started using Groq?
We’re excited you want to get started with Groq. Here are some of the fastest ways to get up and running:
Developer access can be obtained completely self-serve through GroqCloud™ Developer Console. There you can obtain your free API key and access our documentation as well as our terms and conditions on Playground. Join our Discord community here. If you are currently using OpenAI API, you just need three things to convert over to Groq:
- Groq API key
- Endpoint
- Model
Do you need the fastest inference at data center scale? We should chat if you need:
✓ 24/7 support
✓ SLAs
✓ A dedicated account representative
Let’s talk to ensure we can provide the right solution for your needs. Please fill out this form and and our team will be in touch.
The Latest on Groq
Interested in covering Groq? Reach out to our PR team.
01/15/2025
- 01/15/2025
San Francisco, CA
