Groq, a startup creating chips to run generative AI fashions sooner than standard processors, stated on Monday that it’s raised $640 million in a brand new funding spherical led by Blackrock. Neuberger Berman, Kind One Ventures, Cisco, KDDI and Samsung Catalyst Fund additionally participated.
The tranche, which brings Groq’s complete raised to over $1 billion and values the corporate at $2.8 billion, is a significant win for Groq, which reportedly was initially trying to increase $300 million at a barely decrease ($2.5 billion) valuation. It greater than doubles Groq’s earlier valuation (~$1 billion) in April 2021, when the corporate raised $300 million in a spherical led by Tiger International Administration and D1 Capital Companions.
Meta chief AI scientist Yann LeCun will function a technical advisor to Groq and Stuart Pann, the previous head of Intel’s foundry enterprise and ex-CIO at HP, will be a part of the startup as chief working officer, Groq additionally introduced at this time. LeCun’s appointment is a bit sudden, given Meta’s investments in its personal AI chips — but it surely undoubtedly provides Groq a strong ally in a cutthroat area.
Groq, which emerged from stealth in 2016, is creating what it calls an LPU (language processing unit) inference engine. The corporate claims its LPUs can run current generative AI fashions comparable in structure to OpenAI’s ChatGPT and GPT-4o at 10x the pace and on-tenth the vitality.
Groq CEO Jonathan Ross’ declare to fame helps to invent the tensor processing unit (TPU), Google’s {custom} AI accelerator chip used to coach and run fashions. Ross teamed up with Douglas Wightman, an entrepreneur and former engineer at Google mother or father firm Alphabet’s X moonshot lab, to co-found Groq near a decade in the past.
Groq offers an LPU-powered developer platform known as GroqCloud that gives “open” fashions like Meta’s Llama 3.1 household, Google’s Gemma, OpenAI’s Whisper and Mistral’s Mixtral, in addition to an API that enables clients to make use of its chips in cloud situations. (Groq additionally hosts a playground for AI-powered chatbots, GroqChat, that it launched late final yr.) As of July, GroqCloud had over 356,000 builders; Groq says {that a} portion of the proceeds from the spherical will likely be used to scale capability and add new fashions and options.
“Many of those builders are at massive enterprises,” Stuart Pann, Groq’s COO, instructed TechCrunch. “By our estimates, over 75% of the Fortune 100 are represented.”
Because the generative AI growth continues, Groq faces growing competitors from each rival AI chip upstarts and Nvidia, the formidable incumbent within the AI {hardware} sector.
Nvidia controls an estimated 70% to 95% of the marketplace for AI chips used to coach and deploy generative AI fashions, and the agency’s taking aggressive steps to take care of its dominance.
Nvidia has dedicated to releasing a brand new AI chip structure yearly, relatively than each different yr as was the case traditionally. And it’s reportedly establishing a brand new enterprise unit targeted on designing bespoke chips for cloud computing companies and others, together with AI {hardware}.
Past Nvidia, Groq competes with Amazon, Google and Microsoft, all of which provide — or will quickly provide — {custom} chips for AI workloads within the cloud. Amazon has its Trainium, Inferentia and Graviton processors, accessible by means of AWS; Google Cloud clients can use the aforementioned TPUs and, in time, Google’s Axion chip; and Microsoft just lately launched Azure situations in preview for its Cobalt 100 CPU, with Maia 100 AI Accelerator situations to come back within the subsequent a number of months.
Groq might contemplate Arm, Intel, AMD and a rising variety of startups rivals, too, in an AI chip market that might be attain $400 billion in annual gross sales within the subsequent 5 years, in accordance with some analysts. Arm and AMD specifically have blossoming AI chip companies, due to hovering capital spending by cloud distributors to fulfill the capability demand for generative AI.
D-Matrix late final yr raised $110 million to commercialize what it’s characterizing as a first-of-its-kind inference compute platform. In June, Etched emerged from stealth with $120 million for a processor custom-built to hurry up the dominant generative AI mannequin structure at this time, the transformer. SoftBank’s Masayoshi Son is reportedly trying to increase $100 billion for a chip enterprise to compete Nvidia. And OpenAI is stated to be in talks with funding companies to launch an AI chip-making initiative.
To carve out its area of interest, Groq is investing closely in enterprise and authorities outreach.
In March, Groq acquired Definitive Intelligence, a Palo Alto–based mostly agency providing a spread of business-oriented AI options, to type a brand new enterprise unit known as Groq Programs. Inside Groq Programs’ purview is serving organizations, together with U.S. authorities companies and sovereign nations, that want to add Groq’s chips to current knowledge facilities or construct new knowledge facilities utilizing Groq processors.
Extra just lately, Groq partnered with Carahsoft, a authorities IT contractor, to promote its options to public sector purchasers by means of Carahsoft’s reseller companions, and the startup has a letter of intent to put in tens of hundreds of its LPUs at European agency Earth Wind & Energy’s Norway datacenter.
Groq can also be collaborating with Saudi Arabian consulting agency Aramco Digital to put in LPUs in future datacenters within the Center East.
On the identical time it’s establishing buyer relationships, Mountain View, California-based Groq is marching towards the subsequent technology of its chip. Final August, the corporate introduced that it might contract with Samsung’s foundry enterprise to fabricate 4nm LPUs, that are anticipated to ship efficiency and effectivity beneficial properties over Groq’s first-gen 13nm chips.
Groq says it plans to deploy over 108,000 LPUs by the tip of Q1 2025.