AI chip startup Etched has successfully raised $120 million in a Series B funding round.
Founded by Harvard dropouts Gavin Uberti and Chris Zhu, Etched focuses on developing a specialized chip designed exclusively for running transformer-based AI models, such as those used by OpenAI’s ChatGPT and Google Gemini.
Etched introduces its fastest AI chip:
Sohu is >10x faster and cheaper than even NVIDIA’s next-generation Blackwell (B200) GPUs.
One Sohu server runs over 500,000 Llama 70B tokens per second, 20x more than an H100 server (23,000 tokens/sec), and 10x more than a B200 server (~45,000 tokens/sec).
Benchmarks are from… pic.twitter.com/5lqZrksgOk
— Etched (@Etched) June 25, 2024
Etched aims to challenge Nvidia’s dominance in the artificial Intelligence Chip market by offering a more efficient and specialized alternative. Nvidia currently holds approximately 80% of the market share for AI chips, primarily through its versatile but power-hungry GPUs.
In contrast, Etched’s chip, known as Sohu, the fastest AI chip of all time, can process over 500,000 tokens per second with the Llama 70B model, enabling the creation of products previously impossible on GPUs.
An 8xSohu server can replace 160 H100 GPUs, showcasing its remarkable efficiency. As the first ASIC explicitly designed for transformer models, Sohu offers unparalleled performance but cannot run other AI models like CNNs, LSTMs, or SSMs.
what use cases does this unlock? https://t.co/9KZXxte3b8
— andrew gao (@itsandrewgao) June 25, 2024
Given that leading AI products like ChatGPT, Claude, Gemini, and Sora rely on transformers, it’s anticipated that custom chips will soon become the standard for running large AI models.
This focus on transformers makes the Sohu chip significantly more efficient and faster, providing up to ten times the speed of Nvidia’s GPUs for specific tasks.
This is the future for running efficiently AI models at scale – specialized chips each type of model.
Maybe there will be diffusion chips next?
— Justin Trugman (@justin_trugman) June 25, 2024
The startup has partnered with Taiwan Semiconductor Manufacturing Co. (TSMC) for chip fabrication, leveraging TSMC’s advanced 4nm process to produce the Sohu chip.
Etched’s founders believe that their product will reduce the energy consumption associated with AI processing and offer substantial cost savings to companies investing heavily in AI technology.
Comment
byu/geekgodOG from discussion
inLocalLLaMA
The $120 million funding round was led by Primary Venture Partners and Positive Sum Ventures, with notable contributions from Peter Thiel, Stanley Druckenmiller, and Kyle Vogt.
This influx of capital will be crucial for Etched to bring Sohu to market and secure a foothold in an industry dominated by giants like Nvidia.
Comment
byu/geekgodOG from discussion
inLocalLLaMA
Etched is entering the AI chip market at a pivotal moment. With the growing demand for generative AI and the increasing costs associated with traditional GPU solutions, the company’s specialized approach could offer a compelling alternative.
The Sohu chip’s streamlined design eliminates unnecessary components for non-transformer models, enhancing performance and reducing energy consumption. Despite the high stakes, Uberti remains confident in the company’s mission.
Etched’s innovative approach to AI chip design positions it as a formidable contender in a market ripe for disruption. With substantial funding and a clear focus on transformer models, the startup is poised to make significant strides in the AI industry.
As the company moves toward bringing Sohu to market, it faces the dual challenges of scaling production and competing against well-established industry leaders. However, with a strong vision and robust financial backing, Etched is well on its way to shaping the future of AI technology.
Found this interesting? Visit allaboutai.com for more exciting daily AI News like this.
