Etched: Revolutionizing AI with Sohu ASICs

Etched, a groundbreaking two-year-old startup, has made waves by raising $120 million to pioneer a new era in server technology. Their innovative servers leverage custom-designed chips that promise to perform AI inferencing at speeds an order of magnitude faster than Nvidia’s upcoming Blackwell GB200 chip. Central to Etched’s technological breakthrough is their application-specific integrated circuit (ASIC) named Sohu, crafted using TSMC's advanced 4nm process.

The Power of Specialization: Sohu ASIC

Sohu is uniquely designed to excel at running transformer models, a type of neural network architecture that excels at understanding and generating human-like text by analyzing sequences of words. This specialization allows Sohu to achieve exceptional performance, processing over 500,000 tokens per second when running the Llama 70B model. Etched proudly claims that "Sohu lets you build products that are impossible on GPUs," with one 8xSohu server replacing the equivalent power of 160 Nvidia H100 GPUs.

By focusing exclusively on transformer models, Sohu achieves unparalleled efficiency. Unlike general-purpose AI hardware, Sohu cannot run convolutional neural networks (CNNs), long short-term memory networks (LSTMs), state-space models (SSMs), or any other AI model types. This trade-off allows Etched to deliver significantly higher performance for transformer-based applications, which power every major AI product today, including ChatGPT, Claude, Gemini, and Sora.

Future-Proofing AI Infrastructure

Etched envisions a future where every large AI model operates on custom chips like Sohu. To help customers experience the transformative potential of their technology, Etched will soon launch the Sohu Developer Cloud. This platform will enable developers to experiment with and harness the power of Sohu, driving wider adoption and integration of Etched’s groundbreaking technology.

Backed by Visionaries

Etched's ambitious vision is backed by prominent investors, including Peter Thiel, GitHub CEO Thomas Dohmke, Cruise co-founder Kyle Vogt, and Quora co-founder Charlie Cheever. Their support underscores the confidence in Etched’s potential to reshape the landscape of AI hardware and infrastructure.

In summary, Etched's Sohu ASIC represents a significant leap forward in AI inferencing performance, particularly for transformer models. By harnessing the power of specialization, Etched is poised to lead the charge in developing the next generation of AI servers, setting a new standard for efficiency and speed in the industry.