Startup Fractile tackles AI inference bottlenecks with new chip design

A 29-year-old entrepreneur with a Ph.D in robotics is looking to shake up the AI chip industry with an innovative approach that promises to deliver hardware that is 100 times faster, 10 times cheaper, and 20 times more energy efficient than the Nvidia GPUs that dominate the market today.

Walter Goodwin (pictured) says he got the idea for the company when he was at the Oxford Robotics Institute researching ways to use large language models (LLM) to build general-purpose robots that could navigate real-world environments.

Goodwin founded Fractile in 2022 on the premise that there was an untapped market for chips that attack the AI inference performance bottleneck problem by combining storage and compute on one chip. Fractile emerged from stealth in 2024 and has seen its profile rise, particularly after former Intel CEO Pat Gelsinger posted on LinkedIn recently that he had invested in the company and planned to be involved in an advisory role.

Gelsinger wrote: “With the advent of reasoning models, which require memory-bound generation of thousands of output tokens, the limitations of existing hardware roadmaps have been compounded. To achieve our aspirations for AI, we will need radically faster, cheaper and much lower power inference. I’m pleased to share that I’ve recently invested in Fractile, a U.K.-founded AI hardware company who are pursuing a path that’s radical enough to offer such a leap.”

Prabhu Ram, vice president of the Industry Research Group at Cybermedia Research, adds: “I have followed Fractile for its innovative AI chip design, but my interest intensified after Pat Gelsinger’s investment.”

“Fractile is focused on in-memory computing, which enhances AI inference by reducing latency and energy consumption through minimized data transfers. While AI discussions have traditionally centered on training, the focus is now shifting toward inference,” Ram says. He notes that there are other players in the market, but Fractile’s “novel chip design and strong funding position it as a promising alternative.”

From academic to entrepreneur

Goodwin says the transition from academics to running a company has been pretty smooth. He points out that the academic world has a competitive aspect to it.

“You end up a founding team of one for a company that is your thesis effort. You need to ensure that you have a competitive edge and a different angle than other people in the field,” he says.

“The Ph.D is high-altitude training for running a company. As a founder, one of things I like about building a company is all the people that you have to find and persuade,” Goodwin adds. So far, he’s built a workforce of 32 “unbelievably brilliant colleagues.”

What does Fractile do?

There are two pieces to an AI model – training and inference. The training function prepares the LLM for inference, which is the creation of outputs, the response to queries, the generation of images, text, code, etc.

Goodwin explains that the traditional approach to inference is that the AI chip needs to refer back to the training model database, which is stored in DRAM, and could account for terabytes of memory that needs to move from storage to compute. And this process occurs every time the AI system, like ChatGPT, adds a new word to its output.

By fusing memory and inference processing on a single chip, “the benefits are a hundred-fold increase in effective bandwidth and much higher energy efficiency,” says Goodwin.

How could Fractile benefit enterprises?

For enterprise IT, this means that organizations deploying Fractile chips could gain competitive advantage in a number of ways, including answer-bots that produce significantly more words per second, or the ability to serve more customers in parallel for the same costs.

As AI systems themselves improve and get better at reasoning, they will advance from outputs that simply jump to the next word in serial fashion to outputs that constitute longer chains of thought.

“You can serve chatbots to users 100 times faster, or you can serve answers at the same rate, but have it think 100 times harder,” Goodwin says. These types of neural networks can provide organizations with an edge, particularly as AI becomes embedded in more business processes over time. “Your customer base or employee base would have access to the world’s fastest inference models, which would have a transformative effect,” Goodwin says.

The road ahead for Fractile

Goodwin concedes that “building a full hardware and software stack is a heavy lift” that will require a workforce of more than 100 people and more than $25 million in capital. So far, the company has architected the hardware and modeled its behavior. The plan is to further accelerate development efforts.

“We’re running at a good clip. If it all comes together, we should be sending a chip by the end of next year, on the road to a shipping product in 2027.” On the business side, he adds that the company, which has raised around $20 million so far, needs to continue to bring in further investment.

Early backers include Kindred Capital, NATO Innovation Fund, Oxford Science Enterprises, Cocoa VC and Inovia Capital, as well as angel investors and alums of other AI and semiconductor companies.

So, is Fractile audacious enough to think it can knock Nvidia off its perch, or is it targeting a more limited market for custom chips? Goodwin says Fractile wants to “thread the needle,” building a hardware/software stack that supports “a generation of flagship products that serve multiple customers,” including both general purpose and custom workloads.

Ram cautions that for startup Fractile, “the path to market leadership remains challenging.”

“While Fractile still prepares to launch, the hyper-competitive AI hardware landscape continues to evolve, with established players as well as emerging start-ups advancing rapidly,” Ram says. “To challenge industry leaders like Nvidia, Fractile must not only deliver on its ambitious performance claims, but also build a strong developer ecosystem to ensure long-term viability.”