Raising the speed limit on AI’s “information highway”

As part of a new series about what the artificial intelligence economy means for you, “Marketplace Tech” is looking at the infrastructure build-out behind the AI boom. 

Investors have poured billions of dollars into artificial intelligence in recent years and by far the largest share of that spending has gone towards infrastructure.

One of the biggest players in that market is Amazon Web Services (AWS). The cloud computing company builds data centers, software, and hardware on which the AI economy will ultimately run. 

“A network is almost like a data highway, or information highway,” said Satish Vangala the director of network product development at AWS.

AI applications generate significantly more network traffic than traditional cloud servers, which is why AWS and other cloud service providers, including Google and Microsoft, are racing to build more capacity. 

At AWS’ offices in Cupertino, California, Vangala showed Marketplace’s Meghan McCarty around a research and development lab where they are experimenting with new ways of doing that. 

One example is the deployment of “hollow core” fiber cable, which can transport data faster than traditional fiber optic cable. Vangala said that the technology effectively raises the “speed limit” on that metaphorical information highway. 

Another solution has to do with making the process of building networks more efficient for technicians. Vangala pointed to a rack of plugs that link computer servers in data centers together. “Plugging in this many connections…it takes a lot of time, and doing it reliably is also a challenge,” he said. 

To help address that, Vangala’s team designed a solution combining 64 fibers into one plug, so that technicians have fewer connections to manage. “[That] means we can build data centers faster,” said Matt Rehder, Vice President of Core Networking at AWS. “It also improves the reliability of the network.”

Rehder said AWS has invested heavily in designing its own network hardware for more than a decade, but the increased demand from AI has ramped up the speed and scale at which they’re trying to build. “We just need more network,” he said. 

Later this week, “Marketplace Tech” will explore other parts of the AI infrastructure economy, including a data center with historical significance and one company hoping to change the energy equation.