Is your DC AI-Ready

Artificial Intelligence (AI) technologies have started becoming larger parts of data center (DC) workloads, and the global DC community is challenged to make existing and new facilities AI-ready. Before the DC operation team decides on a range of actions, it is crucial to understand what AI is about, the type of expected IT workloads, and the immediate and future market needs. The key steps are evaluating infrastructure components like power, cooling, network capacity, space utilization, and structural elements and proposing technical solutions. After analyzing all the above subjects, technical proposals, and points for further investigation, important conclusions and recommendations are presented here.

Artificial Intelligence (AI) is the theory and development of computer systems’ capability to perceive their environment, reason about their status, and select actions to perform certain tasks, in a similar way humans recognize visual or logical patterns, understand natural language, make decisions, and select actions accordingly. Like humans, any AI system can develop intelligence based on experience and learning. This procedure is widely known as AI model training, where algorithms are fed massive data sets and trained to identify patterns, create experience, and develop non-linear models that relate the input data patterns to output predictions. AI is a general term that encompasses several technologies like machine learning (ML), deep learning (DL), and natural language processing (NLP), all of which use the same workflow to produce models. Today, AI is not just a theoretical concept but a wide range of technologies that power services and products we use daily. Applications like face recognition, online recommendation systems, and real-time communication chatbots are tangible examples of AI systems trained on massive amounts of data to identify face characteristics, gender, age, or tokens of word sequences, create meaning, and respond verbally or by text.

Most of this activity by AI systems is taking place inside data centers. Hundreds of servers, storage units, and network switches process and maintain massive amounts of AI (and non-AI) critical data. The modules are arranged into racks and further into clusters to distribute data and perform parallel processing. Two essential measures of IT data flow and equipment intensity define the data center capacity and operation capabilities:

AI activity mainly includes leveraging various sizes of data sets and data streams through the data center’s computational resources. By performing storage, processing and modeling, the AI workload is: a) Defined in terms of various data processing procedures like training, inference, data analytics and high-performance computing (HPC). b) Quantified through the required IT computing power (kW) needed to implement. The AI workload types managed by a data center facility are the following:

HPC is an area of application mostly related to high-end research and academic applications where either traditional analytical approaches or AI based algorithms leverage large data sets and perform precise and floating-point demanding calculations to solve a problem. In this case, again, the data center infrastructure operator will have to provide stacks of GPUs arranged into high-intensity racks and clusters, which become power-demanding and heat-generative. Recent advancements in medical research with DNA sequence alignment and protein structure prediction indicatively formalize the application of HPC and consequent data center infrastructure requirements.

The projection of AI growth over the next years can be estimated by considering the evaluated growth of GPU sales as one of the main hardware elements in the AI evolution as a point of reference. The global graphic processing unit (GPU) market size is expected to reach USD 773.07 billion by 2032 and is poised to grow at a compound annual growth rate (CAGR) of 33.8% from 2023 to 2032.

Considering the estimation that AI represents 4.3GW of power demand today and projecting this to grow at a CAGR of 33.8% resulting in a total demand of 18.5GW by 2028, as reflected on the graph above, while the annual CAGR for data center non-AI IT loads is estimated to 11%.

Data center infrastructure readiness for AI integration requires a thorough understanding of AI workloads, evaluation of infrastructure components, implementation of technical solutions, continuous investigation, and proactive adaptation. Practically, the DC operation team shall investigate subjects related to power availability, cooling system capacity, network latency, and structural elements like space or raised floors in detail. Essential points are discussed below as starting points for further investigation.

Data The purpose of cooling in any data center is to remove the heat produced by IT equipment inside the data hall area and maintain proper temperature and humidity conditions within the operation range of IT equipment. Most existing data centers use air cooling systems for the IT rooms, where cooled air is provided within the spaces through terminals like CRAC or CRAH units or AHU or even FAN Wall. Air as a means of heat removal has a certain thermodynamic capacity limitation, and air-cooling systems have been proved to be a satisfactory and mature solution for rack densities up to 20 kW. In AI clusters where the rack densities are above this value and can potentially reach figures equal to 50 or 80 kW/rack, alternative technologies are needed, with the most appropriate being liquid cooling (LC). Liquid cooling is represented by three main technologies: the Rear Door Heat Exchanger, Direct-to-Chip (DTC), and Immersive Cooling (IC). All can support extreme rack densities.

AI clusters for model training shall engage a large number of GPUs, providing separate network ports for each. Connection with high-speed network cables will allow for the complete utilization of processing speed. The operators shall coordinate with the IT development team to evaluate network cable options. Using fiber allows larger distances between nodes and clusters, reducing the rack density with fewer servers per rack. On the other hand, in the case of copper cables, limitations on distance will force the team to stack more servers within the same racks to reduce the network latency. In this case, higher rack densities will make the system deployment more complicated and demanding for the DC infrastructure.

Except for power and cooling provisions required by DC operators for an AI-ready data center, several other points shall be equally investigated and verified that can serve or be rectified to accommodate the AI upgrade. Racks sized for AI servers are expected to become larger in width and height to allow enough space for deep learning servers; similarly, in cases where larger PDUs are designed for increased power rating and space for liquid cooling, the manifold will push for deeper racks. Also, the operators shall check the durability of AI-specified racks as more heavy gear is expected to be installed. For DCs with raised floors, the operators shall carefully evaluate the floor-concentrated design load and wheel rolling load. AI racks will become heavier than normal density racks, and special consideration should be given. Main runs of chilled water pipes for connection with DTC liquid-cooled racks or IC tanks are not preferred for most operators to be from above. The DC operation team shall verify whether the existing chilled water system can be used or identify new routes. In the case of raised floor existence, normally, the pipe runs will be underneath, but for facilities without, different options shall be explored through adjacent corridors or by providing trenches on the floor.

Data center operators shall investigate their existing and future market needs in coordination with their business targets and budget allocations. The creation of a medium—and long-term plan related to IT capacity management and rack density distribution will allow them to make critical decisions related to all technical subjects analyzed above. Continuous communication with the market, evaluation of technology trends, and cooperation with existing and future clients are key components of strategy development. In many existing data centers where complete new design and development are not part of business planning, hybrid-type data centers are appropriate for short- to medium-term development. A hybrid DC shall continue to operate all existing facilities, including IT racks and air cooling, but simultaneously will allocate power and space for a small, dedicated AI cluster with increased power supply and liquid cooling following client needs. Hybrid-type solutions create concerns for Tier-certified DCs and the impact on their certification, although discussions for future DC facilities with multiple Tier certificates may arise as a solution. Similarly, the absence of international standards for certification of liquid cooling systems is a subject of concern, and further investigation from the DC community is required, as most of the market-proposed solutions are developed and tested only by the vendors. Institutional certifications for AI hardware and liquid cooling solutions have become a necessity, along with the involvement of international facility certification standards like Uptime Institute Tier rating. Furthermore, the evolution of new technologies, like Tensor Processing Units (TPU), is subject to challenges from GPUs and steers the market even further. Technological trends and different approaches in terms of hybrid cloud models will impact more. Developers tend to use ‘public data’ to develop a foundational model on the cloud and then augment and make it business-specific using private data in their own computing facilities to maintain control of their data. Then, the model will be deployed, and all the inferencing will happen on the edge. This workflow model leaves colocation DCs out of the loop for much of the workload. The future is expected to be very interesting, and the evolution of AI as a service within data centers is currently knocking on the door for facility managers.