When we think about designing and building a new data center in today’s terms, it is not a simple task anymore. Data center facilities are large, capital-intensive fixed assets and requires enormous amount of thought process in it as designing such facility is fraught with risks and cost. Just like building a home, once you get the architecture and foundation completed, whatever shape and form of the house, you will have to live with it for the next 10 to 20 years; For a data center, once you get the architecture right, you create an asset that will facilitate the rapid launch of your new technologies which will give your business that competitive edge.
One of the reasons why it is so difficult to design a data center is the mismatch of the planning horizons. Data center architects are pushed to design and plan for a data center that will last for at least 20 years. However, IT and technology innovation cycle moves at a much faster pace. So, the mismatch here is that we need to design a facility to support future technologies for 20 years, but our insight into the direction of technology is probably only on the horizon of next two or three years.
Therefore, flexibility needs to be one of the key characteristics to be factored in the designing of data centers. Flexibility means the ability to adapt to disruptive technologies, changing trends, adoption of new operational models, changing business cycles, and ability to absorb enormous growth without over-investing in capacity from the start. And as architects, we need to balance flexibility with suitability for today’s existing needs. Overdo it and we’ll end up bursting the budget; Not having sufficient flexibility factored in, and we’ll end up with lots of constraints and issues in the future. For co-location and hosting service provider, this element of flexibility is even more critical (as they will never know who the next customer will be, and what equipment and technologies will they need to house).
In terms of networking for next-generation data centers, it is important that we take into consideration what is happening around us to anticipate the future. Firstly, we all know that IT is growing exponentially, even in businesses that are declining. A simple analogy is this – every day, no matter if your business is profitable or not, no matter if your revenues are growing or not, you will have some transactions. Your business will still incur expenses, pay salaries, pay rent, etc. So, each day, you will have more transaction records, more financial records, and these are data that keeps growing. You can’t run away from it. Capacity will grow. Where businesses are growing, their IT will grow even faster. New business units will demand for new applications and have even more appetite for storage and ultimately, more bandwidth.
The ever-increasing number and scope of applications and the spiraling requirements for data collection, creation and retention drive the growth of servers, storage and bandwidth. A recent survey showed that year-over-year growth, on average, is about 10% in servers, 75% in storage and 99% in bandwidth.
Networks in data centers are no longer just a bunch of corporate workgroup networks, nor low-bandwidth networks. In my younger days, we used terminals to connect to mini-computers or departmental servers, or terminal emulators to submit our jobs. The network is changing and evolving.
One of the other drivers is the changing applications landscape. In the past, we have simple client-server models. Then more and more applications have moved to a three-tier architecture, with Web (presentation layer), application (logic), and the database tier. In the old classic client-server model, traffic flows up and down the network hierarchy. In the three-tier architecture, traffic still flows up and down, but now there may be twice as much traffic between servers at the application and database tiers, as between the client and Web tier. As Service-Oriented Architecture (SOA) adoption increases, it changes the application landscape and the way data and traffic flows, causing application-to-application traffic to dwarf the traffic to/from the client. Traditional network architecture of having a “tree-structure” where there are multiple aggregation/distribution layers, are becoming misaligned with this changing traffic flows. A tree-like structure is good for funneling data up and down the tree efficiently. But when the data flows between servers at the bottom of each branch, data would have to traverse up and down the tree, increasing latency. Hence, it may be necessary to modify the network tree. Removing the layers is one possibility as it helps to improve lateral data flows, however, it may decrease scalability feature. The trend is moving towards collapsing and flattening the network architecture.
Another driver that is evolving is the convergence of network fabrics and protocols. What used to separate networks for storage and data are becoming one now. Storage fibre channels are now merging into IP Ethernets with increasing adoption of FCIP, iSCSI and FCoE (Fibre Channel over Ethernet). Over the years, to reduce costs, unified communications and video conferencing adoption have increased, with voice and video being now absorbed into our IP networks. As virtualization adoption increases, it also becomes a strong driver for storage convergence. For example, one benefit to be gained from server virtualization is network booting, live migration and disaster recovery, where operating systems and application images will be stored on network-based storage. So, virtualization makes storage ubiquity weigh more heavily than raw performance.
As Fibre Channel has slowly progressed from 2Gbps to 4Gbps and beyond, Ethernet performance has grown at even faster pace and even more dramatically, from the days of 100Mbps to 1Gbps to 10Gbps and beyond. Although Ethernet may not be suitable for high-performance networked storage (like in HPC environments), it is increasingly outpacing the evolution of Fibre Channel. The trend is that storage frabric are slowly converging into Ethernet. Network switches that support both storage and Ethernet fabrics can be a solution to bridge the gap and lead to a smoother transition towards a converged network fabric.
Few key considerations (in the following sequence) to note in designing networking and structured cabling for next-generation data centers :
- Modularity: Depending on type of servers, use of networked storage, how flat (or not) is the network, and the balance between port density and management complexity, the topology of future network architectures can vary dramatically. And this in turn affects decisions and options around cabling. So, therefore, one way to overcome the uncertainty is to architect and build the core backbone, then extend the network to a collapsed access layer and distribution layer at the end of each group of racks (in Sun, we call this the Sun POD), for the number of groups of racks you’re going to deploy today and expand this as you grow.
- Future-proofing Cabling: For each modular group of racks (or POD) you deploy, try to pre-positioning and run fibre alongside the twisted pair to each group of racks (or POD). This gives you the flexibility to change and enable future upgrades without messy re-wiring. Short runs of copper twisted pair can easily run 1Gbps, or even up to 10Gbps, but beyond a certain length, 10Gbps requires fibre. Or the other way round, we’ll never know the future of Ethernet may reach even up to 100Gbps (possible?) and perhaps this will require fibre.
- Think inside the box: In the past, traffic caching, acceleration and optimization are used to boost performance between WANs end points. As distribution of users changes, workplace becomes virtual, application landscape evolves, the future may see some of these application delivery and optimization features applied within the data center fabric, likely in the form of what is termed as application delivery subnets. As IT services becomes increasingly more crucial to the business, continuity is critical. This means that it may be a necessity for businesses to plan for DRP with inter-connected data centers via dedicated link circuits or an MPLS network, which provides Layer 2 connectivity or Layer 2 VPLS. Bringing these points together will mean that future networks are no longer following traditional rules anymore. Time to take off that old hat…
Tags: Data Center, Network