Data Coordination
At the center of the data storage software infrastructure, the data coordination functions perform the task of coordinating data access between applications and storage subsystems. These underlying operations serve other storage management functions, such as data protection, policy management, and resource management. In general, the coordination functions sit closer to the movement and organization of the data.
The data coordination functions are transaction-focused areas of storage management. Acting as the interface between applications and subsystems, the primary operations include file services for NAS and volume management for block-accessed storage. In both cases, storage consolidation allows administrators to make the most of the infrastructure through storage pooling or aggregation to a common application presentation. The basic outline of these functions is shown in Figure 3-7.
Figure 3-7. The data coordination path.
3.4.1 File Services for NAS
File services, or file systems, provide users with the basic functions of
A naming scheme for files and directories that enables organization of data for multiple users and applications.
Authorization for access and modification control that includes a breakdown of creating, reading, and writing operations.
Storage allocation that coordinates with volume management functions to determine where files may reside.
File services provide applications a simpler and easier mechanism to access blocks of disk-based storage. By abstracting the working unit of storage from a disk block fixed in size and location, a file system allows applications to work with files that can be stored across multiple disk blocks and locations through the use of volume management. This facilitates file transactions such as varying the file size, creation and deletion, shared identification and use of file data across multiple users, and access and modification control.
Many early computing applications—particularly those in the mainframe world and those focused on databases—were designed to attach directly to disk storage that was accessed in blocks. Today's computing environments rely heavily on the ability to store and retrieve data in file-based formats. File systems may be used directly by operating systems to maintain information such as what parameters should be used to start the machine, including modules to load and network services to initiate. Once the operating system is running, it may continue to use files to keep information temporarily available but out of memory or to log events that occur during the normal course of operation.
The more common understanding of file systems is for the applications themselves. In these cases, applications use files to hold parameters specific to their requirements, common modules also used by other applications, logs on application-related transaction information, and ultimately the data itself.
The file services function can reside throughout the data storage infrastructure. All host operating systems provide their own file systems, and many provide multiple file systems tailored to different needs of media. In addition, specialized file systems can be added to existing host operating system platforms to provide enhanced functionality. On the server, a network file system like NFS of CIFS maps application-level file system operations into transactions over the network with NAS filers. In turn, NAS filers use a variety of internal file systems to hold the file objects that they service. As a specialized server for file-based storage access, NAS provides scalability benefits in multiuser, dynamic-growth storage environments. New systems take file services one step further by providing them within storage networking fabrics. Often referred to a NAS head, by providing file access in the front and block access (volume management) in the back, these devices provide an additional level of fabric flexibility.
3.4.2 Volume Management
Volume management provides operating systems a means to aggregate disks and LUNs into logical storage units called volumes. Volumes can be directly accessed by operating systems, which may be the case with database applications, or through file systems. In either case, each volume can be made up of a set of disks and LUNs that are connected to the appropriate host, directly or through a storage network.
With a software management layer for volumes between the host and storage subsystem, specific features can be offered to the administrator. The volume management layer allows for the creation of dynamic disks, which then enables the logical manipulation of a group of dynamic disks. This disk group can be used to support one large logical volume, or software-based RAID may be implemented.
Additional features of volume management include modification of dynamic disks within the group, access and control features of hosts to volumes, and the ability to redirect data requests across volumes for failover.
Volume management provides users with the basic functions of
A naming scheme for volumes made up of LUNs that enables organization of data for assignment to multiple hosts and applications.
Storage allocation that coordinates with file system or database management functions to determine where files or databases may ultimately reside.
3.4.3 Storage Aggregation
Using file services and volume management, the overriding benefit of these software tools is that they provide an abstraction layer between applications and disks. This provides consolidation capabilities for aggregating file or volume presentation. In a homogeneous environment, file services can bind multiple volumes to a file system, and volume management binds disks to a volume. Another benefit is the ability to provide file or record-locking, or both, across multiple servers.
The basics of virtualization rely upon the storage aggregation capabilities of file services and volume management. For definition purposes, we focus on virtualization primarily in heterogeneous environments, including a variety of device types as well as vendors, covered in Section 3.8, "Virtualization."
Storage aggregation delivers configuration flexibility to storage administrators, providing the ability to layer more sophisticated storage management, such as policy management, within the configuration.
Storage Policy Management
An effective storage management operation needs storage policies to control and enforce the use of resources. While SRM provides a view of storage devices and the ability to change specific device features, policy management takes a wider approach across the infrastructure to define, implement, and enforce a set of rules between users, applications, and storage.
This section focuses on the basic elements of storage policy management within the technical storage networking context. Chapter 11, "Managing the Storage Domain," takes a more global view of the corporate oversight of storage resources from an organizational perspective.
First and foremost, storage policies help administrators overcome the unending growth of storage resources within the organization. From the enterprise to departmental to personal storage, such as workstations and laptops, administrators need automated mechanisms to set usage rules and provide service levels required throughout the organization.
Basic storage policies (see Figure 3-8) can be broken into the following groups:
Security and authentication policies ensure the proper access and manipulation of the data.
Capacity and quota management policies ensure measured allocation across multiple users.
Quality of service policies guarantee service levels for both the storage itself and the access to that storage.
Figure 3-8. Storage policies.
3.5.1 Storage Policy Cycle
Storage policies have a continuous cycle with stages, outlined in Figure 3-9. The exact definition of stages may vary slightly among organizations, but the important common characteristic is the ongoing process improvement built into the cycle.
Figure 3-9. Storage policy cycle.
Assessment involves observing usage and performance patterns to determine areas for improvement. One example might be an application whose data set continues to grow exponentially, reducing the amount of available storage for other applications sharing the same storage. The definition stage requires identification of the anomaly's root cause and a method to solve it. Implementation comes through setting a rule or policy within the software infrastructure. Depending on the cause, effect, and desired results, setting a policy may occur in more than one component of storage software. Enforcement takes place when the desired actions are achieved through the rules. Careful monitoring provides data for evaluation of the set policy and the ability to begin the cycle again with any required modifications.
In many ways, an active storage infrastructure with a broad mix of users, applications, and resources is like an evolving ecosystem. Cause-and-effect relationships rarely remain isolated, and administrators must track implemented policies to watch for unintended effects. Appropriate monitoring through the use of SRM tools, will provide adequate notification for administrators to adjust accordingly.
3.5.2 Capacity, Content, and Quota Management
Every organization aims to maximize the use of resources, and SRM along with storage policies help accomplish that goal. Examples of policies in action include capacity, content, and quota management.
Capacity management involves optimization of storage space on a given device, such as a RAID array, or within a volume. In an ideal world, all devices would operate at or close to 100 percent capacity. However, the reality of our dynamic computing environments dictates otherwise. Unpredictable spikes in storage demand coupled with the need to have extra real-time capacity available means that 100 percent utilization can actually be counterproductive.
The optimal utilization percentage depends on device and storage cost, application needs, and costs of adding more physical capacity. For example, for a high-end RAID array with relatively expensive storage, the optimal utilization might be around 90 percent with 10 percent reserved for unanticipated peak demand. Less expensive storage might be set with utilization peaks of 60 percent, leaving administrators plenty of leeway for surges in capacity needs before new storage devices become necessary.
On the content side, visibility to the types of data and files can help dramatically reduce storage needs. By examining and categorizing content sources, power consumers in the form of applications or individuals can be identified and contained. For example, the rapid proliferation of MP3 music files might indicate improper use of corporate storage resources. Excessive numbers of large files (image, CAD design) with similar names and underlying data structures could be the result of saving inordinate copies, which might be satisfied with a less expensive storage mechanism, such as tape archiving.
Quota management, after identifying storage content by source and data type, allows administrators to pinpoint and curtail unnecessary consumption. This may apply to corporate users from engineering to administration, or to key applications. Of course, quota management requires a set of rules and guidelines that fit with organizational priorities in concert with IT priorities. A cap on engineering storage capacity that impacts development schedules might require adding to new storage purchase budgets. On the other hand, capping resources for email storage might compel individuals to be more conscious about their consumption without impacting productivity.
3.5.3 Security and Authentication
Security for storage (also see Chapter 9, Section 9.5, "Security for Storage Networking") begins by identifying threats and then implementing solutions to address them. Some of these threats and solutions are outlined in Table 3-1.
Table 3-1. Storage Security Threats and Solutions Potential Threat
Solution
Is the entity accessing the storage whom he or she claims to be?
Authentication
Does the entity accessing the storage have the right clearance?
Authorization
Is the data being read or shared unintentionally?
Privacy
Is the data subject to unintentional or malicious modification?
Integrity
Enforcement of these storage security solutions occur throughout the storage management chain, depending on the software in place. While the implementation of these solutions can vary, the objectives remain the same. For example, authentication can be set within the file system, identifying access at the user level, or this function can take place within lower layers of the infrastructure, such as the network fabric. Storage security therefore requires a multifaceted approach across the entire data management chain, from user access, to device access, to device management.
The most basic form of storage security is the physical isolation of the storage environment. Fibre Channel-based storage networks provide some physical security because the technology is separate from more common corporate IP networking. In the early days of Fibre Channel, storage administrators viewed this physical isolation as a feature—the rest of their networking colleagues were unable to understand and meddle with their setups. More common availability of Fibre Channel equipment no longer guarantees such independence.
LUN masking determines which disk drives (or logical unit numbers) are seen by servers. LUN masking can be enforced by HBAs, switches, or storage array controllers. Modification of LUN masking requires password-protected access to the configuration utility. Zoning by port or WWN via SAN switches is one form of LUN masking.
The use of IP networking technologies for storage also delivers a set of well-defined security mechanisms. Virtual LANs (VLANs) and access control lists (ACLs), common in mainstream IP networking, can be used to segment and isolate traffic between end points or network areas of an IP storage network. Existing IPSec functions, such as authentication and data encryption, which are defined in IETF specifications, can be used with IP storage networking. For example, virtual private network equipment can be used between two IP storage switches or gateways to deliver encrypted IP storage traffic between two SANs.
3.5.4 Storage Encryption
While encryption has long been used as a security mechanism for IP networks, that capability has been largely absent from conventional SANs, partly based on the historical lack of support for Fibre Channel encryption. Recent product introductions now make possible Fibre Channel encryption along with the associated management tools required to administer encryption keys.
While security tools such as zoning, ACLs, and authentication address storage access security, they do not address the security of the data payload. In order for the payload to be protected, it must be encrypted with a key that locks (and unlocks) the original data structures. Encryption keys for 3DES (one of the most secure forms of encryption) are made up of 168 bits.
Encryption can be implemented in software and located within Fibre Channel equipment or specialized encryption hardware. Current implementations typically use hardware-based encryption to guarantee high throughput rates. A number of configurations can be deployed to guarantee data security at the payload level. Even if an unauthorized user were to access the data, it would be useless without the key. The basic configurations are shown in Figure 3-10. In addition to these solutions, users can encrypt data at the file-system level before entrusting it to the storage subsystem.
Figure 3-10. Types of storage encryption. (Source: NeoScale Systems)
In a fabric attached deployment, storage encryption takes place within the storage network, allowing multiple storage devices to be protected. All storage that resides behind the encryption device requires key authentication for data access. Simpler configurations such as subsystem attached encryption allow data to be moved to third-party service providers or offsite vaulting with security guarantees. Data that ended up in the wrong hands while being moved offsite would still be protected. A gateway or tunnel implementation allows data to be encrypted at the payload level while traversing MANs or WANs. Application-attached encryption restricts the protection to those applications requiring the highest level of security.
Of course, in any implementation, key management determines the accessibility and recoverability of encrypted data. IT professionals considering payload-level encryption will need to keep keys in multiple secure locations, including legal offices and other venues that guarantee protected access. Loss of a key results in permanent data loss.
3.5.5 Quality of Service for Storage and Storage Area Networks
Quality of service applies to end-storage devices as well as the storage transport, and both must be considered for end-to-end service guarantees. For end-storage devices, quality of service refers to the availability of the storage media—for example, the RAID level. Mission-critical applications require storage devices that have RAID levels set for maximum availability combined with remote mirroring for business continuity in the event of a disaster. Less critical storage may operate sufficiently with RAID levels that provide less availability in favor of increased performance and use tape archiving as a backup mechanism. This provides cost savings with more usable storage per RAID device, forgoing the remote mirror for recovery by using tape.
Storage professionals should assign storage quality of service requirements to individual applications. These service levels can be implemented through storage management software to create an enterprisewide policy. The exercise alone of matching applications to storage service levels will create a useful framework for balancing availability and cost factors. Carried through to the purchasing decisions of new storage devices, this type of framework clearly demarcates the need for high-end, midrange, or entry-level storage devices.
Quality of service also applies to the storage transport, or the storage network. In this case, the interconnect between storage end systems must be adequately provisioned for mission-critical applications. This can apply to allocated bandwidth, multipath availability, or even balancing the long-distance transport across multiple carriers.
Figure 3-11 shows an example of bandwidth prioritization through the use of VLANs and traffic prioritization. Here, the backup of an online transaction processing (OLTP) database needs priority over a less critical backup of corporate files. No matter when the database backup begins, it will be granted the appropriate bandwidth to complete its backup operation within a designated time window.
Figure 3-11. Storage transport quality of service.
3.5.6 Storage Paths
Storage network administrators may also choose to guarantee dual, redundant paths for specific application to storage connections. This practice is often referred to as path management. New software packages with this feature use storage network topology information to create a map of available connections between servers, fabric switches, and storage devices. Criteria such as dual, redundant paths or 2Gb/s Fibre Channel links can be specified in order to meet availability and performance needs.
Attention to storage paths ensures that applications not only have the appropriate data capacity, but also the appropriate service-level transport for data access.
3.5.7 Departmental Segmentation and Accounting
Storage policy management translates to several organizational policies to help assign storage costs. For example, departments can be easily segmented to provide accounting charges based on storage use. Storage administrators can assign storage in terms of both capacity and quality of service, offering options to individual departments and their specific requirements.
Software automation carries this one step further by allowing dynamic changes with minimal manual reconfiguration. For example, capacity can be automatically added on the fly for specific users. Billing systems can then track allocation and keep department heads aware of their capacity usage and expected internal expenses. This focuses attention on storage capacity requirements of the departments through clear, bottom-line financial metrics, eliminating gray areas of infrastructure requirements and allocation.
Data Protection
Data-protection software, from coordinating tape-based backup to real-time disk replication, defends corporate data assets. Through a range of mechanisms that deliver onsite and offsite data copies at various time intervals and across multiple media formats, corporations mitigate the chance of data loss while striving to provide continuous availability. For most companies, the costs of data loss far outweigh the costs of data protection. As such, they invest millions of dollars per year on data protection techniques such as those described in this section.
The primary decisions for data protection compare cost to availability. In this case, availability refers to both the time taken for the backup operation and the time needed to restore the data set in the event of a disaster. Figure 3-12 outlines the position of several data-protection schemes based on their cost and availability measures.
Figure 3-12. Cost-availability trade-off.
For most organizations, the redundancy built into storage architectures spills into the data-protection implementations as well. For example, many companies implement more than one data-protection technique, such as combining tape-based offsite backup with onsite snapshot copies to disk. This gives them near instant recovery onsite as well as the insurance of an offsite recovery if needed.
3.6.1 Tape-Based Backup
Tape archiving is often relegated to the low end of the totem pole of backup technologies. While falling disk prices have made disk-based backup solutions more attractive to a larger segment, tape still fills a valuable need. Density alone makes tape an attractive choice for large backup operations—tape library capacities measure in hundreds of terabytes compared to disk array units in tens of terabytes. Additionally, the flexibility of removable tape media cartridges for both onsite and offsite storage adds another level of protection. Today's IP storage networking options may soon impact transport of tape cartridges. Why bother with moving them between buildings or sites if the data can be easily sent over an IP network? Even with that capability, it may still be more effective to have a tape library for archiving due to device and media capacity.
Storage networks enable sharing of tape libraries among multiple servers, including NAS filers, as shown in Figure 3-13. This type of configuration also requires software to manage access to the tape library, preventing multiple servers from simultaneous access. Backup packages like Veritas NetBackup, Legato Networker, CA Brightstor, and IBM Tivoli for Storage provide the features for managing tape libraries in shared environments, including schedule modules to perform backups at the appropriate intervals, such as daily, weekly, or monthly.
Figure 3-13. Consolidated tape backup across servers and NAS.
In SAN configurations with shared tape libraries, the backup software resides on a backup application server that manages the control information with the individual servers. With clearance established, the backup server grants access to the appropriate server and allows a direct link to the tape library. This process, often referred to as LAN-free backup, facilitates faster backup operations than do more traditional data paths for shared tape libraries through the LAN.
Tape libraries also serve as a useful complement to NAS. Even though NAS delivers file-based storage through its front end, the back end of a NAS system often has a block-based Fibre Channel connection that can be networked to a tape library for block-based backup of the NAS unit. This type of solution provides economical archiving of NAS data and may also operate simultaneously with NAS replication across multiple filers.
Server-free backup (Figure 3-14) is another technique used in conjunction with tape backup, although it can also be used with disk backup. In server-free implementations the data path is completely offloaded to the storage devices, freeing servers to focus on application processing. Server-free backup solutions take advantage of the same LAN-free designs implemented with SANs but also use data-mover agents that reside in the storage fabric to facilitate disk-to-tape or disk-to-disk data paths.
Figure 3-14. Server-free backup through third-party copy.
The data-mover agents employ a third-party copy command called extended copy. This can run off of a fabric-based storage device such as a router or appliance. The backup application server initiates a command to the data mover, and the data mover then assumes control of the copy operation until complete. The ability to conduct a high-speed, image-level backup free of server intervention builds a scalable backup infrastructure. Key benefits include more backup operations within a fixed window, simpler administration, reduced server load, and faster restores.
3.6.2 Disk Backup and Recovery
Disk-based backup ranks high on the simplicity scale but equally high on the required capacity scale. Specifically, a single disk-based backup will double the required disk storage capacity. This may be on top of disk capacity required for RAID features. In the extreme, an organization may find itself with only 25 percent usable capacity of total terabytes owned. This can be a significant cost when compared to tape archiving.
Aside from capacity considerations, disk solutions eclipse tape for achieving short backup and restore windows. For applications requiring guaranteed availability, disk-based solutions are the only mechanism for secure, rapid recovery.
Because of these cost, availability, and performance dynamics, permutations of disk copy, such as point-in-time and snapshot copy as well as replication and mirroring, come into play.
Many disk-backup solutions leverage internal RAID functions built into subsystems. This allows for multilayered redundancy to protect against disk failure, mirror failure, unit failure, and in the case of remote facilities, infrastructure failure.
3.6.3 Point-in-Time and Snapshot Copy
Point-in-time and snapshot copy solutions address the capacity requirement concerns of disk backup while providing short restore times. Point-in-time copies, as the name implies, isolate time-stamped copies of data at regular intervals and provide the ability to revert to those images in the event of application or data failure.
The first point-in-time copy includes the entire data set, doubling the storage capacity requirement. Thereafter, only changes to the data set are copied at regular intervals. Administrators can choose to initiate and maintain multiple point-in-time copies so recovery can roll back to the appropriate data set. These intervals can measure from minutes to hours to days.
With an updated copy available at all times, the recovery process with a point-in-time copy occurs almost immediately. Instead of attempting to restore the database, users are simply redirected to the last stable copy.
Snapshot copies operate in a similar fashion to point-in-time, but with the slight difference that the snapshot volume merely tracks incremental changes with pointers back to the original data set. Like point-in-time, snapshots can be created at regular intervals to provide rollback capabilities in the event of failure. However, snapshots do not replicate the original data set. Therefore, they don't require the upfront capacity increase, although additional capacity is required to maintain the snapshot information. This will vary based on the amount of data changed at each interval and the interval or snapshot frequency. The additional capacity required for a snapshot copy correlates to the amount of original content that must be saved before incremental changes are made.
Like point-in-time, snapshot copies provide quick recovery in the event of disaster, since no restore is required. But snapshot copies alone cannot protect against all disasters, since the copies still point to the parts of the original data set, and a physical storage failure could invalidate several logical snapshot copies. For a complete disk-based backup solution, snapshot would require the addition of replication or mirroring to protect against physical storage failure or site disaster.
3.6.4 Hierarchical Storage Management (HSM)
Serving a data-protection function and also a space optimization function, hierarchical storage management (HSM) helps storage administrators balance the cost per megabyte with the retrieval time of varying storage media. The conventional business driver behind HSM was the relative high cost of disk compared to the relative low cost of tape, coupled with varying types of data and service levels within the organization (see Figure 3-15).
Figure 3-15. Comparisons of media technologies: disk, optical, tape.
For example, a mission-critical e-commerce database must be instantly accessible at all times and must have mirrored or replicated copies available in the event of disaster and disk backup for fast restore. On the other end of the spectrum, large CAD files for products that have been discontinued probably don't merit the same availability, yet designers are reluctant to discard the data—they want it available if needed. This second example fits perfectly with HSM principles: to take less critical data that doesn't require immediate access times and move it to more economical storage, such as tape. In the event that the design team resurrects a particular product, the CAD files could be transferred back to disk during the production cycle to facilitate faster access times.
With disk costs dropping precipitously, HSM can be applied to disk-only environments as well. For example, mission-critical information might reside on high-end Fibre Channel disk drives, while less critical information could be stored on much less expensive serial Advanced Technology Attachment (ATA) drives.
HSM software manages this entire process, allowing administrators to set guidelines and storage categories that determine when and where files and volumes are stored. Automated HSM policies identify files that haven't been accessed recently, maintain a pointer at the original location, and then move the actual data to less expensive media. This migration process happens transparently to users, requiring only a quick restore when the file is accessed, if ever.
The concepts of HSM have been a part of enterprise storage strategies for many years. Today, with new SAN architectures and the explosive growth of data, applying HSM principles through specific HSM packages or other storage management software can lead to significant cost savings in storage capacity purchases.
Monday, December 10, 2007
The Software Spectrum
Across nearly all aspects of enterprise technology, purchasing processes have shifted from hardware-centric decision making to identifying best platforms for software and applications. Servers have essentially become commodity-driven compute engines, amassed in such volume and density that they simply become processing power enclosures. Networking equipment can easily be strung together, but only the effective monitoring and management of that information highway can tap the potential of its capabilities. Similarly with storage—while having ample space always makes solving storage problems easier, effective use of storage resources requires sophisticated organizational monitoring and management systems.
Chapter 2, "The Storage Architectural Landscape," covered the benefits of networked storage architectures from a hardware perspective and the means to deploy effective infrastructure to accommodate torrents of corporate data. This underlying equipment, however, also must be shaped with effective organizational systems that allow data managers to minimize their ongoing administration of the infrastructure and maximize the utilization and efficiency of the overall system. Storage management software helps IT professionals accomplish this task and is the focus of Chapter 3.
With ongoing administration and operation costs far exceeding acquisition costs for storage infrastructure, IT decision makers must account for this imbalance while charting their storage networking roadmap. Careful examination of storage software components can help identify crucial costs early in the deployment phases, resulting in long-term savings. While the concepts of storage management may seem foreign at first, the underlying mechanisms boil down to simplifying organizational and administrative tasks related to storage. Keep in mind that data storage space is just another corporate asset that requires efficient mechanisms to realize maximum returns. As the overall amount of data storage increases, and the ability to assign dedicated personnel decreases, effective software tools emerge as the only viable means for corporations to gain control of their information assets.
Framework for Storage Management Software
The end-to-end equation for storage management software involves numerous components, functions, and product categories. This seemingly complex spectrum of terminology and definitions often leads to unnecessary confusion about the role and objectives of storage management software. Business and technical professionals alike will be well served by a breakdown of storage management categories into a coherent framework, as presented in the following sections. By starting with the basic functional needs and translating those into required actions (as assisted by software applications), IT professionals can create effective storage software deployment strategies.
3.1.1 The Need for Storage Management Software
Businesses face an increasing array of challenges to manage storage infrastructures. First, IT professionals must manage a dazzling array of hardware and networking components that provide the underlying platform for storage. The sheer number of devices far outstrips the capacity of a single individual or group to keep track of them. Second, storage professionals must provide capacity and associated services across a variety of applications, from databases, to email, to transaction processing. Again, keeping track of these requirements and the business fluctuations requires some degree of automated assistance. Finally, all of this must take place with 24/7/365 availability and the capability for immediate recovery in the event of any failure.
These business requirements mandate the automation of the storage management process. Using software tools, storage professionals can effectively manage and monitor their infrastructure, provide capacity and services for application requirements, and guarantee uptime through high-availability configurations with built-in disaster recovery mechanisms. These are the core components of storage management software.
3.1.2 User View of Storage Management Software Components
To accomplish their objectives, storage professionals require assistance in three primary areas for effective storage management—infrastructure management, transaction management, and recovery management—outlined in Figure 3-1. Infrastructure management provides visibility to the entire architecture and the ability to make adjustments to the underlying platforms. As storage moves from direct-attached to networked models, an administrator's visibility extends far beyond the traditional purview. Software tools for SAN management and storage resource management provide visibility and reach across the entire infrastructure.
Figure 3-1. Core components of storage management.
Transaction management represents the application focus storage administrators must maintain to effectively serve the organization. Tools for data coordination, such as volume management or NAS file services, help implement storage resources for applications. Once in place, storage policy management helps ensure that resources are appropriately dedicated, monitored, and managed in a dynamic storage environment.
The availability focus comes in the form of disaster recovery management through data protection. Whether through backup applications or sophisticated real-time replication, storage professionals can guarantee uptime through the use of these software applications.
Virtualization touches several aspects of the storage management infrastructure. The core principal of virtualization separates physical and logical storage, allowing for a variety of infrastructure, transaction, and recovery functions. The physical placement of virtualization in Figure 3-2 symbolizes that virtualization in and of itself provides limited functionality. However, that functionality facilitates many higher level processes outlined in the model. These are more specifically detailed in Section 3.8, "Virtualization."
Figure 3-2. Software elements of storage management.
3.1.3 Piecing Together Storage Management Software
Figure 3-3 shows where certain storage software functions reside from the architectural view. The following sections cover each area in more detail.
Figure 3-3. Storage management software components in the enterprise.
Storage Network Management
Because networked storage infrastructures are required for optimized storage deployments, the corresponding storage network management, or SAN management, takes on critical importance. Today, customers have a choice of SAN management software that can come directly from SAN switch vendors, array vendors, server vendors, or third-party software companies that integrate directly with SAN hardware.
Storage networking implementers will benefit from careful attention to this decision. While SAN management software from equipment vendors typically offer greater functionality and features, a third-party SAN management solution can provide configuration tools and diagnostics across multivendor solutions.
Most SAN management applications operate with an independent set of interfaces that drive control through that application. Efforts are underway in industry organizations like the Storage Networking Industry Association (SNIA) to standardize on common methods for storage network management to facilitate greater interoperability between applications. Ultimately, this will provide user flexibility to use multiple applications for SAN management and the ability to more easily shift between applications.
These standardization efforts fall underneath the overall framework of Web-based Enterprise Management, or WBEM. Within this framework, the Common Information Model (CIM) establishes a standardized means to organize storage network–related information, such as product information characteristics of a SAN switch or disk array. Within these overarching framework models are a set of specific means to exchange information between devices and applications, such as Extensible Markup Language (XML), Hypertext Transfer Protocol (HTTP), and Simple Network Management Protocol (SNMP).
3.2.1 Discovery
The first step towards effective SAN management begins with device discovery. The basic process involves the identification of storage network devices within a storage fabric. For end devices such as HBAs or disk arrays and tape libraries, the initial connectivity and boot process establishes the login to the SAN fabric, typically through a SAN switch. The SAN switches become the initial repository of device information and can then share this data with SAN switch management applications or third-party applications.
Typically, all devices within a SAN have an Ethernet/IP interface dedicated to management. Each device has a specific IP address and communications to other devices or to centralized management agents via an SNMP using Management Information Base (MIB). MIBs are basic frameworks that allow applications and devices to share device-specific information. There are standard MIBs, such as MIB-II, with information that pertains to all devices, such as interface status for a specific port. Hardware vendors also typically provide vendor-specific MIBs that cover unique product features.
Using a combination of information from SAN switches, which have device information through the login process, and direct access to devices through the Ethernet and IP interfaces using SNMP, management applications have a wide array of information to provide to administrators on the general status of devices within the storage network. From this initial discovery process, more sophisticated management, such as manipulation of the storage network, can occur.
3.2.2 Zoning and Configuration
Storage networks create connections between multiple servers and storage devices using a variety of interconnect mechanisms from a single switch or hub to a complex mesh of switches that provide redundancy for high availability. The universal connectivity of storage devices to a common playing field provides tremendous flexibility to architect storage solutions. However, having that connectivity doesn't necessarily mean that one would want every storage device to be able to see every other storage device.
Managed communication between storage devices helps administrators balance the agility of universal accessibility with the business needs of resource allocation, segmentation, security, and controlled access. This managed communication begins with a process of zoning and configuration.
Let's use a simple example of a Windows server and a UNIX server on the same storage network, with two separate disk units (JBOD-A and JBOD-B—each with two individual disks) and one tape library. A typical zoning and configuration application is shown in Figure 3-4. On the right side of the diagram is a list of devices, including the individual disks, tape library, and servers (identified by their HBA interfaces). On the left side of the diagram is a list of zones. Placing devices in a particular zone ensures that only other devices within that zone can "see" each other. SAN switches enforce the zoning through different mechanisms based on the worldwide name (WWN) of the storage device or on the port of the switch to which it is attached. For more detail on zoning, see Section 3.2.3, "SAN Management Guidance: Hard and Soft Zoning."
Figure 3-4. Typical storage networking zone configuration.
In this example, the administrator has separated the disk units for the Windows and UNIX hosts. This avoids any conflicts of one operating system trying to initialize all of the visible storage. However, the tape library has been placed in both zones. To avoid potential conflicts, this configuration would need to be accompanied by a backup software application that can manage the access to the tape library, ensuring access by only one host at time. Sharing the tape library allows its cost to be distributed across a greater number of servers, thereby servicing more direct backups.
3.2.3 SAN Management Guidance: Hard and Soft Zoning
Zoning applications typically operate in two types of device modes: hard zoning and soft zoning. Hard zoning, also referred to as port-based zoning, means that all of the devices connected to a single port remain mapped to that port. For example, three drives on a Fibre Channel arbitrated loop may be allocated via a specific port to Zone A. If another drive is assigned to that loop, and it attaches through the same specified port, it would automatically appear in Zone A.
Another zoning mechanism, soft zoning, or WWN zoning, uses the unique Fibre Channel address of the device. In IP or iSCSI terms, the device has a Worldwide Unique Identifier (WWUI). With soft zoning, devices may be moved and interconnected through different SAN switches but will remain in the same zone.
Hard zoning offers more physically oriented security, while soft zoning offers more flexibility through software-enforced zoning. For very large configurations, customers will likely benefit from the intelligence of soft zoning coupled with the appropriate security and authentication mechanisms.
3.2.4 SAN Topologies
In addition to zoning devices in a SAN, some applications offer the ability to visualize the SAN in a topology view. A sample SAN topology is shown in Figure 3-5.
Figure 3-5. Topology view of a storage area network.
SAN topology views can serve as effective management utilities, allowing administrators to quickly see the entire storage network and to drill down to device levels. In some cases, it may be easier for administrators to assign and allocate storage capacity using topology managers.
SAN topology views also provide more visibility to the network connectivity of SANs. While a zoning and configuration tool helps clarify the communication relationship between storage devices, topology managers help clarify the communication means between storage devices. Specifically, topology managers can show redundant connections between switches, redundant switches, and the available paths that link one storage device to another.
3.2.5 Monitoring
Monitoring allows storage administrators to keep the pulse of the storage network. This includes the general health of the SAN hardware, data activity levels, and configuration changes.
Event and error tracking is used to keep logs of the activity within a storage network. SAN management applications track events such as device additions, zone changes, and network connectivity changes. These logs keep detailed records of SAN activity and can serve as useful tools if problems occur. With hundreds or thousands of SAN configuration operations taking place on any given day, the ability to go back in time to analyze events is invaluable. Similarly, error logs help diagnose the root cause of potential failures within a SAN. Minor errors, such as an unsuccessful first login to a SAN switch may not mean much as a stand-alone event, but a pattern of such errors can help administrators rapidly analyze and repair potential problems in the SAN.
Storage administrators can use alarms and traps to help effectively monitor the storage network. A trap is a set threshold for a certain variable that triggers notification when reached. For example, a trap can be set for a SAN switch to send an alarm when the temperature of the box reaches a "red zone." Such traps are helpful because a problem may not be directly related to failures within the equipment. For example, a broken fan would automatically send an alarm, but if someone placed a large box next to a data center rack, prohibiting airflow, a temperature gauge would be the only mechanism to ensure preemptive awareness of overheating.
SAN management traps typically integrate with large enterprise management systems via SNMP. By tying into these larger software systems, SAN alarms can be directed to the appropriate support staff through existing email, paging, or telephone tracking systems.
Perhaps the most important monitoring component for progressive deployment of storage networking infrastructures is performance. Performance monitoring can be tricky. Ultimately, organizations measure performance of applications, not storage throughput. However, the underlying performance of the SAN helps enable optimized application performance.
Since storage networks carry the storage traffic without much interpretation of the data, SAN performance metrics focus on link utilization, or more specifically, how much available bandwidth is being used for any given connection. An overengineered SAN with low utilization means excess infrastructure and high costs. An overworked SAN with high utilization means more potential for congestion and service outages for storage devices.
3.2.6 SAN GUIDANCE: Protocol Conversion
As outlined in Chapter 2, three transport protocols exist for IP Storage: iSCSI, iFCP, and FCIP. For iSCSI to Fibre Channel conversion, a key part of storage network management, IT professionals can use the framework in evaluating product capabilities. Not all IP storage switches and gateways provide full conversion capabilities. For example, some products may support iSCSI servers to Fibre Channel storage, but not vice versa.
Additionally, storage-specific features for applications like mirroring or advanced zoning capabilities between IP and FC SANs may affect protocol conversion capabilities. This type of connectivity, items 2 and 3 in Figure 3-6, should be specifically examined in multiprotocol environments.
Figure 3-6. Type of IP and FCP conversion.
For Fibre Channel to Fibre Channel interconnect across an IP network, such as item 4 illustrates, the iFCP and FCIP protocols are more suitable because of their ability to retain more of the FCP layer.
3.2.7 Hard Management
Appropriately titled by its frequency of implementation, as opposed to the implementation challenge, is the physical documentation of SANs. No matter how much software is deployed or how detailed the visibility of the applications, nothing protects a business better than clear, easy-to-follow policies and procedures. Obviously, this is more easily said than done, and often the primary challenge in disaster recovery scenarios lies in understanding the design guidelines used by SAN architects. If these architects leave the company or are unavailable, the inherent recovery challenges increase dramatically.
Documenting SAN deployments, including a set of how-to instructions for amateur users, serves as both a self-check on implementation and an invaluable resource for those who may need to troubleshoot an installation sight unseen. At a minimum, items for this documentation would include applications used, vendor support contacts, passwords, authorized company personnel and contact information, backup and recovery procedures, and storage allocation mechanisms.
In conjunction with documenting storage administration policies and procedures, proper cabling and labeling of storage configurations can dramatically save time and effort in personnel costs—one of the largest components of the operational IT budget.
Storage Resource Management
Storage resource management (SRM) is the second component of infrastructure-focused SAN management. While storage network management (SNM) focuses primarily on the connectivity and combinations of storage devices on a network, SRM focuses on the individual devices and the ability to see and modify those devices from a central location. SRM may also be referred to as device management, an appropriate term, given the focus, and an easy way to distinguish from SNM. The primary benefit of SRM is the ability to make more effective use of the resources within an organization, specifically storage capacity and resource efficiency, as outlined at the end of this section.
For clarification purposes, we divide SRM categories into devices and subsystems. Devices are those elements within the overall storage infrastructure that provide access to storage, while subsystems house the actual disk or tape media.
3.3.1 Device Management
In the device category, SRM looks remarkably similar to SNM. Devices under an SRM application may include HBAs, switches, and routers. The SRM application provides a unified view of these devices and the ability to make some changes. In many cases, the SRM application will simply point to the device-specific management interface embedded within the product.
Device management features of SRM include discovery, topology mapping, and configuration planning. These tools provide SRM applications the information to help find potential problems and recommend solutions. Zoning is typically not included in SRM packages.
The primary difference between device management across SNM and SRM applications is the integration with additional multivendor infrastructure. For SRM applications, the visibility typically extends deeper into the subsystems category.
3.3.2 Subsystem Management
SRM features shine when used in large installations of disparate storage subsystems that require centralized administration. In these environments, an SRM application takes a logical view of the storage configuration and provides administrators a clearinghouse to receive information and implement changes.
SRM applications discover storage within the overall infrastructure and provide logical-to-physical maps of storage utilization, such as which servers are using which storage. Further, SRM applications identify operational file systems and file-level detail of such storage. For block-based storage, SRM locates volume groups and provides volume mapping visibility within and across subsystems.
This level of detail can be used by storage administrators to understand asset allocation and identify usage patterns across applications, departments, and individuals. From this viewpoint, administrators now have the required knowledge to drive decisions about optimized storage use within the organization. This leads to the establishment of policies such as quota management, covered in Section 3.5, "Storage Policy Management."
Even with storage resource management focused on the subsystems side, LUN and RAID creation and modification still remain within the subsystem. These features may be accessed through centralized interfaces; however, this basic level of disk aggregation can be effectively accomplished only by the subsystem vendor.
3.3.3 Resource Efficiency
SRM information allows administrators to see the efficiency of their storage resources, such as the amount of storage used compared to the total amount available in the array. This top-level view helps quickly pinpoint overutilized or underutilized storage, allowing administrators to appropriate respond. Additionally, SRM views of the data itself can provide efficiency clues. For example, closer examination of file data may reveal rampant file revision propagation or an absurdly large collection of .mp3 or video files. Once this information is collected and examined, administrators can appropriately respond.
Identifying which servers and applications address which storage helps match the logical and physical pictures. If the accounting department is using 75 percent of the capacity of an array in another building, it may make sense to have that array located within the accounting department yet still available through the storage network to the rest of the organization.
3.3.4 Capacity Planning
Using historical information from SRM applications, specifically trend analysis, administrators can plan for storage capacity more effectively. With visibility into the actual data, including the types of files and applications related to growing volumes, SRM applications can plot and chart areas of future storage growth. Based on such information, administrators can make more accurate decisions based on more than a simple look at the total number of terabytes within the organization.
In addition to helping predict future capacity needs, SRM can help administrators make use of existing capacity, thereby minimizing new storage purchases. For example, the duration for saved email archives can be weighed against the cost of new capacity.
3.3.5 Integration with Other Storage Management Applications
For SRM to be truly effective, the monitoring functions must be incorporated with other functions of storage management, such as data coordination, protection, and policy management. Administrators can begin with basic questions such as the type of storage data, primary users, usage habits, access times and durations, and storage location. From there, policies and actions can be put in place to maximize efficiency, cap reckless data accumulation, and automatically provision and protect storage as needed.
Chapter 2, "The Storage Architectural Landscape," covered the benefits of networked storage architectures from a hardware perspective and the means to deploy effective infrastructure to accommodate torrents of corporate data. This underlying equipment, however, also must be shaped with effective organizational systems that allow data managers to minimize their ongoing administration of the infrastructure and maximize the utilization and efficiency of the overall system. Storage management software helps IT professionals accomplish this task and is the focus of Chapter 3.
With ongoing administration and operation costs far exceeding acquisition costs for storage infrastructure, IT decision makers must account for this imbalance while charting their storage networking roadmap. Careful examination of storage software components can help identify crucial costs early in the deployment phases, resulting in long-term savings. While the concepts of storage management may seem foreign at first, the underlying mechanisms boil down to simplifying organizational and administrative tasks related to storage. Keep in mind that data storage space is just another corporate asset that requires efficient mechanisms to realize maximum returns. As the overall amount of data storage increases, and the ability to assign dedicated personnel decreases, effective software tools emerge as the only viable means for corporations to gain control of their information assets.
Framework for Storage Management Software
The end-to-end equation for storage management software involves numerous components, functions, and product categories. This seemingly complex spectrum of terminology and definitions often leads to unnecessary confusion about the role and objectives of storage management software. Business and technical professionals alike will be well served by a breakdown of storage management categories into a coherent framework, as presented in the following sections. By starting with the basic functional needs and translating those into required actions (as assisted by software applications), IT professionals can create effective storage software deployment strategies.
3.1.1 The Need for Storage Management Software
Businesses face an increasing array of challenges to manage storage infrastructures. First, IT professionals must manage a dazzling array of hardware and networking components that provide the underlying platform for storage. The sheer number of devices far outstrips the capacity of a single individual or group to keep track of them. Second, storage professionals must provide capacity and associated services across a variety of applications, from databases, to email, to transaction processing. Again, keeping track of these requirements and the business fluctuations requires some degree of automated assistance. Finally, all of this must take place with 24/7/365 availability and the capability for immediate recovery in the event of any failure.
These business requirements mandate the automation of the storage management process. Using software tools, storage professionals can effectively manage and monitor their infrastructure, provide capacity and services for application requirements, and guarantee uptime through high-availability configurations with built-in disaster recovery mechanisms. These are the core components of storage management software.
3.1.2 User View of Storage Management Software Components
To accomplish their objectives, storage professionals require assistance in three primary areas for effective storage management—infrastructure management, transaction management, and recovery management—outlined in Figure 3-1. Infrastructure management provides visibility to the entire architecture and the ability to make adjustments to the underlying platforms. As storage moves from direct-attached to networked models, an administrator's visibility extends far beyond the traditional purview. Software tools for SAN management and storage resource management provide visibility and reach across the entire infrastructure.
Figure 3-1. Core components of storage management.
Transaction management represents the application focus storage administrators must maintain to effectively serve the organization. Tools for data coordination, such as volume management or NAS file services, help implement storage resources for applications. Once in place, storage policy management helps ensure that resources are appropriately dedicated, monitored, and managed in a dynamic storage environment.
The availability focus comes in the form of disaster recovery management through data protection. Whether through backup applications or sophisticated real-time replication, storage professionals can guarantee uptime through the use of these software applications.
Virtualization touches several aspects of the storage management infrastructure. The core principal of virtualization separates physical and logical storage, allowing for a variety of infrastructure, transaction, and recovery functions. The physical placement of virtualization in Figure 3-2 symbolizes that virtualization in and of itself provides limited functionality. However, that functionality facilitates many higher level processes outlined in the model. These are more specifically detailed in Section 3.8, "Virtualization."
Figure 3-2. Software elements of storage management.
3.1.3 Piecing Together Storage Management Software
Figure 3-3 shows where certain storage software functions reside from the architectural view. The following sections cover each area in more detail.
Figure 3-3. Storage management software components in the enterprise.
Storage Network Management
Because networked storage infrastructures are required for optimized storage deployments, the corresponding storage network management, or SAN management, takes on critical importance. Today, customers have a choice of SAN management software that can come directly from SAN switch vendors, array vendors, server vendors, or third-party software companies that integrate directly with SAN hardware.
Storage networking implementers will benefit from careful attention to this decision. While SAN management software from equipment vendors typically offer greater functionality and features, a third-party SAN management solution can provide configuration tools and diagnostics across multivendor solutions.
Most SAN management applications operate with an independent set of interfaces that drive control through that application. Efforts are underway in industry organizations like the Storage Networking Industry Association (SNIA) to standardize on common methods for storage network management to facilitate greater interoperability between applications. Ultimately, this will provide user flexibility to use multiple applications for SAN management and the ability to more easily shift between applications.
These standardization efforts fall underneath the overall framework of Web-based Enterprise Management, or WBEM. Within this framework, the Common Information Model (CIM) establishes a standardized means to organize storage network–related information, such as product information characteristics of a SAN switch or disk array. Within these overarching framework models are a set of specific means to exchange information between devices and applications, such as Extensible Markup Language (XML), Hypertext Transfer Protocol (HTTP), and Simple Network Management Protocol (SNMP).
3.2.1 Discovery
The first step towards effective SAN management begins with device discovery. The basic process involves the identification of storage network devices within a storage fabric. For end devices such as HBAs or disk arrays and tape libraries, the initial connectivity and boot process establishes the login to the SAN fabric, typically through a SAN switch. The SAN switches become the initial repository of device information and can then share this data with SAN switch management applications or third-party applications.
Typically, all devices within a SAN have an Ethernet/IP interface dedicated to management. Each device has a specific IP address and communications to other devices or to centralized management agents via an SNMP using Management Information Base (MIB). MIBs are basic frameworks that allow applications and devices to share device-specific information. There are standard MIBs, such as MIB-II, with information that pertains to all devices, such as interface status for a specific port. Hardware vendors also typically provide vendor-specific MIBs that cover unique product features.
Using a combination of information from SAN switches, which have device information through the login process, and direct access to devices through the Ethernet and IP interfaces using SNMP, management applications have a wide array of information to provide to administrators on the general status of devices within the storage network. From this initial discovery process, more sophisticated management, such as manipulation of the storage network, can occur.
3.2.2 Zoning and Configuration
Storage networks create connections between multiple servers and storage devices using a variety of interconnect mechanisms from a single switch or hub to a complex mesh of switches that provide redundancy for high availability. The universal connectivity of storage devices to a common playing field provides tremendous flexibility to architect storage solutions. However, having that connectivity doesn't necessarily mean that one would want every storage device to be able to see every other storage device.
Managed communication between storage devices helps administrators balance the agility of universal accessibility with the business needs of resource allocation, segmentation, security, and controlled access. This managed communication begins with a process of zoning and configuration.
Let's use a simple example of a Windows server and a UNIX server on the same storage network, with two separate disk units (JBOD-A and JBOD-B—each with two individual disks) and one tape library. A typical zoning and configuration application is shown in Figure 3-4. On the right side of the diagram is a list of devices, including the individual disks, tape library, and servers (identified by their HBA interfaces). On the left side of the diagram is a list of zones. Placing devices in a particular zone ensures that only other devices within that zone can "see" each other. SAN switches enforce the zoning through different mechanisms based on the worldwide name (WWN) of the storage device or on the port of the switch to which it is attached. For more detail on zoning, see Section 3.2.3, "SAN Management Guidance: Hard and Soft Zoning."
Figure 3-4. Typical storage networking zone configuration.
In this example, the administrator has separated the disk units for the Windows and UNIX hosts. This avoids any conflicts of one operating system trying to initialize all of the visible storage. However, the tape library has been placed in both zones. To avoid potential conflicts, this configuration would need to be accompanied by a backup software application that can manage the access to the tape library, ensuring access by only one host at time. Sharing the tape library allows its cost to be distributed across a greater number of servers, thereby servicing more direct backups.
3.2.3 SAN Management Guidance: Hard and Soft Zoning
Zoning applications typically operate in two types of device modes: hard zoning and soft zoning. Hard zoning, also referred to as port-based zoning, means that all of the devices connected to a single port remain mapped to that port. For example, three drives on a Fibre Channel arbitrated loop may be allocated via a specific port to Zone A. If another drive is assigned to that loop, and it attaches through the same specified port, it would automatically appear in Zone A.
Another zoning mechanism, soft zoning, or WWN zoning, uses the unique Fibre Channel address of the device. In IP or iSCSI terms, the device has a Worldwide Unique Identifier (WWUI). With soft zoning, devices may be moved and interconnected through different SAN switches but will remain in the same zone.
Hard zoning offers more physically oriented security, while soft zoning offers more flexibility through software-enforced zoning. For very large configurations, customers will likely benefit from the intelligence of soft zoning coupled with the appropriate security and authentication mechanisms.
3.2.4 SAN Topologies
In addition to zoning devices in a SAN, some applications offer the ability to visualize the SAN in a topology view. A sample SAN topology is shown in Figure 3-5.
Figure 3-5. Topology view of a storage area network.
SAN topology views can serve as effective management utilities, allowing administrators to quickly see the entire storage network and to drill down to device levels. In some cases, it may be easier for administrators to assign and allocate storage capacity using topology managers.
SAN topology views also provide more visibility to the network connectivity of SANs. While a zoning and configuration tool helps clarify the communication relationship between storage devices, topology managers help clarify the communication means between storage devices. Specifically, topology managers can show redundant connections between switches, redundant switches, and the available paths that link one storage device to another.
3.2.5 Monitoring
Monitoring allows storage administrators to keep the pulse of the storage network. This includes the general health of the SAN hardware, data activity levels, and configuration changes.
Event and error tracking is used to keep logs of the activity within a storage network. SAN management applications track events such as device additions, zone changes, and network connectivity changes. These logs keep detailed records of SAN activity and can serve as useful tools if problems occur. With hundreds or thousands of SAN configuration operations taking place on any given day, the ability to go back in time to analyze events is invaluable. Similarly, error logs help diagnose the root cause of potential failures within a SAN. Minor errors, such as an unsuccessful first login to a SAN switch may not mean much as a stand-alone event, but a pattern of such errors can help administrators rapidly analyze and repair potential problems in the SAN.
Storage administrators can use alarms and traps to help effectively monitor the storage network. A trap is a set threshold for a certain variable that triggers notification when reached. For example, a trap can be set for a SAN switch to send an alarm when the temperature of the box reaches a "red zone." Such traps are helpful because a problem may not be directly related to failures within the equipment. For example, a broken fan would automatically send an alarm, but if someone placed a large box next to a data center rack, prohibiting airflow, a temperature gauge would be the only mechanism to ensure preemptive awareness of overheating.
SAN management traps typically integrate with large enterprise management systems via SNMP. By tying into these larger software systems, SAN alarms can be directed to the appropriate support staff through existing email, paging, or telephone tracking systems.
Perhaps the most important monitoring component for progressive deployment of storage networking infrastructures is performance. Performance monitoring can be tricky. Ultimately, organizations measure performance of applications, not storage throughput. However, the underlying performance of the SAN helps enable optimized application performance.
Since storage networks carry the storage traffic without much interpretation of the data, SAN performance metrics focus on link utilization, or more specifically, how much available bandwidth is being used for any given connection. An overengineered SAN with low utilization means excess infrastructure and high costs. An overworked SAN with high utilization means more potential for congestion and service outages for storage devices.
3.2.6 SAN GUIDANCE: Protocol Conversion
As outlined in Chapter 2, three transport protocols exist for IP Storage: iSCSI, iFCP, and FCIP. For iSCSI to Fibre Channel conversion, a key part of storage network management, IT professionals can use the framework in evaluating product capabilities. Not all IP storage switches and gateways provide full conversion capabilities. For example, some products may support iSCSI servers to Fibre Channel storage, but not vice versa.
Additionally, storage-specific features for applications like mirroring or advanced zoning capabilities between IP and FC SANs may affect protocol conversion capabilities. This type of connectivity, items 2 and 3 in Figure 3-6, should be specifically examined in multiprotocol environments.
Figure 3-6. Type of IP and FCP conversion.
For Fibre Channel to Fibre Channel interconnect across an IP network, such as item 4 illustrates, the iFCP and FCIP protocols are more suitable because of their ability to retain more of the FCP layer.
3.2.7 Hard Management
Appropriately titled by its frequency of implementation, as opposed to the implementation challenge, is the physical documentation of SANs. No matter how much software is deployed or how detailed the visibility of the applications, nothing protects a business better than clear, easy-to-follow policies and procedures. Obviously, this is more easily said than done, and often the primary challenge in disaster recovery scenarios lies in understanding the design guidelines used by SAN architects. If these architects leave the company or are unavailable, the inherent recovery challenges increase dramatically.
Documenting SAN deployments, including a set of how-to instructions for amateur users, serves as both a self-check on implementation and an invaluable resource for those who may need to troubleshoot an installation sight unseen. At a minimum, items for this documentation would include applications used, vendor support contacts, passwords, authorized company personnel and contact information, backup and recovery procedures, and storage allocation mechanisms.
In conjunction with documenting storage administration policies and procedures, proper cabling and labeling of storage configurations can dramatically save time and effort in personnel costs—one of the largest components of the operational IT budget.
Storage Resource Management
Storage resource management (SRM) is the second component of infrastructure-focused SAN management. While storage network management (SNM) focuses primarily on the connectivity and combinations of storage devices on a network, SRM focuses on the individual devices and the ability to see and modify those devices from a central location. SRM may also be referred to as device management, an appropriate term, given the focus, and an easy way to distinguish from SNM. The primary benefit of SRM is the ability to make more effective use of the resources within an organization, specifically storage capacity and resource efficiency, as outlined at the end of this section.
For clarification purposes, we divide SRM categories into devices and subsystems. Devices are those elements within the overall storage infrastructure that provide access to storage, while subsystems house the actual disk or tape media.
3.3.1 Device Management
In the device category, SRM looks remarkably similar to SNM. Devices under an SRM application may include HBAs, switches, and routers. The SRM application provides a unified view of these devices and the ability to make some changes. In many cases, the SRM application will simply point to the device-specific management interface embedded within the product.
Device management features of SRM include discovery, topology mapping, and configuration planning. These tools provide SRM applications the information to help find potential problems and recommend solutions. Zoning is typically not included in SRM packages.
The primary difference between device management across SNM and SRM applications is the integration with additional multivendor infrastructure. For SRM applications, the visibility typically extends deeper into the subsystems category.
3.3.2 Subsystem Management
SRM features shine when used in large installations of disparate storage subsystems that require centralized administration. In these environments, an SRM application takes a logical view of the storage configuration and provides administrators a clearinghouse to receive information and implement changes.
SRM applications discover storage within the overall infrastructure and provide logical-to-physical maps of storage utilization, such as which servers are using which storage. Further, SRM applications identify operational file systems and file-level detail of such storage. For block-based storage, SRM locates volume groups and provides volume mapping visibility within and across subsystems.
This level of detail can be used by storage administrators to understand asset allocation and identify usage patterns across applications, departments, and individuals. From this viewpoint, administrators now have the required knowledge to drive decisions about optimized storage use within the organization. This leads to the establishment of policies such as quota management, covered in Section 3.5, "Storage Policy Management."
Even with storage resource management focused on the subsystems side, LUN and RAID creation and modification still remain within the subsystem. These features may be accessed through centralized interfaces; however, this basic level of disk aggregation can be effectively accomplished only by the subsystem vendor.
3.3.3 Resource Efficiency
SRM information allows administrators to see the efficiency of their storage resources, such as the amount of storage used compared to the total amount available in the array. This top-level view helps quickly pinpoint overutilized or underutilized storage, allowing administrators to appropriate respond. Additionally, SRM views of the data itself can provide efficiency clues. For example, closer examination of file data may reveal rampant file revision propagation or an absurdly large collection of .mp3 or video files. Once this information is collected and examined, administrators can appropriately respond.
Identifying which servers and applications address which storage helps match the logical and physical pictures. If the accounting department is using 75 percent of the capacity of an array in another building, it may make sense to have that array located within the accounting department yet still available through the storage network to the rest of the organization.
3.3.4 Capacity Planning
Using historical information from SRM applications, specifically trend analysis, administrators can plan for storage capacity more effectively. With visibility into the actual data, including the types of files and applications related to growing volumes, SRM applications can plot and chart areas of future storage growth. Based on such information, administrators can make more accurate decisions based on more than a simple look at the total number of terabytes within the organization.
In addition to helping predict future capacity needs, SRM can help administrators make use of existing capacity, thereby minimizing new storage purchases. For example, the duration for saved email archives can be weighed against the cost of new capacity.
3.3.5 Integration with Other Storage Management Applications
For SRM to be truly effective, the monitoring functions must be incorporated with other functions of storage management, such as data coordination, protection, and policy management. Administrators can begin with basic questions such as the type of storage data, primary users, usage habits, access times and durations, and storage location. From there, policies and actions can be put in place to maximize efficiency, cap reckless data accumulation, and automatically provision and protect storage as needed.
Storage Area Networks
2.6 Storage Area Networks
Entire books have been written about SANs. Two notable volumes are Designing Storage Area Networks and IP SANs, both by Tom Clark. For those interested in the next level of technical detail, they both serve as excellent references.
Achieving the roadmap objectives of IP Storage Networking—Straight to the Core requires a thorough understanding of SANs and the underlying infrastructure of both Fibre Channel and IP SANs. This section covers such material with the intent to quickly move on to the software spectrum that maximizes the impact of a flexible, well-architected infrastructure.
2.6.1 SAN History and Background
In the mid-1990s, IT professionals recognized the gap between DAS and NAS, and momentum grew for a mechanism that combined the performance and direct access of DAS with the network flexibility of NAS. The solution that emerged was Fibre Channel.
Fibre Channel is actually a SCSI-based implementation, only in a serial format. SCSI commands are at the root of all storage exchanges between imitators and targets. Commands such as write, read, and acknowledge provide the rules of engagement between two devices. This SCSI command set has proved invaluable as the lingua franca for storage.
Originally, the SCSI command set could operate only on top of a SCSI parallel bus. This cabling mechanism involved the use of large, high-density cabling with as many as 68 wires per cable. SCSI cabling cannot extend beyond 25 meters and can effectively support only a handful of devices.
Fibre Channel solved SCSI limitations of device count and distance with the Fibre Channel Protocol (FCP) and Fibre Channel networks (see Figure 2-9). It is important to remember that these two components do not necessarily need to operate together, especially when considering implementations of IP storage. With the ability to layer the SCSI command set on to FCP, and then transmit the information across a serial high-speed network built with Fibre Channel, IT professionals could now build networked storage infrastructures where servers have direct access to storage devices, new storage devices could easily be added to the network, and storage capacity could be shared among multiple servers. These SAN benefits, including the ability for storage professionals to manage the ever-increasing amount of data, led to the adoption of this architecture by virtually all large corporations.
Figure 2-9. Basics of Fibre Channel Protocol (FCP).
With the routable packaging in place via FCP shown in Figure 2-9, the network build-out for storage could take place with Fibre Channel SANs, shown in Figure 2-10.
Figure 2-10. Fibre Channel storage area networks.
Fibre Channel SANs proved invaluable for storage professionals looking to expand direct access storage capacity and utilization. However, Fibre Channel SANs came with their own set of issues that left the end-to-end solution incomplete. This includes unresolved interoperability issues between multiple vendors' products. While compatibility between Fibre Channel adapters in servers and Fibre Channel storage devices is mature, the interoperability required to build Fibre Channel fabrics with switches and directors from multiple vendors remains a difficult and complex process.
Fibre Channel SANs, through the inherent nature of introducing a new network technology different from IP and Ethernet, requires new staff training, new equipment, and new management systems. These factors add considerably to the overall total cost of the storage infrastructure.
Finally, Fibre Channel networking was designed as a data center technology, not intended to travel long distances across WANs. With today's requirements for high-availability across geographic areas, Fibre Channel networks cannot compete independently, even with the help of extenders or Dense Wave Division Multiplexing (DWDM) products. At a maximum, these solutions can extend Fibre Channel traffic to around 100 kilometers.
2.6.2 Pure IP Storage Networks
Starting in 2000, the concept of a pure IP SAN took hold. The basic concept of this implementation was to create a method of connecting servers and storage devices, with the direct access block methods applications expected, via an Ethernet and IP network. This would provide IT professionals the ability to utilize existing, well-understood, mature networking technology in conjunction with block-addressable storage devices. The underlying technology enabling this architecture is known as Internet Small Computer Systems Interface, or iSCSI (see Figure 2-11).
Figure 2-11. Basics of Internet Small Computer System Interface (iSCSI).
iSCSI, in its simplest form, represents a mechanism to take block-oriented SCSI commands and map them to an IP network. This protocol can be implemented directly on servers or storage devices to allow native connectivity to IP and Ethernet networks or fabrics. Once accomplished, storage applications requiring direct access to storage devices can extend that connection across an IP and Ethernet network, which has a nearly ubiquitous footprint across corporate and carrier infrastructures. The basics of the iSCSI protocol are outlined in Figure 2-11, while an implementation example is shown in Figure 2-12.
Figure 2-12. Pure iSCSI storage networks.
This wholehearted embrace of underlying IP and Ethernet technologies—mature network technology, installed base, software integration, industry research—should guarantee iSCSI will succeed as a storage transport. However, networking technologies succeed based on provision of a transition path. The same applies for iSCSI, where the transition path may occur from internal server storage to iSCSI SANs, parallel SCSI devices to iSCSI, or enterprise Fibre Channel storage to iSCSI SAN.
Adoption of new technologies occurs most frequently with those likely to gain. With iSCSI, that would indicate adoption first in mid-size to large corporations that have already deployed a Fibre Channel SAN; recognize the benefits of direct access, networked storage infrastructures; and need the ability to scale to large deployments across geographically dispersed locations. This profile often accompanies a significant storage budget that could be considerably trimmed in cost and complexity savings introduced by an IP-centric approach.
For a pure iSCSI storage network, no accommodation is made for Fibre Channel. More specifically, the iSCSI specification, as ratified by the IETF, makes no reference to Fibre Channel. That protocol conversion is outside the purview of the standards organization and left completely to vendor implementation.
Several mechanisms exist to enhance current Fibre Channel SANs with IP. Optimized storage deployment depends on careful analysis of these solutions, attention to control points within the architecture, and recognition that aggressively embracing IP networking can lead to dramatic total cost benefits. These considerations are covered in Section 2.6.5, "Enhancing Fibre Channel SANs with IP."
2.6.3 SAN GUIDANCE: Comparing Ethernet/IP and Fibre Channel Fabrics
With the introduction of IP storage as a capable storage network transport, many comparisons have been drawn between the two technologies. For fair comparison, one must recognize that Fibre Channel and Ethernet are both Layer 2 networking technologies virtually equivalent in performance. They both provide point-to-point, full-duplex, switched architectures. It is very hard to differentiate networking technologies with these common characteristics. Therefore, the speed and feed comparisons between Fibre Channel and Ethernet are largely irrelevant.
However, Ethernet is the largest home for the Internet Protocol, and IP ubiquity dramatically expands fabric flexibility. The equivalent in Fibre Channel would be the FCP, but since that works only on Fibre Channel fabrics, it can't compete with IP as an intelligent packaging mechanism. With that in mind, the Table 2-1 depicts some fabric differences.
Fibre Channel SANs function well in data center environments and are useful for massive Fibre Channel end-device consolidation. But for reasons identified above, Fibre Channel requires complementary technologies such as IP and Ethernet to help integrate storage into overall corporate cost and management objectives.
Table 2-1. Comparison of Fibre Channel and Ethernet/IP Fabrics Pure Fibre Channel Fabric
Ethernet and IP Fabric
Distance
Approximately 100Km, including DWDM
Unlimited
Size limitations
Fabric switch limit of 239
Unlimited
Network performance
Equivalent (point-to-point, full-duplex, switched)
Equivalent (point-to-point, full-duplex, switched)
Transport
Fibre Channel, DWDM
Ethernet, ATM, Packet-over-SONET, T-1, T-3, DS-3, DWDM
Fault Isolation
Not available; single fabric solution
Protection via autonomous regions
Positioning
High performance
Network convergence
Total cost of ownership
Independent to SAN
Integrated with corporate IP networking
Network management
Independent
Integrated to IP
Network reach
Data center, Metro SAN
SAN, LAN, MAN, WAN
2.6.4 Host Bus Adapters and Network Interface Cards
Regardless of the networking interconnect, servers and storage devices need methods to take data from the CPU or the disk drives out to the wire. Servers use HBAs or network interface cards (NICs) for such functions, and storage subsystems use controllers. Both types of devices are functionally similar. Frequently, end-system connectivity gets inappropriately mixed in with network connectivity to describe SANs, but the two are fundamentally different. With the ability to do wire-speed conversion within networking fabrics between different cabling and protocols, end-system interconnects and network fabrics need not be identical.
With the dramatic differences between traditional IP-based LAN traffic using NICs and traditional storage-based traffic using Fibre Channel HBAs, it helps to clarify how these fit with new IP storage adapters, such as iSCSI. In the traditional IP case, TCP/IP segmentation resided in the host application layer. When the server wanted to send information out on the wire (link layer), it would have to segment the file into smaller frames that would be sent to the NIC driver layer. The driver layer had the simple task of relaying those packets to the wire. With large files, however, the host-based process of parsing the information into smaller units consumed considerable CPU cycles—and ultimately led to poor performance.
Fibre Channel-based HBAs compensated for this inefficiency by moving the data segmentation directly to the card and letting the card do the heavy lifting of converting to smaller frames for the link layer. This freed CPU cycles for application processing and created a highly efficient mechanism for moving data out to the wire.
A combination of these two approaches results in the efficiency of the Fibre Channel approach, through a process called TCP/IP offload. TCP/IP offload can be used for network based servers using TCP/IP, including iSCSI or NAS. The processing for data segmentation once completed in the host now moves to the card, freeing CPU cycles. Also, the data comes out of the server directly to a TCP/IP and Ethernet link using iSCSI, providing native access to IP networks and the corresponding universal reach. IP storage can run on traditional IP NICs through software drivers, but the CPU utilization will be high and could impact application performance during operating hours. During off-peak times, when CPU cycles are available, this approach may be sufficient.
IP storage adapters can be further categorized by the types of offload mechanisms. At the basic level, all IP storage adapters employ some form of TCP/IP offloading. Further, some adapters will offload the iSCSI process from the host as well. The intricacies among these implementations require examining several features during evaluation. First and foremost, IP storage adapters should provide wire-speed throughput with CPU utilization of less than 10 percent. This guarantees that even in times of peak processing, the adapter will perform without consuming more that 10 percent of the available CPU resources. Additionally, features like integrated storage and network traffic processing on a single card should be evaluated. Even if such features aren't used all the time, common interface cards for network and storage connectivity cut redundancy costs dramatically. For example, two integrated cards could operate together, one in network mode the other in storage mode. If one fails, the other takes over in both modes. While performance may be impacted, the cost savings for redundancy likely far outweigh that trade-off. Finally, dual-pathing and link aggregation of multiple adapters may be required for high-performance, mission-critical applications. These features should be present in IP storage adapters. Figure 2-13 diagrams the basics of traditional IP network, traditional storage, and new IP storage adapters.
Figure 2-13. Network interface cards, host bus adapters, IP storage adapters.
2.6.5 Enhancing Fibre Channel SANs with IP
Critical decisions face storage and networking professionals as they try to balance Fibre Channel storage and SAN investment with IP networking investment. The buildup of two incompatible switched, full-duplex, point-to-point networking architectures simply cannot continue long term. It behooves everyone in the organization, from the CIO to storage managers, to question maintaining separate, independent networking topologies based on different technologies, such as one based on Fibre Channel and one based on Ethernet.
Traditionally, there has been a distinct separation between networking groups and storage groups within corporate IT departments. Some storage groups have claimed that networking technologies cannot support their requirements and that independent storage traffic requires special networks. The reality of the situation is that IP and Ethernet technologies have supported storage traffic for some time (in file-level formats), and recent innovations such as iSCSI provide block-level transport as well. Going forward, networking professionals interested in an overall view of the IT infrastructure will benefit from a closer look at storage, and vice versa for storage professionals.
Whether or not the merging of Fibre Channel and IP technologies meets certain organizational structures is largely irrelevant. Market forces behind the integration of these technologies are upon us, and savvy technologists and business leaders will embrace this dynamic change.
2.6.6 Making Use of the IP Core
Enhancing Fibre Channel solutions with IP starts through an understanding of the IP core. This part of the network provides the most performance, scalability, and intelligence to facilitate the Internet-working of any devices. Equipment used in the IP core can include routers, chassis-based enterprise Ethernet switches, and optical networking equipment using standard Ethernet or DWDM. Other networking technologies include ATM, Frame Relay, and SONET, all of which have mature mechanisms for handling IP transport.
At the edge of the network reside Fibre Channel devices, including servers, storage, and Fibre Channel SAN switches and directors. This equipment provides access to Fibre Channel devices, specifically end-device aggregation, and the fan-out of storage ports.
In order to make use of the existing IP core, which spans from enterprise, to campus, to metropolitan area networks (MANs) and wide area networks (WANs), an IP storage distribution layer integrates the Fibre Channel devices. This intelligent mechanism for protocol conversion enables Fibre Channel to extend its reach and capabilities in ways that it could not accomplish single-handedly. The basics of this model are shown in Figure 2-14.
Figure 2-14. Using IP to enhance Fibre Channel.
2.6.7 Linking Fibre Channel to Fibre Channel with IP
The most common initial deployments for Fibre Channel using IP include remote replication for large Fibre Channel disk subsystems and linking two Fibre Channel SANs across IP, otherwise known as SAN extension. In the first case, IP storage switches or gateways front-end each Fibre Channel subsystem, providing Fibre Channel access to the disks and IP and Ethernet connectivity to the network. Typically, a subsystem-based replication application (such as EMC's SRDF, Hitachi's TrueCopy, HP/Compaq's DRM, XIOtech's REDI SAN Links) then operates on top of this physical link. Across a pair of subsystems, one will assume the initiator status while another assumes the target status, and the appropriate data is replicated through synchronous or asynchronous mechanisms.
Similar architectures exist that may also include Fibre Channel SANs, such as switches and directors. Those Fibre Channel SAN islands can also connect via IP using IP storage switches or gateways. Both of these options are shown in Figure 2-15. Servers and storage within each SAN can be assigned to view other devices locally as well as across the IP connection.
Figure 2-15. Mechanisms for linking Fibre Channel SANs and devices with IP networks.
The benefits of these architectures include the ability to use readily available and cost-effective IP networks across all distances. Typically, these costs can be an order of magnitude less than the costs associated with dedicated dark fiber-optic cabling or DWDM solutions required by pure Fibre Channel SANs. Also, adoption of IP technologies within the core prepares for the addition of native iSCSI devices.
2.6.8 Integrating iSCSI with Fibre Channel across IP
The addition of iSCSI devices means that more of the connection between initiators and targets can be IP-centric. Storage planners gain tremendous flexibility and management advantages through this option, maximizing the use of IP networking and localizing the Fibre Channel component to the attachment of storage devices.
Using iSCSI devices with native IP interfaces allows the devices to talk to each other across any IP and Ethernet network. However, iSCSI makes no provision for Fibre Channel, and the conversion between iSCSI devices and Fibre Channel devices must reside within the network. This protocol conversion process typically takes place within an IP storage switch or gateway that supports both iSCSI and Fibre Channel. It can also take place in a server with specialized software or even within a chip that is embedded in a storage subsystem, as shown in Figure 2-16.
Figure 2-16. Mechanisms for linking Fibre Channel SANs to iSCSI with IP Networks.
For a single device-to-device session, such as server to disk, most multiprotocol products can easily accomplish this task. However, when considering more sophisticated storage device-layer interaction, such as device-to-device interconnect for remote mirroring or SAN extension/interconnect, the protocol conversion process may have more impact on the overall operation. Since iSCSI and Fibre Channel have separate methods of SCSI command encapsulation, the various "states" of each storage session differ between the implementations. And while in a perfect world, the conversion process should not affect the overall integrity of the transaction, there is no practical way to preserve complete integrity across two translations (Fibre Channel to iSCSI and iSCSI to Fibre Channel). Therefore, using iSCSI to interconnect two Fibre Channel disk subsystems or two Fibre Channel SANs adds unnecessary risk to the overall system, particularly when considering the storage error conditions that must be kept in check. This is shown in Figure 2-17. Mitigating this risk requires a complementary approach to iSCSI, embracing the IP network, and aiming for a transition path to pure IP SANs.
Figure 2-17. iSCSI and iFCP/FCIP to connect two Fibre Channel end nodes.
IP SAN Protocols
Throughout 2001 and 2002, the IP storage industry spent considerable time addressing the underlying protocols for IP storage. The SNIA IP Storage Forum (www.ipstorage.org), an industry group within the SNIA, provides valuable informational and educational materials on IP storage, including the protocols. The standards ratification process for IP storage is conducted by the IETF (www.ietf.org), the same organization that has standardized and ratified the protocols relating to IP and the Internet.
The three IP storage transport protocols currently under consideration by the IETF reflect unique strategies for block data over TCP/IP networks. They are iSCSI, iFCP, and FCIP, shown in Figure 2-18.
Figure 2-18. Protocol options for IP storage.
The technologies fall into two categories:
The Fibre Channel over IP (FCIP) protocol is a Fibre Channel perpetuation strategy for linking Fibre Channel fabrics over distance. FCIP has minimal IP content facilities, since its aim is not to replace Fibre Channel with IP but simply to facilitate further deployments of Fibre Channel fabrics via IP tunneling.
The Internet Fibre Channel Protocol (iFCP) and Internet SCSI (iSCSI) protocols, by contrast, are IP strategies for supplementing or replacing Fibre Channel fabrics with Gigabit Ethernet and IP infrastructures.
This basic distinction between Fibre Channel and IP strategies reveals fundamental assumptions about market directions and the solutions that customers may deem most viable. As a Fibre Channel strategy, FCIP assumes that current Fibre Channel fabrics are appropriate for data center SANs and that IP is only required to accommodate distances that exceed the capabilities of native Fibre Channel. Both iFCP and iSCSI make the opposite assumption. iFCP and iSCSI address the limitations of Fibre Channel fabrics for both data center and remote SAN connectivity. Since iFCP and iSCSI are native IP solutions for storage end devices, Fibre Channel fabrics are no longer required. The iFCP protocol is designed to support Fibre Channel end devices, and it substitutes an IP infrastructure in place of Fibre Channel. It also has the capability to incorporate existing Fibre Channel hubs and switches, if present. The iSCSI protocol ultimately replaces both storage end devices and the storage transport with IP technology. Thus while FCIP represents Fibre Channel perpetuation, iFCP represents migration transition from Fibre Channel to IP SANs, and iSCSI represents the ultimate goal of homogeneous IP storage networking.
As native IP storage protocols, iFCP and iSCSI are complementary. The iFCP protocol addresses the current SAN market in which both server platforms and storage devices use Fibre Channel interfaces. The current HBAs, Fibre Channel RAIDs, JBODs, and tape subsystems enjoy a much higher degree of interoperability and stability than Fibre Channel fabric switches. The iFCP protocol enables these Fibre Channel end systems to be joined over an IP network, both for data center applications and for wide area requirements. It offers customers the option of using familiar and more manageable Gigabit Ethernet switches and IP routers to build enterprise SANs. This allows SANs to be more easily incorporated into mainstream data communications implementation and management. Aside from overcoming the complexity and interoperability issues of Fibre Channel fabrics, iFCP enables customers to start building IP SANs today. By attaching current Fibre Channel end systems to IP networks, IP storage switches supporting iFCP (and iSCSI) create a foundation on which data center, metropolitan, and wide area storage applications can be built over time. This foundation is the common architecture that will support iSCSI end devices as well.
While iFCP provides the migration transition path from present to future SANs, the iSCSI protocol is the enabling technology for homogeneous IP SANs. The development of iSCSI storage NICs and iSCSI interfaces on storage and tape devices will allow customers to treat storage as just additional nodes on the network and to fully utilize the functionality of IP networks for storage. As with iFCP, leveraging IP and Gigabit Ethernet technology for the SAN fabric allows iSCSI to implement advanced services for storage, such as quality of service levels and encryption of storage data. As iSCSI products based on Gigabit and 10 Gigabit Ethernet come to market, customers will be able to synchronize their requirements for network messaging and storage backbones, and deploy a common network infrastructure for a variety of application needs.
It is worth noting that iSCSI affords customers the option of building native iSCSI solutions without relying on other technologies. Customers can build iSCSI SANs or use iSCSI devices alongside NAS storage.
While Fibre Channel-based storage currently represents the upper layer of the enterprise market, sustained adoption is a market reality. Customer awareness for best of breed Fibre Channel storage deployment options combined with best of breed IP and Ethernet storage networking will drive the adoption of native iSCSI. Customers will always gravitate to solutions that are more familiar, more manageable, and more easily supported. Use of iFCP in IP storage switches or gateways allows customers to minimize their Fibre Channel exposure and maximize benefits of mainstream IP networking. Products based on iSCSI will take this transition a step further by creating iSCSI SANs connected to IP networks. In combination, iFCP and iSCSI solutions thus take customers from where they are today to where they need to be tomorrow.
Entire books have been written about SANs. Two notable volumes are Designing Storage Area Networks and IP SANs, both by Tom Clark. For those interested in the next level of technical detail, they both serve as excellent references.
Achieving the roadmap objectives of IP Storage Networking—Straight to the Core requires a thorough understanding of SANs and the underlying infrastructure of both Fibre Channel and IP SANs. This section covers such material with the intent to quickly move on to the software spectrum that maximizes the impact of a flexible, well-architected infrastructure.
2.6.1 SAN History and Background
In the mid-1990s, IT professionals recognized the gap between DAS and NAS, and momentum grew for a mechanism that combined the performance and direct access of DAS with the network flexibility of NAS. The solution that emerged was Fibre Channel.
Fibre Channel is actually a SCSI-based implementation, only in a serial format. SCSI commands are at the root of all storage exchanges between imitators and targets. Commands such as write, read, and acknowledge provide the rules of engagement between two devices. This SCSI command set has proved invaluable as the lingua franca for storage.
Originally, the SCSI command set could operate only on top of a SCSI parallel bus. This cabling mechanism involved the use of large, high-density cabling with as many as 68 wires per cable. SCSI cabling cannot extend beyond 25 meters and can effectively support only a handful of devices.
Fibre Channel solved SCSI limitations of device count and distance with the Fibre Channel Protocol (FCP) and Fibre Channel networks (see Figure 2-9). It is important to remember that these two components do not necessarily need to operate together, especially when considering implementations of IP storage. With the ability to layer the SCSI command set on to FCP, and then transmit the information across a serial high-speed network built with Fibre Channel, IT professionals could now build networked storage infrastructures where servers have direct access to storage devices, new storage devices could easily be added to the network, and storage capacity could be shared among multiple servers. These SAN benefits, including the ability for storage professionals to manage the ever-increasing amount of data, led to the adoption of this architecture by virtually all large corporations.
Figure 2-9. Basics of Fibre Channel Protocol (FCP).
With the routable packaging in place via FCP shown in Figure 2-9, the network build-out for storage could take place with Fibre Channel SANs, shown in Figure 2-10.
Figure 2-10. Fibre Channel storage area networks.
Fibre Channel SANs proved invaluable for storage professionals looking to expand direct access storage capacity and utilization. However, Fibre Channel SANs came with their own set of issues that left the end-to-end solution incomplete. This includes unresolved interoperability issues between multiple vendors' products. While compatibility between Fibre Channel adapters in servers and Fibre Channel storage devices is mature, the interoperability required to build Fibre Channel fabrics with switches and directors from multiple vendors remains a difficult and complex process.
Fibre Channel SANs, through the inherent nature of introducing a new network technology different from IP and Ethernet, requires new staff training, new equipment, and new management systems. These factors add considerably to the overall total cost of the storage infrastructure.
Finally, Fibre Channel networking was designed as a data center technology, not intended to travel long distances across WANs. With today's requirements for high-availability across geographic areas, Fibre Channel networks cannot compete independently, even with the help of extenders or Dense Wave Division Multiplexing (DWDM) products. At a maximum, these solutions can extend Fibre Channel traffic to around 100 kilometers.
2.6.2 Pure IP Storage Networks
Starting in 2000, the concept of a pure IP SAN took hold. The basic concept of this implementation was to create a method of connecting servers and storage devices, with the direct access block methods applications expected, via an Ethernet and IP network. This would provide IT professionals the ability to utilize existing, well-understood, mature networking technology in conjunction with block-addressable storage devices. The underlying technology enabling this architecture is known as Internet Small Computer Systems Interface, or iSCSI (see Figure 2-11).
Figure 2-11. Basics of Internet Small Computer System Interface (iSCSI).
iSCSI, in its simplest form, represents a mechanism to take block-oriented SCSI commands and map them to an IP network. This protocol can be implemented directly on servers or storage devices to allow native connectivity to IP and Ethernet networks or fabrics. Once accomplished, storage applications requiring direct access to storage devices can extend that connection across an IP and Ethernet network, which has a nearly ubiquitous footprint across corporate and carrier infrastructures. The basics of the iSCSI protocol are outlined in Figure 2-11, while an implementation example is shown in Figure 2-12.
Figure 2-12. Pure iSCSI storage networks.
This wholehearted embrace of underlying IP and Ethernet technologies—mature network technology, installed base, software integration, industry research—should guarantee iSCSI will succeed as a storage transport. However, networking technologies succeed based on provision of a transition path. The same applies for iSCSI, where the transition path may occur from internal server storage to iSCSI SANs, parallel SCSI devices to iSCSI, or enterprise Fibre Channel storage to iSCSI SAN.
Adoption of new technologies occurs most frequently with those likely to gain. With iSCSI, that would indicate adoption first in mid-size to large corporations that have already deployed a Fibre Channel SAN; recognize the benefits of direct access, networked storage infrastructures; and need the ability to scale to large deployments across geographically dispersed locations. This profile often accompanies a significant storage budget that could be considerably trimmed in cost and complexity savings introduced by an IP-centric approach.
For a pure iSCSI storage network, no accommodation is made for Fibre Channel. More specifically, the iSCSI specification, as ratified by the IETF, makes no reference to Fibre Channel. That protocol conversion is outside the purview of the standards organization and left completely to vendor implementation.
Several mechanisms exist to enhance current Fibre Channel SANs with IP. Optimized storage deployment depends on careful analysis of these solutions, attention to control points within the architecture, and recognition that aggressively embracing IP networking can lead to dramatic total cost benefits. These considerations are covered in Section 2.6.5, "Enhancing Fibre Channel SANs with IP."
2.6.3 SAN GUIDANCE: Comparing Ethernet/IP and Fibre Channel Fabrics
With the introduction of IP storage as a capable storage network transport, many comparisons have been drawn between the two technologies. For fair comparison, one must recognize that Fibre Channel and Ethernet are both Layer 2 networking technologies virtually equivalent in performance. They both provide point-to-point, full-duplex, switched architectures. It is very hard to differentiate networking technologies with these common characteristics. Therefore, the speed and feed comparisons between Fibre Channel and Ethernet are largely irrelevant.
However, Ethernet is the largest home for the Internet Protocol, and IP ubiquity dramatically expands fabric flexibility. The equivalent in Fibre Channel would be the FCP, but since that works only on Fibre Channel fabrics, it can't compete with IP as an intelligent packaging mechanism. With that in mind, the Table 2-1 depicts some fabric differences.
Fibre Channel SANs function well in data center environments and are useful for massive Fibre Channel end-device consolidation. But for reasons identified above, Fibre Channel requires complementary technologies such as IP and Ethernet to help integrate storage into overall corporate cost and management objectives.
Table 2-1. Comparison of Fibre Channel and Ethernet/IP Fabrics Pure Fibre Channel Fabric
Ethernet and IP Fabric
Distance
Approximately 100Km, including DWDM
Unlimited
Size limitations
Fabric switch limit of 239
Unlimited
Network performance
Equivalent (point-to-point, full-duplex, switched)
Equivalent (point-to-point, full-duplex, switched)
Transport
Fibre Channel, DWDM
Ethernet, ATM, Packet-over-SONET, T-1, T-3, DS-3, DWDM
Fault Isolation
Not available; single fabric solution
Protection via autonomous regions
Positioning
High performance
Network convergence
Total cost of ownership
Independent to SAN
Integrated with corporate IP networking
Network management
Independent
Integrated to IP
Network reach
Data center, Metro SAN
SAN, LAN, MAN, WAN
2.6.4 Host Bus Adapters and Network Interface Cards
Regardless of the networking interconnect, servers and storage devices need methods to take data from the CPU or the disk drives out to the wire. Servers use HBAs or network interface cards (NICs) for such functions, and storage subsystems use controllers. Both types of devices are functionally similar. Frequently, end-system connectivity gets inappropriately mixed in with network connectivity to describe SANs, but the two are fundamentally different. With the ability to do wire-speed conversion within networking fabrics between different cabling and protocols, end-system interconnects and network fabrics need not be identical.
With the dramatic differences between traditional IP-based LAN traffic using NICs and traditional storage-based traffic using Fibre Channel HBAs, it helps to clarify how these fit with new IP storage adapters, such as iSCSI. In the traditional IP case, TCP/IP segmentation resided in the host application layer. When the server wanted to send information out on the wire (link layer), it would have to segment the file into smaller frames that would be sent to the NIC driver layer. The driver layer had the simple task of relaying those packets to the wire. With large files, however, the host-based process of parsing the information into smaller units consumed considerable CPU cycles—and ultimately led to poor performance.
Fibre Channel-based HBAs compensated for this inefficiency by moving the data segmentation directly to the card and letting the card do the heavy lifting of converting to smaller frames for the link layer. This freed CPU cycles for application processing and created a highly efficient mechanism for moving data out to the wire.
A combination of these two approaches results in the efficiency of the Fibre Channel approach, through a process called TCP/IP offload. TCP/IP offload can be used for network based servers using TCP/IP, including iSCSI or NAS. The processing for data segmentation once completed in the host now moves to the card, freeing CPU cycles. Also, the data comes out of the server directly to a TCP/IP and Ethernet link using iSCSI, providing native access to IP networks and the corresponding universal reach. IP storage can run on traditional IP NICs through software drivers, but the CPU utilization will be high and could impact application performance during operating hours. During off-peak times, when CPU cycles are available, this approach may be sufficient.
IP storage adapters can be further categorized by the types of offload mechanisms. At the basic level, all IP storage adapters employ some form of TCP/IP offloading. Further, some adapters will offload the iSCSI process from the host as well. The intricacies among these implementations require examining several features during evaluation. First and foremost, IP storage adapters should provide wire-speed throughput with CPU utilization of less than 10 percent. This guarantees that even in times of peak processing, the adapter will perform without consuming more that 10 percent of the available CPU resources. Additionally, features like integrated storage and network traffic processing on a single card should be evaluated. Even if such features aren't used all the time, common interface cards for network and storage connectivity cut redundancy costs dramatically. For example, two integrated cards could operate together, one in network mode the other in storage mode. If one fails, the other takes over in both modes. While performance may be impacted, the cost savings for redundancy likely far outweigh that trade-off. Finally, dual-pathing and link aggregation of multiple adapters may be required for high-performance, mission-critical applications. These features should be present in IP storage adapters. Figure 2-13 diagrams the basics of traditional IP network, traditional storage, and new IP storage adapters.
Figure 2-13. Network interface cards, host bus adapters, IP storage adapters.
2.6.5 Enhancing Fibre Channel SANs with IP
Critical decisions face storage and networking professionals as they try to balance Fibre Channel storage and SAN investment with IP networking investment. The buildup of two incompatible switched, full-duplex, point-to-point networking architectures simply cannot continue long term. It behooves everyone in the organization, from the CIO to storage managers, to question maintaining separate, independent networking topologies based on different technologies, such as one based on Fibre Channel and one based on Ethernet.
Traditionally, there has been a distinct separation between networking groups and storage groups within corporate IT departments. Some storage groups have claimed that networking technologies cannot support their requirements and that independent storage traffic requires special networks. The reality of the situation is that IP and Ethernet technologies have supported storage traffic for some time (in file-level formats), and recent innovations such as iSCSI provide block-level transport as well. Going forward, networking professionals interested in an overall view of the IT infrastructure will benefit from a closer look at storage, and vice versa for storage professionals.
Whether or not the merging of Fibre Channel and IP technologies meets certain organizational structures is largely irrelevant. Market forces behind the integration of these technologies are upon us, and savvy technologists and business leaders will embrace this dynamic change.
2.6.6 Making Use of the IP Core
Enhancing Fibre Channel solutions with IP starts through an understanding of the IP core. This part of the network provides the most performance, scalability, and intelligence to facilitate the Internet-working of any devices. Equipment used in the IP core can include routers, chassis-based enterprise Ethernet switches, and optical networking equipment using standard Ethernet or DWDM. Other networking technologies include ATM, Frame Relay, and SONET, all of which have mature mechanisms for handling IP transport.
At the edge of the network reside Fibre Channel devices, including servers, storage, and Fibre Channel SAN switches and directors. This equipment provides access to Fibre Channel devices, specifically end-device aggregation, and the fan-out of storage ports.
In order to make use of the existing IP core, which spans from enterprise, to campus, to metropolitan area networks (MANs) and wide area networks (WANs), an IP storage distribution layer integrates the Fibre Channel devices. This intelligent mechanism for protocol conversion enables Fibre Channel to extend its reach and capabilities in ways that it could not accomplish single-handedly. The basics of this model are shown in Figure 2-14.
Figure 2-14. Using IP to enhance Fibre Channel.
2.6.7 Linking Fibre Channel to Fibre Channel with IP
The most common initial deployments for Fibre Channel using IP include remote replication for large Fibre Channel disk subsystems and linking two Fibre Channel SANs across IP, otherwise known as SAN extension. In the first case, IP storage switches or gateways front-end each Fibre Channel subsystem, providing Fibre Channel access to the disks and IP and Ethernet connectivity to the network. Typically, a subsystem-based replication application (such as EMC's SRDF, Hitachi's TrueCopy, HP/Compaq's DRM, XIOtech's REDI SAN Links) then operates on top of this physical link. Across a pair of subsystems, one will assume the initiator status while another assumes the target status, and the appropriate data is replicated through synchronous or asynchronous mechanisms.
Similar architectures exist that may also include Fibre Channel SANs, such as switches and directors. Those Fibre Channel SAN islands can also connect via IP using IP storage switches or gateways. Both of these options are shown in Figure 2-15. Servers and storage within each SAN can be assigned to view other devices locally as well as across the IP connection.
Figure 2-15. Mechanisms for linking Fibre Channel SANs and devices with IP networks.
The benefits of these architectures include the ability to use readily available and cost-effective IP networks across all distances. Typically, these costs can be an order of magnitude less than the costs associated with dedicated dark fiber-optic cabling or DWDM solutions required by pure Fibre Channel SANs. Also, adoption of IP technologies within the core prepares for the addition of native iSCSI devices.
2.6.8 Integrating iSCSI with Fibre Channel across IP
The addition of iSCSI devices means that more of the connection between initiators and targets can be IP-centric. Storage planners gain tremendous flexibility and management advantages through this option, maximizing the use of IP networking and localizing the Fibre Channel component to the attachment of storage devices.
Using iSCSI devices with native IP interfaces allows the devices to talk to each other across any IP and Ethernet network. However, iSCSI makes no provision for Fibre Channel, and the conversion between iSCSI devices and Fibre Channel devices must reside within the network. This protocol conversion process typically takes place within an IP storage switch or gateway that supports both iSCSI and Fibre Channel. It can also take place in a server with specialized software or even within a chip that is embedded in a storage subsystem, as shown in Figure 2-16.
Figure 2-16. Mechanisms for linking Fibre Channel SANs to iSCSI with IP Networks.
For a single device-to-device session, such as server to disk, most multiprotocol products can easily accomplish this task. However, when considering more sophisticated storage device-layer interaction, such as device-to-device interconnect for remote mirroring or SAN extension/interconnect, the protocol conversion process may have more impact on the overall operation. Since iSCSI and Fibre Channel have separate methods of SCSI command encapsulation, the various "states" of each storage session differ between the implementations. And while in a perfect world, the conversion process should not affect the overall integrity of the transaction, there is no practical way to preserve complete integrity across two translations (Fibre Channel to iSCSI and iSCSI to Fibre Channel). Therefore, using iSCSI to interconnect two Fibre Channel disk subsystems or two Fibre Channel SANs adds unnecessary risk to the overall system, particularly when considering the storage error conditions that must be kept in check. This is shown in Figure 2-17. Mitigating this risk requires a complementary approach to iSCSI, embracing the IP network, and aiming for a transition path to pure IP SANs.
Figure 2-17. iSCSI and iFCP/FCIP to connect two Fibre Channel end nodes.
IP SAN Protocols
Throughout 2001 and 2002, the IP storage industry spent considerable time addressing the underlying protocols for IP storage. The SNIA IP Storage Forum (www.ipstorage.org), an industry group within the SNIA, provides valuable informational and educational materials on IP storage, including the protocols. The standards ratification process for IP storage is conducted by the IETF (www.ietf.org), the same organization that has standardized and ratified the protocols relating to IP and the Internet.
The three IP storage transport protocols currently under consideration by the IETF reflect unique strategies for block data over TCP/IP networks. They are iSCSI, iFCP, and FCIP, shown in Figure 2-18.
Figure 2-18. Protocol options for IP storage.
The technologies fall into two categories:
The Fibre Channel over IP (FCIP) protocol is a Fibre Channel perpetuation strategy for linking Fibre Channel fabrics over distance. FCIP has minimal IP content facilities, since its aim is not to replace Fibre Channel with IP but simply to facilitate further deployments of Fibre Channel fabrics via IP tunneling.
The Internet Fibre Channel Protocol (iFCP) and Internet SCSI (iSCSI) protocols, by contrast, are IP strategies for supplementing or replacing Fibre Channel fabrics with Gigabit Ethernet and IP infrastructures.
This basic distinction between Fibre Channel and IP strategies reveals fundamental assumptions about market directions and the solutions that customers may deem most viable. As a Fibre Channel strategy, FCIP assumes that current Fibre Channel fabrics are appropriate for data center SANs and that IP is only required to accommodate distances that exceed the capabilities of native Fibre Channel. Both iFCP and iSCSI make the opposite assumption. iFCP and iSCSI address the limitations of Fibre Channel fabrics for both data center and remote SAN connectivity. Since iFCP and iSCSI are native IP solutions for storage end devices, Fibre Channel fabrics are no longer required. The iFCP protocol is designed to support Fibre Channel end devices, and it substitutes an IP infrastructure in place of Fibre Channel. It also has the capability to incorporate existing Fibre Channel hubs and switches, if present. The iSCSI protocol ultimately replaces both storage end devices and the storage transport with IP technology. Thus while FCIP represents Fibre Channel perpetuation, iFCP represents migration transition from Fibre Channel to IP SANs, and iSCSI represents the ultimate goal of homogeneous IP storage networking.
As native IP storage protocols, iFCP and iSCSI are complementary. The iFCP protocol addresses the current SAN market in which both server platforms and storage devices use Fibre Channel interfaces. The current HBAs, Fibre Channel RAIDs, JBODs, and tape subsystems enjoy a much higher degree of interoperability and stability than Fibre Channel fabric switches. The iFCP protocol enables these Fibre Channel end systems to be joined over an IP network, both for data center applications and for wide area requirements. It offers customers the option of using familiar and more manageable Gigabit Ethernet switches and IP routers to build enterprise SANs. This allows SANs to be more easily incorporated into mainstream data communications implementation and management. Aside from overcoming the complexity and interoperability issues of Fibre Channel fabrics, iFCP enables customers to start building IP SANs today. By attaching current Fibre Channel end systems to IP networks, IP storage switches supporting iFCP (and iSCSI) create a foundation on which data center, metropolitan, and wide area storage applications can be built over time. This foundation is the common architecture that will support iSCSI end devices as well.
While iFCP provides the migration transition path from present to future SANs, the iSCSI protocol is the enabling technology for homogeneous IP SANs. The development of iSCSI storage NICs and iSCSI interfaces on storage and tape devices will allow customers to treat storage as just additional nodes on the network and to fully utilize the functionality of IP networks for storage. As with iFCP, leveraging IP and Gigabit Ethernet technology for the SAN fabric allows iSCSI to implement advanced services for storage, such as quality of service levels and encryption of storage data. As iSCSI products based on Gigabit and 10 Gigabit Ethernet come to market, customers will be able to synchronize their requirements for network messaging and storage backbones, and deploy a common network infrastructure for a variety of application needs.
It is worth noting that iSCSI affords customers the option of building native iSCSI solutions without relying on other technologies. Customers can build iSCSI SANs or use iSCSI devices alongside NAS storage.
While Fibre Channel-based storage currently represents the upper layer of the enterprise market, sustained adoption is a market reality. Customer awareness for best of breed Fibre Channel storage deployment options combined with best of breed IP and Ethernet storage networking will drive the adoption of native iSCSI. Customers will always gravitate to solutions that are more familiar, more manageable, and more easily supported. Use of iFCP in IP storage switches or gateways allows customers to minimize their Fibre Channel exposure and maximize benefits of mainstream IP networking. Products based on iSCSI will take this transition a step further by creating iSCSI SANs connected to IP networks. In combination, iFCP and iSCSI solutions thus take customers from where they are today to where they need to be tomorrow.
The Storage Architectural Landscape
Storage Building Blocks
The basic elements of a computing storage system haven't changed in the last 30 or so years. The equipment and areas that control the primary interaction may have moved around, but at the end of the day, an operating system still initiates a request to a file system, which in turns talks to a volume management layer, and ultimately the storage devices themselves, as shown in Figure 2-1.
Figure 2-1. Basic storage building blocks.
The earliest server implementations of this architecture took place internal to the system but have since expanded outside the box into specialized equipment and software platforms. Intelligent disk arrays, switching infrastructure, and network transports have shattered the traditional blueprint for storage designs. This newfound geographic freedom for a wide variety of storage functions presents IT professionals with previously unavailable architectural options. In today's global computing environments, these options help save time and money.
The SNIA shared storage model serves as an excellent reference for outlining storage architectures. Specifically, the model provides a navigable map for the variety of functions that make up a complete storage system.
2.1.1 STORAGE GUIDANCE: File Systems, Volumes, and Blocks
The model in Figure 2-2 provides an abstraction of how applications address storage, allowing IT administrators to effectively plan complete architectures. It carries the simplified model in Figure 2-1 further by calling out specific mechanisms that file systems and volume management use to access storage devices. In the case of the SNIA model, one can roughly equate volume management with block aggregation.
Figure 2-2. The SNIA shared storage model.
Applications are the common starting point for the model. From there, all other mechanisms fall into the storage domain—database management systems (DBMS), file systems (FS), and all of the ways to connect to disks. The connection to the physical disk drive may be direct from the server or host, through a storage network, or through a device such as a NAS head or NAS server. Each of these areas—host, network, and device—may perform the block aggregation functions required to deliver blocks to the upper-layer file or record layer.
One benefit of the SNIA shared storage model is the ability to map a variety of storage architectures onto a single framework. In the case of large corporations with storage deployments spanning direct-attached storage (DAS), NAS, and SAN-attached storage, viewing these seemingly separate hardware infrastructure across common dimensions can help rationalize purchases and resources across the storage organization. Further, this optimized view helps IT managers plan and architect corporatewide solutions serving multiple departments, enabling greater efficiency.
Figure 2-3 highlights the versatility of the model in depicting multiple architectures within one framework. Item one, DAS, connects directly from a host, or server, with its own logical volume management (LVM) to a set of disks. In this case, the host provides software redundant array of independent disks (RAID) functionality to the unintelligent disks, often referred to as just a bunch of disks (JBOD). File system and volume management functions reside on the host. Note that the host box covers the entire space of the file/record layer, and covers host block aggregation in the block layer of the diagram. Item two is similar, but now the host connects to a disk array across a SAN. It still has some host block aggregation (i.e., volume management), but now some of the aggregation, such as the RAID functions, are shared by the device, which overlaps with device block aggregation in the block layer of the diagram.
Figure 2-3. The SNIA shared storage model depicting direct-attached, SAN-attached, and network-attached storage.
Items three and four change the storage access mechanisms to file-based, as opposed to block-based. Note that a local area network (LAN) resides between the host and the NAS head and NAS server, and that each NAS unit overlaps into the file/record layer. The communication requests from the hosts to the NAS units are file-based, used for corporate file sharing, for example, compared to block-based hosts (items one and two), which use a SCSI-based access mechanism (SCSI, Fibre Channel, iSCSI) to communicate with the disks.
Internal Storage/Server-based storage
In early computing designs, storage devices were located within the computing system itself, such as a server. In this case, all storage components (file system, volume management, and storage devices) are internal. This simple implementation is easy for administrators to deploy, with each server having specific storage allocations met by internal disk drives.
Disk drive interfaces vary between SCSI (Small Computer System Interface), IDE (Integrated Device Electronics), and ATA (Advanced Technology Attachment), with each providing a cost/performance optimization suited to varying market segments. Inside an enclosure, the disk drive interface has little effect on the overall enterprise computing system design. But placing individual storage devices within each server leads to scalability issues. In an internal-only storage environment, each application is largely constrained by the amount of storage that can fit inside the box. Beyond that, the only options left are moving outside the server.
External Storage (JBOD, RAID, Tape, Optical, Other)
External to the server, storage devices have the flexibility to evolve to highly specific task-oriented machines, from JBODs at the low end of the disk-based media spectrum to high-end RAID systems. Similarly, tape, optical, solid-state, and other forms of storage come in all shapes and sizes to suit specific needs. This section covers the basics of external storage devices leading up to transition from direct-attached to networked storage.
An understanding of these basic storage components provides significant value as IT professionals begin to chart their storage networking roadmaps. Since external storage devices vary in size, cost, and complexity, effective use of these basic building blocks helps offset potential pitfalls as the entire storage infrastructure scales.
2.3.1 Just a Bunch of Disks (JBOD)
While some may place JBODs at the low end of the storage food chain, a careful look at the overall storage networking picture reveals more sophisticated uses for these devices. With a relatively low cost per megabyte for disk-based storage, JBODs provide excellent value where redundancy and intelligence can be added elsewhere in the overall architecture, for example within a host or network.
The basic JBOD is an enclosure for 3.5-inch hard disk drives that provides a daisy-chain mechanism to link the devices together. In SCSI terms, the disk drives are independent devices on a parallel SCSI bus. In Fibre Channel terms, the disk drives are independent devices on a Fibre Channel loop. A typical JBOD architecture based on Fibre Channel loop is outlined in Figure 2-4.
Figure 2-4. Basic JBOD design.
2.3.2 Redundant Array of Independent/Inexpensive Disks (RAID)
RAID can be implemented directly in storage hardware devices or in storage controllers, such as host bus adapters (HBAs), or in software. While these options offer choice in terms of cost and performance, the basic principles of RAID remain the same. RAID architectures protect disk drive-based data with several redundancy mechanisms to protect against disk failure. Implemented properly, storage systems using RAID will be unaffected by disk drive loss and can go uninterrupted through the cycle of disk failure and repair.
RAID grew considerably with the introduction of inexpensive 5.25-inch and 3.5-inch drives. Manufacturers valued the opportunity to use less expensive components with a cost curve similar to the PC industry, but also had to contend with the potential failure of such low-end components. The specific terminology used to describe the life of a disk drive is mean time between failure (MTBF). By removing individual disk drive failure from the overall system MTBF, RAID systems could guarantee higher availability and continuous uptime for users.
Today, high-end RAID systems offer a variety of redundancy measures and storage management features that contend directly with similar offerings in the rest of the enterprise. A mirror, or exact copy of the data on an additional set of disk drives, can operate directly within a RAID system for optimized performance. Similar functions can also be accomplished directly with host-based software or within a storage network.
The primary RAID functions include striping, mirroring, and parity. IT professionals should be familiar with these functions and terminology, as outlined next in Section 2.3.3.
2.3.3 STORAGE GUIDANCE: RAID Levels
Basic RAID features break down into a series of defined RAID levels as originally documented by David A. Patterson, Garth A. Gibson, and Randy H. Katz of the University of California at Berkeley in 1988. Six basic levels were defined—RAID 0 through 5 (see Figure 2-5). Each represents a unique mechanism to achieve higher performance and availability with common disk drive components. The numbers do not directly correlate with degrees of performance or availability and should be viewed categorically. In practice, RAID 0, 1, and 5 are the most commonly implemented RAID mechanisms. RAID 2 required special disk features. Given the economics of volume production, the cost of such features favored simply using other RAID levels. RAID 3 and 4 also are not frequently implemented due to the performance impact of dedicated parity disks.
Figure 2-5. RAID levels.
RAID levels, outlined in Figure 2-5, can be combined to create even higher levels of availability. For example, a RAID 0 array may also be mirrored. Known as RAID 0+1, this provides the performance of RAID 0 striping with the redundancy of RAID 1 mirroring. Another configuration, RAID 10, stripes data across two independent mirrored disk volumes. Similar combinations exist with RAID 0 and RAID 5.
2.3.4 Tape Drives and Tape Libraries
Purchasing decisions for storage devices measure performance, features, capacity, and cost. At the high end of the performance, feature, and cost equation are large RAID arrays. While it would be convenient for enterprises to standardize on a single set of media devices, economics come into play and create opportunities for other device types.
Tape drives and tape libraries offer high-capacity, low-cost storage when compared to disk arrays. Tape density for single tape drives and cartridge density for automated tape libraries far outweigh the equivalent capacity of a disk-based unit with a similar footprint. Further, the ability to swap tape cartridges provides flexibility to add capacity and the ability to remotely locate tapes for disaster recovery purposes.
Today, SANs allow consolidated sharing of tape libraries across a number of servers. Whereas previously, each server required its own tape drive, or servers had to share traffic on the LAN for access to a central backup server, SANs allow direct connections between servers and tape libraries for simplified backup, offloading a potentially overburdened LAN. Coupled with the appropriate backup software package, this provides an economical, easily managed backup system.
Tape media comes in a variety of forms, such as digital linear tape (DLT), linear tape-open (LTO), and advanced intelligent tape (AIT), each offering its own capacity, performance, and cost equation that will manifest itself in the overall system specifications. Choice of a tape library should focus more on system features within the context of the enterprise storage architecture rather than choice of a specific media type. However, purchasers should be advised that media types do have a life of their own. Yearly checks to assure both media type and subsystem availability should be part of data center operating procedures. To protect against outdated media types and potential data loss due to tape degradation or magnetic interference, IT managers often re-record data to fresh tapes every year. This guarantees recovery mechanisms but also adds the cost of tape replenishment.
2.3.5 Other Storage (Optical, Solid State)
Other media technologies include optical, which is often compared to tape, but since it uses electromagnetic recording mechanisms, the shelf life of the discs far exceeds that of tape. Electromagnetic mechanisms for optical recording operate at much higher temperatures than traditional magnetic disk and tape recording. This prevents interference from nearby magnetic material or environmental impacts due to low or high temperatures. In fact, if kept in a properly controlled environment, optical media suffers virtually no degradation over time.
Optical media comes in recordable CD/DVD form, 5.25-inch WORM (write once, read many), and 12-inch optical disks. These magnetic-optic (MO) drives operate independently or can be assembled in to a library (often called a jukebox) to provide high-capacity optical storage similar to tape libraries. In an automated jukebox, mechanics allow for recording on both sides of the optical disk.
Solid-state storage media sits highest on both the performance measure and cost measure. It is often used for performance-critical applications that operate from a small data set. As the name implies, solid-state storage relies upon memory chips to provide capacity. This implementation provides extremely high throughput and I/O operations per second. However, due to cost and capacity limitations, solid-state storage is used in limited market segments.
Direct-Attached Storage
DAS expands server storage capacity. It provides easily configurable storage that looks local, or internal, to a server, yet has scalable capacity benefits of being outside the server itself. External storage can extend server life by providing not only additional capacity but a range of storage features, such as redundancy, that help protect data.
With DAS, servers access blocks of data. The file system and volume management layers are host-based. Other data storage functions implemented independent of the server, such as RAID, provide greater performance and leave the server free to focus on its priority tasks, such as application processing. Most external storage devices attach to servers via high speed parallel SCSI or Fibre Channel cabling. Traditionally, the speed difference between SCSI and Fibre Channel compared to LANs has been significant, although today's high-speed networking interconnects, such as Gigabit Ethernet, level the playing field. See Section 2.6.3, "SAN GUIDANCE: Comparing Ethernet/IP and Fibre Channel Fabrics," for a more complete overview of storage and networking interconnects.
However, even DAS has its limits. Adding additional storage usually requires incremental servers, and one server does not have direct access to another server's storage. This captive storage model makes resource sharing difficult and leaves servers susceptible to single points of failure. The basic DAS model is shown in Figure 2-6.
Figure 2-6. Direct-attached storage.
Reaching the limits of the captive storage model, solutions with additional scalability arose that networked storage together. This development took shape in the mid-1980s, well before the introduction of high-speed serial storage technologies such as Fibre Channel and iSCSI. Therefore, without a means to serialize the high-speed parallel SCSI interconnects between servers and DAS devices, the only other available network was the Ethernet and IP-based LAN. This led to the introduction of NAS.
Network-Attached Storage
NAS devices overcame several barriers when introduced to the storage market. First, NAS broke the DAS captive model. Second, it provided a mechanism to link storage across Ethernet and IP networks, a completely new footprint for storage administrators. Most importantly, and in part to support the architectural change, NAS moved the file system and volume management functions from the server to a NAS device, often called a filer. Keep in mind that the "filing" function remains no matter what the storage model. The location of this function has traditionally separated the distinction of NAS and SAN. See the following section for a more complete outline of the differences between NAS and SAN.
2.5.1 STORAGE GUIDANCE: SAN and NAS
A discussion about accessing storage via IP and Ethernet, such as with IP storage adapters, invariably begs the question about the difference between SAN and NAS. The SNIA shared storage model outlines file-level and block-level distinctions, which are further clarified in Figure 2-7.
Figure 2-7. Comparing SAN and NAS.
SANs provide scalable performance with direct access to storage devices. Since the storage system resides on the distributed hosts, this architecture works well for databases and online transaction processing.
NAS provides crossplatform scalability, but not the performance scalability due to its use of a high level of file abstraction between applications and storage. While a NAS filer offloads some server workload required to map files into physical storage, it adds significant workload to the filer itself. NAS delivers a central storage system (file system) that is ideal for file sharing, Web serving, and similar applications.
A NAS device is essentially a highly optimized file-delivery server. Depending on the specific implementation, NAS devices may have internal storage (similar to a traditional server), or they may be directly connected to a RAID device for additional capacity or to a tape library for backup functions. Additionally, some NAS devices can connect directly to a block-based storage network.
Traditional servers access NAS devices through the Ethernet and IP network by means of a file request. Since the file system resides on the NAS device, servers with different operating systems can all easily access the same data. This crossplatform access gives NAS a considerable advantage in terms of simplicity. It also allows the NAS device to provide lock functions for particular files. Since the device knows if one server is accessing a file, the second server will not be able to write or modify that file.
NAS has made tremendous inroads to certain market segments where the architecture suits the applications. Web-related functions, such as serving HTTP requests, are ideal applications. The relatively small file size combined with the crossplatform access across Ethernet and IP networks makes NAS a winner.
Similarly, corporate file sharing is easily addressed by NAS. To add more storage capacity to such a configuration, a storage administrator would simply need to plug another NAS device into the IP network. Of course, if for the new NAS device to be completely integrated with other NAS devices in a network—for example, where the NAS devices could pool storage capacity—purchasing the unit from the same vendor becomes a requirement.
Figure 2-8 outlines a basic NAS architecture. As shown, application servers access NAS filers over the IP network, providing greater scalability and crossplatform access than DAS provides. However, NAS doesn't alleviate LAN congestion, and the expansion of block-oriented storage remains disruptive. Additionally, each filer owns the storage that it maintains (except in some homogeneous, pooled NAS environments), and in general, NAS has a unique OS that must be maintained across all filers for ease of management. Finally, the high level of file abstraction inherent to NAS design doesn't suit certain application requirements accustomed to block-level storage.
Figure 2-8. Network-attached storage.
The gap between high-performance DAS and flexible NAS became the sweet spot for SANs. Traditionally, block-oriented SANs solved a combination of problems between DAS and NAS with Fibre Channel networks. More recently, SANs have been able to fill this gap with block-oriented IP SANs, using IP storage protocols such as iSCSI. The following sections cover the details of both Fibre Channel and IP SAN infrastructure.
The basic elements of a computing storage system haven't changed in the last 30 or so years. The equipment and areas that control the primary interaction may have moved around, but at the end of the day, an operating system still initiates a request to a file system, which in turns talks to a volume management layer, and ultimately the storage devices themselves, as shown in Figure 2-1.
Figure 2-1. Basic storage building blocks.
The earliest server implementations of this architecture took place internal to the system but have since expanded outside the box into specialized equipment and software platforms. Intelligent disk arrays, switching infrastructure, and network transports have shattered the traditional blueprint for storage designs. This newfound geographic freedom for a wide variety of storage functions presents IT professionals with previously unavailable architectural options. In today's global computing environments, these options help save time and money.
The SNIA shared storage model serves as an excellent reference for outlining storage architectures. Specifically, the model provides a navigable map for the variety of functions that make up a complete storage system.
2.1.1 STORAGE GUIDANCE: File Systems, Volumes, and Blocks
The model in Figure 2-2 provides an abstraction of how applications address storage, allowing IT administrators to effectively plan complete architectures. It carries the simplified model in Figure 2-1 further by calling out specific mechanisms that file systems and volume management use to access storage devices. In the case of the SNIA model, one can roughly equate volume management with block aggregation.
Figure 2-2. The SNIA shared storage model.
Applications are the common starting point for the model. From there, all other mechanisms fall into the storage domain—database management systems (DBMS), file systems (FS), and all of the ways to connect to disks. The connection to the physical disk drive may be direct from the server or host, through a storage network, or through a device such as a NAS head or NAS server. Each of these areas—host, network, and device—may perform the block aggregation functions required to deliver blocks to the upper-layer file or record layer.
One benefit of the SNIA shared storage model is the ability to map a variety of storage architectures onto a single framework. In the case of large corporations with storage deployments spanning direct-attached storage (DAS), NAS, and SAN-attached storage, viewing these seemingly separate hardware infrastructure across common dimensions can help rationalize purchases and resources across the storage organization. Further, this optimized view helps IT managers plan and architect corporatewide solutions serving multiple departments, enabling greater efficiency.
Figure 2-3 highlights the versatility of the model in depicting multiple architectures within one framework. Item one, DAS, connects directly from a host, or server, with its own logical volume management (LVM) to a set of disks. In this case, the host provides software redundant array of independent disks (RAID) functionality to the unintelligent disks, often referred to as just a bunch of disks (JBOD). File system and volume management functions reside on the host. Note that the host box covers the entire space of the file/record layer, and covers host block aggregation in the block layer of the diagram. Item two is similar, but now the host connects to a disk array across a SAN. It still has some host block aggregation (i.e., volume management), but now some of the aggregation, such as the RAID functions, are shared by the device, which overlaps with device block aggregation in the block layer of the diagram.
Figure 2-3. The SNIA shared storage model depicting direct-attached, SAN-attached, and network-attached storage.
Items three and four change the storage access mechanisms to file-based, as opposed to block-based. Note that a local area network (LAN) resides between the host and the NAS head and NAS server, and that each NAS unit overlaps into the file/record layer. The communication requests from the hosts to the NAS units are file-based, used for corporate file sharing, for example, compared to block-based hosts (items one and two), which use a SCSI-based access mechanism (SCSI, Fibre Channel, iSCSI) to communicate with the disks.
Internal Storage/Server-based storage
In early computing designs, storage devices were located within the computing system itself, such as a server. In this case, all storage components (file system, volume management, and storage devices) are internal. This simple implementation is easy for administrators to deploy, with each server having specific storage allocations met by internal disk drives.
Disk drive interfaces vary between SCSI (Small Computer System Interface), IDE (Integrated Device Electronics), and ATA (Advanced Technology Attachment), with each providing a cost/performance optimization suited to varying market segments. Inside an enclosure, the disk drive interface has little effect on the overall enterprise computing system design. But placing individual storage devices within each server leads to scalability issues. In an internal-only storage environment, each application is largely constrained by the amount of storage that can fit inside the box. Beyond that, the only options left are moving outside the server.
External Storage (JBOD, RAID, Tape, Optical, Other)
External to the server, storage devices have the flexibility to evolve to highly specific task-oriented machines, from JBODs at the low end of the disk-based media spectrum to high-end RAID systems. Similarly, tape, optical, solid-state, and other forms of storage come in all shapes and sizes to suit specific needs. This section covers the basics of external storage devices leading up to transition from direct-attached to networked storage.
An understanding of these basic storage components provides significant value as IT professionals begin to chart their storage networking roadmaps. Since external storage devices vary in size, cost, and complexity, effective use of these basic building blocks helps offset potential pitfalls as the entire storage infrastructure scales.
2.3.1 Just a Bunch of Disks (JBOD)
While some may place JBODs at the low end of the storage food chain, a careful look at the overall storage networking picture reveals more sophisticated uses for these devices. With a relatively low cost per megabyte for disk-based storage, JBODs provide excellent value where redundancy and intelligence can be added elsewhere in the overall architecture, for example within a host or network.
The basic JBOD is an enclosure for 3.5-inch hard disk drives that provides a daisy-chain mechanism to link the devices together. In SCSI terms, the disk drives are independent devices on a parallel SCSI bus. In Fibre Channel terms, the disk drives are independent devices on a Fibre Channel loop. A typical JBOD architecture based on Fibre Channel loop is outlined in Figure 2-4.
Figure 2-4. Basic JBOD design.
2.3.2 Redundant Array of Independent/Inexpensive Disks (RAID)
RAID can be implemented directly in storage hardware devices or in storage controllers, such as host bus adapters (HBAs), or in software. While these options offer choice in terms of cost and performance, the basic principles of RAID remain the same. RAID architectures protect disk drive-based data with several redundancy mechanisms to protect against disk failure. Implemented properly, storage systems using RAID will be unaffected by disk drive loss and can go uninterrupted through the cycle of disk failure and repair.
RAID grew considerably with the introduction of inexpensive 5.25-inch and 3.5-inch drives. Manufacturers valued the opportunity to use less expensive components with a cost curve similar to the PC industry, but also had to contend with the potential failure of such low-end components. The specific terminology used to describe the life of a disk drive is mean time between failure (MTBF). By removing individual disk drive failure from the overall system MTBF, RAID systems could guarantee higher availability and continuous uptime for users.
Today, high-end RAID systems offer a variety of redundancy measures and storage management features that contend directly with similar offerings in the rest of the enterprise. A mirror, or exact copy of the data on an additional set of disk drives, can operate directly within a RAID system for optimized performance. Similar functions can also be accomplished directly with host-based software or within a storage network.
The primary RAID functions include striping, mirroring, and parity. IT professionals should be familiar with these functions and terminology, as outlined next in Section 2.3.3.
2.3.3 STORAGE GUIDANCE: RAID Levels
Basic RAID features break down into a series of defined RAID levels as originally documented by David A. Patterson, Garth A. Gibson, and Randy H. Katz of the University of California at Berkeley in 1988. Six basic levels were defined—RAID 0 through 5 (see Figure 2-5). Each represents a unique mechanism to achieve higher performance and availability with common disk drive components. The numbers do not directly correlate with degrees of performance or availability and should be viewed categorically. In practice, RAID 0, 1, and 5 are the most commonly implemented RAID mechanisms. RAID 2 required special disk features. Given the economics of volume production, the cost of such features favored simply using other RAID levels. RAID 3 and 4 also are not frequently implemented due to the performance impact of dedicated parity disks.
Figure 2-5. RAID levels.
RAID levels, outlined in Figure 2-5, can be combined to create even higher levels of availability. For example, a RAID 0 array may also be mirrored. Known as RAID 0+1, this provides the performance of RAID 0 striping with the redundancy of RAID 1 mirroring. Another configuration, RAID 10, stripes data across two independent mirrored disk volumes. Similar combinations exist with RAID 0 and RAID 5.
2.3.4 Tape Drives and Tape Libraries
Purchasing decisions for storage devices measure performance, features, capacity, and cost. At the high end of the performance, feature, and cost equation are large RAID arrays. While it would be convenient for enterprises to standardize on a single set of media devices, economics come into play and create opportunities for other device types.
Tape drives and tape libraries offer high-capacity, low-cost storage when compared to disk arrays. Tape density for single tape drives and cartridge density for automated tape libraries far outweigh the equivalent capacity of a disk-based unit with a similar footprint. Further, the ability to swap tape cartridges provides flexibility to add capacity and the ability to remotely locate tapes for disaster recovery purposes.
Today, SANs allow consolidated sharing of tape libraries across a number of servers. Whereas previously, each server required its own tape drive, or servers had to share traffic on the LAN for access to a central backup server, SANs allow direct connections between servers and tape libraries for simplified backup, offloading a potentially overburdened LAN. Coupled with the appropriate backup software package, this provides an economical, easily managed backup system.
Tape media comes in a variety of forms, such as digital linear tape (DLT), linear tape-open (LTO), and advanced intelligent tape (AIT), each offering its own capacity, performance, and cost equation that will manifest itself in the overall system specifications. Choice of a tape library should focus more on system features within the context of the enterprise storage architecture rather than choice of a specific media type. However, purchasers should be advised that media types do have a life of their own. Yearly checks to assure both media type and subsystem availability should be part of data center operating procedures. To protect against outdated media types and potential data loss due to tape degradation or magnetic interference, IT managers often re-record data to fresh tapes every year. This guarantees recovery mechanisms but also adds the cost of tape replenishment.
2.3.5 Other Storage (Optical, Solid State)
Other media technologies include optical, which is often compared to tape, but since it uses electromagnetic recording mechanisms, the shelf life of the discs far exceeds that of tape. Electromagnetic mechanisms for optical recording operate at much higher temperatures than traditional magnetic disk and tape recording. This prevents interference from nearby magnetic material or environmental impacts due to low or high temperatures. In fact, if kept in a properly controlled environment, optical media suffers virtually no degradation over time.
Optical media comes in recordable CD/DVD form, 5.25-inch WORM (write once, read many), and 12-inch optical disks. These magnetic-optic (MO) drives operate independently or can be assembled in to a library (often called a jukebox) to provide high-capacity optical storage similar to tape libraries. In an automated jukebox, mechanics allow for recording on both sides of the optical disk.
Solid-state storage media sits highest on both the performance measure and cost measure. It is often used for performance-critical applications that operate from a small data set. As the name implies, solid-state storage relies upon memory chips to provide capacity. This implementation provides extremely high throughput and I/O operations per second. However, due to cost and capacity limitations, solid-state storage is used in limited market segments.
Direct-Attached Storage
DAS expands server storage capacity. It provides easily configurable storage that looks local, or internal, to a server, yet has scalable capacity benefits of being outside the server itself. External storage can extend server life by providing not only additional capacity but a range of storage features, such as redundancy, that help protect data.
With DAS, servers access blocks of data. The file system and volume management layers are host-based. Other data storage functions implemented independent of the server, such as RAID, provide greater performance and leave the server free to focus on its priority tasks, such as application processing. Most external storage devices attach to servers via high speed parallel SCSI or Fibre Channel cabling. Traditionally, the speed difference between SCSI and Fibre Channel compared to LANs has been significant, although today's high-speed networking interconnects, such as Gigabit Ethernet, level the playing field. See Section 2.6.3, "SAN GUIDANCE: Comparing Ethernet/IP and Fibre Channel Fabrics," for a more complete overview of storage and networking interconnects.
However, even DAS has its limits. Adding additional storage usually requires incremental servers, and one server does not have direct access to another server's storage. This captive storage model makes resource sharing difficult and leaves servers susceptible to single points of failure. The basic DAS model is shown in Figure 2-6.
Figure 2-6. Direct-attached storage.
Reaching the limits of the captive storage model, solutions with additional scalability arose that networked storage together. This development took shape in the mid-1980s, well before the introduction of high-speed serial storage technologies such as Fibre Channel and iSCSI. Therefore, without a means to serialize the high-speed parallel SCSI interconnects between servers and DAS devices, the only other available network was the Ethernet and IP-based LAN. This led to the introduction of NAS.
Network-Attached Storage
NAS devices overcame several barriers when introduced to the storage market. First, NAS broke the DAS captive model. Second, it provided a mechanism to link storage across Ethernet and IP networks, a completely new footprint for storage administrators. Most importantly, and in part to support the architectural change, NAS moved the file system and volume management functions from the server to a NAS device, often called a filer. Keep in mind that the "filing" function remains no matter what the storage model. The location of this function has traditionally separated the distinction of NAS and SAN. See the following section for a more complete outline of the differences between NAS and SAN.
2.5.1 STORAGE GUIDANCE: SAN and NAS
A discussion about accessing storage via IP and Ethernet, such as with IP storage adapters, invariably begs the question about the difference between SAN and NAS. The SNIA shared storage model outlines file-level and block-level distinctions, which are further clarified in Figure 2-7.
Figure 2-7. Comparing SAN and NAS.
SANs provide scalable performance with direct access to storage devices. Since the storage system resides on the distributed hosts, this architecture works well for databases and online transaction processing.
NAS provides crossplatform scalability, but not the performance scalability due to its use of a high level of file abstraction between applications and storage. While a NAS filer offloads some server workload required to map files into physical storage, it adds significant workload to the filer itself. NAS delivers a central storage system (file system) that is ideal for file sharing, Web serving, and similar applications.
A NAS device is essentially a highly optimized file-delivery server. Depending on the specific implementation, NAS devices may have internal storage (similar to a traditional server), or they may be directly connected to a RAID device for additional capacity or to a tape library for backup functions. Additionally, some NAS devices can connect directly to a block-based storage network.
Traditional servers access NAS devices through the Ethernet and IP network by means of a file request. Since the file system resides on the NAS device, servers with different operating systems can all easily access the same data. This crossplatform access gives NAS a considerable advantage in terms of simplicity. It also allows the NAS device to provide lock functions for particular files. Since the device knows if one server is accessing a file, the second server will not be able to write or modify that file.
NAS has made tremendous inroads to certain market segments where the architecture suits the applications. Web-related functions, such as serving HTTP requests, are ideal applications. The relatively small file size combined with the crossplatform access across Ethernet and IP networks makes NAS a winner.
Similarly, corporate file sharing is easily addressed by NAS. To add more storage capacity to such a configuration, a storage administrator would simply need to plug another NAS device into the IP network. Of course, if for the new NAS device to be completely integrated with other NAS devices in a network—for example, where the NAS devices could pool storage capacity—purchasing the unit from the same vendor becomes a requirement.
Figure 2-8 outlines a basic NAS architecture. As shown, application servers access NAS filers over the IP network, providing greater scalability and crossplatform access than DAS provides. However, NAS doesn't alleviate LAN congestion, and the expansion of block-oriented storage remains disruptive. Additionally, each filer owns the storage that it maintains (except in some homogeneous, pooled NAS environments), and in general, NAS has a unique OS that must be maintained across all filers for ease of management. Finally, the high level of file abstraction inherent to NAS design doesn't suit certain application requirements accustomed to block-level storage.
Figure 2-8. Network-attached storage.
The gap between high-performance DAS and flexible NAS became the sweet spot for SANs. Traditionally, block-oriented SANs solved a combination of problems between DAS and NAS with Fibre Channel networks. More recently, SANs have been able to fill this gap with block-oriented IP SANs, using IP storage protocols such as iSCSI. The following sections cover the details of both Fibre Channel and IP SAN infrastructure.
Subscribe to:
Posts (Atom)