White PaperHow To Cost-Effectively OptimizeYour Investment with Serial-Attached SCSISAS AdvantagesSAS ExpanderComparisonCase StudyCopyright 2007 Advanced Industrial Computer, Inc (AIC). All Rights Reserved. Advanced Industrial Computer andAdvanced Industrial Computer logo are trademarks used under license by Advanced Industrial Computer, Inc.
How To Cost-Effectively OptimizeYour Investment with Serial-Attached SCSISince the emergence of Serial-Attached SCSI two years ago, many high-end storage solution providersand OEMs have recognized the advantages of SAS over its parallel predecessor. Vertical markets thathave since adopted SAS technology are entertainment industries, medical, financial institutions, andresearch & government institutions, which typically need performance and/or capacity. SAS can offergood value, performance, and scalability.SAS AdvantagesIn general, everyone recognizes the following advantages SAS can offer: Superior aggregated performance over 4-lane wide port -- this is like offering 12 Gb/s SAS Dual-ported design with fail-over capabilities Support both SAS and SATA HDD in one single enclosure to avoid redundant qualificationand certification efforts, as well as the ability to offer tiered storage Available edge and fan-out expander technologies allowing for easier expansion andunmatched scalability Longer interconnect distance allowed compared to SATA for low-cost expansionenclosure solutionsAlthough overall market share is still relatively small comparing SATA, parallel SCSI, and evenFiber-Channel, the majority of the next generation solutions currently being designed by the OEMs arebased on SAS technology. Lack of full understanding on each of these advantages is being blamed forthe slower-than-desired market growth rate. Among these, the advantages of what SAS Expandertechnology can offer is the least understood thus misleads many people when making decisions on whichstorage technology to choose.A SAS Expander is basically designed to allow the Maximization of Available Bandwidth Utilization.In other words, it maximizes the value on investment.Since SAS does not use a parallel bus to attach multiple devices to the same data path, the only way to“share” the bandwidth would be to design a “bridge device” to connect multiple devices to the availablebandwidth. This bridge device is a SAS expander.
SAS ExpanderParallelSerialDRAWING BSerialExpanderDRAWING ADRAWING CFigure 1As we have seen, Figure 1 (Drawing B) above, a typical 8-port SATA RAID controller can offer atotal bandwidth of 24 Gb/s (3.0b/s x8). However, one can only connect eight HDDs to thiscontroller, even though each HDD can barely fill 20% of the available bandwidth per each port ofSATA. What this means is the average throughput on a SATA 3.0G HDD is no more than 60 MB/sunder the best case scenario of 100% sequential read. This is only 20% of 300MB/s each SATAport is designed to support. It is not a good way to connect more HDDs to each SATA port unlessa SATA II port multiplier is used. Since SATA II port multipliers are not widely supported by manymanufacturers, 80% of the total available bandwidth is wasted. This is not a goodvalue in anyone’s opinion.
Case Study 14 x lanesPCI-E 8XMotherboard8-port SATA RAID4 x lanesPCI-E 8X8-port SATA RAID16x lanes of PCI-E 40Gb/sSATA port isdesigned tosupport only 20%of bandwidth oneach port.4 x lanes4 x lanes16x bay chassis with non-expander based backplaneIn the case where large numbers of low-cost SATA disk drives are desired for applications such asDisk-to-Disk Backup or Virtual Disk Library, say 24, 36, or even 48 drives, one would either have touse a large port-count SATA controller (such as a 24-port one) or multiple smaller port-count ones.Let’s look at a real system configuration for a case study and compare thevalue and SAS can each bring to the table. The diagram above is a verytypical system with 16x SATA hard disks.Figure 2Cost for the total solution can add up quickly as more controllers are needed to support the drivecapacity the system is calling for. Also, when using multiple controllers that do not supportRAID-spanning, a software striping will need to be implemented across multiple RAID arrays tobring all the disks under one volume. This will add more loading on the server board CPU thushampering the overall performance.Case Study 1:An 8-port SATA RAID controller is retailed at around 600. A typical 16-bay SATA or non-expanderbased SAS storage server enclosure is priced at around 1,500. To support 16x SATA HDD, wewill need two 8-port controllers for a total cost of 2,700 for the solution minus the drives andmotherboard. Since SAS controllers now mostly come with 8x ports only, we will stay with 8-portSATA controllers only for comparison purposes. Please note two PCI-E 8x lanes for total of 40Gb/sbandwidth are occupied on the server board to support these 16x hard disks. This means eachHDD is allocated around 250MB/s of bandwidth to use, even though actual throughput on the HDDis not even close to that.
Now let’s look at a SAS expander based solution below:Case Study 24x lanes to External Expansion Enclosure16x lanesMotherboardPCI-E 8X8-port SATA RAID4 x lanes24-port Expander4 x lanes4x uplink 4x uplink 16x device links 24x portsIllustration with 24-portSAS expander.Figure 3Case Study 2:By using a 24-port SAS expander we only need 4x lanes (one cable) from an 8-port RAID controller,and leave the other 4x lanes for future expansion using external enclosures. Although this controlleris using a PCI-E 8x slot, it is using only half of the total bandwidth of 20Gb/s to support these 16x harddisks. This is equivalent to around 1GB/s of data path (60 MB/s per HDD allocated) which is a muchbetter utilization of available bandwidth. Should the actual performance exceed 1 GB/s aggregatingthe performance of 16x hard disks, a second SAS RAID controller with 8x internal ports can be usedinstead to double the bandwidth, provided that the controller supports link aggregation.
How To Optimize Your Investment with Serial-Attached SCSI Cost-EffectivelyComparisonFor the overall cost comparison, a SAS expander based server chassis with a single 24-portexpander is retailed at around 2,200. Together with an 8-port SAS RAID controller priced at around 1,000, we are looking at a total cost of 3,200 without MB and HDD. Although this solution is 500more than using a pure SATA solution, it leaves another precious PCI-E 8x slot open on the MB forother add-on card use. When comparing to a SAS/SATA solution using two SAS RAID controllersand a non-expander based enclosure, the net cost of this solution is actually less expensive ( 3,500versus 3,200). More importantly, the ability to further expand out of the same 8-port controllerexternally and the better utilization of total bandwidth brings much better value to solution.Top View of Storage ServerNon-Expander Based SAS/SATA StorageExpander Based SAS/SATA StorageSAS/SATA RAID ControllerDriveDriveSAS ExpanderSATA RAID ControllerFigure 4One last important advantage of a SAS expander is the ability to provide external expansionsolutions using direct SAS links. Until this was available, users had to use FC-SATA or SCSI-SATAsolutions which are all very expensive, and in most cases they are not daisy-chainable. Fordisk-based archive solutions where throughput is not a concern, the ability to daisy-chain fromenclosure to enclosure using one single SAS RAID controllers, adds even more value to SAS. Theease of manageability across the enclosures is also unmatched by other solutions. However, thereis a typical limitation of around 128 devices each controller can support, but in terms of 16x bayenclosures that is still eight enclosures a single controller can support.The ability to recognize and take advantage of SAS technology combined with available full-featuredSAS enclosures, a storage system architect can minimize the Total Cost of Ownership and maximizethe value on investment.
Cost Analysis and ComparisonRSC-3EC2-2SAS/SATA RAID ControllerCase Study 1 (Figure 2)SAS ExpanderCase Study 2 (Figure 3)SATA only orNon-Expander based SAS/SATA SystemExpander-based SAS/SATA SystemTwo PCI-E 8X slots are occupied.(Only 1/4 of 40Gb/s bandwidth is used, 3/4bandwidth is wasted.)One PCI-E 8X bandwidth (20Gb/s) splits into half(1GB/s) for internal storage, allowing 60 MB/s perHDD bandwidth.Another half of PCI-E 8X bandwidth (10Gb/s) couldbe used for storage expansion to external JBOD.Performance2nd PCI-E 8X slot can be used for other add-oncontroller card. 1,500 - SAS/SATA Non-Expander based Enclosure 1,200 - 2x SATA RAID Controllers-------- 2,700Cost (*) 2,200 - SAS/SATA Expander based Enclosure 1,000 - 1x SAS RAID Controller--------- 3,200OR 1,500 - SATA Enclosure 2,000 - 2x SAS RAID Controllers(MSRP)--------*SATARAIDController 600 3,700(MSRP)*SAS RAID Controller 1,000*SAS RAID Controller 1,000 Simpler cabling enables better enclosure coolingOthers Maximization of available bandwidth utilization Bandwidth-sharing using “Bridge Device” akaSAS ExpanderDisclaimer: *Cost estimation is for reference only. Product specifications and pricing are subject to change without notice.For more information on our sas products, please visit our SAS splash page at http://www.aicipc.com/sas
How To Optimize Your Investment with Serial-Attached SCSI Cost-Effectively For the overall cost comparison, a SAS expander based server chassis with a single 24-port expander is retailed at around 2,200. Together with an 8-port SAS RAID controller priced at around 1,000, we