In today’s global economy the concept of “after-hours” no longer exists. The days of running reports after business hours to analyze data representing market pressures and opportunities is no longer feasible. Capital market and financial services firms must be able to perform real-time analysis of market data in order to anticipate and respond to market swings and gain a competitive edge.
These businesses are demanding the latest technology solutions in order to perform data analysis and modeling at high speeds with low latency, and against all relevant market variables. Technology solutions must also drive operational efficiencies to new levels while helping to meet risk and compliance requirements.
Financial institutions are turning to the performance and low-latency benefits of NAND flash memory to accelerate access to data and mitigate the delays associated with traditional hard drive-based storage. Flash storage solutions deliver the performance required to quickly traverse massive data sets and provide real-time analytics and reporting.
Flash in Enterprise Storage Architectures
The use of flash technology in enterprise server and storage architectures is experiencing tremendous growth, and is forecasted to continue to grow rapidly throughout the next decade. Initially, flash technology was packaged as solid state drives (SSDs) since they utilize the same packaging and interface as hard disk drives (HDDs). This allowed SSDs to be easily integrated into existing storage solutions as a HDD replacement. Without the delay of a spinning disk, SSDs support lower latencies and higher transaction rates. However, before organizations move to SSDs, several cautions should be observed.
First, check with your storage array or RAID card provider to verify that the specific SSD you select will be supported. Some storage arrays simply will not use the SSD unless you purchase them from the array manufacturer.
Second, SSD robustness is a concern. Extensive error correction and limited write life mitigation strategies are required for SSDs to provide enterprise reliability. Even major suppliers will limit their implementations, so read available reviews and check with other colleagues about their experiences.
One symptom frequently mentioned is “write cliff,” which occurs when write performance to flash suddenly slows. Writing data to areas on the flash that were previously written is slower than reading data because of the many maintenance tasks. To mitigate this delay, vendors are reserving some areas on the flash for sustained writes. In general, the larger a reserve size, the better the performance. When a vendor’s specification sheet shows a difference between the “raw” and “usable” capacity, that difference usually indicates the size of the reserve area. To save costs, some SSDs will minimize the reserve area, but this negatively impacts write performance.
Third, the interface used by the SSD can limit performance. When accessing SSDs installed in an external storage array, the type of SAN connection used may also limit SSD performance. Typically, Fibre Channel, iSCSI or SAS are commonly used.
A recent study conducted by Demartek found that when iSCSI is used, the SSD performance can be bottlenecked. As iSCSI must encapsulate transactions within a LAN protocol, an appreciable delay in sending the transaction will occur. Compared to a Fibre Channel or SAS connection to the storage array, iSCSI is much slower. While the iSCSI overhead is largely insignificant when using HDDs, it becomes significant when accessing SSDs.
Technologies such as LSI SandForce flash storage processors are designed to address most of the shortcomings of flash in the enterprise. Extensive error correction increases flash reliability to the levels required by enterprise datacenter customers, and techniques that ensure data is uniformly written on the chip extend the lifetime of flash drives and devices. Offloading computation-intensive flash management tasks such as these onto dedicated chips also helps to ensure the performance of financial applications by not requiring the server to perform these lower-level maintenance tasks.