It’s been over 20 years since NAND flash was supposed to take over the world, so why is the ‘ultra-portable’ laptop I’m writing this on still using HDD? And why, nearly 30 years after the invention of flash, are we still debating its applications vs. the rotating platter? Given the performance advantages of flash and the mechanical shortcomings of HDD – things like wear and susceptibility to shock and vibration, not to mention the latency associated with spinning up for reads – it’s no wonder that flash has long been the darling of memory technology futurists. So why after all this time has flash not replaced older technologies? I’ll give you a hint: It’s all about the money. But it’s not only about the money – check out our list of five top barriers to flash adoption:

1. Cost – Despite years of oversupply in the flash market, and the corresponding reductions in price, flash is still relatively expensive when compared to HDD, especially on a $/bit basis. To make matters worse, the current economic climate has taken its toll on the flash industry, spurring several rounds of consolidation and requiring flash vendors to curb manufacturing costs by shrinking portfolios and closing fabs. Predictably, these changes in the supply landscape are causing prices rise in many cases, making the cost factor an even bigger problem for flash.

2. Shrinking lithography = lower endurance – One way for flash manufacturers to remain competitive is to use smaller die size to reduce raw material costs. Just a couple years ago, the vast majority of NAND flash was manufactured with 90nm lithography. Most vendors are now planning to move to 30nm technology either this year or next. An unfortunate side-effect of smaller lithography is significantly decreased endurance. SLC NAND, which had 100K + erase cycles, is now predicted to be in the 50-70k range. The biggest impact is on MLC NAND where the endurance has gone from 10k erase cycles to around 3k (a 70% reduction!).

3. Increasing ECC – Another side-effect of shrinking lithography is an increase in error rates for flash, requiring stronger correction codes. Most SLC NAND flash today requires 1-bit correction. That number is predicted to increase to 4-bit on 30nm NOR parts. And the ECC outlook for MLC NAND is even worse, requiring ECCs greater than 12-bit (compared to 4-bit or 8-bit today). These increased ECC requirements mean the controller design for managing flash will become more complicated, and more difficult for OEMs to implement. Performance will also be impacted, especially if the ECC is done in software running on the host processor.

4. Vendor volatility – Churn or volatility in the flash market, the products of a difficult economic climate, are making it difficult for OEMs to find a reliable source of flash parts. Examples are everywhere; A major flash supplier is currently under Chapter 11. There are merger talks happening between SanDisk and Samsung. Asian vendors have been hit especially hard, particularly those also in the DRAM business. OEMs are rightfully concerned about interruptions to their production cycles in the midst of all this turmoil.

5. Lack of killer application – While NAND flash densities have continued to increase, the industry is still waiting for the killer application to gobble up these immense quantities of flash. For long SSDs have been viewed as that application but they have not taken off as fast as the flash industry would have liked.

In spite of the obstacles faced by the industry, flash remains a strong and growing choice for data storage and has put breakthrough devices like MP3 players and smart phones (iPhone!) into the hands of millions of consumers. Early adopters of SSD technology in laptop computers, netbooks and enterprise applications are making a solid case for mass market potential there, which should significantly drive flash adoption in the next few years. Visit the FlashFX Tera page to learn how Datalight is making flash easier and more competitive.

Image via Wikipedia

Being in the flash memory management space for 15+ years, a very high number of our customers use our products on flash memory (NAND, NOR, NAND controllers, Fusion flash like Samsung OneNAND, etc). Now FlashFX Pro is designed only for flash memory but Datalight Reliance is a file system that works on all block devices. This includes hard drives, USB flash drives, removable cards like SD, CF, solid state drives (SSD), etc. The advantage Reliance brings to these devices is of reliability against data corruption, fast mount times and fast I/O throughput. It also mandates certain requirements on the physical media to ensure reliability against data corruption. We have had customers use Reliance on hard drives before and I want to share some requirements for Reliance to provide high reliability on rotating media. This post is specific to Linux but the general concepts should be applicable to all OSes.

Reliance is a transactional file system and at each transaction point it flushes all its internal caches and commits the data to disk in atomic operations. Primary requirement for Reliance to function on hard drives is that the hardware and the ATA driver must support the “FLUSH CACHE” command. The Linux IDE disk driver checks bits 12 and 13 of word 83 in the IDENTIFY DEVICE information to determine whether FLUSH CACHE is supported.  These bits are defined by the ATA-6 specification, and are not set in earlier drives.  The IDE disk driver will report whether it has detected this capability in a drive.  This is available in the system log.  A typical message will look like:

Jun  9 09:49:23 billr-qa kernel: [   18.621740] hda: cache flushes supported

Since there are a vast number of hard disks on the market and new ones are constantly being introduced (and old ones discontinued), it is a little difficult for Datalight to qualify all hard drives and recommend a specific one. Generally any disk that conforms to the ATA-6 specification and reports that it supports FLUSH CACHE should work correctly with Reliance.  Reliance reports whether it is able to use flush to ensure correct operation, the system log typically looks like this:

Jun  9 09:52:44 billr-qa kernel: [  240.283463] relfs: block device supports flush.

If this message appears in the log, Reliance should operate correctly when power is interrupted unexpectedly.

Datalight’s power interruption testing has been performed on a Western Digital AC29100D using kernel version 2.6.21.1

If you have any questions on the FLUSH CACHE on an OS other than Linux, please leave a comment.

Adoption by Industrial & Mil-Aero Promises Some Rewards & Major Issues

MLC NAND is experiencing a high rate of adoption and within the consumer electronics sector – MP3 players, digital cameras, smart phones, flash cards and USB drives – it is everywhere you look. However, other embedded segments (industrial, automotive, military, aerospace, etc), are hesitating to take advantage of MLC’s low-cost, high-density attributes. There are good reasons behind the cautious stance; these applications are often mission critical, have a low tolerance for failure, and are expected to perform consistently over a much longer lifespan than their counterparts in the nearly-disposable consumer world. These requirements are in direct conflict with some of MLC’s known shortcomings: shorter lifespan, shorter data retention times, higher error rates, more complex (and consequently slower) error detection and correction.

On the topic of lifespan, traditional single-level NOR parts are typically expected to endure up to 100,000 cycles, which could translate to 20 years of use in a typical embedded application.  Most MLC NAND is rated for 10,000 cycles, rendering these parts unusable in 2 years under the same use case.  While 2 years is a long time for many consumer grade products, it is unacceptably short for the vast majority of industrial products. Similarly, data retention requirements differ.  Traditional flash data retention rates have been 20 years, but recently some flash parts are being introduced with only a 10 or 15 year rating. Applications involving products with life times in the 10 year range need to consider such limitations.

Lower erase cycle endurance is conceptually easy to manage: track high use areas and occasionally swap the data within those areas with a low use area. However, a major difficulty is brewing that involves how errors are introduced and the performance impact of detecting and correcting them.

When writing pages within an erase block, disturb errors may be introduced, causing some number of bits to be flipped in pages other than the one being written to. The time required to read and verify the contents of the entire erase block can cause unacceptable delays, leading programmers to defer the detection until the next read operation, which may occur infrequently. Consequently, bit errors can exist in these “not written to” pages for a long time before they are detected.

And the issues with MLC error rates will worsen, as each new generation of chips pushes the cell size down even further. Future generations of MLC NAND devices beyond the 35nm range may have to distinguish between only a few hundred electrons on each cell. With so few electrons, discerning among the multiple levels of charge in a cell will be a time-consuming, error-prone process.

The somewhat obvious solution is to put in place a process to read and verify areas in the vicinity of writes in an attempt to detect disturb errors earlier.  A solution like this must be carefully balanced with the system performance requirements.

MLC NAND has many compelling reasons for adoption, but until its challenges are successfully dealt with, it will not be broadly accepted by industrial, mil-aero, and automotive device designers as a viable replacement for tried and true technologies of SLC NAND and NOR.

At Datalight, we are focused on easing many of the problems of MLC NAND. For more information on our intelligent flash management solutions, please visit our resources page.