A week of SSD, NAND questions and New Technology

It’s hard to believe that it’s been 20 years since the invention of flash memory, but the 20-foot timeline documenting its milestones that was displayed at this year’s Flash Memory Summit offered ample evidence of the progress the technology has made. Attendance at this third Summit once again broke records set by previous shows and the energy of the attendees was high. At more than 1,300 registered attendees, 2008 was at least 25% larger than 2007. Having been a sponsor since the show’s inception, we can confirm that it was even more packed this year. Most of the keynote presentations spilled out of the main hall, and people were stacked up into the hallway trying to hear what was being said inside. The quality of the presentations once again proved worth the trip to Santa Clara, but it was obvious to everyone that the show is quickly outgrowing the Santa Clara Marriott.

The unofficial theme this year appeared to be solid-state drives (SSD), but beyond the SSD buzz, there were many presentations on designing NAND-based products, software optimization of flash, future technologies, and other flash-related topics of interest. Datalight gave four presentations (Flash Interfaces 101), and organized a forum on using flash in embedded as well as a full-day “executive update.”

There were a couple of keynote addresses we found particularly interesting and entertaining:

Dean Klein from Micron gave a speech entitled, “A Closer Look at NAND Flash.” Highlights of Dean’s keynote included a map of NANDs progress through the Gartner Cycle of Hype,

Gartner Cycle of Hype – Source: Gartner Research

in which he asserted that NAND is over the “peak of inflated expectations,” heading down into the “trough of disillusionment.” Klein jokingly referred to hard-drives several times as “rotating rust,” and the address featured an entertaining series of video clips along the lines of Apple’s Mac vs. PC ads, in which Hard Drive was escorted by Flash to a therapy session to talk about his sluggishness, forgetfulness, narcolepsy, and overeating (power consumption).

Eli Harari from SanDisk gave a keynote called “Changing the World: The Flash Memory Revolution.” Eli’s speech was less humorous than Dean’s, but no less interesting to listen to. He showed several timelines describing the evolution of flash applications, and a chart predicting that NAND demand will outstrip supply by 2011, allowing flash vendors to raise prices (!) and finally get a return on their investment in the technology. He also compared the progress of NAND density to Moore’s law, showing that NAND is tracking far ahead of where Moore’s law says it should be. He theorized that the next flash technology will be 3D NAND, and gave a fascinating demonstration of how it’s built and how it works, including photographs of 3D NAND’s unique architecture.

Spansion showed their ecoRAM: basically flash in a DIMM form factor. Cool. And eco-friendly, apparently. This new class of flash promises to reduce the power requirements for large server applications, using an eighth of the energy of DRAM, with better reliability, and read performance fast enough to meet the rapid access requirements of large-scale server installations.

Which highlights another key theme of the Summit: Power. How can flash help reduce global warming? Can SSDs make data centers run more economically? Uh, did I say “data centers?” Yes, surprise! While last year’s Summit saw the invasion of the laptop, this year a significant portion of the sessions addressed opportunities in the Enterprise segment.

But the “Big E” didn’t totally eclipse the “Little e” (embedded). There is still a growing need for low power, high performance flash soldered onto boards and into removable cards for embedded systems. The embedded track had presentations ranging from the basics of flash interfaces and differences between NOR and NAND to complex design methodology and frequency sources for flash memory applications.

Speaking of last year’s Summit, where Hybrid Hard Drives (traditional hard disk drive with flash caching) battled SSDs for attention, whatever happened to the HHDs? The only sign of them we saw was a presentation from Seagate wherein they said there is still work to do, particularly on the software (i.e., Windows Vista).

Our take overall? Industry insiders’ perspectives are essential for long term planning and this show is the place to get them. But all that crystal-ball-gazing can be a bit out of phase with where customers are today. SSDs are interesting and undoubtedly will be a key component in many future designs, but the reality is that migration from NOR-only systems to those that include both NAND and NOR or just NAND continues to be the mainstay for today’s designs. An analyst from IDC cashed a reality check on the SSD hype when he put up a slide showing relative market sizes of flash memory (big), hard disk drives (huge) and SSDs (tiny).

While many flash manufacturers are in an oversupply situation on NAND, others have parts on allocation. The industry as a whole is looking for ways to reduce costs and keep (or get) fabs profitable. This causes lower volume, lower margin product lines to be discontinued, sometimes just as designs using them are about to go to market.

Bottom line? The Flash Memory Summit provides a great opportunity to step outside our day-to-day reality and consider the possibilities promised by emerging technology. Next year’s Summit is sure to be a must-attend event for gaining planning perspective. We hope to see you there!

If you missed the Summit, presentations should be available soon at www.FlashMemorySummit.com. Bill’s presentation on flash interfaces, complete with narration, is available now: Flash Interfaces 101

Demonstrating how system software work in a visual manner is an interesting problem, especially in embedded space. There is no UI or visual effects to WOW the audience. To evaluate the value system software components bring to an embedded design, the customer usually needs to configure our software on his embedded development board. This provides for comprehensive evaluation but can be a significant effort. . We run into this very often at Datalight since our primary products (FlashFX Pro and Reliance) are system level embedded software. One way we have attempted to demonstrate our software is to make a full-functional version of our software run on user’s desktop PC. This allows the customer to run the software without any special hardware in matter of minutes and understand the core working of our software. Once they believe there is value, the can request source code access and try it on the actual embedded hardware.

The first demonstration that we have built is for Datalight Reliance. This demo is available to all users who are registered for MyDatalight account. The video below shows how you can use this software on any Windows PC and understand how Reliance unique transactional design allows for 100% reliability against data corruption and how Dynamic Transaction Point technology allows developers to tune their file system even while the device is running.

Bothell, Wash., – August 12, 2008 – Datalight announced today that it has released new versions of Reliance™ and FlashFX® Pro, with pre-configured support for  MontaVista® Linux®. The new install experience includes simplified integration within MontaVista DevRocket, an Eclipse-based IDE that streamlines common embedded  development tasks. Dropped into DevRocket, Datalight products build as kernel loadable modules that work with a project’s OS image. Sample bootstrap code is also  supplied for developers who need to integrate the products into a boot loader.

“Developers choose MontaVista Linux for faster time-to-market, integration, and stable, fully tested code. We are pleased that Datalight has extended these benefits at  the flash memory file system level, and to provide embedded Linux developers with added performance and reliability,” said Dan Cauchy, Senior Director of Market Development, MontaVista Software.

Other upgrades include a read-only version of Reliance inside the Datalight Loader. This small footprint version permits a bootloader to load an OS image directly from a reliance partition. Devices benefit from risk-free “in-place OS upgrades” enabled by the application-controlled transaction point feature of Reliance. The new versions also feature enhancements in reliability, as well as support for a wide range of new flash parts. FlashFX Pro now supports Spansion NS-P, Samsung FlexOneNAND, Micron 55nm flash parts, and all CFI-compliant NOR parts. The Datalight flash file system solution is comprised of the Reliance file system and FlashFX Pro intelligent flash media manager. Reliance was designed from the ground up for high reliability applications. Dynamic Transaction Point™ technology provides 100% immunity from file corruption, even after unexpected system interruption. Embedded applications can benefit from faster boot times that remain consistent for the life of the product, regardless of disk size.

FlashFX Pro features pre-written support for over 200 flash parts, works with virtually any NAND controller, and features wear leveling, bad block management, and garbage compaction for unrivaled performance. Datalight flash file system products are also available on other operating systems and integrated development environments.

Bothell, Wash., – August 12, 2008 – Datalight announced today that it has released new versions of Reliance™ and FlashFX® Pro with preconfigured support for Wind River Linux. The new install experience includes simplified integration with Wind River Workbench, an award-winning, Eclipse-based development platform. Dropped into Wind River Workbench, Datalight products build as kernel loadable modules that work with a project’s OS image. Sample bootstrap code is also supplied for users needing to integrate the products into a boot loader.

“Developers choose Wind River Linux for our broad range of hardware support and stable code base. Our VxWorks customers have enjoyed the benefits of the Datalight  flash file system platform for years, and we are pleased that Wind River Linux customers now have the option of adding Datalight performance and reliability to their designs,” said Paul Anderson, vice president of Linux product engineering at Wind River.

Other upgrades include a read-only version of Reliance inside the Datalight Loader. This small footprint version permits a boot loader to load an OS image directly from a  reliance partition. Devices benefit from risk-free “in-place OS upgrades” enabled by the application-controlled transaction point feature of Reliance.

The new versions also feature enhancements in reliability as well as support for a wide range of new flash parts. FlashFX Pro now supports Spansion NS-P, Samsung  FlexOneNAND, Micron 55nm flash parts and all CFI-compliant NOR parts.

The Datalight flash file system solution is comprised of the Reliance file system and FlashFX Pro intelligent flash media manager. Reliance was designed from the ground up for high reliability applications. Dynamic Transaction Point technology provides 100 percent immunity from file corruption, even after unexpected system interruption.  embedded applications can benefit from faster boot times that remain consistent for the life of the product regardless of disk size. FlashFX™ Pro features prewritten  support for more than 200 flash parts, works with virtually any NAND controller and features wear leveling, bad block management and garbage compaction for unrivaled performance. Datalight flash file system products are also available on other operating systems including Wind River VxWorks.

Datalight FlashFX Pro ships as an evaluation version in all VxWorks distributions since version 6.5. Customers who need support for NAND flash on VxWorks chose FlashFX Pro for abstracting the intricacies of working with flash memory. FlashFX Pro provides Bad Block Management (BBM), Wear Leveling, Garbage collection and background compaction functionalities on VxWorks along with support for 200+ flash parts. VxWorks is one of the top OS amongst Datalight Reliance customers. This is because Reliance provides a 100% reliable file system that provides fast performance.

Given the popularity of VxWorks amongst our customer based, we have prospective VxWorks customers ask us how our products are installed and configured on that platform. This short blog post will try and answer these questions. If you have more questions, please leave a comment and we will get back to you.

Q: How does a customer install Datalight VxWorks products
We ship our products with an image of what would be put on a CD, so customers can extract our installation ZIP files into a temporary directory, and run SETUP.  This will install our product into a directory that they tell us (usually c:\dl\flashfx or c:\dl\reliance) and will add our CDF (WorkBench catalog files) in the appropriate place in the Workbench tree.
Q: How do they configure it?  If it’s a menu structure (like Tornado’s Project) where is the menu, what are the options, and what are their effect?

In Workbench and Tornado, FlashFX Pro shows up in the Catalog right next to the TFFS flash driver, and Reliance shows up right next to dosFS.  Our manuals for VxWorks provide step-by-step guidance on how our products can be configured using the VxWorks development IDEs
Q: What effort is involved in using the Datalight Reliance or FlashFX Pro file system on VxWorks?
Developers use the standard file system APIs of VxWorks (file open, file close, etc) like they would if dosFS or another file system and don’t need to know anything specific about the API of Reliance.  Same with FlashFX Pro – once we’re working in the environment, we’re a “disk”, and the people writing applications don’t need to know any special APIs.

Image via Wikipedia

WindowsForDevices has an interesting article today that shows the results from a recent ABI research study on high-end handset sales. It is interesting to note that the economy has not slowed down sale of higher-end mobile phones and the overall mobile market has been strong as well. This bodes well for flash memory vendors, especially for NOR flash vendors like Spansion and Numonyx since mobile handset industry has been a NOR stronghold (though lately NAND is making heavy inroads - call it the iPhone effect).

Image via Wikipedia

Being in the flash memory management space for 15+ years, a very high number of our customers use our products on flash memory (NAND, NOR, NAND controllers, Fusion flash like Samsung OneNAND, etc). Now FlashFX Pro is designed only for flash memory but Datalight Reliance is a file system that works on all block devices. This includes hard drives, USB flash drives, removable cards like SD, CF, solid state drives (SSD), etc. The advantage Reliance brings to these devices is of reliability against data corruption, fast mount times and fast I/O throughput. It also mandates certain requirements on the physical media to ensure reliability against data corruption. We have had customers use Reliance on hard drives before and I want to share some requirements for Reliance to provide high reliability on rotating media. This post is specific to Linux but the general concepts should be applicable to all OSes.

Reliance is a transactional file system and at each transaction point it flushes all its internal caches and commits the data to disk in atomic operations. Primary requirement for Reliance to function on hard drives is that the hardware and the ATA driver must support the “FLUSH CACHE” command. The Linux IDE disk driver checks bits 12 and 13 of word 83 in the IDENTIFY DEVICE information to determine whether FLUSH CACHE is supported.  These bits are defined by the ATA-6 specification, and are not set in earlier drives.  The IDE disk driver will report whether it has detected this capability in a drive.  This is available in the system log.  A typical message will look like:

Jun  9 09:49:23 billr-qa kernel: [   18.621740] hda: cache flushes supported

Since there are a vast number of hard disks on the market and new ones are constantly being introduced (and old ones discontinued), it is a little difficult for Datalight to qualify all hard drives and recommend a specific one. Generally any disk that conforms to the ATA-6 specification and reports that it supports FLUSH CACHE should work correctly with Reliance.  Reliance reports whether it is able to use flush to ensure correct operation, the system log typically looks like this:

Jun  9 09:52:44 billr-qa kernel: [  240.283463] relfs: block device supports flush.

If this message appears in the log, Reliance should operate correctly when power is interrupted unexpectedly.

Datalight’s power interruption testing has been performed on a Western Digital AC29100D using kernel version 2.6.21.1

If you have any questions on the FLUSH CACHE on an OS other than Linux, please leave a comment.

Datalight Sponsoring and Presenting August 12-14 in Santa Clara

Every August, flash memory zealots converge on Santa Clara to trade war stories, find out what’s new, and debate the relative merits of a mind-boggling assortment of flash technologies. Now in its third year, the Flash Memory Summit is set to break attendance and sponsorship records yet again, and Datalight will be there waving the banner of fault-tolerance, flexible design and raw performance. Other sponsors include intel, Spansion, Samsung, Micron, Toshiba, Numonyx, Dell and many others.

We hope you can join us at the world’s only conference dedicated entirely to flash memory technology. This year we’re excited to present four tutorial sessions at the show, given by veteran Datalight engineers Keith Garvin and Bill Roman:

View Session Overviews

Image by daslive.blogspot.com via Flickr

Gartner just announced their latest research on mobile phone sales. Here are some of the highlights

1. Q1 2008 worldwide sales – 294.3 Million
2. Growth in emerging markets [Asia/Pacific, Latin America] and slowdown in mature ones [US, Europe, Japan]
3. Nokia maintains market leadership with 39.1% market share
4. Motorola woes continue – share drops from 18.4% to 10.2%
5. LG and Samsung capitalize on Motorola’s challenges.

The entire press release is located here and is very informative. If you are in the mobile software/hardware business, I’d strongly recommend you check it out.

Contrary to popular belief, flash memory does not last forever. Every flash part in existence comes with a finite number of write and erase cycles before the data stored becomes corrupted and the flash part unusable. Most flash file systems on the market today include a basic type of wear leveling, but all wear leveling algorithms are not created equal. Chiefly, wear leveling strategies can be broken into two camps: Dynamic wear leveling monitors high and low-use areas of the flash, and after a certain set point, will swap out high-use erase blocks with low-use erase blocks. Large areas of a disk may be occupied by rarely-changing data, forcing frequent system writes/erases to occur on the remainder of the disk and increasing the wear on those areas. Static wear leveling deals with this by moving static data to higher-use areas of the flash, thereby balancing the load. The idea system would use both kinds of wear leveling. Check out some real-world examples by reading our whitepaper on the topic:

A Short Study on Wear‑Leveling

Over the past fifteen years, flash memory has been widely adopted in mainstream consumer grade products having short lifetimes, often measured in months. In recent years however, flash memory has begun to break into more industrial and commercial grade devices with lifetimes counted in years. There are many unique characteristics of flash memory that have fueled its growth across these varying market segments, such as its ability to retain data without continued power; this benefit, however, comes at a cost of a finite lifetime and endurance. The hardware architecture and software technologies that extend the life of a flash chip are often ill‑considered or, at times, given more worry than necessary. While the limited lifetime of flash memory may or may not be problematic for products that are expected to last ten or more years, flash management software can expand the breadth of available flash parts for your project.

This paper focuses on determining when the limitations of flash memory lifetime become significant and what can be done about them.

Flash Lifetime Metrics

Flash memory lifetimes are described in two primary metrics which are generally touted on the first page of any flash memory manufacturers’ data sheets:

• Data retention
• Endurance cycles

Data retention is often listed at 20 years at a given operating temperature. Increased temperature ranges reduce the data retention period which further decrease as the flash memory is used at or near its specified operating temperatures. It is important to note that data retention is measured from the time data is successfully programmed.

The second metric, endurance cycles, is a measure of the number of write and erase cycles that the flash memory can endure before becoming unreliable. Flash memories are organized into a number of erase blocks or sectors and each must be erased prior to writing data. A typical erase block is 128KB in size, however may range from 512B to 2,048KB or even more. Any given address within an erase block cannot be rewritten without an intervening erase. Erase cycles are cumulative and affect only those erase blocks being cycled. In other words, an error in any erase block is constrained to the data of that block.

Erase cycles range from 1,000 to 1,000,000. While these ranges have an order of magnitude difference, it is the application the flash is placed into that will primarily define the product lifetime.

What is Wear-Leveling?

Wear‑leveling is a process to ensure that an entire flash memory device or an array of devices is used in a uniform fashion in order to extend the overall lifetime of the flash.

For a simplistic example of wear-leveling, let’s look at a data recorder device with the following characteristics:

• Application: The device collects and stores the past 24 hours of field data by simply writing and rewriting the data to the same location on the flash.
• File size of data to be recorded: 128KB
• Erase block size (of the flash): 128KB
• Flash memory endurance: 1,000 cycles

With one spare area, the device is assumed one cycle per day each year:

(1,000 cycles ÷ 365 days) * 1 spare area = 2.74 years

In this example, it would take about 2.74 years to cycle that one erase sector 1,000 times.

For the data recorder device to accommodate the write‑erase rules of flash memory, it would have to complete an erase operation to start writing the next day’s set of data. To make the data recorder more robust – to ensure that it doesn’t lose a whole day’s worth of data – we can set aside a second erase block, and erase the first block only after the second set of data was recorded. The resulting side effect is the introduction of a simple wear-leveling scheme.

With two spare areas, the device is assumed one cycle every two days each year:

(1,000 cycles ÷ 365 days) * 2 spare areas = 5.48 years

With these parameters, the period of time prior to cycling the flash to its lifetime has just been increased to almost 5.5 years!

This simple example shows how distributing a fixed set of writes across more flash sectors can increase the period of time prior to cycling the flash to its specified limits. The following sections describe how to account for the important variables associated with wear-leveling techniques, and determine the expected lifetime of the flash in any application.

Continue: A Short Study on Wear-Leveling