Sea Change Hits Consumer Electronics as Customers Demand Long-term Value

For the first time in more than a decade, people are saving again. In 2007 and years prior, the savings rate hovered around zero as we maxed our credit cards and lines of credit, driving the savings rate into the red and giving the world’s manufacturing base an almost unbelievable boom. In January 2009 though, something unexpected happened; the US savings rate suddenly moved above 5%, the highest in decades. As news of our cloudy economic picture has emerged, consumer behavior is shifting away from status-seeking luxury purchases toward more value-based buying patterns, forcing manufacturers around the world to take notice. And after decades of excess, the shift to thrift is looking like a lasting trend.

But what does this mean for Embedded? As consumers focus on needs over wants, they will increasingly seek out products that are proven durable and reliable.

This will have broad implications for manufacturers of everything from cars to clothing, refrigerators to embedded devices. Today’s consumers are choosing efficiency, durability and value over gee-whiz gadgetry. Consumer mobile OEMs too must focus on delivering value and fewer, more targeted features. Rather than packing devices full of a laundry list of apps and expensive hardware, this means streamlined offerings and more segmented products, while making sure the consumer doesn’t feel like they’re missing out. Motorola’s new EM330 is a prime example of this kind of pared-down, demographic-specific approach. The phone, called the MOTOROKR STAR is marketed specifically toward music lovers, offering a basic clamshell with music recognition software and download-on-the-go at a price point in the sub-$200 range.

As OEMs scramble to add value and enhance their reputations for durability and reliability, Datalight responds with products that support those goals. The combination of flexible flash management that lowers bill of material costs, wear-leveling algorithms extend flash life by several times, and the rock-solid reliability of our file system become essential components of a strategy to provide value to customers.

Many have remarked that markets are driven by a combination of fear and greed. Though the pendulum has recently taken a dramatic –and we believe temporary– move in the direction of fear, ultimately we know a move away from excess is good for all of us and good for the world we live in. Here’s hoping the trend toward value and quality is a long-lasting one.

5 Reasons for Ditching Managed NAND
Everyone knows that NAND has challenges: from factory bad blocks and spontaneous bit failures to endurance limits, etc. That’s why a few years ago managed NAND (NAND flash plus an integrated controller) seemed to be the answer, offering the density of raw NAND, while mitigating many of its inherent limitations. What many device manufacturers may not realize is that the management hardware comes with significant costs, both in terms of dollars per part as well as design limitations. In the world of tradeoffs in which every system designer lives, there are solid technical reasons to consider using raw NAND and leaving the management to software instead. While there are clear commercial advantages for Datalight (as a provider of vendor-neutral software-based flash management) to advocate this approach, we also believe that there are strong technical reasons that flash silicon vendors would do well to embed ECC capabilities into their NAND flash devices rather than relying on ‘total hardware’ solutions such as eMMC or other complex and costly controllers. Beyond the benefits outline below, this approach would allow the flash manufacturers the ability to continue to differentiate their products from others in the industry.

1.    Cost
The cost of managed NAND parts is coming down, but the stuff still sells at a premium over its raw NAND brethren. 

2.    Flash Optimization
There are many new features of NAND available to us today.  Performance features such as cached reads, multi-plane operations, concurrency, and others are becoming invaluable to keep performance at the ever-increasing demands of portable media. The Open NAND Flash Interface (ONFI) has defined a standard method to query the capabilities and characteristics of NAND flash which can be put to use by both software and hardware systems. A software media manager offers the flexibility to take advantage of the most current flash memory features and put them to use efficiently, or to avoid certain features that may be unproven or problematic. A software solution will allow a developer to take full advantage of the media’s characteristics and features unburdened by the indirection or inability for the hardware to expose them.

3.    Visibility/Flexibility
Software (in general) is easily inspected and validated. Features such as wear-leveling move data around the flash device to optimize its life expectancy. Without the ability to inspect source code, a managed NAND solution makes it difficult to validate wear-leveling operation and/or characterize its effect on performance and reliability.  Hardware implementations are often generalized to suit a majority of use cases, while a software solution is easily tailored to the specific use case during development.

4.    Performance
Speaking of use cases, there are many system features that are not available to hardware that may make a generalized hardware solution less advantageous to a specific use case.  For example, system idle time can be used to improve the media performance by scheduling background cache operations and compaction to occur then.  Coordination between the file system and flash media manager can further optimize operations by freeing space when it will no longer be needed and having the media manager code cache certain regions of the flash where meta data might be held. Migrating flash management features to hardware removes this ability to coordinate with other components of the software stack, such as file systems.

5.    Reliability
Lest you think we believe that everything is better left to software, consider error detection and correction (EDC). Error rates are increasing substantially as flash manufacturers push the limits of physics.  Errors can be introduced externally by heat or other radiation, during writes or reads of data, and even to data that was successfully written at a different time. Historically SLC NAND flash required only a single bit error detection and correction (a hamming code is usually sufficient), while MLC parts require minimally four bit EDC.  As the die sizes continue to shrink, error rates will continue to increase, even for SLC flash. 
Calculating the codes to detect and correct such errors is getting increasingly complex and solving such a solution in software for higher-bit EDCs (above 4-bit) is time consuming and often unacceptably slow.  Hardware ECC is a necessary requirement for systems with high EDC requirements and where performance is a concern.

Many of the processors on the market today are incorporating EDC in their NAND controllers. Choosing one of these processors (e.g. TI OMAP 35xx) in combination with raw NAND flash and software management can give you the high-performance EDC to handle next generation flash while maintaining the design flexibility that a software manager provides.

Flash manufacturers have much to gain by adding ECC code into their NAND flash parts.  They know better than anyone what kind of ECC is necessary for a specific part and by adding just that one piece of hardware to their offerings, rather than the jack of all trades, master of none approach of complete flash management, they will better serve the markets. 

In short, features should reside where they can be handled most efficiently; ECCs belong in hardware, other flash management functions belong in software.  While managed NAND certainly has its place and its appeal in the market, we believe the best combination of value, performance and flexibility lies in using a combination of raw NAND and hardware with built-in ECC capabilities.

Datalight FlashFX Pro ships as an evaluation version in all VxWorks distributions since version 6.5. Customers who need support for NAND flash on VxWorks chose FlashFX Pro for abstracting the intricacies of working with flash memory. FlashFX Pro provides Bad Block Management (BBM), Wear Leveling, Garbage collection and background compaction functionalities on VxWorks along with support for 200+ flash parts. VxWorks is one of the top OS amongst Datalight Reliance customers. This is because Reliance provides a 100% reliable file system that provides fast performance.

Given the popularity of VxWorks amongst our customer based, we have prospective VxWorks customers ask us how our products are installed and configured on that platform. This short blog post will try and answer these questions. If you have more questions, please leave a comment and we will get back to you.

Q: How does a customer install Datalight VxWorks products
We ship our products with an image of what would be put on a CD, so customers can extract our installation ZIP files into a temporary directory, and run SETUP.  This will install our product into a directory that they tell us (usually c:\dl\flashfx or c:\dl\reliance) and will add our CDF (WorkBench catalog files) in the appropriate place in the Workbench tree.
Q: How do they configure it?  If it’s a menu structure (like Tornado’s Project) where is the menu, what are the options, and what are their effect?

In Workbench and Tornado, FlashFX Pro shows up in the Catalog right next to the TFFS flash driver, and Reliance shows up right next to dosFS.  Our manuals for VxWorks provide step-by-step guidance on how our products can be configured using the VxWorks development IDEs
Q: What effort is involved in using the Datalight Reliance or FlashFX Pro file system on VxWorks?
Developers use the standard file system APIs of VxWorks (file open, file close, etc) like they would if dosFS or another file system and don’t need to know anything specific about the API of Reliance.  Same with FlashFX Pro – once we’re working in the environment, we’re a “disk”, and the people writing applications don’t need to know any special APIs.

Contrary to popular belief, flash memory does not last forever. Every flash part in existence comes with a finite number of write and erase cycles before the data stored becomes corrupted and the flash part unusable. Most flash file systems on the market today include a basic type of wear leveling, but all wear leveling algorithms are not created equal. Chiefly, wear leveling strategies can be broken into two camps: Dynamic wear leveling monitors high and low-use areas of the flash, and after a certain set point, will swap out high-use erase blocks with low-use erase blocks. Large areas of a disk may be occupied by rarely-changing data, forcing frequent system writes/erases to occur on the remainder of the disk and increasing the wear on those areas. Static wear leveling deals with this by moving static data to higher-use areas of the flash, thereby balancing the load. The idea system would use both kinds of wear leveling. Check out some real-world examples by reading our whitepaper on the topic:

A Short Study on Wear‑Leveling

Over the past fifteen years, flash memory has been widely adopted in mainstream consumer grade products having short lifetimes, often measured in months. In recent years however, flash memory has begun to break into more industrial and commercial grade devices with lifetimes counted in years. There are many unique characteristics of flash memory that have fueled its growth across these varying market segments, such as its ability to retain data without continued power; this benefit, however, comes at a cost of a finite lifetime and endurance. The hardware architecture and software technologies that extend the life of a flash chip are often ill‑considered or, at times, given more worry than necessary. While the limited lifetime of flash memory may or may not be problematic for products that are expected to last ten or more years, flash management software can expand the breadth of available flash parts for your project.

This paper focuses on determining when the limitations of flash memory lifetime become significant and what can be done about them.

Flash Lifetime Metrics

Flash memory lifetimes are described in two primary metrics which are generally touted on the first page of any flash memory manufacturers’ data sheets:

• Data retention
• Endurance cycles

Data retention is often listed at 20 years at a given operating temperature. Increased temperature ranges reduce the data retention period which further decrease as the flash memory is used at or near its specified operating temperatures. It is important to note that data retention is measured from the time data is successfully programmed.

The second metric, endurance cycles, is a measure of the number of write and erase cycles that the flash memory can endure before becoming unreliable. Flash memories are organized into a number of erase blocks or sectors and each must be erased prior to writing data. A typical erase block is 128KB in size, however may range from 512B to 2,048KB or even more. Any given address within an erase block cannot be rewritten without an intervening erase. Erase cycles are cumulative and affect only those erase blocks being cycled. In other words, an error in any erase block is constrained to the data of that block.

Erase cycles range from 1,000 to 1,000,000. While these ranges have an order of magnitude difference, it is the application the flash is placed into that will primarily define the product lifetime.

What is Wear-Leveling?

Wear‑leveling is a process to ensure that an entire flash memory device or an array of devices is used in a uniform fashion in order to extend the overall lifetime of the flash.

For a simplistic example of wear-leveling, let’s look at a data recorder device with the following characteristics:

• Application: The device collects and stores the past 24 hours of field data by simply writing and rewriting the data to the same location on the flash.
• File size of data to be recorded: 128KB
• Erase block size (of the flash): 128KB
• Flash memory endurance: 1,000 cycles

With one spare area, the device is assumed one cycle per day each year:

(1,000 cycles ÷ 365 days) * 1 spare area = 2.74 years

In this example, it would take about 2.74 years to cycle that one erase sector 1,000 times.

For the data recorder device to accommodate the write‑erase rules of flash memory, it would have to complete an erase operation to start writing the next day’s set of data. To make the data recorder more robust – to ensure that it doesn’t lose a whole day’s worth of data – we can set aside a second erase block, and erase the first block only after the second set of data was recorded. The resulting side effect is the introduction of a simple wear-leveling scheme.

With two spare areas, the device is assumed one cycle every two days each year:

(1,000 cycles ÷ 365 days) * 2 spare areas = 5.48 years

With these parameters, the period of time prior to cycling the flash to its lifetime has just been increased to almost 5.5 years!

This simple example shows how distributing a fixed set of writes across more flash sectors can increase the period of time prior to cycling the flash to its specified limits. The following sections describe how to account for the important variables associated with wear-leveling techniques, and determine the expected lifetime of the flash in any application.

Continue: A Short Study on Wear-Leveling

Continuing the conversation started in Flash File Systems and JFFS2 blog posts, this post talks about a YAFFS, another Linux flash file system alternative. YAFFS (Yet Another Flash File System) was designed to solve some of the performance issues suffered by JFFS2 on NAND flash. Later, YAFFS was upgraded (to YAFFS2) to work with modern, high-density NAND flash. Like JFFS2, YAFFS2 is a log-structured flash file system. YAFFS2 is licensed under the GPL for use with Linux; it also can be ported to and licensed for non-GPL environments, if needed

Interesting facts about YAFFS

1. Reliability against data corruption – As a log-structured file system, YAFFS2 is intended to be power-fail safe, though there have been reports of data corruption during the garbage collection process and cases where YAFFS2 has lost directories.

2. Wear Leveling – YAFFS2 only implements dynamic wear leveling. Wear leveling is not performed for static data. This may cause a higher number of blocks to be rendered useless at a faster rate than if both static and dynamic wear-leveling scheme were available. [For more information on static and dynamic wear-leveling, see our whitepaper on the topic at www.Datalight.com/whitepapers].

3. Performance: According to the YAFFS development team, YAFFS2 will perform best on disks that are greater than 64MB, while JFFS2 is still preferred for smaller disks.

For a detailed look at YAFFS, there is a great presentation on YAFFS by Wookey at Embedded Linux Conference 2007.

The read, write and erase timing characteristics of flash hardware specifications are useful for comparing different products, but don’t tell the whole story about what you will get from your real-world devices. When Flash memory is incorporated into a system, the performance of the system depends on a number of factors. One key factor that can reduce the effective performance of flash memory involves the shared bus topology of your system. Optimal flash performance depends on the speed and availability of the bus that connects the flash to the system. Also critical are the manner in which the operating system handles interrupts and whether the flash device is connected to the system’s interrupt architecture.

The published read, write, and erase timing characteristics of flash hardware specifications are useful for comparing different products, but don’t tell the whole story about what you will get from your real-world devices. When Flash memory is incorporated into a system, the performance of the system depends on a number of factors in addition to the capabilities of the flash hardware.

One key factor that can reduce the effective performance of flash memory involves the shared bus topology of your system. Optimal flash performance depends on the speed and availability of the bus that connects the flash to the system. For example, if your flash shares a bus with parts that operate at slower clock speeds, the timing of the accesses to the flash part may be extended to match. On the other hand, your flash part may be competing for bus availability with other demanding high-speed system components.

RAM memory, network interfaces, and LCD screens are demanding components that can compete with flash for bus and CPU bandwidth. The use of certain features of the processor and operating system, such as DMA and caching, can have a similar impact. As more components, peripherals, and device drivers are added to the system, more opportunities arise for the bus to be shared. The proliferation of high performance audio and video features, now common on mobile devices, can further tax a shared bus system on a general purpose chipset. For this reason special-purpose chipsets designed for a specific application, as well as tuning the characteristics of your flash management software to meet your specific needs, will generally enable higher levels of flash performance.

Well designed hardware bus topology can alleviate the issue of shared bus contention, yet other factors may still impact flash memory performance. Even if the flash part has full speed access to the processor’s external bus, the availability of the CPU to service that bus is still a question. Bus arbitration may take CPU cycles away from the flash bus in favor of other system busses or internal accesses. Operating system timer interrupts and other peripheral device driver interrupts can interfere with flash software operations, as can a CPU that is simply overloaded by running complex applications.

Also critical are the manner in which the operating system handles interrupts and whether the flash device is connected to the system’s interrupt architecture. Some flash is connected to processors in such a way that the signal generated by the flash is connected to a GPIO, or not connected at all. This may have little impact on flash performance, but it will limit the ability of the CPU to execute other flash-related software, such as garbage collection, or even unrelated tasks. Additionally, many systems have an explicit or implied interrupt priority that must be considered at the system level. Responsiveness requirements of all interrupt-driven components in the system must be carefully weighed against the desire to maximize flash performance.

An equally significant factor affecting flash performance that might be easily overlooked is the flash management software itself. There is a necessary amount of overhead inherent in running software to manage your flash memory, and there are some complex operations that the software needs to accomplish well in order to optimize flash performance. The software provided by your flash vendor may or may not provide satisfactory performance for your particular application.

While flash memory often appears to the end user like a virtual hard drive, the underlying technology is quite different and presents certain challenges. Flash management software can do more than bad block management and wear leveling, it can increase the effective performance of the flash part by addressing these challenges:

1. Flash performance can be impeded by the need for a slow erase operation before writing new data, but software that intelligently performs background garbage collection during idle time can solve that problem.
2. Fragmented data can degrade performance in applications such as streaming media from NAND memory, but compaction software that de-fragments the data can improve performance in these situations.
3. With some algorithms, throughput is maximized for performance until a percentage of the flash memory is used, at which point performance can degrade. The percentage of the flash that is used before performance suffers can be tuned in some implementations, by allowing the system designer to reserve a specified amount of ‘cushion’ of unused memory.
4. In some solutions, maintenance operations such as garbage collection can preempt high-priority read requests. Implementations that make careful use of multithreading operating systems’ capabilities to manage this issue can reduce read latency by orders of magnitude.

Several factors will affect the performance of flash memory in your real-world system, some of which may be beyond your control. Chipset hardware and system bus topology decisions may have been made already. No matter whether your hardware is specially designed for your application or you are using a general-purpose hardware design, though, the effective performance of your flash memory can be improved through software methods. Datalight FlashFX is a multithreading memory management software solution that enables garbage collection, data compaction, memory cushion, and high priority read interrupts to allow the highest real-world flash performance your hardware configuration can support.

Flash memory has established itself as the technology of choice for device data storage on embedded devices. The advantages it brings in terms of storage capacity, I/O throughput, power consumption and board space savings are significant. In 2007, flash memory was a $7.7 billion industry. Analysts predict a 23% growth of the flash memory market between 2007 and 20111; surpassing the history-making growth of DRAM ten times over.

One of the barriers to flash memory adoption is its perceived complexity of integration into a product design. With the flash memory market branching to multiple product lines beyond traditional NAND and NOR devices, this perception, along with a concern about the reliability of flash, is becoming magnified. Basic flash management software can lessen the complexity of integration, and sophisticated flash software can ensure the optimum lifetime and reliability of a flash device.

The challenges of integrating flash memory are broad, including operations from the seemingly simple – like reading, writing, and overwriting data – to the exceedingly complex – such as bad block management and wear-leveling. When flash memory is not accompanied by an intelligent software manager, the system will suffer from slow reads and writes, data corruption, and a short usable life.

There has been a lot of interpretations for the term “Flash File System”. Some consider it as the combination of flash management software and a block file system. For some it is just the flash management piece. The following diagram shows the different layers involved in managing data on flash memory and the corresponding terminologies for software components

· Flash Driver: Basic software that provides rudimentary read/write access to flash; this software is often acquired from the chip provider, and is usually part-specific.

· Flash Manager: In addition to the functionality of a flash driver, a flash manager also intelligently determines which part is being used, and handles it accordingly – whether it is NAND, NOR, or a fusion of the two (i.e. Samsung OneNAND, or Spansion ORNAND). Bad block management, wear-leveling, garbage collection, and error detection and correction are features that a flash manager provides. A flash manager may also be designed to take advantage of unique performance or technical characteristics a specific part provides. Flash managers are sometimes referred to as FTL (flash translation layers).

· Flash File System: Contains the flash driver and the flash manager aspects, but also incorporates a file system that is designed for use with flash memory. In the way of performance optimizations, a flash file system includes a discard interface which ensures that erased blocks are immediately available for use by both the file system and the flash manager without additional queries to those blocks.

The following diagram shows the flash file system in perspective of an embedded device

Hope this post was useful in understanding the layers of flash management. In the next post in the series, we will look at various flash file systems for one of the most talked-about embedded OS – Linux.