For the first time in more than a decade, people are saving again. In 2007 and years prior, the savings rate hovered around zero as we maxed our credit cards and lines of credit, driving the savings rate into the red and giving the world’s manufacturing base an almost unbelievable boom. In January 2009 though, something unexpected happened; the US savings rate suddenly moved above 5%, the highest in decades. As news of our cloudy economic picture has emerged, consumer behavior is shifting away from status-seeking luxury purchases toward more value-based buying patterns, forcing manufacturers around the world to take notice. And after decades of excess, the shift to thrift is looking like a lasting trend.

But what does this mean for Embedded? As consumers focus on needs over wants, they will increasingly seek out products that are proven durable and reliable.

This will have broad implications for manufacturers of everything from cars to clothing, refrigerators to embedded devices. Today’s consumers are choosing efficiency, durability and value over gee-whiz gadgetry. Consumer mobile OEMs too must focus on delivering value and fewer, more targeted features. Rather than packing devices full of a laundry list of apps and expensive hardware, this means streamlined offerings and more segmented products, while making sure the consumer doesn’t feel like they’re missing out. Motorola’s new EM330 is a prime example of this kind of pared-down, demographic-specific approach. The phone, called the MOTOROKR STAR is marketed specifically toward music lovers, offering a basic clamshell with music recognition software and download-on-the-go at a price point in the sub-$200 range.

As OEMs scramble to add value and enhance their reputations for durability and reliability, Datalight responds with products that support those goals. The combination of flexible flash management that lowers bill of material costs, wear-leveling algorithms extend flash life by several times, and the rock-solid reliability of our file system become essential components of a strategy to provide value to customers.

Many have remarked that markets are driven by a combination of fear and greed. Though the pendulum has recently taken a dramatic –and we believe temporary– move in the direction of fear, ultimately we know a move away from excess is good for all of us and good for the world we live in. Here’s hoping the trend toward value and quality is a long-lasting one.

Given the popularity of VxWorks amongst our customer based, we have prospective VxWorks customers ask us how our products are installed and configured on that platform. This short blog post will try and answer these questions. If you have more questions, please leave a comment and we will get back to you.

Q: How does a customer install Datalight VxWorks products
We ship our products with an image of what would be put on a CD, so customers can extract our installation ZIP files into a temporary directory, and run SETUP.  This will install our product into a directory that they tell us (usually c:\dl\flashfx or c:\dl\reliance) and will add our CDF (WorkBench catalog files) in the appropriate place in the Workbench tree.
Q: How do they configure it?  If it’s a menu structure (like Tornado’s Project) where is the menu, what are the options, and what are their effect?

In Workbench and Tornado, FlashFX Pro shows up in the Catalog right next to the TFFS flash driver, and Reliance shows up right next to dosFS.  Our manuals for VxWorks provide step-by-step guidance on how our products can be configured using the VxWorks development IDEs
Q: What effort is involved in using the Datalight Reliance or FlashFX Pro file system on VxWorks?
Developers use the standard file system APIs of VxWorks (file open, file close, etc) like they would if dosFS or another file system and don’t need to know anything specific about the API of Reliance.  Same with FlashFX Pro – once we’re working in the environment, we’re a “disk”, and the people writing applications don’t need to know any special APIs.

WindowsForDevices has an interesting article today that shows the results from a recent ABI research study on high-end handset sales. It is interesting to note that the economy has not slowed down sale of higher-end mobile phones and the overall mobile market has been strong as well. This bodes well for flash memory vendors, especially for NOR flash vendors like Spansion and Numonyx since mobile handset industry has been a NOR stronghold (though lately NAND is making heavy inroads – call it the iPhone effect).

Being in the flash memory management space for 15+ years, a very high number of our customers use our products on flash memory (NAND, NOR, NAND controllers, Fusion flash like Samsung OneNAND, etc). Now FlashFX Pro is designed only for flash memory but Datalight Reliance is a file system that works on all block devices. This includes hard drives, USB flash drives, removable cards like SD, CF, solid state drives (SSD), etc. The advantage Reliance brings to these devices is of reliability against data corruption, fast mount times and fast I/O throughput. It also mandates certain requirements on the physical media to ensure reliability against data corruption. We have had customers use Reliance on hard drives before and I want to share some requirements for Reliance to provide high reliability on rotating media. This post is specific to Linux but the general concepts should be applicable to all OSes.

Reliance is a transactional file system and at each transaction point it flushes all its internal caches and commits the data to disk in atomic operations. Primary requirement for Reliance to function on hard drives is that the hardware and the ATA driver must support the “FLUSH CACHE” command. The Linux IDE disk driver checks bits 12 and 13 of word 83 in the IDENTIFY DEVICE information to determine whether FLUSH CACHE is supported.  These bits are defined by the ATA-6 specification, and are not set in earlier drives.  The IDE disk driver will report whether it has detected this capability in a drive.  This is available in the system log.  A typical message will look like:

Jun  9 09:49:23 billr-qa kernel: [   18.621740] hda: cache flushes supported

Since there are a vast number of hard disks on the market and new ones are constantly being introduced (and old ones discontinued), it is a little difficult for Datalight to qualify all hard drives and recommend a specific one. Generally any disk that conforms to the ATA-6 specification and reports that it supports FLUSH CACHE should work correctly with Reliance.  Reliance reports whether it is able to use flush to ensure correct operation, the system log typically looks like this:

Jun  9 09:52:44 billr-qa kernel: [  240.283463] relfs: block device supports flush.

If this message appears in the log, Reliance should operate correctly when power is interrupted unexpectedly.

Datalight’s power interruption testing has been performed on a Western Digital AC29100D using kernel version 2.6.21.1

If you have any questions on the FLUSH CACHE on an OS other than Linux, please leave a comment.

A Short Study on Wear‑Leveling

Over the past fifteen years, flash memory has been widely adopted in mainstream consumer grade products having short lifetimes, often measured in months. In recent years however, flash memory has begun to break into more industrial and commercial grade devices with lifetimes counted in years. There are many unique characteristics of flash memory that have fueled its growth across these varying market segments, such as its ability to retain data without continued power; this benefit, however, comes at a cost of a finite lifetime and endurance. The hardware architecture and software technologies that extend the life of a flash chip are often ill‑considered or, at times, given more worry than necessary. While the limited lifetime of flash memory may or may not be problematic for products that are expected to last ten or more years, flash management software can expand the breadth of available flash parts for your project.

This paper focuses on determining when the limitations of flash memory lifetime become significant and what can be done about them.

Flash Lifetime Metrics

Flash memory lifetimes are described in two primary metrics which are generally touted on the first page of any flash memory manufacturers’ data sheets:

• Data retention
• Endurance cycles

Data retention is often listed at 20 years at a given operating temperature. Increased temperature ranges reduce the data retention period which further decrease as the flash memory is used at or near its specified operating temperatures. It is important to note that data retention is measured from the time data is successfully programmed.

The second metric, endurance cycles, is a measure of the number of write and erase cycles that the flash memory can endure before becoming unreliable. Flash memories are organized into a number of erase blocks or sectors and each must be erased prior to writing data. A typical erase block is 128KB in size, however may range from 512B to 2,048KB or even more. Any given address within an erase block cannot be rewritten without an intervening erase. Erase cycles are cumulative and affect only those erase blocks being cycled. In other words, an error in any erase block is constrained to the data of that block.

Erase cycles range from 1,000 to 1,000,000. While these ranges have an order of magnitude difference, it is the application the flash is placed into that will primarily define the product lifetime.

What is Wear-Leveling?

Wear‑leveling is a process to ensure that an entire flash memory device or an array of devices is used in a uniform fashion in order to extend the overall lifetime of the flash.

For a simplistic example of wear-leveling, let’s look at a data recorder device with the following characteristics:

• Application: The device collects and stores the past 24 hours of field data by simply writing and rewriting the data to the same location on the flash.
• File size of data to be recorded: 128KB
• Erase block size (of the flash): 128KB
• Flash memory endurance: 1,000 cycles

With one spare area, the device is assumed one cycle per day each year:

(1,000 cycles ÷ 365 days) * 1 spare area = 2.74 years

In this example, it would take about 2.74 years to cycle that one erase sector 1,000 times.

For the data recorder device to accommodate the write‑erase rules of flash memory, it would have to complete an erase operation to start writing the next day’s set of data. To make the data recorder more robust – to ensure that it doesn’t lose a whole day’s worth of data – we can set aside a second erase block, and erase the first block only after the second set of data was recorded. The resulting side effect is the introduction of a simple wear-leveling scheme.

With two spare areas, the device is assumed one cycle every two days each year:

(1,000 cycles ÷ 365 days) * 2 spare areas = 5.48 years

With these parameters, the period of time prior to cycling the flash to its lifetime has just been increased to almost 5.5 years!

This simple example shows how distributing a fixed set of writes across more flash sectors can increase the period of time prior to cycling the flash to its specified limits. The following sections describe how to account for the important variables associated with wear-leveling techniques, and determine the expected lifetime of the flash in any application.

Continue: A Short Study on Wear-Leveling

Interesting facts about YAFFS

1. Reliability against data corruption – As a log-structured file system, YAFFS2 is intended to be power-fail safe, though there have been reports of data corruption during the garbage collection process and cases where YAFFS2 has lost directories.

2. Wear Leveling – YAFFS2 only implements dynamic wear leveling. Wear leveling is not performed for static data. This may cause a higher number of blocks to be rendered useless at a faster rate than if both static and dynamic wear-leveling scheme were available. [For more information on static and dynamic wear-leveling, see our whitepaper on the topic at www.Datalight.com/whitepapers].

3. Performance: According to the YAFFS development team, YAFFS2 will perform best on disks that are greater than 64MB, while JFFS2 is still preferred for smaller disks.

For a detailed look at YAFFS, there is a great presentation on YAFFS by Wookey at Embedded Linux Conference 2007.

Read Datalight whitepaper Choosing NAND or NOR Flash Memory: Tradeoffs and Strategies to Learn More

Consumer electronics and embedded software devices are using larger amounts of flash memory for nonvolatile storage than ever before. One important decision in designing such devices is what kind of flash memory to use: NAND or NOR?

NOR flash memory has traditionally been used to store relatively small amounts of executable code for embedded computing devices such as PDAs and cell phones. NOR is well suited to use for code storage because of its reliability, fast read operations, and random access capabilities. Because code can be directly executed in place, NOR is ideal for storing firmware, boot code, operating systems, and other data that changes infrequently.

NAND flash memory has become the preferred format for storing larger quantities of data on devices such as USB Flash drives, digital cameras and MP3 players. Higher density, lower cost, and faster write and erase times, and a longer re-write life expectancy make NAND especially well suited for consumer media applications in which large files of sequential data need to be loaded into memory quickly and replaced with new files repeatedly.

The choice between using NAND and NOR Flash may not be a simple one for the complex embedded devices being developed today. While ever-larger media files are driving increased demand for inexpensive NAND, powerful new operating systems and intricate applications running on fast processors call for the kind of fast-executing code NOR can support. An important example is a smart phone or PDA that combines a tremendous need for storage with a demanding set of application performance requirements. In some cases an optimal design might call for both types of flash memory in the same device.

Whichever type of flash is used in a device, there are certain negative performance characteristics that need to be mitigated. NOR is fast to read current data but markedly slower to erase it and write new data. NAND is fast to erase and write, but slow to read non-sequential data through its serial interface. NAND is also prone to single-bit errors, requiring rigorous algorithms for error detection and correction.

Well-designed software strategies can be very effective in increasing the performance and reliability of Flash hardware. The goals of flash memory management software include:

Avoid loss of data. Perhaps the most important goal in managing flash memory is to assure that no data is ever lost as a result of an interrupted operation or the failure of a memory block.  There are several ways that flash management software can achieve this goal. Rewrite operations, for example, can be managed in such a way that new data is written and verified before the old data is deleted, so that no power loss or other interruption can result in the loss of both old and new data. Bad block management is another important safeguard to prevent data being written to memory blocks that have failed. Software can check for bad blocks shipped from the factory, as is typical with NAND, and avoid writing to those blocks from the beginning. When blocks go bad over time they can be identified and managed so that they are no longer used. Finally, as the end of media life nears, good memory management software can implement a graceful strategy such as placing the entire flash unit in a read-only state, thereby avoiding data loss when the number of block errors exceeds a predefined number.

Improve effective performance. Two ways media management software can improve performance are background compaction and multithreading. Compaction reclaims space by identifying blocks that have obsolete data that can be erased, copying any valid data to a new location, then erasing the blocks to make them available for reuse. Such compaction increases the amount of usable space on the media and improves write performance. Compaction may also help to defragment noncontiguous data for improved performance on read operations. The space recovery is particularly valuable for the more costly NOR memory and the defragmentation benefits the slower-reading NAND. Compaction is best performed in the background during idle time, however, or it can interfere with critical operations and degrade performance. This is where a multithreading system becomes important. By allowing high-priority read requests to interrupt low-priority maintenance operations, a multithreading system can reduce read latency by orders of magnitude compared to a single-thread solution.

Maximize media lifespan. When some blocks of memory contain fixed content, such as binary code, the remaining blocks will experience increased demand for erase and write operations, leading to earlier failure. Wear-leveling algorithms can prevent overuse of memory blocks and prevent a “stalemate” scenario in which a small region of memory becomes locked in a pattern of repeated writing and compaction. Wear leveling software can monitor block usage to identify high-use areas and low-use areas containing static data, then swap the static data into the high use areas. It can also balance write operations across all available blocks by choosing the optimal location for each write operation.

The decision between NAND and NOR memory will ultimately depend on both technical and pricing requirements of the device being built. Whatever type or combination of flash is used, it is prudent to include memory management software to prevent data loss while improving the performance and maximizing the lifespan of the memory.

One of the ways to reduce such risk is to include support for multiple flash parts and use multi-sourcing to source flash parts from 3-4 flash vendors. If you are using an intelligent flash manager like FlashFX Pro , you are already covered since FlashFX pro supports 200+ flash parts from all top flash vendors. For others, it can still be done with some serious effort during planning and design time. Consider this work as an insurance against an event such as today’s.

MLC NAND is experiencing a high rate of adoption and within the consumer electronics sector – MP3 players, digital cameras, smart phones, flash cards and USB drives – it is everywhere you look. However, other embedded segments (industrial, automotive, military, aerospace, etc), are hesitating to take advantage of MLC’s low-cost, high-density attributes. There are good reasons behind the cautious stance; these applications are often mission critical, have a low tolerance for failure, and are expected to perform consistently over a much longer lifespan than their counterparts in the nearly-disposable consumer world. These requirements are in direct conflict with some of MLC’s known shortcomings: shorter lifespan, shorter data retention times, higher error rates, more complex (and consequently slower) error detection and correction.

On the topic of lifespan, traditional single-level NOR parts are typically expected to endure up to 100,000 cycles, which could translate to 20 years of use in a typical embedded application.  Most MLC NAND is rated for 10,000 cycles, rendering these parts unusable in 2 years under the same use case.  While 2 years is a long time for many consumer grade products, it is unacceptably short for the vast majority of industrial products. Similarly, data retention requirements differ.  Traditional flash data retention rates have been 20 years, but recently some flash parts are being introduced with only a 10 or 15 year rating. Applications involving products with life times in the 10 year range need to consider such limitations.

Lower erase cycle endurance is conceptually easy to manage: track high use areas and occasionally swap the data within those areas with a low use area. However, a major difficulty is brewing that involves how errors are introduced and the performance impact of detecting and correcting them.

When writing pages within an erase block, disturb errors may be introduced, causing some number of bits to be flipped in pages other than the one being written to. The time required to read and verify the contents of the entire erase block can cause unacceptable delays, leading programmers to defer the detection until the next read operation, which may occur infrequently. Consequently, bit errors can exist in these “not written to” pages for a long time before they are detected.

And the issues with MLC error rates will worsen, as each new generation of chips pushes the cell size down even further. Future generations of MLC NAND devices beyond the 35nm range may have to distinguish between only a few hundred electrons on each cell. With so few electrons, discerning among the multiple levels of charge in a cell will be a time-consuming, error-prone process.

The somewhat obvious solution is to put in place a process to read and verify areas in the vicinity of writes in an attempt to detect disturb errors earlier.  A solution like this must be carefully balanced with the system performance requirements.

MLC NAND has many compelling reasons for adoption, but until its challenges are successfully dealt with, it will not be broadly accepted by industrial, mil-aero, and automotive device designers as a viable replacement for tried and true technologies of SLC NAND and NOR.

At Datalight, we are focused on easing many of the problems of MLC NAND. For more information on our intelligent flash management solutions, please visit our resources page.

Image via Wikipedia

Linux has been slowly but surely establishing itself as the predominant OS in the embedded industry. ABI research report suggested that 23% of Smartphones will be based on Linux by 2013. High-profile industry support from Android and the LiMo foundation has put the spotlight back on embedded Linux.

In a previous post, we talked about flash memory and the various layers of flash management. In this post, I will talk about JFFS2, the most popular flash file systems available on the Linux platform

JFFS2

The Journaling Flash File System version 2 (JFFS2) is a log-structured file system that was originally designed in 1999. The original JFFS was developed by Axis Communications (and later enhanced by Red Hat) to provide support for NOR flash devices. The current version has been updated to include support for NAND flash. JFFS2 is open-source software, distributed under the terms of GPL license.

JFFS2 Strengths

1. Portability to Development Environments: Included with the Linux kernel since version 2.4.10, JFFS2 has become a de facto standard flash file system for Linux developers. Today, it is included in most commercial Linux distributions (such as MontaVista and Wind River Linux). Because of this wide distribution and use, it has been integrated into many varying environments and is known to be relatively easy to build.

2. Reliability: As changes are made to the file system, a “log” is built; this log provides information about where a file and its associated metadata are located on the flash chip.^[1] As the log is consistently maintained, it will be read back in the event of an unexpected power loss to determine the location of a missing file. Although the log structure provides a level of data reliability, this is accomplished at a cost to performance

3. Support for disk-wide compression: The benefit, or cost, of using compression depends on each specific use case. Compression will be useful in making efficient use of disk space when several text-only or code data (OS, etc.) files are being stored. Media files are already compressed (in *.jpg, *.mpg, or other formats), so the time used to attempt data compression is wasted. It is even possible that media files may take up more space after an unnecessary compression than originally needed. Disk usage and performance for the type and number of files to be maintained must be considered by the device designer in order to determine whether compression will yield a benefit or not

JFFS2 Shortcomings

1. Resource usage: RAM usage by JFFS2 increases in linear proportion to the number of nodes. Hence on large flash volumes, the system resources required by JFFS2 can be very significant

2. System performance: For devices that primarily use the file system for read operations, JFFS2 performance may be acceptable. However, for multi-functional devices whose applications perform a continuous mix of read and write operations, it is likely that the performance of a system using JFFS2 will not pass a rigorous standard. In addition to slow writes, the flash disk mount times of JFFS2 are exceptionally slow and worsen as the amount of data stored increases. Upon start-up after an unexpected power failure, JFFS2 must reconstruct the file system structure from the log. This is a costly operation that requires several seconds – more for volumes that are large or near-capacity. The device will be halted during this check operation, as any data that is stored on the disk will not be ready for use until the start-up completes

Informative links on JFFS2

1. JFFS2 Technical paper [PDF]

2. JFFS2 RAM usage [ppt] – Presentation at Embedded Linux Conf 2007

In the next post in this series, we will look at another popular Linux flash file system – YAFFS.