While the software that we develop integrates at system level in devices, we do keep an eye open for developments happening at the application level primarily because the heavy data storage use most modern applications make. Understanding what kind of applications are making headway in the market helps us determine the type of data profile to expect at the file system and flash management level. This helps is in creating new system level functionality that benefits all applications. One example of this is the work done in Reliance Nitro. We observed a lot of device applications and how they use files. We found that apart from some data logging / music playback apps, most apps do not indulge in sequential reads and writes that often. Most of the time, apps are primarily engaged in file operations (file open being the most common). Hence we designed the tree-based metadata architecture in Reliance Nitro to provide radical performance improvements in the file operation speeds.

When we talk about apps on devices, we cannot ignore the 800-pound gorilla – the Apple Appstore. The appstore has created a huge ecosystem for apps on the iPhone and the iPod touch by having a one-stop shop for discovering, reviewing and purchasing apps. Combine this usability with the popularity of the hardware, the app ecosystem has blossomed which has forced rivals to pay attention and start something similar of their own. Google’s Android marketplace is probably the next biggest player while Palm, Microsoft and Nokia are just getting started. While Apple does indeed have a head start of 12-14 months, what is interesting to note is the huge difference between it and its closest rival. This article by Fierce Developer highlights the significant difference in revenue that app developers are seeing on the Apple v/s the Android platform.

Source: AdMob’s July 2009 Mobile Metrics Report

What do these stats mean for device developers? In order to catch up or surpass Apple, innovation needs to happen on multiple fronts

1. Device hardware: Apple innovated with the capacitive touchscreen and no-keyboard design. OEMs need to find the next game changer on the hardware front

2. Device user experience: Hardware on its own does not win over users. There have been several iPhone competitors who have had better hardware of paper but without matching software capabilities, the devices have not met with spectacular success. Using an optimized data storage stack can help improve device responsiveness which users rank as one of their top requirements. Creating new simple user interface paradigms will also be critical.

3. Developer support: Use of standard tools and languages for writing apps along with a low cost of entry is critical to attract app developers to your platform

4. Discovery and purchase of apps: How do users of your devices find and purchase apps? How can you do it better and faster? What is the average price-point for the buyer profile of your device. All of these are critical questions to answer before creating an app store.

It will need strong execution on all of the above fronts for device manufacturers to catch up to the Apple success. We will be keeping an eye open to developments here; look out for further postings on this topic in the future.

Attendance at FMS broke records for the fourth year in a row – truly remarkable, given our current economic situation. I think even the organizers were pleasantly surprised by the confirmation that FMS is now the venue of choice for flash industry leaders to come together and for the rest of the industry to learn what is new in the world of flash technology. As a sponsor of the show since its inception, Datalight contributed to the Summit by organizing a half-day executive update, presenting on wear-leveling, moderating a round-table on endurance at the beer & pizza forum, and delivering a product demonstration at the FMS Theater.
Once again SSDs, thought by many to be the savior of the beleaguered flash industry, dominated the agenda. This year talk of application-specific SSDs was everywhere, specifically optimizations for interfaces like SAS, SATA, and fiber channel.

Presumably, the idea behind application-specific SSDs is to drive faster adoption by tuning them for the use cases presented by different applications. For example, in a random write intensive application, additional RAM may be added to mitigate lackluster random write performance by buffering writes.
As expected, there was a fair amount of preaching to the choir about the benefits of SSDs over HDDs, including read/write performance and the power savings. Intel touted 17% longer battery life and reduced failure rates (less than .5% for SSD vs. 5% for HDD). Other manufacturer’s boasted instant-on boot time, smaller footprint, and lower DRAM requirements.

On the topic of barriers to adoption, SSDs were declared still too expensive for mass market appeal. Also, many of the benefits promised by SSDs are being undone by bottlenecks in the OS (and file system!). As a result, it was postulated that SSDs will cannibalize 15k RPM HDDs, where customers are willing to pay a premium for speed, rather than replacing HDDs as a whole, at least in the short term,
In the emerging technologies track, 3-D NAND was promoted by both Micron and SanDisk, in part because they are able to leverage the technology used to build DRAM for manufacturing 3-D NAND, making it easier and cheaper to produce. It was particularly interesting to hear SanDisk say that 4 bit per cell will have limited application in the future, especially since it had been speculated that the 4 b/c technology was a key reason for them to acquire M-Systems. PCM and FeRAM were also discussed as next generation flash technologies. Overall, we were impressed by the level of participation from across the industry and we’re already looking forward to next year’s Summit!

Denali’s Lane Mason has a very good article on NAND flash pricing. Strongly recommended reading.

http://www.denali.com/wordpress/index.php/dmr/2009/07/16/nand-forward-prices-rate-of-decline-will

It’s been over 20 years since NAND flash was supposed to take over the world, so why is the ‘ultra-portable’ laptop I’m writing this on still using HDD? And why, nearly 30 years after the invention of flash, are we still debating its applications vs. the rotating platter? Given the performance advantages of flash and the mechanical shortcomings of HDD – things like wear and susceptibility to shock and vibration, not to mention the latency associated with spinning up for reads – it’s no wonder that flash has long been the darling of memory technology futurists. So why after all this time has flash not replaced older technologies? I’ll give you a hint: It’s all about the money. But it’s not only about the money – check out our list of five top barriers to flash adoption:

1. Cost – Despite years of oversupply in the flash market, and the corresponding reductions in price, flash is still relatively expensive when compared to HDD, especially on a $/bit basis. To make matters worse, the current economic climate has taken its toll on the flash industry, spurring several rounds of consolidation and requiring flash vendors to curb manufacturing costs by shrinking portfolios and closing fabs. Predictably, these changes in the supply landscape are causing prices rise in many cases, making the cost factor an even bigger problem for flash.

2. Shrinking lithography = lower endurance – One way for flash manufacturers to remain competitive is to use smaller die size to reduce raw material costs. Just a couple years ago, the vast majority of NAND flash was manufactured with 90nm lithography. Most vendors are now planning to move to 30nm technology either this year or next. An unfortunate side-effect of smaller lithography is significantly decreased endurance. SLC NAND, which had 100K + erase cycles, is now predicted to be in the 50-70k range. The biggest impact is on MLC NAND where the endurance has gone from 10k erase cycles to around 3k (a 70% reduction!).

3. Increasing ECC – Another side-effect of shrinking lithography is an increase in error rates for flash, requiring stronger correction codes. Most SLC NAND flash today requires 1-bit correction. That number is predicted to increase to 4-bit on 30nm NOR parts. And the ECC outlook for MLC NAND is even worse, requiring ECCs greater than 12-bit (compared to 4-bit or 8-bit today). These increased ECC requirements mean the controller design for managing flash will become more complicated, and more difficult for OEMs to implement. Performance will also be impacted, especially if the ECC is done in software running on the host processor.

4. Vendor volatility – Churn or volatility in the flash market, the products of a difficult economic climate, are making it difficult for OEMs to find a reliable source of flash parts. Examples are everywhere; A major flash supplier is currently under Chapter 11. There are merger talks happening between SanDisk and Samsung. Asian vendors have been hit especially hard, particularly those also in the DRAM business. OEMs are rightfully concerned about interruptions to their production cycles in the midst of all this turmoil.

5. Lack of killer application – While NAND flash densities have continued to increase, the industry is still waiting for the killer application to gobble up these immense quantities of flash. For long SSDs have been viewed as that application but they have not taken off as fast as the flash industry would have liked.

In spite of the obstacles faced by the industry, flash remains a strong and growing choice for data storage and has put breakthrough devices like MP3 players and smart phones (iPhone!) into the hands of millions of consumers. Early adopters of SSD technology in laptop computers, netbooks and enterprise applications are making a solid case for mass market potential there, which should significantly drive flash adoption in the next few years. Visit the FlashFX Tera page to learn how Datalight is making flash easier and more competitive.

We have talked about managed NAND in a few blog posts before. Usually a combination of raw NAND flash (SLC or MLC) combined with a hardware controller that performs flash management features like bad block management, ECC and wear leveling is referred to as managed NAND. The term covers a huge spectrum of flash-based storage devices so in this post we will try and highlight some of the more prevalent types of managed NAND

The following is an enumeration of some of the popular managed NAND form factors. Please note that the list covers flash technologies used for resident storage and does not cover removable storage like USB flash, SD, etc.

•    eMMC
•    eSD
•    CompactFlash
•    Solid State Drives
•    BA NAND
•    Adaptable NAND
•    Specialized
–    Specially designed controller + raw flash

CompactFlash is included here because it is used both as resident and removable storage. CF comes with a Fixed-drive option which allows it to be used a resident managed NAND.

The above technologies differ from each other on several attributes

•    Form factor – managed NAND can come is several form factors. An SSD may sport a standard 2.5” drive enclosure whereas a CF card will take a 1.0” card form factor.
•    Plug-in interface: What interface does the managed NAND use to connect to the device platform
–    MMC
–    SD
–    ATA
–    Custom
•    Cost: Cost depends on several elements
–    Type of flash used: SLC is much more expensive than MLC
–    Type of controller used: consumer grade controllers (used for consumer grade CF for example) are much cheaper than specialized industrial grade controllers
•    Performance
–    Performance varies depending on the flash type, the controller attributes and the interface.

Some of the big players in the managed NAND business are

•    eMMC
–    Micron, Numonyx
•    eSD
–    SanDisk, Toshiba
•    BA NAND
–    Toshiba
•    Solid State Drives, CompactFlash
–    Too many players in these markets

This was a brief view of the managed NAND landscape. If there is interest, we will do a follow up going in details about the specific categories and interfaces

Once again confirming our suspicion that he doesn’t sleep, Datalight Director of Engineering Ken Whitaker has just published his second book on managing the software development process, this time with a focus on incorporating the ‘agile’ project management techniques that have been so successful at Datalight and elsewhere in Ken’s long distinguished career. At over 400 pages, we haven’t read it yet, but it’s bound to become a favorite of technology management gurus around the world. Here’s some praise from one of Ken’s colleagues:

“Whitaker explains how to run development as the critical business function that it is. Get Principals of Software Development Leadership if you want to lead a technical team to success – or buy a copy for your boss if you want to work on a successful team.” – Steve Johnson, VP, Pragmatic Marketing

Sea Change Hits Consumer Electronics as Customers Demand Long-term Value

For the first time in more than a decade, people are saving again. In 2007 and years prior, the savings rate hovered around zero as we maxed our credit cards and lines of credit, driving the savings rate into the red and giving the world’s manufacturing base an almost unbelievable boom. In January 2009 though, something unexpected happened; the US savings rate suddenly moved above 5%, the highest in decades. As news of our cloudy economic picture has emerged, consumer behavior is shifting away from status-seeking luxury purchases toward more value-based buying patterns, forcing manufacturers around the world to take notice. And after decades of excess, the shift to thrift is looking like a lasting trend.

But what does this mean for Embedded? As consumers focus on needs over wants, they will increasingly seek out products that are proven durable and reliable.

This will have broad implications for manufacturers of everything from cars to clothing, refrigerators to embedded devices. Today’s consumers are choosing efficiency, durability and value over gee-whiz gadgetry. Consumer mobile OEMs too must focus on delivering value and fewer, more targeted features. Rather than packing devices full of a laundry list of apps and expensive hardware, this means streamlined offerings and more segmented products, while making sure the consumer doesn’t feel like they’re missing out. Motorola’s new EM330 is a prime example of this kind of pared-down, demographic-specific approach. The phone, called the MOTOROKR STAR is marketed specifically toward music lovers, offering a basic clamshell with music recognition software and download-on-the-go at a price point in the sub-$200 range.

As OEMs scramble to add value and enhance their reputations for durability and reliability, Datalight responds with products that support those goals. The combination of flexible flash management that lowers bill of material costs, wear-leveling algorithms extend flash life by several times, and the rock-solid reliability of our file system become essential components of a strategy to provide value to customers.

Many have remarked that markets are driven by a combination of fear and greed. Though the pendulum has recently taken a dramatic –and we believe temporary– move in the direction of fear, ultimately we know a move away from excess is good for all of us and good for the world we live in. Here’s hoping the trend toward value and quality is a long-lasting one.

OEMs Position Themselves for the Economic Turnaround

As everyone’s mother used to say, “When life hands you lemons…” And at this particular time in embedded technology and elsewhere, it seems there is no shortage of them.  In just the last three weeks, we’ve heard about Nortel filing for bankruptcy, Motorola planning to cut 7,000 jobs, and Sony Ericsson’s dramatic profitability swing from $1.48 billion in 2007 to negative $96 million in 2008. And yet, one thing we know about recessions is that they inevitably end in expansion. The only question is when. With the fresh winds of optimism blowing in from Washington this week, now is as good a time as any to start planning your strategy for survival followed by world domination.

Indeed, if there is a silver lining to our current situation, it has to be the unprecedented opportunities for well-positioned, forward thinking companies to dominate their markets. Clearly, this takes grit, an eye on keeping costs down, and a truly differentiated product strategy. It’s more important now than ever before for OEMs to focus on their core competencies, keep bill of materials costs low and work with vendors who have experience weathering economic storms and possess the fortitude to survive tough times.

Datalight was born in 1983 during one of the worst recessions in the post-war era, and we remember well the challenges of the early 90’s and the aftermath of 9/11/01. While many of our competitors are pulling resources from flash file system development, we are moving ahead with a full schedule of product releases and innovation for 2009. While at CES this year, we discovered one of our competitors is pulling out of the flash file system business altogether. I won’t bore you with a long-winded sales pitch, because chances are if you’re reading this newsletter you already know Datalight can help you build better products with lower development costs and give you the ability to choose between the least expensive of over 300 NAND and NOR flash parts.

Like our new president, we are cautiously optimistic about the coming year. As President Obama said so well, this economy is going to take hard work and a little time to turn around. One thing we know for sure though, is that the turnaround is coming. Companies that design products their customers need and want, continue to invest in technology and communication, and prepare themselves to capitalize on a business environment with fewer competitors will do extremely well in the coming years.

5 Reasons for Ditching Managed NAND
Everyone knows that NAND has challenges: from factory bad blocks and spontaneous bit failures to endurance limits, etc. That’s why a few years ago managed NAND (NAND flash plus an integrated controller) seemed to be the answer, offering the density of raw NAND, while mitigating many of its inherent limitations. What many device manufacturers may not realize is that the management hardware comes with significant costs, both in terms of dollars per part as well as design limitations. In the world of tradeoffs in which every system designer lives, there are solid technical reasons to consider using raw NAND and leaving the management to software instead. While there are clear commercial advantages for Datalight (as a provider of vendor-neutral software-based flash management) to advocate this approach, we also believe that there are strong technical reasons that flash silicon vendors would do well to embed ECC capabilities into their NAND flash devices rather than relying on ‘total hardware’ solutions such as eMMC or other complex and costly controllers. Beyond the benefits outline below, this approach would allow the flash manufacturers the ability to continue to differentiate their products from others in the industry.

1.    Cost
The cost of managed NAND parts is coming down, but the stuff still sells at a premium over its raw NAND brethren. 

2.    Flash Optimization
There are many new features of NAND available to us today.  Performance features such as cached reads, multi-plane operations, concurrency, and others are becoming invaluable to keep performance at the ever-increasing demands of portable media. The Open NAND Flash Interface (ONFI) has defined a standard method to query the capabilities and characteristics of NAND flash which can be put to use by both software and hardware systems. A software media manager offers the flexibility to take advantage of the most current flash memory features and put them to use efficiently, or to avoid certain features that may be unproven or problematic. A software solution will allow a developer to take full advantage of the media’s characteristics and features unburdened by the indirection or inability for the hardware to expose them.

3.    Visibility/Flexibility
Software (in general) is easily inspected and validated. Features such as wear-leveling move data around the flash device to optimize its life expectancy. Without the ability to inspect source code, a managed NAND solution makes it difficult to validate wear-leveling operation and/or characterize its effect on performance and reliability.  Hardware implementations are often generalized to suit a majority of use cases, while a software solution is easily tailored to the specific use case during development.

4.    Performance
Speaking of use cases, there are many system features that are not available to hardware that may make a generalized hardware solution less advantageous to a specific use case.  For example, system idle time can be used to improve the media performance by scheduling background cache operations and compaction to occur then.  Coordination between the file system and flash media manager can further optimize operations by freeing space when it will no longer be needed and having the media manager code cache certain regions of the flash where meta data might be held. Migrating flash management features to hardware removes this ability to coordinate with other components of the software stack, such as file systems.

5.    Reliability
Lest you think we believe that everything is better left to software, consider error detection and correction (EDC). Error rates are increasing substantially as flash manufacturers push the limits of physics.  Errors can be introduced externally by heat or other radiation, during writes or reads of data, and even to data that was successfully written at a different time. Historically SLC NAND flash required only a single bit error detection and correction (a hamming code is usually sufficient), while MLC parts require minimally four bit EDC.  As the die sizes continue to shrink, error rates will continue to increase, even for SLC flash. 
Calculating the codes to detect and correct such errors is getting increasingly complex and solving such a solution in software for higher-bit EDCs (above 4-bit) is time consuming and often unacceptably slow.  Hardware ECC is a necessary requirement for systems with high EDC requirements and where performance is a concern.

Many of the processors on the market today are incorporating EDC in their NAND controllers. Choosing one of these processors (e.g. TI OMAP 35xx) in combination with raw NAND flash and software management can give you the high-performance EDC to handle next generation flash while maintaining the design flexibility that a software manager provides.

Flash manufacturers have much to gain by adding ECC code into their NAND flash parts.  They know better than anyone what kind of ECC is necessary for a specific part and by adding just that one piece of hardware to their offerings, rather than the jack of all trades, master of none approach of complete flash management, they will better serve the markets. 

In short, features should reside where they can be handled most efficiently; ECCs belong in hardware, other flash management functions belong in software.  While managed NAND certainly has its place and its appeal in the market, we believe the best combination of value, performance and flexibility lies in using a combination of raw NAND and hardware with built-in ECC capabilities.