Recently we ran benchmarks comparing ext3 and ext4 to Reliance Nitro on eMMC and Linux. Here’s what we learned….

IOzone Test – Reliance Nitro compared to ext3/4 on eMMC

Test Environment

Operating System: Linux 2.6.39 with eMMC enhancements
Reference Platform: OMAP 4430 PandaBoard
Memory: Socketed eMMC via SD adapter Linux SanDisk 4GB iNAND with 3 partitions

• 64 MB boot partition
• 512 MB root file system
• 3.4 GB test partition

Test Configurations

Testing was run on three different PandaBoards: two Rev A3 boards and one Rev A2 board in two different configurations that reflect distinctly different operating environments that Linux file systems may experience.
Each test configuration was run against each file system twice on each target board, for a total of six tests per file system/configuration. The six sets of test results for each file system/configuration were then averaged together, graphed and analyzed.

The configurations were as follows:

Configuration 1 – Low RAM, fsync
In this configuration the target system had little available RAM, and the test issued the fsync command to the file system to force data to be written out to the media. This measures the performance of the file system when only small amounts of data can be cached and the data must be written out to the media. The goal of this configuration is to measure the file system overhead.

• Designed to show hardware throughput
• Embedded Device or High Reliability Device Use Case
• Boot Arguments:
  root=/dev/mmcblk0p2 rw rootwait mem=64M console=ttyO2,115200n8 init=/sbin/init earlyprintk
• Test Arguments:
  iozone –Raze -+w 1 –q 16m –g 128m

Configuration 2 – High RAM, no fsync
This test configuration had large amounts of available RAM and did include the fsync command. This measures the performance of the file system when large amounts of data can be cached and the data need not be written to the disk; therefore measuring the file system interaction with the cache.

• Reflects performance through internal caches
• Closer to desktop or data center use case
• Boot arguments:
  root=/dev/mmcblk0p2 rw rootwait mem=463M console=ttyO2,115200n8 init=/sbin/init earlyprintk
• Test Arguments:
  iozone –Raz -+w 1 –q 16m –g 128m

Limitations

• Benchmarks not a normal use case
  • IOzone creates a single file, then deletes it before creating the next
  • Tests were run on freshly formatted disk
• Test runs were single threaded
  • IOzone can test multithreaded
• Reliance Nitro is protecting user data; ext3/ext4 journals only the file metadata

It is important to note that IOzone and the tests run are for benchmarking purposes and do not attempt to measure real world use case performance. IOzone creates a single file, and deletes it before creating the next, and the test was run against a freshly formatted disk. Both test configurations used were single-threaded, while this is the default “automatic” behavior for IOzone, it is unlikely for a real world use case. Target systems have many files that are accessed in different ways, often all at once.

Additionally, IOzone measures performance and in no way compares the data at risk for a given file system. For all tests run, Reliance Nitro is protecting user data and file data via transaction points while ext3 and ext4 are only protecting file metadata via the journal.

Performance Data

Performance data was generated using IOzone v3.397.

The test results are graphed in a manner that allows side-by-side comparison of each file system’s performance in each test case. The test executes the test cases against file sizes that are powers of 2 from 64 KB – 128 MB, and record sizes that are also powers of 2 from 4 KB – 16 MB.

The vertical axis of the graphs shows the performance measure during the test, in units of kilobytes per second (KB/s).

The horizontal axis shows the file sizes, and the data points between the file sizes represent the record sizes.  For example, the first data point (immediately above the 64 KB mark) is for 64 KB files with 4 KB records, the next is for 64 KB files with 8 KB records, and so on.

Configuration 2

In both test configurations, the read performance of all three file systems is largely comparable.

With the sequential read test, all three file systems appear to be operating outside the cache at 32 MB file size and greater which is the point where the files are too large to be cached. Ext3 and ext4 are operating within the cache on 8 MB and 16 MB file sizes. All three file systems are within the cache below 8 MB. A likely reason for Reliance Nitro operating outside the cache at smaller file sizes than ext3 and ext4 is that Reliance Nitro allocates more RAM for private structures therefore starving the cache sooner.

Sequential writes are comparable in Configuration 1, in Configuration 2 ext3 and ext4 perform better due to more optimized integration with cache.

With random writes in Configuration 1, Reliance Nitro outperforms at larger file sizes perhaps due to the extent based design, this could also explain ext4’s advantage over ext3 since it is also extent based. In Configuration 2, ext3 and ext4 perform random writes faster due to better integration with the cache.

IOzone Testing Conclusions

• Read performance of all three file systems is comparable
• Configuration 1: sequential writes are comparable, random writes Reliance Nitro outperforms at larger file sizes
• Configuration 2: ext3 and ext4 perform better on sequential and random writes due to Linux cache integration

Linux has a very cache-centric I/O architecture that, with proper support in the file system, leverages available RAM to avoid doing actual I/O to the storage media.  The media used in this test is capable of reading at about 25 MB/s and writing at about 11 MB/s.  Measured performance in excess of those constraints indicates that some or all of the data is being cached.

In Configuration 1, all three file systems tested perform comparably on reads and sequential writes. For random writes Reliance Nitro outperforms ext3 and ext4 and 16 MB file sized and above likely due to the extent based design of Reliance Nitro.

In Configuration 2, there is ample available RAM to cache the test files and the test is not forcing the data to be written to the disk with fflush/fsync, so the read and write performance of all three file systems consistently exceeds the disk’s capabilities. Ext3 and ext4 perform the same in almost all test cases, but Reliance Nitro performs slower in write cases.  This is because Reliance Nitro integration with the Linux page cache is not optimized, resulting in a less efficient cached performance.

While optimal use of the Linux cache is valuable in terms of performance; cache utilization and its costs and risks must be considered for the individual use case. Using the cache is RAM intensive meaning it can add both BOM cost in the form of extra memory needed as well as additional power consumption cost. Cached data also adds a reliability risk. When data is cached, it is not flushed to the media so it is vulnerable in the event of power loss. Reliance Nitro can dynamically mitigate this risk with transaction points, although further cache optimization is required to boost performance for some use cases.

Last week we attended Storage Visions, a show adjacent to CES that focuses on data storage solutions. This year’s theme was Heavy Storage for thin Clients.
We were honored to be a finalist for the Storage Visions award “New Enabling Consumer Storage Technology,” for the Datalight Reliance Nitro fault-tolerant file system. Although we didn’t get to take home the trophy, we wanted to congratulate our partner, Micron, for winning the award with their Real SSD C400 Self Encrypting Drive.

Learn more about Datalight Embedded File Systems

This article entitled Advances in Nonvolatile Memory Interfaces Keep Pace with the Data Volume, recently published in RTC Magazine, gives a nice overview of maintaining performance on newer technologies.

Learn more about Datalight and ClearNAND

Today we are excited to announce Reliance Nitro 2.5. This release was focused on the requirements of “Converged Mobile Devices” or multi-function mobile device types. To meet the requirements of the market, Reliance Nitro 2.5 continues to be a competitive advantage for any device storing critical data, plus now improves responsiveness, security and control of the device. For more information visit our Reliance Nitro 2.5 landing page or view our press releases:
Datalight Releases File Level Secure Delete on eMMC 4.4x
and
Datalight Improves User Experience with Low-Latency Reliable File System

Read more about our exciting new release

Lately you may have noticed a lot of talk about eMMC on our blog and website. If so you may be thinking, “Why is Datalight so excited about eMMC?” Here are 5 top reasons we’re jumping on the eMMC bandwagon:

#5) Standards: Having a standard specification for hardware like eMMC enables our customers to spend less time integrating new hardware and more time focusing on their core competencies.

#4) ECC: As NAND flash lithography shrinks and density increases, error correction has become exponentially more complex and therefore difficult to do in software. Performing ECC in the hardware, like eMMC (or OnFI EZ-NAND parts like Micron’s ClearNAND), overcomes this problem and offers much simpler integration.

#3) Performance: In combining an accepted standard with on-board ECC, eMMC offers excellent performance and flexibility for customers.

#2) Advanced features like secure delete: With more critical data being stored, data security is becoming a much bigger issue. We’re happy to see a standards body like JEDEC focus on tackling this problem. And since customers will need a file system capable of enabling this feature, we’re delighted to be the first to offer support for eMMC 4.41 features like secure erase and secure trim.

#1) Customers: More and more of our customers are either evaluating or using eMMC. Whenever we see our customers using a new technology we ask ourselves how we can help enable them to take full advantage of it. Exploiting the features of eMMC at the file system level is a win for all of us.

Learn more about Datalight flash management solutions

It’s always gratifying when you run benchmarks and discover your product actually does outperform the competition. Months and months of development effort went in to making Reliance Nitro and FlashFX Tera run flawlessly in an open source environment. We were pretty sure our transactional architecture beat the pants off YAFFS2, JFFS2, and UBIFS, but until you run the final benchmarks, you really don’t know for certain. Recently we ran tests on two platforms, a ConnectCore Wi-i.MX51 (Cortex-A8) and an NVidia Tegra 2 (Cortex ARM9). The Flash part used for all tests was a Samsung 512 MB part. The specific test used was IOZone, with a specified file size sufficient to be larger than the Linux cache, in order to better reflect the raw throughput. The results speak for themselves:

Also see an article weighing the pros and cons of JFFS2

If you’ve noticed the numerous posts lately on the Datalight blog regarding JEDEC and eMMC, you might be wondering why we’re so excited about this particular standard. There are many features that this “smarter” memory will enable for OEMs; In this post I’ll focus on one of those features in the eMMC specification –secure delete.

Securely deleting information on flash memory is more complicated than it seems. For one thing, files are constantly being moved around to ensure even wear of the flash, resulting in multiple copies of file data on the media. Furthermore, when a file is marked for delete, it is typically not physically deleted, rather the space is only marked as available to be overwritten. Until that happens, the “deleted” data is still present and recoverable on the media. In fact, the University of California San Diego Non-volatile systems lab has produced an in depth study of file deletion on flash memory, where they found significant data still present on the media even after deleting the files. A copy of the report can be found at: http://cseweb.ucsd.edu/users/swanson/papers/Fast2011SecErase.pdf

In order to securely delete a file on raw flash, you must use a controller that will either track every block where the file has been stored, or will overwrite the space the file was stored in each time it is moved. The latter describes exactly the secure erase and secure trim features found in the eMMC 4.41 standard. This means that the hardware will finally be capable of securely deleting files –brilliant! There is just one problem: Who has software to support this functionality? As of this writing, there is no file system which supports the feature. While an application can make a call to the media to delete a file securely, the file system may have a backup copy stored somewhere. Fact is, the file system must support the secure delete capabilities of the hardware in order for these features to function correctly.

If an OEM wants to take advantage of the secure erase and secure trim features, their application will need to communicate with the eMMC driver, which may differ from part to part. As the only software company that is an active member of JEDEC, we are excited offer support for quite a few eMMC features. File system support for secure erase and secure trim will be coming later this summer!

Learn more about Datalight Embedded File Systems

The hot topics in the consumer electronics segments today are Android, installable applications, sexy user interfaces, sensors like GPS receivers, gyroscopes and accelerometers and larger capacity/smaller size storage. Ever since I read Roy’s blog post on CES, I’ve been asking myself when we’ll start seeing the impact of trends in consumer electronics on the more general embedded market. Just as buying a car causes you to see that model everywhere, it seems like asking the question has made it so. In the past few weeks we’ve seen a dramatic increase in the number of customers considering a move from traditional RTOS’s to Linux – more specifically, Android. Many are in the “tire kicking” stage and may never make the switch, but the tide has clearly turned in that direction. The appeal seems to be a combination of user interface – which becomes more important as data use, which has also taken a leap skyward on embedded devices, increases – and ease of application development, perhaps even leveraging the burgeoning application marketplaces that have developed around smartphones. Then again, much of what’s happening in the consumer electronics space is old hat in embedded systems: working in resource constrained environments, power efficiency and dealing with ruggedness requirements.

I guess the real migration is that consumer electronics are combining technologies proven in embedded into multifunction devices, smaller packages with slicker form factors to meet the trendy demands of consumers.

Learn more about Datalight for Linux

We’ve received a bit of feedback on our Bootstrapping Linux from NAND Flash with FlashFX Tera and Reliance Nitro whitepaper. Questions that have come up regarding this whitepaper include: “The sample project included in the whitepaper demonstrated booting from NOR rather than NAND, is it possible for the bootloader to reside in NAND?” and “Is booting from NAND reliable?”

I talked with Datalight Software Architect Bill Roman, the author of the whitepaper, to answer these questions…

Question: The sample project included in the whitepaper demonstrated booting from NOR rather than NAND, is it possible for the bootloader to reside in NAND?

Bill:

Yes, you can boot from NAND. The sample for the whitepaper was done on NOR partly so the example would be more simple and thus easier to understand, and because of the functionality of the board that we were working with. The main topic was how to read the Linux kernel from a Reliance Nitro root file system on flash managed by FlashFX Tera. How the bootloader initially gets loaded into RAM is an implementation detail. Some platforms make booting directly from NAND easy, while some can’t do this at all, and thus require some NOR flash.

To boot from NAND there are two stages required. The details of the first-stage loader are very platform-specific. Typically, the processor manufacturer would provide documentation and help to do this –this is why the focus for the whitepaper was on the second stage.

Question: Is booting from NAND reliable?

Bill:

Loading the initial bootloader from NAND can be made quite reliable. Most NAND manufacturers guarantee the first erase block to be entirely error-free for some limited number of erase cycles. This allows an unchanging first-stage loader to be stored there, which then typically finds and loads the second stage loader (the main topic of the white paper).

If you’d like to check out the whitepaper, download here:

“To the Datalight team: I really appreciate the hard work, good technical feedback, and general professionalism. Thank you for your time and effort spent on this issue; for getting to the bottom of it (never doubted you would); and for getting us the patches so quickly. Outstanding support, as always!”

“The overall experience is very positive. Specifically, the sales person and the technical support team are very knowledgeable and helpful.”

“I wanted to express my appreciation for the dedication and hard work of Gary, Tony and the rest of the Datalight team on resolving this important issue.  This is a great example of Datalight’s professionalism, customer support ethic and the value we receive from Datalight.”

“I have been completely satisfied with Datalight. We are working on mission critical flight software for a spacecraft and the insight and support provided by Datalight has been exceptional.”

Read even more comments here