I for one am happy to see people talking about file system features, especially Data Integrity, knowledge of Flash Media, and faster access through B+ trees. Of course, Datalight’s own Reliance Nitro file system has had all this and more for some time now…

Microsoft has a new term for a thing we’ve seen often in the case of unexpected power loss – a “Torn Write”. They point this out as a specific problem for their journalling file system, NTFS, but updating any file system metadata in place can be problematic. It looks to me like this new file system, ReFS, handles this by bundling the metadata writes with other metadata writes or with the file data. If the former, this demonstrates the trade-off between Reliability and Performance that we are very familiar with at Datalight. Bundling smaller writes will help with spinning media and flash. In time we will see how much control the application developer has over this configuration – another important point for our customers.

One of the commenters posted that error correction belongs at the block device layer, and I tend to agree. Microsoft’s design goal “to detect and correct corruption” is a noble one, but how would they detect corruption for user data? Additional file checksums and ECC algorithms would be intrusive and potentially time consuming. Keeping a watch on vital file system structures is important, of course, and a good backup in case block level error detection fails.

I look forward to reading more from Microsoft’s file system team in the future, and especially hope to see a roadmap for when these important changes will make it down to the embedded space.

Learn more about what happens during a power interruption.

Also see an article weighing the pros and cons of JFFS2

Fragmentation is usually caused when modifying a file. Overwriting the file or making it larger means storing a fragment of the file in a new place, unless the file system creates a complete new copy of the file. Databases are particularly susceptible here – they are usually large files and often updated in the middle.

Another way fragmentation happens is when the file system initially stores the file in pieces. This could happen if the file system is not configured to keep file blocks together, or if the media is fairly full and there are no spaces of sufficient size for the new file.

What about the impact of fragmentation? In the days of rotating media, a fragmented file meant extra head movement and platter rotation to read the file. With flash media, the extra overhead is just additional block reads – a far smaller cost.

Avoiding fragmentation if you’re using Reliance Nitro can be as simple as customizing your transaction points. Instead of transacting on a timed basis, create a new transaction point only when the entire file is on the media, at “file close”. Similar settings may be available on other file systems.

If your use case causes fragmentation, a valid workaround might be to reformat the media after backing up the database files. A fresh file format is fairly quick on modern hardware, and can be coupled with a bad block test as well.

Read more about Reliance Nitro

“With the growth in adoption of Linux for data-intensive embedded devices, OEMs need a flash file system that better supports their reliability and performance requirements.” said Roy Sherrill, Datalight CEO. “By supporting the most recent kernel versions of Linux we’re filling that gap in the market with a robust, commercial-grade solution backed by our reputation for responsive, high-quality support.”

FlashFX Tera supports the full range of flash technologies including NAND, NOR, and MLC NAND flash in a single driver. Its patented wear-leveling and bad block management extend the useful life of devices using flash. While FlashFX Tera can be used with virtually any file system, pairing it with Reliance Nitro provides an optimized data storage software stack to simplify system development.

FlashFX Tera 1.2 and Reliance Nitro 1.2 are available immediately from Datalight and the Datalight worldwide network of channel partners. Please visit us at http://www.datalight.com/partners/worldwide-sales-partners to find a reseller near you.

The Reliance family of file systems and FlashFX family of flash media managers comprise the Datalight flash file system solution. Reliance was designed from the ground up for high reliability applications. Dynamic Transaction Point™ technology gives developers full control over performance and data protection characteristics, protecting users from file system corruption, even after unexpected system interruption. Embedded applications can benefit from faster boot times that remain consistent for the life of the product, regardless of disk size. FlashFX™ Tera features pre-written support for over 300 flash parts, works with virtually any NAND controller, and features wear leveling, bad block management, and background compaction for unrivaled performance.

For information, contact:
Kerri McConnell, Director of Marketing
425.686.1069
kerri.mcconnell@datalight.com

“Right now we are in the process of testing the impact of Reliance Nitro in our application. Apparently, we noticed some boost in the performance:  faster write speed, significant speed increase of transaction point creation, faster read speed, and significantly faster directory read (we typically have 1000 files in the directory). So, in conclusion, the overall performance of the system is boosted quite significantly.”

Another real world example of how Reliance Nitro boosts performance in directories with a large number of files. To learn how Reliance Nitro does it, check out the whitepaper.