Home >

Transactional File System Operations

24. January 2009

UPDATE: The use of TxF and KTM for long running transactions is not good.

Most of us are familiar with Transactions from the ADO.NET. To briefly explain what a transaction is, I can say that in a transaction, all operations occur in all-or-none basis. If an exception occurs and transaction is rolled back, then the system is in exactly the state it was before all those operations.

During our work on BlogSharp, I realized that I need to have transaction support for File System Operations. The scenario was simple, the author tries to create a post, upload some images and files. The user may save the post, or cancels it. Until now, it is easy to deal with those cases, but there is the third case: User may close the browser directly(or because of an exception). Keeping track of file and deleting them when they are no longer used is easy but it is not the way it should be. A better solution is possible with the use of transactions.

I searched if someone else had the same need, and found out that Martijn Boland(from Cuyahoga) gave it a try here using Castle Transactions. He uses a temporary directory to do the file operations, and after commit he copies them back to final location. This is definitely a good way of doing it.

However, I wanted to see if the file system supports transactions natively, and the answer turned out to be yes and no. NTFS of XP doesn’t support transactions while that of Vista(and Win2k8) does. I also learnt that Vista has kernel transactions via Kernal Transaction Manager which can be used against registry, too. Imagine an installer which doesn’t use its own transaction mechanism but KTM. When an error occurs, there will be no left overs since the transaction is not committed. The MSDN Magazine has an article that goes deep in this topic and provides some managed classes around TxF but I don’t like it. I then found Michael Kennedy's implementation here and it was better than the previous one. However, it still doesn’t provide any abstraction which I need for testing purposes. I gave it a go, too and here is the result:

[Fact]
public void Open_write_returns_stream_to_be_written()
{
    string data;
    using (TransactionScope scope = new TransactionScope())
    {
        
        using (var fileStream = fileService.OpenFileForWrite(this.textfile))
        {
            Assert.True(fileStream.CanWrite);
            Assert.False(fileStream.CanRead);
            Assert.True(fileStream.CanWrite);
            fileStream.Write(new byte[] {126}, 0, 1);
        }
        data = GetFileContentTransactional(this.textfile);
        Assert.Equal("~lah", data);
    }
    data = GetFileContentNonTransactional(this.textfile);
    Assert.Equal("blah",data);
}


I change the file content in transaction, and this change is only valid within the transaction. Unless we complete the scope, the change doesn’t take effect.

Please note that this is a managed wrapper with many many simplifications which may lead to improper results. It is by no means a production quality code. I have written many tests for basic cases but not advanced ones which may include file security problems. Moreover, there is no exception handling mechanism taking place to convert native I/O exceptions to human readable ones. There may also be memory leaks somehow(even though i paid special attention to this). Those mean that use it at your own risk. All feedback is welcome.

The native methods for transactional file operations can be found here.
The managed wrapper with FS abstractions can be found here

kick it on DotNetKicks.comShout it

, ,

Comments

1/24/2009 6:46:01 PM #
Trackback from DotNetKicks.com

Transactional File System Operations with C#
1/25/2009 9:06:13 PM #
Transaction support is a great idea, but there is a compatibility issue. Maybe support of XP isn't important; but non-NTFS systems and Windows 2003 compatibilities are vital. I think you should provide an alternative method for non-Vista (and 2008) operating systems.


1/25/2009 9:36:30 PM #
"but there is the third case: User may close the browser directly(or because of an exception)."

I really don't agree with this. The correct solution to this should be draft system. I would curse you, if your transaction system would delete my precious images, documents (which I found in Google after a deep search) because my browser terminated because of an exception. And also i sometimes need immediate restart or shutdown situations. So, if something unexpected happens while i try to post a blog; a draft mechanism should handle it not a razor transaction system.
1/25/2009 9:46:18 PM #
OK, that is acceptable and is a better solution, agreed! For your first comment, one can use Martijn Boland's solution. Interfaces are there, it is a matter of plugging that implementation. Not a big deal, I guess.
1/25/2009 10:03:46 PM #
Trackback from DotNetShoutout

Transactional File System Operations
1/25/2009 10:59:34 PM #
It is also not very good to keep transaction open for a looong while. So I guess I will only use it at when the user presses OK button, and remove temporary stuff(aka draft).
1/26/2009 4:55:55 AM #
Trackback from Transactional File System Operations - DotNetBurner

Transactional File System Operations
1/26/2009 8:29:36 AM #
What you have here is a classic "long running transaction" which is best handled by a "saga". The way this works is that when you do any action in the overall transaction, you also emit a timed undo step. Then later, when they commit the work, you delete the undo steps. Conversely, if they abort, or the timeout happens, you execute the undo steps.

For concert tickets, as an example, you start by asking what seats are available that match your parameters. The system finds available seats, marks them as unavailable and inserts an "put them back to available" action in  the queue. You get a token that says these seats are reserved, which when you actually pay for them will cause the "put them back" action to be deleted. If you don't purchase them before the timeout, the "put them back" is executed.

For you situation, you would allow the file upload (possibly to a temporary location). At that time, insert a "delete that uncommitted file" action into the queue with whatever timeout you like (say 20 minutes). When they finish the posting, just delete the "delete it" action from the queue. Otherwise, you've got a background thread that queries every once in a while for any "due" actions and it picks up the delete.

All of this is covered extremely well in Bernstein and Newcomer's book Principles of Transaction Processing here:
http://snurl.com/transactions
1/26/2009 8:36:33 AM #
Thanks Marc, it is helpful. I'll look into that but i don't want to complicate the stuff yet, so it will take a while for me to take a look at it.
8/31/2010 1:47:18 AM #
Support for the 2008 OS is not vital.
11/25/2010 11:02:43 AM #
thats good article!!
12/11/2010 7:48:18 PM #
We are also very happy to now offer pakistan hosting services, regardless of whether or not you want us to build you a website;
There's no need to go scouring the net for a great hosting service to compliment your new site…
GFsoul does hosting and takes care of all the important details.
Our design, development and hosting services perfectly compliment each other; we are offering customers true "one-stop shopping" experience, and at extremely affordable prices for design, development and hosting for pakistan and worldwide. You won't find customized design and hosting at these amazingly low prices anywhere buy here.
Our hosting services features all of the same appointments that other (better known) hosting services can offer you, but with lots of great tech support, customization and a lower overall cost.
1/4/2011 7:42:15 AM #
Your blog is really excellent. It would be great if you can provide more details about it.
1/10/2011 3:54:38 AM #
yep. there is a compatibility issue, but the transaction support is a great idea!
1/14/2011 3:48:51 AM #
e-Xperts Solutions is an IT service provider offering web design,development,software development, professional graphics designing and network management services to its clients across the globe. Our extensive expertise and client-centric approach ensures maximum client satisfaction. Enjoying a knowledge-rich base, e-Xperts is in a unique position to bring to life innovative solutions to a diverse range of clients throughout the world.
2/23/2011 12:49:32 PM #
Hi, Thanks for sharing such a wonderful piece of information. I must say that while reading your post I found my thoughts in agreement with the topic that you have discussed, which happens very rare.
3/21/2011 10:59:05 PM #
There are always 2 sides of the same coin, but I must say, I am learning a lot from your site. Sometimes others have very limited mindset, that's why they fail to understand the others.
3/21/2011 11:07:38 PM #
Wow the colors are amazing!
3/22/2011 1:32:59 AM #
Wow that was really interesting, altought i would add few more deeper thoughts
3/22/2011 12:08:47 PM #
The Zune concentrates on being a Portable Media Player. Not a web browser. Not a game machine. Maybe in the future it'll do even better in those areas, but for now it's a fantastic way to organize and listen to your music and videos, and is without peer in that regard. The iPod's strengths are its web browsing and apps. If those sound more compelling, perhaps it is your best choice.
3/22/2011 2:27:26 PM #
This is probably one of the best mentions of this topic I've seen in quite a while. It's obvious that your knowledge of the subject is deep and this made for a very interesting read.
3/22/2011 6:11:27 PM #
This article is good but I disagree with some certain points in it. Oh well, I am not a complete expert on the matter.  Thanks for the article anyways.
3/22/2011 6:11:53 PM #
I see there is some very good information in this article. Please continue with the work that you are doing
3/22/2011 7:46:40 PM #
Now this is in fact lots of great information! I really try to continue to keep up with your site's info on our Google Reader.
3/22/2011 8:07:28 PM #
What a great opportunity to finally read something worth reading.  thanks a lot and keep posting articles like this for us to read.
3/22/2011 8:11:47 PM #
I see there is some very good information in this article. Please continue with the work that you are doing
3/23/2011 1:02:35 AM #
Hello. I just became alerted to %BLOGTITLE% through Ask.com, and found that it's truly informative. I am gonna bookmark it at dig. I will appreciate if you continue this in the future. Numerous people will be informed from your info. Cheers! ;)
3/23/2011 4:30:21 AM #
Once again I have enjoyed your most recent post.  Thanks for posting it for your readers.  It is always a pleasure and I plan on returning for each of the new writings.
3/23/2011 6:47:24 AM #
This is getting a bit more subjective, but I much prefer the Zune Marketplace. The interface is colorful, has more flair, and some cool features like 'Mixview' that let you quickly see related albums, songs, or other users related to what you're listening to. Clicking on one of those will center on that item, and another set of "neighbors" will come into view, allowing you to navigate around exploring by similar artists, songs, or users. Speaking of users, the Zune "Social" is also great fun, letting you find others with shared tastes and becoming friends with them. You then can listen to a playlist created based on an amalgamation of what all your friends are listening to, which is also enjoyable. Those concerned with privacy will be relieved to know you can prevent the public from seeing your personal listening habits if you so choose.
3/23/2011 2:17:09 PM #
Hello, I found your blog in a new directory of blogs. I dont know how your blog came up, must have been a typo, Your blog looks good. Have a nice day.
3/23/2011 4:06:43 PM #
Wow! what an idea ! What a concept ! Beautiful .. Amazing … Smile
Comments are closed