Sunday, August 30, 2009

F# - Asynchronous Directory.GetFiles and File.Copy

The Problem:
My wife recently decided to take on the task of ordering family photo albums. This primarily involved wading through the deluge of digital photos that have been piling up over the last three years, selecting the photos that were worthy to be placed in the album, uploading those photos to a certain web site, organizing the photos in the albums, and finally making the purchase. The particular imaging software, that was used to import/organize/modify the photos, stores the images in a directory structure that looks something like this ..\<Main Directory>\<Year>\<Month>\<Day>. While this would not normally be an issue, it made the task of uploading the images to the photo album creation site quite arduous.

This of course is a simple problem to solve in most programming languages. The following examples show how it could be solved with F# both synchronously and asynchronously. While this is definitely not production ready, it provides a few samples of how asynchronous workflows can be used in the wild.

The Synchronous Way:
(Note: A sleep statement has been added to emphasize the speed difference between the synchronous/asynchronous approaches.)
open System.IO
let destinationDirectory = @"C:\temp\picDest\"
let sourceDirectoryRoot = @"C:\temp\pic"
let searchPattern = @"*.jpg";
let getFileName sourceFile = FileInfo(sourceFile).Name
let getSourceImages = Directory.GetFiles(sourceDirectoryRoot, searchPattern, 
                          SearchOption.AllDirectories)
let getDestinationFileName sourceFile destinationDirectory = 
    destinationDirectory + getFileName sourceFile
let copyImage sourceFile destinationDirectory = 
    File.Copy(sourceFile, getDestinationFileName sourceFile destinationDirectory, true) 
    |> ignore  

do printfn 
    "Starting the image consolidation process with base directory: %s" sourceDirectoryRoot
for image in getSourceImages do
    System.Threading.Thread.Sleep(5000)
    do printfn "[.NET Thread %d] %s" System.Threading.Thread.CurrentThread.ManagedThreadId image
    copyImage image destinationDirectory
do printfn "The images have been consolidated into directory %s." destinationDirectory
do printfn "Press [Enter] close this command prompt." 
System.Console.ReadKey() |> ignore
The Asynchronous Way:
(Note: A sleep statement has been added to emphasize the speed difference between the synchronous/asynchronous approaches.)
open System
open System.IO

type Directory with
    static member AsyncGetFiles(path:string, searchPattern:string, searchOption:SearchOption) = 
        let fn = new Func <string * string * SearchOption, string[]>(Directory.GetFiles)    
        Async.BuildPrimitive((path, searchPattern, searchOption), fn.BeginInvoke, fn.EndInvoke)
type File with
    static member AsyncCopy(sourceFile:string, destinationFile:string, overwrite:bool) = 
        let fn = new Func<string * string * bool, unit>(File.Copy)
        Async.BuildPrimitive((sourceFile, destinationFile, overwrite), fn.BeginInvoke, fn.EndInvoke)   

let destinationDirectory = @"C:\temp\picDest\"
let sourceDirectoryRoot = @"C:\temp\pic"
let searchPattern = @"*.jpg";
let getFileName sourceFile = FileInfo(sourceFile).Name
let getSourceImages imageDirectory searchPattern searchOption = 
    async { return! Directory.AsyncGetFiles(imageDirectory, searchPattern, 
                searchOption) }
let getDestinationFileName sourceFile destinationDirectory = 
    destinationDirectory + getFileName sourceFile
let copyImage sourceFile destinationDirectory overwrite = 
    async {
        System.Threading.Thread.Sleep(5000)
        do printfn 
            "[.NET Thread %d] %s" Threading.Thread.CurrentThread.ManagedThreadId sourceFile
        return! File.AsyncCopy(sourceFile, 
            getDestinationFileName sourceFile destinationDirectory, overwrite) }

do printfn 
    "Starting the image consolidation process with base directory: %s" 
    sourceDirectoryRoot

let sourceImages = 
    getSourceImages sourceDirectoryRoot searchPattern SearchOption.AllDirectories
    |> Async.RunSynchronously

for image in sourceImages do
    copyImage image destinationDirectory true
    |> Async.Start
       
do printfn 
    "The images will be consolidated into the following directory: %s" destinationDirectory 
System.Console.ReadKey() |> ignore