How to Limit No Of Files in Mule File Connector

File Connector is very important and useful utility provided by MULE ESB . We can use this to monitor a directory , and process the files as it comes in directory . But , if there are hundreds files pumped in the source directory , mule tries to load all of them by creating different threads . To control the number of files , mule can read once , we have to write our own custom File Receiver .

First , change your file connector in xml , to use custom file receiver :

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd 
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd 
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
 <file:connector name="myFileConnector" >
 <service-overrides messageReceiver="InputFileMessageReceiver"/>
 </file:connector>

    <flow name="fileInboundTestFlow1" doc:name="fileInboundTestFlow1">
        <file:inbound-endpoint path="E:/fileTest" responseTimeout="10000" doc:name="File" pollingFrequency="5000" connector-ref="myFileConnector"/>
        <byte-array-to-object-transformer doc:name="Byte Array to Object"/>
    </flow>
</mule>

Use Byte-Array-To-Object-Transformer carefully as it loads the full file in memory , it might create Out Of Memory Error if the file is too large .

And here is our custom file receiver which will allow only two files at a time .

import java.io.File;
import java.io.FileFilter;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import org.mule.api.construct.FlowConstruct;
import org.mule.api.endpoint.InboundEndpoint;
import org.mule.api.lifecycle.CreateException;
import org.mule.api.routing.filter.Filter;
import org.mule.api.transport.Connector;
import org.mule.transport.ConnectException;
import org.mule.transport.file.FileConnector;
import org.mule.transport.file.FileMessageReceiver;

/**
 * Created.
 * User: Abhishek Somani
 * Date: 14/05/2014
 * Time: 2:19 PM
 */
public class InputFileMessageReceiver extends FileMessageReceiver {
    
    private int maxFiles = 2;

    public InputFileMessageReceiver(Connector connector, FlowConstruct flowConstruct, InboundEndpoint endpoint, String readDir, String moveDir, String moveToPattern, long frequency) throws CreateException, ConnectException, IOException {
        super(connector, flowConstruct, endpoint, readDir, moveDir, moveToPattern, frequency);
    }
    
    protected void basicListFiles(File currentDirectory, List<File> discoveredFiles)
    {
        File[] files;
        Filter filter = endpoint.getFilter(); 
        if ( filter instanceof FileFilter)
        {
            files = currentDirectory.listFiles((FileFilter)filter);
        }
        else if(filter instanceof FilenameFilter)
        {
            files = currentDirectory.listFiles((FilenameFilter)filter);
        }
        else
        {
         files = currentDirectory.listFiles();
        }

        // the listFiles calls above may actually return null (check the JDK code).
        if (files == null)
        {
            return;
        }

        for (File file : files)
        {
            if (!file.isDirectory())
            {
                discoveredFiles.add(file);
                if(discoveredFiles.size() >= maxFiles)
                 return ;
            }
            else
            {
                if (((FileConnector)connector).isRecursive())
                {
                    this.basicListFiles(file, discoveredFiles);
                }
            }
        }
    }

}

So this is how we can limit the number of file to be processed by Mule File Connector .


Post Comments And Suggestions !!!