Watch Entire Directory with NIO WatchService

dangxunb

Active Member
10/4/15
140
33
28
24
I've created a WatchDirService class to simplicity the effort with NIO's WatchService. I also added the feature to watch entire classes include subdir. The process of recursively register directory with WatchService is a lightweight process and somewhat interesting me, so I include it into my class.

Usage Example:
PHP:
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardWatchEventKinds;


/**
*
* @author NguyenHaiDang
*/
public class Test {
    public static void main(String[] args) throws IOException, InterruptedException {
        //monitor CREATE and DELETE event
        WatchDirService watchService = new WatchDirService(true, StandardWatchEventKinds.ENTRY_CREATE,
                                                            StandardWatchEventKinds.ENTRY_DELETE);
   
        //register a directory, can call multiple times to register multiple directories
        watchService.registerDir(Paths.get("E:\\"));
   
        //register an action when a new file/dir created
        watchService.setOnCreatedAction(new WatchEventAction() {
            @Override
            public void doAction() {
                //get the Path object of the file which make event happent
                Path path = watchService.getCurrentFilePath();
           
                //print out its absolute path
                System.out.println(path.toAbsolutePath().toString());
            }
        });
   
        //register an action when a new file/dir deleted
        watchService.setOnDeletedAction(new WatchEventAction() {
            @Override
            public void doAction() {
                //get the Path object of the file which make event happent
                Path path = watchService.getCurrentFilePath();
           
                //print out its absolute path
                System.out.println(path.toAbsolutePath().toString());
            }
        });
   
        //start it in a seperate thread
        new Thread(watchService).start();
    }
}

Source code: Gist
PHP:
package com.activestudy.Utility.file.watchservice;

import java.io.IOException;
import java.nio.file.*;
import java.util.*;
import static java.nio.file.StandardWatchEventKinds.*;
import java.nio.file.WatchEvent.Kind;
import java.nio.file.attribute.BasicFileAttributes;

/**
 * A class based on NIO's WatchService to watch entire folder.
 *
 * @author NguyenHaiDang@ActiveStudy
 */
public class WatchDirService implements Runnable {

    private final WatchService watchService;
    private final Map<WatchKey, Path> keyPathMap = new HashMap<>();
    private final WatchEvent.Kind[] eventKinds;
    private Path currentFilePath;
    private WatchEventAction onCreatedAction;
    private WatchEventAction onModifiedAction;
    private WatchEventAction onDeletedAction;
    private final boolean all;

    public WatchDirService(boolean all, WatchEvent.Kind... eventKinds) {
        try {
            this.watchService = FileSystems.getDefault().newWatchService();
        } catch (IOException ex) {
            throw new RuntimeException("Cannot create watch service. Reason: ", ex);
        }
        this.eventKinds = eventKinds;
        this.all = all;
    }

    /**
     * This blocking method continously watch given directories until an
     * exception throw or it have no job to do anymore (ex: folder deleted,
     * permission changed...)
     *
     * @throws IOException
     * @throws InterruptedException
     */
    private void watch() throws IOException, InterruptedException {
        MAIN_LOOP:
        while (true) {
            final WatchKey key = watchService.take();

            INNER_LOOP:
            for (WatchEvent<?> watchEvent : key.pollEvents()) {
                final Kind<?> kind = watchEvent.kind();

                WatchEvent<Path> watchEventPath = (WatchEvent<Path>) watchEvent;
                Path fileName = watchEventPath.context();
                Path parentDir = keyPathMap.get(key);
                Path child = parentDir.resolve(fileName);
                setCurrentFilePath(child);

                if (kind == ENTRY_CREATE) {
                    if (Files.isDirectory(child, LinkOption.NOFOLLOW_LINKS) && all) {
                        registerDir(child);
                    }
                    performAction(onCreatedAction);
                    continue;
                }

                if (kind == ENTRY_MODIFY) {
                    performAction(onModifiedAction);
                    continue;
                }

                if (kind == ENTRY_DELETE) {
                    performAction(onDeletedAction);
                }
            }

            boolean validKey = key.reset();

            if (!validKey) {
                keyPathMap.remove(key);
                if (keyPathMap.isEmpty()) {
                    watchService.close();
                    break MAIN_LOOP;
                }
            }
        }
    }

    private void registerPath(Path path) throws IOException {
        WatchKey key = path.register(watchService, eventKinds);
        keyPathMap.put(key, path);
    }

    /**
     * Register a directory to watch
     *
     * @param dir Path, the Path object represent the directory
     * @throws IOException
     */
    public void registerDir(Path dir) throws IOException {
        if (Files.isDirectory(dir, LinkOption.NOFOLLOW_LINKS)) {
            if (all) {
                Files.walkFileTree(dir, new SimpleFileVisitor<Path>() {
                    @Override
                    public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
                            throws IOException {
                        registerPath(dir);
                        return FileVisitResult.CONTINUE;
                    }

                    public FileVisitResult visitFileFailed(Path file, IOException exc)
                            throws IOException {
                        return FileVisitResult.CONTINUE;
                    }
                });
            } else registerPath(dir);
        }
    }

    private void setCurrentFilePath(Path file) {
        currentFilePath = file;
    }

    /**
     *
     * @return the Path object of the file which is making current event
     */
    public Path getCurrentFilePath() {
        return currentFilePath;
    }

    public void setOnCreatedAction(WatchEventAction onCreatedAction) {
        this.onCreatedAction = onCreatedAction;
    }

    public void setOnModifiedAction(WatchEventAction onModifiedAction) {
        this.onModifiedAction = onModifiedAction;
    }

    public void setOnDeletedAction(WatchEventAction onDeletedAction) {
        this.onDeletedAction = onDeletedAction;
    }

    private void performAction(WatchEventAction ea) {
        ea.doAction();
    }

    @Override
    public void run() {
        try {
            watch();
        } catch (IOException | InterruptedException ex) {
            throw new RuntimeException(ex);
        }
    }
}
The WatchEventAction interface:
PHP:
/**
* Provide an action when a WatchEvent happen
* @author NguyenHaiDang
*/
public interface WatchEventAction {
    void doAction();
}
I don't see any sensible performance problem with this API. As the example above, I register entire E:\ drive with it (79.5GB size, ~2500 file, 1100 folder) and quickly making event in a deep directory. It interact very fast. No overhead at start time.
I appropriate any help to improve this class design and effeciency. If forum's admins accept this, I hope it can join congdongjava's library. Thank you :D
 
Sửa lần cuối:

Joe

Thành viên VIP
21/1/13
2,701
1,246
113
Good work, but it's the Job of JackV or Tranhuyvc. I'm just a guest admin with limited admin-rights. That's good so and suits me.
 

dangxunb

Active Member
10/4/15
140
33
28
24
My main purpose when I upload this is to learn class design and code effeciency :D Add to congdongjava library to help people who want to easily use it is admin's job :)
 

Joe

Thành viên VIP
21/1/13
2,701
1,246
113
My main purpose when I upload this is to learn class design and code effeciency :D Add to congdongjava library to help people who want to easily use it is admin's job :)
You don't seem to read my comment. I said "limited admin-rights" and that meant a lot.
 

dangxunb

Active Member
10/4/15
140
33
28
24
You don't seem to read my comment. I said "limited admin-rights" and that meant a lot.
Joe, I know that, and I don't mean anything, esspecially I don't point to you :( Again, I'm happy here to learn and share my knowledge, mostly from you, and don't really care much about other things :(
 

Nancru

CongDongJava Project Leader
Staff member
9/10/11
1,640
307
83
I don't see any sensible performance problem with this API. As the example above, I register entire E:\ drive with it (79.5GB size, ~2500 file, 1100 folder) and quickly making event in a deep directory. It interact very fast. No overhead at start time.
The performance is good because your IO is low, how about you create an application to read multiple files at the same time(let's say 10 files a time, start with small numbers or your PC will be froze). And see what happens?

On Windows, to monitor IO you can type perfmon in cmd
On Linux, you can type iostats -x 1 in terminal to check it
 
Sửa lần cuối:

dangxunb

Active Member
10/4/15
140
33
28
24
The performance is good because your IO is slow, how about you create an application to read multiple files at the same time(let's say 10 files a time, start with small numbers or your PC will be froze). And see what happens?

On Windows, to monitor IO you can type perfmon in cmd
On Linux, you can type iostats -x 1 in terminal to check it
I think you should read carefully the WatchService API and you will understand why it's fast.
The implementation that observes events from the file system is intended to map directly on to the native file event notification facility where available
My API is just based on the WatchService API and I was said that
The process of recursively register directory with WatchService is a lightweight process
With the test above, when the app first run, IO jump slowly from 0% to 18% in 2.5s (because of the recursive register dir) then the disk IO come back to 0% when it's monitoring. I think it's an acceptable performance with entire drive E:\. I posted it here to improve it. Please share with me. Thank you :)
 
Sửa lần cuối:

Nancru

CongDongJava Project Leader
Staff member
9/10/11
1,640
307
83
Lol, Sorry, I mean your IO is low, not slow. I mean that you should test when your PC's IO is high or very high, or at 100%. The best way to test is Let It Crash.
I sometimes face with the IO problems, one time I feel hopeless when all the optimization I gave in my codes, but the system is still crash.
 
  • Like
Reactions: dangxunb

dangxunb

Active Member
10/4/15
140
33
28
24
Thank you :D Can you give me some simple way to make my PC IO work at 100%? :D
 

Nancru

CongDongJava Project Leader
Staff member
9/10/11
1,640
307
83
Read some files at the same time, you can make an application to read files with multiple threads.
And make endless loop in every thread to keep it always busy.
However I hope that your CPUs are powerful, otherwise it's hard to test it when your number of CPUs is just about two and they are weak. In that case, you are testing the CPU performance.
 

Joe

Thành viên VIP
21/1/13
2,701
1,246
113
Hi all,
The recursive algorithm is RARELY the best way to obtain the best performance, it's merely the best way to solve a problem. That's the principle difference and it's one of the basic POPL -Problem Oriented Programming Language. The best method to solve trickiest problem is rarely be the same to the best way to obtain the most optimal performance. IOs are always the same -regardless of the way how they are read. The reason is simple: Modern CPU processing time is million times faster than one IO access.

@Dangxunb
As I said, only the bosses, JackV and Tranhuyvc, can decide what member's software is "worth" for CongdongJava Library. I abstain from such a decision. OK?
 
Sửa lần cuối: