- Configuration and Testing
- Microservice Configuration
- Polling Action
- Compute Paths relative to Directory
- Source Directory
- Destination Directory
- Use Working Directory
- Working Directory
- File name
- Archive File Name
- Action if same file exist
- Extensions of the file to be filtered
- Include Sub Directory
- Sub Directory Regex
- Maintain Sub Directory Order
- Delete After Move/Archive
- Microservice Configuration
- Functional Demonstration
- Useful Tips
FilePoller polls for files with the specified name or specified file pattern to move it to a specified directory. It provides a set of actions that allow moving files between two directories as well as it scans a specified directory, checks for the existence of any new files or modification of a file for polling.
Configuration and Testing
Microservice configuration can be done in the CPS (Configuration Property Sheet). The microservice supports adding multiple configurations with each configuration having the necessary details for polling.
The figure below illustrates the panel with a single configuration (Polling) added.
Figure 1: File Poller Configuration properties
It includes three options to choose from depending upon the type of operation:
Move: If selected, matched files will be moved/copied from the source to the destination directory .
Poll: If selected, the directory will be polled for created/updated/deleted files.
- Archive: If selected, matched files will get archived and archived directory gets placed in the destination folder .
It is prompted after choosing the "Poll" option from the Action property. It includes various options to choose files from the directory to poll.
- Create: This option gives the list of files that are created within a given scheduling interval.
- Update: This option gives the list of files that are updated within a given scheduling interval.
- Delete: This option gives the list of files that are deleted within a given scheduling interval.
- All: This option gives the list of files that are created, update and deleted within a given scheduling interval.
- List: This option gives the list of all the matched files with the File Name specified.
Compute Paths relative to Directory
The path of the directory relative to which the paths of Source Directory and Destination Directory are computed. By default, this points to the FIORANO_HOME directory. If the paths specified for Source and Destination Directories are not absolute, their paths are calculated relative to the directory specified here.
The directory which holds the file(s) to be polled/move/archive has to be specified in the Source Directory. All the files in this directory whose names match the pattern specified for the File Name property will be processed.
An absolute path or a path relative to the directory specified in the Compute Paths relative to Directory can be provided.
The path of the directory where the target file (with the data received on the input port) is created. An absolute path of the destination directory or a path relative to the directory provided in Compute Paths relative to Directory can be specified here.
Use Working Directory
This option is shown only if MOVE action was chosen. An absolute path of the destination directory or a path relative to the directory provided in Compute Paths relative to Directory can be specified here.
Shown only when 'Use Working Directory' is enabled. Working directory is used for processing the files, files are copied/moved to working directory temporarily. Files are not copied/moved to working directory.
The name of the file which needs to be polled in the source directory. This property supports patterns such as *.csv and abc *.*, while multiple patterns are not allowed.
Archive File Name
Name of the archived file. This is included only when the Archive functionality is enabled.
Action if same file exist
If the file with the same name exists in the Destination folder, choose the action to be taken from the below options:
- Overwrite: Overwrites the existing file and hence the old file gets replaced with the new one.
- Rename: Renames the file in the format - "filename + MMddyyHHmmss + counter(optional)" retaining the previously existing files.
Figure 2: Options of Actions in case file name matches
If the Rename option is selected, the following properties will be displayed:
- Append TimeStamp
- Append date-time format
- Append Counter
Figure 3: Append settings
Enabling this appends timestamp in the format specified in the Append date-time format property.
- If enabled, the microservice adds a time stamp whose format is provided through the Timestamp format property
- If disabled, no timestamp is added to the file name
Append date-time format
The format in which the date and time are to be appended to the file name.
A counter added to the destination file name. This is used when the file need not be overwritten.
Extensions of the file to be filtered
Extension of the files for which the transfer of files needs to be restricted. This property accepts a comma-separated list of file extensions. Example: *.zip, *.exe, *.dat.
Include Sub Directory
Files will be searched inside subdirectories present inside the source directory. If this option is disabled, only the matching files present inside source directory will be moved/copied.
Sub Directory Regex
By default, the subdirectory regex pattern is [A-Za-z0-9_-]*, which searches in all the subdirectories for the pattern and includes the subdirectories matching the pattern.
Maintain Sub Directory Order
If the matched files are inside the sub directory, then the subdirectory order will be preserved in the target directory on moving the file. On disabling this property, matched files which are inside the subdirectory get moved/copied to the target directory directly.
Delete After Move/Archive
Enabling this option results in deletion of files from the source directory after moving to the destination directory. If disabled, files get copied to the destination directory.
Enable this property to overcome the polling operation data loss that occurs when Peer/HA server crashes. It creates a file in the $FIORANO_HOME/installer/runtimedata/PeerServers/profile1/FPS/run/components/FilePoller/4.0/tif_<EventProcessName>_<VersionNumber>/<FilePollerMicroserviceName> directory which stores the list of files polled under the directory specified in the CPS.
If the option is not enabled, the polling operation information will be lost at the time of peer or HA server crashes.
Scheduler Service Configuration
Please refer to the respective section in the Common Configurations page.
Figure 4: Scheduler Service Configuration Panel
Moving/Copying a specified set of files in the Source directory to the Destination directory.
Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "MOVE" for the property Action.
Figure 5: Scenario demonstration
Archive a specified set of files in the Source directory to the Destination directory.
Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "ARCHIVE" for the property Action.
Figure 8: CPS configuration when action is set to 'ARCHIVE'.
Polling a set of files in the Source directory based on the Polling Action chosen.
Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "POLL" for the property Action.
Figure 10: CPS configuration when action is set to 'POLL'.
Figure 11: Output message received
- The microservice runs on the peer server and therefore the file paths and directories mentioned in the CPS should be valid on the machine where the peer server is running. If the microservice fails over to another peer.
- Ensure that the machine on which the secondary peer server runs have the same path available.