Configuration

Configuration

General Configuration Steps

1. Navigate to the config directory and copy the example configuration file to a new file:

   cd config
   cp config.yml.example config.yml
   cp archive_config.yml.example archive_config.yml

2. Update the configuration files with the necessary details:

   • config.yml: Add the FTP server credentials and meter configuration data.
   • archive_config.yml: Add the source and destination directories and other relevant details.

3. Secure the configuration files so that only the owner can read and write:

   chmod 600 config.yml
   chmod 600 archive_config.yml

Data Pipeline (download) Configuration File

# General settings for the application
application:
  name: "Event Data Collection"
  version: ""  # Version number of the application

# Default download directory (overridden by -d/--download_dir flag if specified)
download_directory: ""

# Configuration settings
location: ""    
data_type: ""  

# bytes/s, 0 means unlimited, Suffixes are supported, e.g. 100K means 102400.
bandwidth_limit: "0"

# Optional, leave empty to download all available
max_age_days: 

# Default FTP credentials (overridden by meter-specific credentials)
credentials:
  username: ""  # Default FTP username
  password: ""  # Default FTP password

# Meter configurations with optional username/password overrides
meters:
  # Meter 1
  - ip: ""       # IP address of the meter
    id: ""       # Unique identifier for the meter
    type: ""     # Type of the meter (e.g., "SEL735")
    credentials:
      username: ""  # Optional FTP username override for this meter
      password: ""  # Optional FTP password override for this meter

Explanation of Fields

• bandwidth_limit:
  • Description: Specifies the maximum download speed.
  • Usage: Set in bytes per second. Use suffixes like K for Kilobytes (e.g., 100K for 102400 bytes).
  • Example: bandwidth_limit: "500K" limits the download speed to 500 KB/s.
• max_age_days:
  • Description: Limits the age of the files to be downloaded.
  • Usage: Specify the maximum age of files to download, in days. If left empty, all available files are downloaded.
  • Example: max_age_days: 30 downloads only files from the last 30 days.
• credentials (default and override):
  • Description: Default specifies the FTP credentials. Override credentials can be provided for specific meters.
  • Usage: Set default FTP credentials under credentials. Use the credentials field under each meter to override the default credentials if needed. These overrides are optional and can be left empty.

Archive Configuration File

Configures file archiving from multiple local sources to their respective remote destinations.

# Hostname or IP address of the destination server
host: ""
credentials:
  user: ""
  ssh_key_path: ""

# Optional bandwidth limit in Kb/s
bandwidth_limit: ""

# Enable or disable cleanup process
enable_cleanup: true

directories:
  - source: ""
    destination: ""
    retention_days:

  - source: ""
    destination: ""
    retention_days: 

Explanation of Fields

• bandwidth_limit:
  • Description: Specifies the maximum transfer rate, specified in units per second.
  • Usage: The value can be suffixed with a string to indicate a size multiplier and may be a fractional value (e.g., --bwlimit=1.5m). If no suffix is specified, the default value is K. A value of 0 or no value specifies no limit.
  • Example: bandwidth_limit: "500" limits the transfer speed to 500 Kb/s.
• retention_days:
  • Description: Specifies the number of days to retain files before they are cleaned up.
  • Usage: Set the retention period for files in each directory.
  • Example: retention_days: 30 retains files for 30 days before they are removed by the cleanup process.