solarpowerlog trunk
[LOGGER] CVSDataLogger: Logging to CSV Files

Overview

The CSV Data Logger takes some or all data and writes it to a regular comma- sepearated-file.

The data to be logged can be selected by specifying the identifiers or all data.

Solarpowerlog honors the RFC 4180. However, some feature, like the "log all" feature causes that for example the number of columns changes during runtime. (However, patches will be accepted to fix this: For example, on a new feature, one could just reparse the file and add the missing datas.

If some data is unavailable, it will be logged with an empty value.

To increase the use of the logfile, a (ISO 8601)-like timestamp will be inserted as the first column.

Configuration

As every logger, the CSV Logger is configured using the Loggers Section.

Option Type Mandatory Default Value Description
name string yes  

Names the Logger. Used to identify the logger.

type string yes   Selects the LoggerType. To get a CSVLogger, this must be CSVLogger.
datasource string yes   Name of the datasource. Must be either a name of a Inverter (Inverter Section) or a name of another DataFilter/logger.
logfile string yes   Defines the target file for this CSV file. See below for additional information.
rotate bool   false Create a new logfile at midnight. Also see the notes on logfile below.
compact_csv bool   false tries to keep the files compact if the data is not changing. Done by not writing lines with the exact same content.
flush_file_buffer_immediatly bool   true if true, do not cache information but immediately write to disk. If you are "only logging" this might be fine to set to false, if you do some kind of real-time data processing, make this false, as it might take some times for the data to enter the disk. (If unsure, say "true".) One use of this option disabled is if you log to flash memory or if you want to avoid spinning up disks. Be aware that you might lose some data if solarpowerlog crashes or power is lost.
format_timestamp string   "%Y-%m-%d %T" How should the timestamp be rendered? You can use the options as described here: http://www.boost.org/doc/libs/1_37_0/doc/html/date_time/date_time_io.html#date_time.format_flags
However, the default set the date in the ISO 8601 format, for example 2009-12-20 13:34:56.
data2log string or array   all If the string reads "all", everything is logged. If an array is given, the data identified by the array will be logged. See below for details.

logfile: With lofile the filename to log to will be specified. If "rotate" is enabled, this logger will begin a new logfile at midnight . To avoid overwriting the old logfile it will add the current date to the filename, using by default an ISO 8601 format: YYYY-MM-DD. To specify *where* the timestamp should be placed, use "%s". If s is not given, it will be appended *at the end* of the filename. For example

     logfile="Inverter_1_%s.csv"

will create a logfile like "Inverter_1_2009-07-04.csv"

To set the format of the timestamp see the option format_timestamp.

data2log:

To specify which data should be logged, their identifiers have to be listed in a array.

The identifiers supported can be retrieved by either the inverter's documentation, in the documentation of the intermediate datafilters.

One can also use the DumbDumper logger or run the CVS Logger in the "all" mode to obtain a list.

Note:
The identifiers are case sensitive!

Array-Example:

  data2log = [
     "Current Grid Feeding Power",
     "Energy produced today (kWh)",
     "Energy produced this month (kWh)",
     "DC voltage in (V)"
 ];

If you want to use the "log all" feature, just say "all" or do not specify the data to log (as "all" is the default)

Note:
When selecting "all features" and new features are detected at runtime, the CSV-header will be written again with the new data added to a new column. If you want to avoid having ever-changing tables, please configure all the data you want to see in the log file. Data which is not present all the time will then still gets its placeholder in the output file.