data.hpp File Reference

Go to the source code of this file.

Description

Manage a table with data records, stored persistently as CSV.

In the context of observations, configuration, calibration and QA, a series of measurement data taken over time is often evaluated statistically, to distill typical averages, variances and trends. Short of using a database, a modest amount of numeric data can be maintained in CSV files, which also allows for further manual evaluation within a spreadsheet or statistics application. The CSV format as such can be quite elaborate, yet for the purpose of saving and later reading back some values generated by the application itself, supporting a limited format flavour is sufficient:

• first line is a header line and used to verify the storage format
• one record per line, embedded line breaks prohibited
• fields separated by comma, semicolon tolerated
• fields are trimmed and may be empty
• a field may be double quoted
• only quoted fields may contain whitespace or comma
• no escaping of quotes, i.e. no quotes within quotes

As a fundamental building block, this header provides a data table template with flexible column configuration to hold arbitrary, explicitly typed values. This solution is statically typed and does not carry any runtime type information; the actual data table object is then defined and accessed by means of accessor components for each column of data. A tuple of current values corresponding to the most recent row of data can be accessed directly through these sub-components.

Usage

Create an actual instantiation of the DataTable template, passing a structure with util::Column descriptors. You may then directly access the values of the actual column or save/load from a persistent CSV file.

Note

mandatory to define a method allColumns()

struct Storage
  {
    Column<string> name{"theName"};
    Column<int>    n{"counter"};
    Column<double> x{"X value"};
    Column<double> y{"Y value"};

    auto allColumns(){ return std::tie(name,n,x,y); }
  };

using Dataz = lib::stat::DataTable<Storage>;

Dataz daz("filename.csv");

daz.x = 123e-4;
daz.y = -12345e-6;

std::vector<int>& counters = daz.n.data;

Variations

The standard case is to have a table backed by persistent file storage, which can be initially empty. Under some conditions, especially for tests

• the DataTable can be created without filename
• it can be created from a CSVData, which is a std::vector of CSV-strings
• it can be rendered into CSV strings
• a (new) storage file name can be given later

See also

DataCSV_test

Definition in file data.hpp.

#include "lib/error.hpp"
#include "lib/nocopy.hpp"
#include "lib/stat/csv.hpp"
#include "lib/stat/file.hpp"
#include "lib/format-string.hpp"
#include "lib/util.hpp"
#include <type_traits>
#include <utility>
#include <fstream>
#include <vector>
#include <string>
#include <limits>
#include <deque>

Classes
struct	Column< VAL >
	Descriptor and Accessor for a data column within a DataTable table. More...
class	DataTable< TAB >
	Table with data values, stored persistently as CSV file. More...

Namespaces
	lib
	Implementation namespace for support and library code.