Difference between revisions of "PRO History"

From meteoplug
Jump to: navigation, search
(Editing Data of a Minute)
(Export Data)
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
<languages /><translate>
 
<languages /><translate>
 
__NOTOC__
 
__NOTOC__
<br />One of the features that sets Meteobridge PRO apart from  the regular Meteobridges is the ability to store 8 GB of sensor data on local USB storage. This is the foundation for lots of additional features to come. Before looking too much into the future let's see what the PRO can offer right now.
+
<br />One of the features that sets Meteobridge PRO apart from  the regular Meteobridges is the ability to store 1 GB (2 GB in case of NANO SD) of sensor data on local storage. Being able to store data locally is the foundation for additional services.
  
Meteobridge PRO stores data in intervals of one minute and automatically aggregates every minutes update to hourly, daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored.
+
Meteobridge PRO and NANO SD store data short term (last two days) in per-minute intervals and long-term in hourly intervals and automatically aggregate hourly data update to daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored.
  
 
Foundation for making use of stored data is that there are management functions available that allow to easily inspect, edit and cleanup data. Picture below shows Meteobridge PRO "History" tab that is the data management console of Meteobridge PRO.
 
Foundation for making use of stored data is that there are management functions available that allow to easily inspect, edit and cleanup data. Picture below shows Meteobridge PRO "History" tab that is the data management console of Meteobridge PRO.
  
 
==Inspect Data==
 
==Inspect Data==
Inspect data field allows you to select a piece of sensor data you are interested in and to drill down by selecting year, month, days, hour to the minute data that then can be changed, inserted or deleted.
+
Inspect data field allows you to select a piece of sensor data you are interested in and to drill down by selecting year, month, days to the hourly data that then can be changed, inserted or deleted.
  
 
'''Sensor''' allows to select the sensor to inspect. Without selecting a sensor a further drill-down is not possible.  
 
'''Sensor''' allows to select the sensor to inspect. Without selecting a sensor a further drill-down is not possible.  
Line 14: Line 14:
 
'''Year''' shows all years beginning at 2014. When a year contains data (for the selected sensor) it is marked with an asterisk "*". When a year is selected the graph on the right does show all months of the year and the average, min and max values of that specific sensor for the year. If it is a cumulative sensor (like rain totals) only the total sum is presented. Example below shows 2015 with data for th0temp in months November and December.  
 
'''Year''' shows all years beginning at 2014. When a year contains data (for the selected sensor) it is marked with an asterisk "*". When a year is selected the graph on the right does show all months of the year and the average, min and max values of that specific sensor for the year. If it is a cumulative sensor (like rain totals) only the total sum is presented. Example below shows 2015 with data for th0temp in months November and December.  
  
[[File:page7a.png]]
+
[[File:page7a.png|685px]]
  
 
'''Month''' shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month.
 
'''Month''' shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month.
  
[[File:page7b.png]]
+
[[File:page7b.png|685px]]
  
 
'''Day''' selects a day in the month and summarizes all hours of the day in the graph on the right.
 
'''Day''' selects a day in the month and summarizes all hours of the day in the graph on the right.
  
[[File:page7c.png]]
+
[[File:page7c.png|685px]]
  
'''Hour''' is the final drill-down step that lists average, minimum and maximum values for every minute of the hour. Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field.
+
Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field.
  
===Editing Data of an Hinute===
+
===Editing Data of an Hour===
A click onto a minute time stamp starts deletion of the sensor data for that minute, when you confirm the pop-up dialog to do so.
+
A click onto an hour time stamp starts deletion of the sensor data for that hour, when you confirm the pop-up dialog to do so.
A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, pleasa add "F", "in", "mph", "inhg" to your values. A field that shows "--" indicates that the is no sensor data available for this minute. When data repeats across average, minimum, maximum the min/max fields are marked with a double quote to avoid visual overloading by repeating data.
+
A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, please add "F", "in", "mph", "inhg" to your values. A field that shows "--" indicates that the is no sensor data available for this minute. When data repeats across average, minimum, maximum the min/max fields are marked with a double quote to avoid visual overloading by repeating data.
  
[[File:page7d.png]]
+
[[File:page7d.png|685px]]
  
 
===Quick Relative Navigation===  
 
===Quick Relative Navigation===  
* '''Now''' button directs to the current hour and shows data of that hour for the selected sensor.
+
* '''Now''' button directs to the current day and shows data of that hour for the selected sensor.
* '''< <''' button skips to the past. If drill-down level is on hour level, previous hour is selected. When drill-down level is days, previous day is selected, and so on.
+
* '''< <''' button skips to the past. If drill-down level is on daily level, previous day is selected. When drill-down level is months, previous month is selected, and so on.
 
* '''> >''' button skips one step to the future using the current drill-down level.
 
* '''> >''' button skips one step to the future using the current drill-down level.
  
Line 42: Line 42:
 
* '''Delete All''' button deletes data for all sensors.
 
* '''Delete All''' button deletes data for all sensors.
 
Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered.
 
Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered.
 +
 +
==Import Data==
 +
Meteobridge PRO and NANO SD can inport data from various sources, including
 +
* Meteohub monthly raw data
 +
* Meteoplug monthly esport data
 +
* Meteotemplate
 +
* Cumulus
 +
* weewx
 +
* WSWIN
 +
Import operations can take some time as thousands of data sets need to be stored in Meteobridge's internal database. In order to import a file tit needs to be dropped into the "import" folder of Meteobridge's PC-network share. Import can be told to stop on errors or to ignore those. Import operation is logged in messages section of "System" tab.
 +
 +
[[File:page7e.png|685px]]
  
 
==Export Data==
 
==Export Data==
Meteobridge PRO's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp").  
+
Meteobridge PRO's or NANO SD's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp").  
  
 
===Export Template Format===
 
===Export Template Format===
Line 60: Line 72:
  
 
Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings:
 
Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings:
* '''emptylines=no''' blocks lines that don't have any sensor data from being printed into the target file. Default is that empty lines are printed (emptylines=yes).
+
* '''emptylines=n''' blocks lines that don't have any sensor data from being printed into the target file. Default is that empty lines are printed (emptylines=y).
 +
* '''futuredata=n''' blocks lines that have time stamps in the future. Default is futuredata=n.
 +
* '''crlf=y''' provides Windows style line ends (CR + LF) instead of Linux style (LF only). Default is crlf=n.
 +
* '''log=y''' logs data export execution in Meteobridsge system messages. Default is log=y. 
 +
* '''fillgap=120''' gives a time period data of a sensor is assumed to be recent. During that period the data of sensors is repeated into following lines of export data in case the sensor has not provided fresh data. Default is 600 seccond (10 minutes).
 +
* '''exportfile=myexport.txt''' specifies a dedicated name of the export file. By default the name of the template and the chosen time period is used to generate a file name automatically.
  
 
===Starting Exports===   
 
===Starting Exports===   
Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of an hour to data of a whole year. This exports can be done in resolution of a minute, hour, day or month. Combination of export of a whole year's data and resolution down to a minute is not allowed as this will produce too much data. All other combinations can be done.     
+
Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of a day to data of a whole year. This exports can be done in resolution of a minute, hour, day or month. Combination of export of a whole year's data and resolution down to a minute is not allowed as this will produce too much data. All other combinations can be done.     
  
 
Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this:
 
Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this:

Latest revision as of 16:11, 12 January 2019


One of the features that sets Meteobridge PRO apart from the regular Meteobridges is the ability to store 1 GB (2 GB in case of NANO SD) of sensor data on local storage. Being able to store data locally is the foundation for additional services.

Meteobridge PRO and NANO SD store data short term (last two days) in per-minute intervals and long-term in hourly intervals and automatically aggregate hourly data update to daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored.

Foundation for making use of stored data is that there are management functions available that allow to easily inspect, edit and cleanup data. Picture below shows Meteobridge PRO "History" tab that is the data management console of Meteobridge PRO.

Inspect Data

Inspect data field allows you to select a piece of sensor data you are interested in and to drill down by selecting year, month, days to the hourly data that then can be changed, inserted or deleted.

Sensor allows to select the sensor to inspect. Without selecting a sensor a further drill-down is not possible.

Year shows all years beginning at 2014. When a year contains data (for the selected sensor) it is marked with an asterisk "*". When a year is selected the graph on the right does show all months of the year and the average, min and max values of that specific sensor for the year. If it is a cumulative sensor (like rain totals) only the total sum is presented. Example below shows 2015 with data for th0temp in months November and December.

Page7a.png

Month shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month.

Page7b.png

Day selects a day in the month and summarizes all hours of the day in the graph on the right.

Page7c.png

Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field.

Editing Data of an Hour

A click onto an hour time stamp starts deletion of the sensor data for that hour, when you confirm the pop-up dialog to do so. A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, please add "F", "in", "mph", "inhg" to your values. A field that shows "--" indicates that the is no sensor data available for this minute. When data repeats across average, minimum, maximum the min/max fields are marked with a double quote to avoid visual overloading by repeating data.

Page7d.png

Quick Relative Navigation

  • Now button directs to the current day and shows data of that hour for the selected sensor.
  • < < button skips to the past. If drill-down level is on daily level, previous day is selected. When drill-down level is months, previous month is selected, and so on.
  • > > button skips one step to the future using the current drill-down level.

Data Deletion

Depending on the drill-down level data for an hour, day or month can be deleted.

  • Delete One button deletes data for the selected sensor only.
  • Delete All button deletes data for all sensors.

Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered.

Import Data

Meteobridge PRO and NANO SD can inport data from various sources, including

  • Meteohub monthly raw data
  • Meteoplug monthly esport data
  • Meteotemplate
  • Cumulus
  • weewx
  • WSWIN

Import operations can take some time as thousands of data sets need to be stored in Meteobridge's internal database. In order to import a file tit needs to be dropped into the "import" folder of Meteobridge's PC-network share. Import can be told to stop on errors or to ignore those. Import operation is logged in messages section of "System" tab.

Page7e.png

Export Data

Meteobridge PRO's or NANO SD's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp").

Export Template Format

A template file consist of lines beginning with a "#", which are copied 1:1 (without the leading "#") to the output files, and lines that are handled as a variable templates where variable definitions are replaced with sensor data from the archive. The example below shows an example that exports

  • a date and time stamp
  • outdoor temperature in °F
  • outdoor humidity in %
  • dew point in °F
  • sea level pressure in inHg
## Standard CSV template with data in imperial units
##
## date, time, temperature[F], humidity[%], dew point[F], sealevel pressure[inHg]
[YYYY]-[MM]-[DD],[hh]:[mm],[th0temp-avg=F.1:],[th0hum-avg.0:],[th0dew-avg=F.1:],[thb0seapress-avg=inhg.1:]

Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings:

  • emptylines=n blocks lines that don't have any sensor data from being printed into the target file. Default is that empty lines are printed (emptylines=y).
  • futuredata=n blocks lines that have time stamps in the future. Default is futuredata=n.
  • crlf=y provides Windows style line ends (CR + LF) instead of Linux style (LF only). Default is crlf=n.
  • log=y logs data export execution in Meteobridsge system messages. Default is log=y.
  • fillgap=120 gives a time period data of a sensor is assumed to be recent. During that period the data of sensors is repeated into following lines of export data in case the sensor has not provided fresh data. Default is 600 seccond (10 minutes).
  • exportfile=myexport.txt specifies a dedicated name of the export file. By default the name of the template and the chosen time period is used to generate a file name automatically.

Starting Exports

Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of a day to data of a whole year. This exports can be done in resolution of a minute, hour, day or month. Combination of export of a whole year's data and resolution down to a minute is not allowed as this will produce too much data. All other combinations can be done.

Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this:

# Standard CSV template with data in metric units
#
# date, time, temperature[C], humidity[%], dew point[C], sealevel pressure[hPa]
2015-11-01,00:00,,,,
2015-11-02,00:00,,,,
2015-11-03,00:00,,,,
2015-11-04,00:00,,,,
2015-11-05,00:00,,,,
2015-11-06,00:00,,,,
2015-11-07,00:00,,,,
2015-11-08,00:00,,,,
2015-11-09,00:00,,,,
2015-11-10,00:00,,,,
2015-11-11,00:00,13.8,87,11.6,1018.8
2015-11-12,00:00,13.0,85,10.5,1021.3
2015-11-13,00:00,,79,7.3,1015.6
2015-11-14,00:00,7.9,86,5.7,1015.7
2015-11-15,00:00,9.9,89,8.1,1004.4
2015-11-16,00:00,11.9,88,10.0,1007.9
2015-11-17,00:00,11.4,89,9.5,1002.7
2015-11-18,00:00,11.9,87,9.8,1002.2
2015-11-19,00:00,10.4,88,8.5,1002.4
2015-11-20,00:00,8.6,89,6.8,999.3
2015-11-21,00:00,6.3,88,4.5,994.1
2015-11-22,00:00,2.6,86,0.5,1001.2
2015-11-23,00:00,3.8,87,1.9,1019.1
2015-11-24,00:00,4.5,86,2.3,1012.9
2015-11-25,00:00,5.3,89,3.7,1003.1
2015-11-26,00:00,6.0,88,4.1,1013.7
2015-11-27,00:00,5.4,86,3.4,1013.7
2015-11-28,00:00,6.3,86,4.2,1004.1
2015-11-29,00:00,,,,
2015-11-30,00:00,,,,