Differences between revisions 8 and 9
Revision 8 as of 2013-01-08 22:41:18
Size: 13322
Editor: dslb-084-058-143-059
Comment: correction wde1_read.sh
Revision 9 as of 2013-01-09 05:47:03
Size: 14157
Editor: dslb-084-058-129-179
Comment: added sytem mail explanation
Deletions are marked like this. Additions are marked like this.
Line 103: Line 103:
# Adjust serial port # Setup serial port parameters
Line 110: Line 110:
    #echo $line > wde1_raw.dat
Line 114: Line 115:
    data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g' | sed 's/:0:0/:0/'`     # 1. add prefix "N", remove trailing 0
    # 2. substitude all (g=global) strings "::" by ":U:"
    # 3. again point 2., why?
    data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g'`
Line 117: Line 121:
    # update rrd
Line 120: Line 123:
    # update rrdmc
Line 141: Line 145:
'''Note:''' In case of an '''error''' in the '''crontab call''', you will get a '''system mail''', announced at the next RETURN on the command line. Then check your system mail for the error message:
{{{
# Announcement of a system mail
Sie haben Post in /var/mail/rudi.
rudi@rudiswiki:~/Install/rrdtool$ ls -ls

# check system mail
$ mail
"/var/mail/rudi": 40 messages 2 new 5 unread
...
N 40 Cron Daemon Wed Jan 9 06:11 19/705 Cron <rudi@rudiswiki> $HOME/Install/rrdtool/wde1_read.sh >

# Read mail
? RETURN
...
ERROR: weather.rrd: found extra data on update argument: 0

# delete mail
? d

# next mail
? n
...
At EOF

# Quit mail program
? q
}}}

..

Temperature Measurement

The target was to collect weather data via radio sensors, display them via web interface and monitor the inside temperature for a specified minimum and maximum threshold. In case of leaving the allowed bandwidth, an email will be send for an alarm.

The resulting temperature and humidity graphs are made over a time span of a day and week, please see the links at the top right of this page.

Components

The used computer should be a low cost Linux capable device with network connection and USB interface. Actually a Seagate Freeagent Dockstar box is used, please see CategoryDockStar. A Raspberry Pi may also be usable.

The computer network port must be connected to a router, in order to allow Internet access.

The web interface is realized with the wiki software http://moinmo.in.

The radio sensors for temperature and humidity are named S300TH, please see here for reference.

The radio receiver with USB port is the module WDE1 (Wetter Daten Empfänger 1 = weather data receiver 1) from company ELV, please see here for reference.

Software

The software should be build with available modules, as much as possible. What was collected:

  • Bash shell script wde1_read.sh for data read in, started from /etc/rc.local.

  • Database rrdtool for collecting the data in weather.rrd.

  • Shell script rrd_graph_all.sh for updating the graphs every 15 minutes via cron.

  • Python program temp_alarm.py to check regularly the temperature limit and sending an email alarm.

  • Wiki software moinmoin for web interface.

graphviz-SomeGraph-a06813222963e7d3979feeb0c43b759886e5e79e.png

Database setup

If not already done install package rrdtool.

# for example:
$ sudo apt-get install rrdtool

Before collecting data, the RRD (Round Robin database) must be setup first. This is done with the script file rrd_create_wde1.sh:

.
#!/bin/sh
rrdtool create weather.rrd --step 300 \
DS:temps1:GAUGE:1200:-40:50 \
DS:temps2:GAUGE:1200:-40:50 \
DS:temps3:GAUGE:1200:-40:50 \
DS:temps4:GAUGE:1200:-40:50 \
DS:temps5:GAUGE:1200:-40:50 \
DS:temps6:GAUGE:1200:-40:50 \
DS:temps7:GAUGE:1200:-40:50 \
DS:temps8:GAUGE:1200:-40:50 \
DS:hums1:GAUGE:1200:0:100 \
DS:hums2:GAUGE:1200:0:100 \
DS:hums3:GAUGE:1200:0:100 \
DS:hums4:GAUGE:1200:0:100 \
DS:hums5:GAUGE:1200:0:100 \
DS:hums6:GAUGE:1200:0:100 \
DS:hums7:GAUGE:1200:0:100 \
DS:hums8:GAUGE:1200:0:100 \
DS:temps9:GAUGE:1200:-40:50 \
DS:hums9:GAUGE:1200:0:100 \
DS:winds9:GAUGE:1200:0:200 \
DS:rains9:DERIVE:1200:0:U \
DS:israins9:GAUGE:1200:0:1 \
RRA:AVERAGE:0.5:1:4032 \
RRA:MIN:0.5:96:3600 \
RRA:MAX:0.5:96:3600 \
RRA:AVERAGE:0.5:96:7300
# Datensatz alle 5 min: step 300
# 14 Tage Einzelwerte: 12 x 24 x 14 = AVERAGE 4032
# 20 Jahre Mittelwerte: 365 x 20 = AVERAGE 7300

WDE1 data read, crontab

  • Start method: by crontab line

In order to access device /dev/ttyUSB0 you have to be a member of the group dialout, if not, you will get a permission problem, please see TempMess#Test_WDE1_Data_Reception .

The WDE1 weather data are read in via shell script wde1_read.sh.

# File: wde1_read.sh
# Receive remote weather data from USB-WDE1 and store into RRD database
# Start with crontab
# Adopted from: http://www.kompf.de/weather/technik.html
# 2013-01-08 RudolfReuter

cd $HOME/Install/rrdtool

# Read data from USB-WDE1
# Setup serial port parameters
stty -F /dev/ttyUSB0 cstopb -ixon raw speed 9600 > /dev/null

read line < /dev/ttyUSB0

# Check for header = "$1"
if [[ "${line%%;*}" == '$1' ]] ; then
    #echo $line > wde1_raw.dat
    # remove trailing "0"
    line=`echo "${line%?}"`
    # format data, remove prefix "$1;1;", replace ";," by ":."
    tmp=`echo "${line#?1;1;}" | tr ';,' ':.'`
    # 1. add prefix "N", remove trailing 0
    # 2. substitude all (g=global) strings "::" by ":U:"
    # 3. again point 2., why?
    data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g'`
    # remove trailing ":"
    data=${data%%:}
    # save last data values
    echo $data > rrd_data.dat
    # update rrdmc
    rrdtool update weather.rrd $data
    # save last indoor temperature for alarm compare
    cut -d ":" -f 6 rrd_data.dat > temp5.dat
fi

Note: The bash shell must be used, in order to work properly.

The shell script is started with crontab:

$ crontab -e

# append
# read every 5 minutes the temp. and hum. values from WDE1
*/5 * * * * $HOME/Install/rrdtool/wde1_read.sh > /dev/null

# check with
$ crontab -l

# for a manual test do
$ ./wde1_read.sh

Note: In case of an error in the crontab call, you will get a system mail, announced at the next RETURN on the command line. Then check your system mail for the error message:

# Announcement of a system mail
Sie haben Post in /var/mail/rudi.
rudi@rudiswiki:~/Install/rrdtool$ ls -ls

# check system mail
$ mail
"/var/mail/rudi": 40 messages 2 new 5 unread
...
N 40 Cron Daemon Wed Jan 9 06:11 19/705 Cron <rudi@rudiswiki> $HOME/Install/rrdtool/wde1_read.sh >

# Read mail
? RETURN
...
ERROR: weather.rrd: found extra data on update argument: 0

# delete mail
? d

# next mail
? n
...
At EOF

# Quit mail program
? q

WDE1 data read, rc.local

  • Start method: by /etc/rc.local

  • Note: This is no longer used, because of unreliable behavior.

If not already done install package socat.

# for example:
$ sudo apt-get install socat

In order to access device /dev/ttyUSB0 you have to be a member of the group dialout, if not, you will get a permission problem, please see TempMess#Test_WDE1_Data_Reception .

The WDE1 weather data are read in via shell script wde1_read2.sh.

.
#!/bin/bash
# File: wde1_read2.sh
# Receive remote weather data from USB-WDE1 and store it into database
# Author: http://www.kompf.de/weather/technik.html

# Loop forever to read data from USB-WDE1
socat /dev/ttyUSB0,b9600 STDOUT | \
while read line
do
    if [[ "${line%%;*}" == '$1' ]] ; then
        # format data
        tmp=`echo "${line#?1;1;}" | tr ';,' ':.'`
        data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g'`
        data=${data%%:}
        # update rrd
        rrdtool update weather.rrd $data
    fi
done

Note: The bash shell must be used, in order to work properly.

The shell script is started at boot time with a line in file /etc/rc.local:

start-stop-daemon --background --chuid rudi:rudi --start  --exec /home/rudi/Install/rrdtool/wde1_read2.sh --chdir /home/rudi/Install/rrdtool

# for a manual start do
$ sudo /etc/rc.local

Note: The shell script should be started with user rights (--chuid) in a specified folder (--chdir) and must run in the background (--background).

Sensor channels

The WDE1 receiver does have the capacity for 8 different sensors.

  • Channel 5 is used for Indoor temperature and humitdity.

  • Channel 6 is used for Outdoor temperature and humitdity.

Generate data graphs

Now you have some data in the database, next you will see in a graph how the trend is.

For me it was the easiest method to update all useful graphs every 15 minutes and display them in a web interface (wiki moinmoin). This is done with the script rrd_graph_all.sh:

.
#!/bin/sh
# file: rrd_graph_all.sh
# exec all rrd_graph* files at once, for cron
# 2013-01-07 RudolfReuter

cd $HOME/Install/rrdtool/
./rrd_graph_h1d.sh
./rrd_graph_h1w.sh
./rrd_graph_t1d.sh
./rrd_graph_t1w.sh

The crontab entry was setup with:

$ crontab -e

# append the following line:
*/15 * * * * $HOME/Install/rrdtool/rrd_graph_all.sh > /dev/null

# check with:
$ crontab -l

The single graphs are defined with the following 4 files.

.
#!/bin/sh
# File: rrd_graph_h1d.sh
rrdtool graph hum1d.png --end now --start end-1d --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n" 

.
#!/bin/sh
# File: rrd_graph_h1w.sh
rrdtool graph hum1w.png --end now --start end-1w --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n"

.
#!/bin/sh
# File: rrd_graph_t1d.sh
rrdtool graph temp1d.png --end now --start end-1d --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n" 

.
#!/bin/sh
# File: rrd_graph_t1w.sh
rrdtool graph temp1w.png --end now --start end-1w --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n"

Check for Alarm

If not already done install package python-rrdtool.

# for example:
$ sudo apt-get install python-rrdtool

The indoor temperature check for exceeding the thresholds is done with a Python program temp_alarm.py.

In case of a transition from a temperature value inside the bandwidth to outside, an alarm email is send. In order to avoid unnecessary alarm email, a hysteresis is used.

All parameters are written in a configuration file temp_alarm.cfg.

The state of the program is written in file temp_alarm.state.

Web Interface

The web interface is realized with the wiki software moinmoin, see MoinSetup.

Test WDE1 Data Reception

Connect the WDE1 to the USB plug and check if it is recognized:

# see if the USB device is seen
$ ls -ls /dev/ttyU*
0 crw-rw---- 1 root dialout 188, 0 Dez 26  2012 /dev/ttyUSB0

# check if your user is in the group "dialout"
$ groups

# if the user is not in the group "dialout", add it
$ sudo adduser rudi dialout

# logout and login to have the group attached to the user
$ groups

Check if you have reception of the sensors:

# if utility "socat" is not installed, do so
$ sudo apt-get install socat

# check for the data telegram
# 2 sensors at address 5 and 6
$ socat /dev/ttyUSB0,b9600 stdout
$1;1;;;;;;19,8;9,2;;;;;;;59;85;;;;;;;;0

# inside temperature = 19,8 °C, outside temperature =  9,2 °C
# inside humidity    = 59 %,    outside humidity    = 85 %

Troubleshooting WDE1

If the USB device WDE1 is not seen:

# check for needed kernel modules
$ lsmod
...
cp210x                 17514  0 
usbserial              37173  1 cp210x
...

# check for USB device
$ lsusb
...
Bus 002 Device 004: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP210x Composite Device

List of pages in this category:

-- RudolfReuter 2013-01-06 18:55:50


Go back to CategoryDockStar or FrontPage ; KontaktEmail (ContactEmail)

TempMess (last edited 2013-01-12 05:48:24 by dslb-084-058-178-181)