Differences between revisions 36 and 37
Revision 36 as of 2015-05-24 11:42:12
Size: 0
Editor: RudolfReuter
Comment: added ash2200 picture
Revision 37 as of 2016-07-16 06:37:34
Size: 27335
Editor: RudolfReuter
Comment: added Quick Help and Network config
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
##master-page:DockStarTemplate
#format wiki
#language en
<<Navigation(children)>> ..

== Temperature Measurement ==
||<tablestyle="float:right;margin:0px;"style="padding:0.5em;border:0px none;font-size:100%;"><<TableOfContents>> ||


The '''target''' was to '''collect weather data''' via radio sensors, display them via '''web interface''' and '''monitor''' the inside temperature for a specified minimum and maximum threshold. In case of leaving the allowed bandwidth, an '''email''' will be send for an '''alarm'''.

The resulting '''temperature''' and '''humidity''' graphs are made over a time span of a '''day''' and '''week''', please see the '''links''' at the top right of this page.


=== Quick help ===
 1. '''shut down''' computer for maintenance
{{{
$ sudo shutdown now
}}}
 1. '''reboot''' computer
{{{
$ sudo shutdown -r now
}}}

=== Components ===
The used '''computer''' should be a low cost '''Linux''' capable device with '''network connection''' and USB interface. Actually a Seagate Freeagent '''Dockstar''' box is used, please see CategoryDockStar. A [[http://www.raspberrypi.org/|Raspberry Pi]] may also be usable.

The computer network port must be connected to a router, in order to allow '''Internet access'''.

The '''web interface''' is realized with the '''wiki''' software http://moinmo.in.

The '''radio sensors''' for '''temperature''' and '''humidity''' are named S300TH, please see [[http://www.elv.de/funk-temperatur-luftfeuchteaussensensor-s-300-th-fuer-z-b-elv-ws-200-300-elv-ws-300-pc-usb-wde1-und-ipwe-1.html|here]] for reference.

The '''radio receiver''' with USB port is the module '''WDE1''' (Wetter Daten Empfänger 1 = weather data receiver 1) from company ELV, please see [[http://www.elv.de/output/controller.aspx?cid=74&detail=10&detail2=29870|here]] for reference.

The '''logical channels''' of the '''sensors/transmitters''' in this application are the following:
 * Channel 5 (1-8) - in house
 * Channel 6 (1-8) - outside

=== WDE1 Superhet ===
||<tablestyle="float: right;">[[attachment:WDE1_Superhet_DSC05885.jpg|{{attachment:WDE1_Superhet_DSC05885.jpg||width="400"}}]]||
In the original ELV '''WDE1''' receiver (868.35 MHz) there was a '''super-regenerative receiver'''. This was good up to the time, '''LTE''' mobile radio appeared. Now the large bandwidth of the receiver is a problem.

The solution to this problem is the [[http://www.elv.de/output/controller.aspx?cid=74&detail=10&detail2=42432|RX868SH-DV receiver]]. You can '''retrofit''' an old WDE1 with it. It is in principle pin compatible to the old receiver, but needs a jumper between '''EN and +UB'''. Instead of winding the short antenna wire to fit the housing, I put the wire straight out, see the '''picture''' on the right.

The new USB receiver '''WDE1-2''' from ELV already comes with the new '''superhet receiver''' included.

||<tablestyle="float: right;">[[attachment:ASH2200_DSC05886.jpg|{{attachment:ASH2200_DSC05886.jpg||width="320"}}]]||
=== Temperature sensor ===
The '''low cost''' temperature and humidity sensor/transmitter '''S300TH''' is no longer available (2015-05). Now you can buy the '''ASH2200''' sensor/transmitter from ELV for about 30 EUR, see the '''picture''' on the right.

=== Software ===
The software should be build with '''available modules''', as much as possible. What was collected:

 * [[http://www.kompf.de/weather/technik.html|Bash shell script]] `wde1_read.sh` for data read in, started from `crontab`.
 * Database [[http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html|rrdtool]] for collecting the data in `weather.rrd`.
 * Shell script `rrd_graph_all.sh` for updating the graphs every 15 minutes via '''cron'''.
 * Python program `temp_alarm.py` to check regularly the temperature limit and sending an email alarm.
 * Wiki software [[http://moinmo.in|moinmoin]] for web interface.

{{{#!GraphViz dot
digraph SomeGraph {
  $STD_GRAPH_HEADER
  "cron" -> { "rrd_graph_all.sh" "wde1_read.sh" "temp_alarm.py" };
  read [label="weather.rrd\ntemp5.dat\nrdd_data.dat"];
  "wde1_read.sh" -> read;
  graphs [label="hum1d.png\nhum1w.png\ntemp1d.png\ntemp1w.png"];
  "rrd_graph_all.sh" -> graphs;
  standaloneserver [label="moin\nstandalone\nWeb- Server"];
  "/etc/rc3.d/S81moin" -> standaloneserver;
  standaloneserver -> standaloneserver;
  alert [label="temp_alarm.cfg\ntemp_alarm.dat\ntemp5.dat"];
  "temp_alarm.py" -> alert;
  alert -> "temp_alarm.py";
}
}}}
=== Database setup ===
If not already done install package '''rrdtool'''.

{{{
# for example:
$ sudo apt-get install rrdtool
}}}
Before collecting data, the '''RRD''' (Round Robin database) must be setup first. This is done with the script file `rrd_create_wde1.sh`:

{{{#!bash
#!/bin/sh
rrdtool create weather.rrd --step 300 \
DS:temps1:GAUGE:1200:-40:50 \
DS:temps2:GAUGE:1200:-40:50 \
DS:temps3:GAUGE:1200:-40:50 \
DS:temps4:GAUGE:1200:-40:50 \
DS:temps5:GAUGE:1200:-40:50 \
DS:temps6:GAUGE:1200:-40:50 \
DS:temps7:GAUGE:1200:-40:50 \
DS:temps8:GAUGE:1200:-40:50 \
DS:hums1:GAUGE:1200:0:100 \
DS:hums2:GAUGE:1200:0:100 \
DS:hums3:GAUGE:1200:0:100 \
DS:hums4:GAUGE:1200:0:100 \
DS:hums5:GAUGE:1200:0:100 \
DS:hums6:GAUGE:1200:0:100 \
DS:hums7:GAUGE:1200:0:100 \
DS:hums8:GAUGE:1200:0:100 \
DS:temps9:GAUGE:1200:-40:50 \
DS:hums9:GAUGE:1200:0:100 \
DS:winds9:GAUGE:1200:0:200 \
DS:rains9:DERIVE:1200:0:U \
DS:israins9:GAUGE:1200:0:1 \
RRA:AVERAGE:0.5:1:4032 \
RRA:MIN:0.5:96:3600 \
RRA:MAX:0.5:96:3600 \
RRA:AVERAGE:0.5:96:7300
# Datensatz alle 5 min: step 300
# 14 Tage Einzelwerte: 12 x 24 x 14 = AVERAGE 4032
# 20 Jahre Mittelwerte: 365 x 20 = AVERAGE 7300
}}}
=== Database conversion ===
If the database file should be copied to another computer, it may give an '''error''':

 * ERROR: This RRD was created on another architecture

Then the database must be converted:

{{{
# Dump database on source computer
$ rrdtool dump weather.rrd >weather.xml

# Copy XML file to target computer

# Restore database on target computer
$ rrdtool restore weather.xml weather.rrd
}}}
=== WDE1 data read, crontab ===
 * '''Start method''': by crontab line

In order to access device `/dev/ttyUSB0` you have to be a member of the group '''dialout''', if not, you will get a permission problem, please see TempMess#Test_WDE1_Data_Reception .

The WDE1 weather data are read in via shell script `wde1_read.sh`.

{{{#!bash
#!/bin/bash
# File: wde1_read.sh
# Receive remote weather data from USB-WDE1 and store into RRD database
# Start with crontab
# Adopted from: http://www.kompf.de/weather/technik.html
# 2013-01-08 RudolfReuter

cd $HOME/Install/rrdtool

# Read data from USB-WDE1
# Setup serial port parameters
stty -F /dev/ttyUSB0 cstopb -ixon raw speed 9600 > /dev/null

read line < /dev/ttyUSB0

# Check for header = "$1"
if [[ "${line%%;*}" == '$1' ]] ; then
    #echo $line > wde1_raw.dat
    # remove trailing "0", stop character
    line=`echo "${line%?}"`
    # format data, remove prefix "$1;1;", replace ";," by ":."
    tmp=`echo "${line#?1;1;}" | tr ';,' ':.'`
    # 1. add prefix "N" (date stamp), remove trailing 0
    # 2. substitude all (g=global) strings "::" by ":U:"
    # 3. again point 2., why?
    data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g'`
    # remove trailing ":"
    data=${data%%:}
    # save last data values
    echo $data > rrd_data.dat
    # update rrdmc
    rrdtool update weather.rrd $data
    # save last indoor temperature for alarm compare
    cut -d ":" -f 6 rrd_data.dat > temp5.dat
fi
}}}
'''Note:''' The '''bash''' shell must be used, in order to work properly.

The shell script is started with crontab:

{{{
$ crontab -e

# append
# read every 5 minutes the temp. and hum. values from WDE1
*/5 * * * * $HOME/Install/rrdtool/wde1_read.sh &> /dev/null

# check with
$ crontab -l

# for a manual test do
$ ./wde1_read.sh
}}}
=== Cron System Mail ===
In case of an '''error''' in the '''crontab call''', you will get a '''system mail''', announced at the next RETURN on the command line. Then check your system mail for the error message:

{{{
# Announcement of a system mail
Sie haben Post in /var/mail/user.
$ ls -ls

# check system mail
$ mail
"/var/mail/user": 40 messages 2 new 5 unread
...
N 40 Cron Daemon Wed Jan 9 06:11 19/705 Cron <user@rudiswiki> $HOME/Install/rrdtool/wde1_read.sh >

# Read mail
? RETURN
...
ERROR: weather.rrd: found extra data on update argument: 0

# delete mail
? d

# next mail
? n
...
At EOF

# Quit mail program
? q
}}}
=== Cron logging disable ===
Normally '''cron''' logs every job. In order to avoid that, you can modify the log level:

{{{
# example: Apr 10 18:03:01 FADS11 cron.info /USR/SBIN/CRON[3209]: (peter) CMD ($HOME/Install/rrdtool/rrd_graph_all.sh > /dev/null)

# Edit file /etc/default/cron
# add "-L 0" for no logging (errors are logged regardless)
EXTRA_OPTS="-L 0"

# restart cron
sudo /etc/init.d/cron restart
}}}
Additionally there is a message: "pam_unix(cron:session): session opened for user", which can be suppressed by:

{{{
# Edit file /etc/pam.d/common-session-noninteractive
# Find line "session required pam_unix.so"
# add a line before
session [success=1 default=ignore] pam_succeed_if.so service in cron quiet use_uid

# restart cron
$ sudo invoke-rc.d cron restart
}}}

=== Sensor channels ===
The WDE1 receiver does have the capacity for 8 different sensors.

 * Channel 5 is used for '''Indoor''' temperature and humitdity.
 * Channel 6 is used for '''Outdoor''' temperature and humitdity.

=== Generate data graphs ===
Now you have some data in the database, next you will see in a '''graph''' how the '''trend''' is.

For me it was the easiest method to '''update''' all useful graphs every 15 minutes and '''display''' them in a web interface (wiki moinmoin). This is done with the script `rrd_graph_all.sh`:

{{{#!bash
#!/bin/sh
# file: rrd_graph_all.sh
# exec all rrd_graph* files at once, for cron
# 2013-01-07 RudolfReuter

cd $HOME/Install/rrdtool/
./rrd_graph_h1d.sh
./rrd_graph_h1w.sh
./rrd_graph_t1d.sh
./rrd_graph_t1w.sh

# copy graphs to rudsiwiki.de
scp -i $HOME/.ssh/id_rsa *.png rudi@www.rudiswiki.de:Install/rrdtool
}}}
The `crontab` entry was setup with:

{{{
$ crontab -e

# append the following lines:
# run every 15 minutes, but with offset of 3 minutes
3,18,33,48 * * * * $HOME/Install/rrdtool/rrd_graph_all.sh &> /dev/null

# check with:
$ crontab -l
}}}
The single graphs are defined with the following 4 files.

{{{#!bash
#!/bin/sh
# File: rrd_graph_h1d.sh
rrdtool graph hum1d.png --end now --start end-1d --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n"
}}}
{{{#!bash
#!/bin/sh
# File: rrd_graph_h1w.sh
rrdtool graph hum1w.png --end now --start end-1w --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n"
}}}
{{{#!bash
#!/bin/sh
# File: rrd_graph_t1d.sh
rrdtool graph temp1d.png --end now --start end-1d --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
GPRINT:temps6:LAST:"Cur %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n"
}}}
{{{#!bash
#!/bin/sh
# File: rrd_graph_t1w.sh
rrdtool graph temp1w.png --end now --start end-1w --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n"
}}}
=== Check for Alarm ===
 * '''Start method''': by crontab line

If not already done install package '''python-rrdtool'''.

{{{
# for example:
$ sudo apt-get install python-rrdtool
}}}
The '''indoor temperature check''' for exceeding the thresholds is done with a Python program `temp_alarm.py`.

In case of a '''transition''' from a temperature value inside the bandwidth to outside, an '''alert email''' is send. In order to avoid unnecessary alert emails, a '''hysteresis''' is used.

All parameters are stored in a configuration file `temp_alarm.cfg`.

The '''state''' of the '''alert''' is written in file `temp_alarm.dat`.

The `crontab` entry was setup with:

{{{
$ crontab -e

# append the following lines:
#
# run every 15 minutes, but with offset of 4 minutes
4,19,34,49 * * * * $HOME/Install/rrdtool/temp_alarm.py &> /dev/null

# check with:
$ crontab -l
}}}
The program is not so long, therefore it is listed here:

{{{#!python
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# File: temp_alarm.py
# Author: RudolfReuter
# Copyright: GPL2
# Source: http://www.rudiswiki.de
# Version: 2013-01-09
# Version: 2013-05-30, catch "string to float" error in line 98
# Features:
# - computer: Dockstar-Debian
# - Email and Text Alerts for High and Low Temperatures
# - Data Logging and Analysis using RRDTool
# - Custom Configuration parameter
#------------------------------------------
# Import Libaries
import datetime
from smtplib import SMTP
import time
import ConfigParser # to read cfg file
from email.MIMEText import MIMEText
import sys # for exit
import os # for file delete, cd

#----- Functions -------------------

def email_alert(subject, message):
    msg = MIMEText(message + ", see http://www.peterswiki.dd-dns.de")
    msg['From'] = myEmailFrom
    msg['To'] = myEmailTo
    msg['Subject'] = subject
    server = SMTP(myEmailSMTPsrv)
    server.login(myEmailFrom, myEmailPwd)
    try:
        print "EmailAlert"
        server.sendmail(myEmailFrom, myEmailTo, msg.as_string())
    finally:
        server.quit()
    try:
        fpw = open("temp_alert.dat","w")
    except:
        print ("Error: File open AlertSend.dat failed")
        sys.exit(0)
    fpw.write(message)
    fpw.close()

# Begin main program --------------------

# If called from crontab, change directory is needed.
try:
    home = os.environ['HOME']
    os.chdir(home + "/Install/rrdtool")
    print(os.getcwd())
except:
    print ("Error: cd HOME does not work")
    sys.exit(0)

# Read in configuration file using Config Parser
aConfig = ConfigParser.RawConfigParser()
aConfig.read('temp_alarm.cfg')

myAlertLabel = aConfig.get('cfg', 'alertlabel')

# Alert level for Temperature
alertHighTemp = aConfig.getfloat('cfg', 'alerthightemp')
alertLowTemp = aConfig.getfloat('cfg', 'alertlowtemp')
alertHyst = aConfig.getfloat('cfg', 'alerthyst')

# Email parameters
myEmailFrom = aConfig.get('cfg', 'emailfrom')
myEmailPwd = aConfig.get('cfg', 'emailpwd')
myEmailSMTPsrv = aConfig.get('cfg', 'emailsmtpsrv')
myEmailTo = aConfig.get('cfg', 'emailto')

DateNow = time.strftime('%Y-%m-%d %H:%M:%S', (time.localtime(time.time())))

# Read the actual indoor temperature
try:
    fpr = open("temp5.dat")
except:
    print ("Error at open temp5.dat")
    sys.exit(0)
temp5 = fpr.readline()
fpr.close()
try:
    ftemp5 = float(temp5)
except ValueError,e:
    # fake temperature in case of receive error
    ftemp5 = 20.0
print("Temperature: " + temp5)

# Check for Alert recovery
try:
    fp = open("temp_alert.dat")
    # Check for temperature recovery with hysteris
    if ftemp5 > (alertLowTemp + alertHyst):
        if ftemp5 < (alertHighTemp - alertHyst):
            try:
                os.remove("temp_alert.dat")
            except:
                print("Error: File AlertSend.dat delete failed")
                sys.exit(0)
except:
    # Alerts checking for high and low temp
    if ftemp5 < 50.0:
        if ftemp5 < alertLowTemp:
            print(DateNow + " Alert: Low Temp °C " + temp5)
            email_alert(myAlertLabel, DateNow + " Low Temperature: " + temp5)
        if ftemp5 > alertHighTemp:
            print(DateNow + " Alert: High Temp °C " + temp5)
            email_alert(myAlertLabel, DateNow + " High Temperature: " + temp5)
}}}
The '''configuration file''' is used to change quickly parameters, with no need to touch the program.

{{{
[cfg]
alertlabel = Temperatur Alarm Vogelhaus
alerthightemp = 30.0
alertlowtemp = 15.0
alerthyst = 1.0
emailto = reuterx@xxxx.de
emailfrom = reuterx@xxxx.de
emailpwd = xxx
emailsmtpsrv = mail.xxx.de
}}}
=== Web Interface ===
The web interface is realized with the wiki software '''moinmoin''', see MoinSetup.

=== Test WDE1 Data Reception ===
Connect the WDE1 to the USB plug and check if it is recognized:

{{{
# see if the USB device is seen
$ ls -ls /dev/ttyU*
0 crw-rw---- 1 root dialout 188, 0 Dez 26 2012 /dev/ttyUSB0

# check if your user is in the group "dialout"
$ groups

# if the user is not in the group "dialout", add it
$ sudo adduser user dialout

# logout and login to have the group attached to the user
$ groups
}}}
Check if you have '''reception''' of the sensors:

{{{
# if utility "socat" is not installed, do so
$ sudo apt-get install socat

# check for the data telegram
# 2 sensors at address 5 and 6
$ socat /dev/ttyUSB0,b9600 stdout
$1;1;;;;;;20,5;6,8;;;;;;;57;84;;;;;;;;0

# inside temperature = 20,5 °C, outside temperature = 6,8 °C
# inside humidity = 57 %, outside humidity = 84 %

# some times I will receive also a foreign "Kombi sensor" from the neighborhood:
$1;1;;;;;;20,4;7,3;;;;;;;57;84;;;6,1;73;0,0;550;0;0

# Kombi Temperature = 6,1 °C, Kombi Humidity = 73 %
# Kombi Wind speed 0.0 kmph, Kombi Rain volume = 550 rocker beats
# Kombi Rain = 0 = no
}}}
=== Troubleshooting WDE1 ===
If the USB device WDE1 is not seen:

{{{
# check for needed kernel modules
$ lsmod
...
cp210x 17514 0
usbserial 37173 1 cp210x
...

# check for USB device
$ lsusb
...
Bus 002 Device 004: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP210x Composite Device
}}}
=== Copy Graphs per scp ===
In order to show the graphs also on a remote computer, you can copy the .png graphs to the remote computer with '''scp'''. In order to avoid the '''password question''' in a script file you can use an '''ssh ident file''' (key):

{{{
# generate public key
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/peter/.ssh/id_rsa): ENTER
Enter passphrase (empty for no passphrase): ENTER
Enter same passphrase again: ENTER
Your identification has been saved in /home/peter/.ssh/id_rsa.
Your public key has been saved in /home/peter/.ssh/id_rsa.pub.
The key fingerprint is:
df:0e:12:85:c6:ac:c2:21:e4:7c:18:12:31:74:e4:93 peter@FADS11
The key's randomart image is:
+--[ RSA 2048]----+
...

# on the remote computer, a folder /home/rudi/.ssh should already exist, otherwise make one with:
$ ssh rudi@192.168.17.72 mkdir .ssh/
# enter password

# transfer public key, do NOT overwrite an already existing file "authorized_keys"
$ ssh-copy-id -i $HOME/.ssh/id_rsa.pub rudi@192.168.17.72
# enter password
or
$ cat $HOME/.ssh/id_rsa.pub | ssh rudi@192.168.17.72 "cat >>.ssh/authorized_keys"
# enter password

# copy .png files to /home/rudi/Install/rrdtool/
$ cd Install/rrdtool
# Test with internal network
$ scp -i $HOME/.ssh/id_rsa *.png rudi@192.168.17.72:Install/rrdtool

# Test with internet access
$ scp -i $HOME/.ssh/id_rsa *.png rudi@www.rudiswiki.de:Install/rrdtool

# append the previous line to the file "rdd_graph_all.sh" to get an automatic refresh
}}}
'''Problem solving''', see at [[http://h30499.www3.hp.com/t5/System-Administration/SSH-key-not-working-for-1-account-but-working-for-another/td-p/5776097|Re: SSH key not working for 1 account but working for another]]:

{{{
# If you get still asked the password of the remote server, have a look at remote /var/log/auth.log
# If you see: "Authentication refused: bad ownership or modes for directory /home/user"
# you will have TOO much permissions on your home directory -> it must be 755
# change with:
$ chmod 755 /home/user
}}}
=== Start methods at boot ===
The start of the bash script `wde1_read2.sh` at boot time, running in an endless loop was not reliable. But I will describe two easy methods to start the script at boot time, just for reference. The start|stop method in `/etc/init.d` was too much effort for this job.

 * '''Start method''': by `/etc/rc.local`

The shell script is started at boot time with a line in file `/etc/rc.local`:

{{{
start-stop-daemon --background --chuid user:group --start --exec /home/user/Install/rrdtool/wde1_read2.sh --chdir /home/user/Install/rrdtool

# for a manual start do
$ sudo /etc/rc.local
}}}
'''Note:''' The shell script should be started with user rights (--chuid) in a specified folder (--chdir) and must run in the background (--background).

 * '''Start method''': by '''crontab'''

{{{
$ crontab -e

# append
# start once at boot time
@reboot $HOME/Install/rrdtool/wde1_read2.sh > /dev/null
}}}
'''Note:''' Take care to make a '''change directory''' in the script to the working directory, or use an absolute path.

=== Maintenance ===
Installed '''fail2ban''' because of break in trials, see [[http://www.rudiswiki.de/wiki9/ServerUbuntu1204#Setup_of_fail2ban|setup of fail2ban]].

 * check regular `/var/volatile/fail2ban.log`
 * because the '''busybox-syslogd''' normally has a RAM circular buffer (-C128) fail2ban can not check the log. This has to be changed to a file logging in a RAM-disk, in order to reduce the access to the USB flash.

{{{
# edit /etc/default/busybox-syslogd
SYSLOG_OPTS="-O /var/volatile/messages"

# append to file /etc/fstab
tmpfs /var/volatile tmpfs defaults,size=10%,mode=755 0 0
# after a restart the RAM-disk is working.
}}}
 * Unfortunately the '''busybox-syslogd''' was compiled without the possible '''logrotate''' feature, the package '''logrotate''' must be installed. The configuration file for the '''messages''' logging looks like:

{{{
# edit file /etc/logrotate.d/messages
/var/volatile/messages {
    daily
    rotate 1
    copytruncate
    missingok
    sharedscripts
    postrotate
        day=$(date +%Y-%m-%d)
        # Path "~" will give "/root" instead of "/home/rudi"!
        mv /var/volatile/messages.1 /home/peter/log/messages-$day.log
    endscript

}
}}}
So, a daily copy of the '''messages''' is kept in the user folder '''log'''.

 * The '''fail2ban.log''' is also rotated, but only the last copy is kept. The configuration file looks like:

{{{
# edit /etc/logrotate.d/fail2ban
/var/volatile/fail2ban.log {

    weekly
    rotate 1
    compress

    delaycompress
    missingok
    postrotate
        fail2ban-client set logtarget /var/volatile/fail2ban.log >/dev/null
    endscript

    # If fail2ban runs as non-root it still needs to have write access
    # to logfiles.
    # create 640 fail2ban adm
    create 640 root adm
}
}}}


=== Network setup ===
If the local network '''IP segment address''' differs between development environment and usage environment, you must set it up in the file `/etc/resolv.conf`, in order to get in both cases a '''name server''' access. In my case the file looks like :
{{{
search fritz box
nameserver 192.168.178.1
nameserver 192.168.17.1
}}}

=== Dynamic DNS ===
For access via Internet, a '''Dynamic DNS Server''' was needed. In order to have a '''.de domain''' the provider [[https://www.twodns.de|twodns.de]] was selected. The login data are stored in '''!KeePass'''. The '''domain name''' in the '''Fritz!Box''' parameter window must be '''peterswiki.dd-dns'''. For security reason, the '''token''' is used. Parameters:
{{{
# IP-address
$ sudo ifconfig
  inet address: 192.168.178.29

# port mapping:
  22 SSH (remote shell)
  80 http (web server, wiki, TempMess)
  8080 http (web server, volkszaehler)
}}}

=== Replication of the Graphs ===
I wanted to show the actual '''Graphs''' also on another moin wiki. The setup was done with:

 * Place a '''symbolic link''' of the '''Graph''' file (source) into the moin web static folder. It is easiest done with the '''mc''' (Midnight Commander).

{{{
e.g. moin197/MoinMoin/web/static/htdocs/temp1d.png (symbolic link)
}}}
 * Embed in the local wiki the remote static '''Graph''' file, please see HelpOnConfiguration (url_prefix_static) and HelpOnLinking#Embedding. Unfortunately the link to the source is moin '''version dependent''', so keep in mind.

{{{
e.g. {{http://www.peterswiki.dd-dns.de/moin_static197/temp1d.png}}
}}}
=== Links ===
 1. [[http://www.ibm.com/developerworks/library/l-bash-parameters/|Bash parameter examples]] - good examples
 1. [[http://www.linuxjournal.com/article/8600|Pass on Passwords with scp]]
 1. [[http://kittblog.com/article/6/webserver/syslog-pam-unix-cron-session-session-opened-closed-for-user-root-by-uid-0-deaktivieren/545/syslog-pam-unix-cron-session-session-opened-closed-for-user-root-by-uid-0-deaktivieren/|deactivate syslog pam cron_unix session]]
 1. [[https://www.twodns.de|Dynamic DNS Provider twodns.de]]

=== List of pages in this category: ===
<<FullSearch(category:CategoryDockStar)>>

-- RudolfReuter <<DateTime(2013-01-06T18:55:50Z)>>

----
Go back to CategoryDockStar or FrontPage ; KontaktEmail (ContactEmail)

..

Temperature Measurement

The target was to collect weather data via radio sensors, display them via web interface and monitor the inside temperature for a specified minimum and maximum threshold. In case of leaving the allowed bandwidth, an email will be send for an alarm.

The resulting temperature and humidity graphs are made over a time span of a day and week, please see the links at the top right of this page.

Quick help

  1. shut down computer for maintenance

$ sudo shutdown now
  1. reboot computer

$ sudo shutdown -r now

Components

The used computer should be a low cost Linux capable device with network connection and USB interface. Actually a Seagate Freeagent Dockstar box is used, please see CategoryDockStar. A Raspberry Pi may also be usable.

The computer network port must be connected to a router, in order to allow Internet access.

The web interface is realized with the wiki software http://moinmo.in.

The radio sensors for temperature and humidity are named S300TH, please see here for reference.

The radio receiver with USB port is the module WDE1 (Wetter Daten Empfänger 1 = weather data receiver 1) from company ELV, please see here for reference.

The logical channels of the sensors/transmitters in this application are the following:

  • Channel 5 (1-8) - in house
  • Channel 6 (1-8) - outside

WDE1 Superhet

attachment:WDE1_Superhet_DSC05885.jpg

In the original ELV WDE1 receiver (868.35 MHz) there was a super-regenerative receiver. This was good up to the time, LTE mobile radio appeared. Now the large bandwidth of the receiver is a problem.

The solution to this problem is the RX868SH-DV receiver. You can retrofit an old WDE1 with it. It is in principle pin compatible to the old receiver, but needs a jumper between EN and +UB. Instead of winding the short antenna wire to fit the housing, I put the wire straight out, see the picture on the right.

The new USB receiver WDE1-2 from ELV already comes with the new superhet receiver included.

attachment:ASH2200_DSC05886.jpg

Temperature sensor

The low cost temperature and humidity sensor/transmitter S300TH is no longer available (2015-05). Now you can buy the ASH2200 sensor/transmitter from ELV for about 30 EUR, see the picture on the right.

Software

The software should be build with available modules, as much as possible. What was collected:

  • Bash shell script wde1_read.sh for data read in, started from crontab.

  • Database rrdtool for collecting the data in weather.rrd.

  • Shell script rrd_graph_all.sh for updating the graphs every 15 minutes via cron.

  • Python program temp_alarm.py to check regularly the temperature limit and sending an email alarm.

  • Wiki software moinmoin for web interface.

graphviz-SomeGraph-ea319bef754f3470a394796cc41a5e3b8763e4db.png

Database setup

If not already done install package rrdtool.

# for example:
$ sudo apt-get install rrdtool

Before collecting data, the RRD (Round Robin database) must be setup first. This is done with the script file rrd_create_wde1.sh:

#!/bin/sh
rrdtool create weather.rrd --step 300 \
DS:temps1:GAUGE:1200:-40:50 \
DS:temps2:GAUGE:1200:-40:50 \
DS:temps3:GAUGE:1200:-40:50 \
DS:temps4:GAUGE:1200:-40:50 \
DS:temps5:GAUGE:1200:-40:50 \
DS:temps6:GAUGE:1200:-40:50 \
DS:temps7:GAUGE:1200:-40:50 \
DS:temps8:GAUGE:1200:-40:50 \
DS:hums1:GAUGE:1200:0:100 \
DS:hums2:GAUGE:1200:0:100 \
DS:hums3:GAUGE:1200:0:100 \
DS:hums4:GAUGE:1200:0:100 \
DS:hums5:GAUGE:1200:0:100 \
DS:hums6:GAUGE:1200:0:100 \
DS:hums7:GAUGE:1200:0:100 \
DS:hums8:GAUGE:1200:0:100 \
DS:temps9:GAUGE:1200:-40:50 \
DS:hums9:GAUGE:1200:0:100 \
DS:winds9:GAUGE:1200:0:200 \
DS:rains9:DERIVE:1200:0:U \
DS:israins9:GAUGE:1200:0:1 \
RRA:AVERAGE:0.5:1:4032 \
RRA:MIN:0.5:96:3600 \
RRA:MAX:0.5:96:3600 \
RRA:AVERAGE:0.5:96:7300
# Datensatz alle 5 min: step 300
# 14 Tage Einzelwerte: 12 x 24 x 14 = AVERAGE 4032
# 20 Jahre Mittelwerte: 365 x 20 = AVERAGE 7300

Database conversion

If the database file should be copied to another computer, it may give an error:

  • ERROR: This RRD was created on another architecture

Then the database must be converted:

# Dump database on source computer
$ rrdtool dump weather.rrd >weather.xml

# Copy XML file to target computer

# Restore database on target computer
$ rrdtool restore weather.xml weather.rrd

WDE1 data read, crontab

  • Start method: by crontab line

In order to access device /dev/ttyUSB0 you have to be a member of the group dialout, if not, you will get a permission problem, please see TempMess#Test_WDE1_Data_Reception .

The WDE1 weather data are read in via shell script wde1_read.sh.

#!/bin/bash
# File: wde1_read.sh
# Receive remote weather data from USB-WDE1 and store into RRD database
# Start with crontab
# Adopted from: http://www.kompf.de/weather/technik.html
# 2013-01-08 RudolfReuter

cd $HOME/Install/rrdtool

# Read data from USB-WDE1
# Setup serial port parameters
stty -F /dev/ttyUSB0 cstopb -ixon raw speed 9600 > /dev/null

read line < /dev/ttyUSB0

# Check for header = "$1"
if [[ "${line%%;*}" == '$1' ]] ; then
    #echo $line > wde1_raw.dat
    # remove trailing "0", stop character
    line=`echo "${line%?}"`
    # format data, remove prefix "$1;1;", replace ";," by ":."
    tmp=`echo "${line#?1;1;}" | tr ';,' ':.'`
    # 1. add prefix "N" (date stamp), remove trailing 0
    # 2. substitude all (g=global) strings "::" by ":U:"
    # 3. again point 2., why?
    data=`echo "N${tmp%%0}" | sed 's/::/:U:/g' | sed 's/::/:U:/g'`
    # remove trailing ":"
    data=${data%%:}
    # save last data values
    echo $data > rrd_data.dat
    # update rrdmc
    rrdtool update weather.rrd $data
    # save last indoor temperature for alarm compare
    cut -d ":" -f 6 rrd_data.dat > temp5.dat
fi

Note: The bash shell must be used, in order to work properly.

The shell script is started with crontab:

$ crontab -e

# append
# read every 5 minutes the temp. and hum. values from WDE1
*/5 * * * * $HOME/Install/rrdtool/wde1_read.sh &> /dev/null

# check with
$ crontab -l

# for a manual test do
$ ./wde1_read.sh

Cron System Mail

In case of an error in the crontab call, you will get a system mail, announced at the next RETURN on the command line. Then check your system mail for the error message:

# Announcement of a system mail
Sie haben Post in /var/mail/user.
$ ls -ls

# check system mail
$ mail
"/var/mail/user": 40 messages 2 new 5 unread
...
N 40 Cron Daemon Wed Jan 9 06:11 19/705 Cron <user@rudiswiki> $HOME/Install/rrdtool/wde1_read.sh >

# Read mail
? RETURN
...
ERROR: weather.rrd: found extra data on update argument: 0

# delete mail
? d

# next mail
? n
...
At EOF

# Quit mail program
? q

Cron logging disable

Normally cron logs every job. In order to avoid that, you can modify the log level:

# example: Apr 10 18:03:01 FADS11 cron.info /USR/SBIN/CRON[3209]: (peter) CMD ($HOME/Install/rrdtool/rrd_graph_all.sh > /dev/null)

# Edit file /etc/default/cron
# add "-L 0" for no logging (errors are logged regardless)
EXTRA_OPTS="-L 0"

# restart cron
sudo /etc/init.d/cron restart

Additionally there is a message: "pam_unix(cron:session): session opened for user", which can be suppressed by:

# Edit file /etc/pam.d/common-session-noninteractive
# Find line "session required pam_unix.so"
# add a line before
session [success=1 default=ignore] pam_succeed_if.so service in cron quiet use_uid

# restart cron
$ sudo invoke-rc.d cron restart

Sensor channels

The WDE1 receiver does have the capacity for 8 different sensors.

  • Channel 5 is used for Indoor temperature and humitdity.

  • Channel 6 is used for Outdoor temperature and humitdity.

Generate data graphs

Now you have some data in the database, next you will see in a graph how the trend is.

For me it was the easiest method to update all useful graphs every 15 minutes and display them in a web interface (wiki moinmoin). This is done with the script rrd_graph_all.sh:

#!/bin/sh
# file: rrd_graph_all.sh
# exec all rrd_graph* files at once, for cron
# 2013-01-07 RudolfReuter

cd $HOME/Install/rrdtool/
./rrd_graph_h1d.sh
./rrd_graph_h1w.sh
./rrd_graph_t1d.sh
./rrd_graph_t1w.sh

# copy graphs to rudsiwiki.de
scp  -i $HOME/.ssh/id_rsa *.png rudi@www.rudiswiki.de:Install/rrdtool

The crontab entry was setup with:

$ crontab -e

# append the following lines:
# run every 15 minutes, but with offset of 3 minutes
3,18,33,48 * * * * $HOME/Install/rrdtool/rrd_graph_all.sh &> /dev/null

# check with:
$ crontab -l

The single graphs are defined with the following 4 files.

#!/bin/sh
# File: rrd_graph_h1d.sh
rrdtool graph hum1d.png --end now --start end-1d --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n"

#!/bin/sh
# File: rrd_graph_h1w.sh
rrdtool graph hum1w.png --end now --start end-1w --height 125 \
DEF:hums5=weather.rrd:hums5:AVERAGE \
LINE2:hums5#000000:Raum. \
DEF:min=weather.rrd:hums5:MIN \
DEF:max=weather.rrd:hums5:MAX \
VDEF:First=hums5,FIRST \
GPRINT:hums5:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums5:AVERAGE:"Avg %3.0lf" \
GPRINT:hums5:MAX:"Max %3.0lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:hums6=weather.rrd:hums6:AVERAGE \
LINE2:hums6#FF0000:Außen \
DEF:min2=weather.rrd:hums6:MIN \
DEF:max2=weather.rrd:hums6:MAX \
GPRINT:hums6:MIN:"Humidity %% Min %3.0lf" \
GPRINT:hums6:AVERAGE:"Avg %3.0lf" \
GPRINT:hums6:MAX:"Max %3.0lf" \
COMMENT:"\\n"

#!/bin/sh
# File: rrd_graph_t1d.sh
rrdtool graph temp1d.png --end now --start end-1d --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
GPRINT:temps6:LAST:"Cur %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n"

#!/bin/sh
# File: rrd_graph_t1w.sh
rrdtool graph temp1w.png --end now --start end-1w --height 125 \
DEF:temps5=weather.rrd:temps5:AVERAGE \
LINE2:temps5#000000:Raum. \
DEF:min=weather.rrd:temps5:MIN \
DEF:max=weather.rrd:temps5:MAX \
VDEF:First=temps5,FIRST \
GPRINT:temps5:MIN:"Temp. Min %5.1lf" \
GPRINT:temps5:AVERAGE:"Avg %5.1lf" \
GPRINT:temps5:MAX:"Max %5.1lf" \
GPRINT:First:"from %Y-%m-%d\\n":strftime \
DEF:temps6=weather.rrd:temps6:AVERAGE \
LINE2:temps6#FF0000:Außen \
DEF:min2=weather.rrd:temps6:MIN \
DEF:max2=weather.rrd:temps6:MAX \
GPRINT:temps6:MIN:"Temp. Min %5.1lf" \
GPRINT:temps6:AVERAGE:"Avg %5.1lf" \
GPRINT:temps6:MAX:"Max %5.1lf" \
HRULE:0#0000ff \
COMMENT:"\\n"

Check for Alarm

  • Start method: by crontab line

If not already done install package python-rrdtool.

# for example:
$ sudo apt-get install python-rrdtool

The indoor temperature check for exceeding the thresholds is done with a Python program temp_alarm.py.

In case of a transition from a temperature value inside the bandwidth to outside, an alert email is send. In order to avoid unnecessary alert emails, a hysteresis is used.

All parameters are stored in a configuration file temp_alarm.cfg.

The state of the alert is written in file temp_alarm.dat.

The crontab entry was setup with:

$ crontab -e

# append the following lines:
#
# run every 15 minutes, but with offset of 4 minutes
4,19,34,49 * * * * $HOME/Install/rrdtool/temp_alarm.py &> /dev/null

# check with:
$ crontab -l

The program is not so long, therefore it is listed here:

   1 #!/usr/bin/env python
   2 # -*- coding: utf-8 -*-
   3 # File:      temp_alarm.py
   4 # Author:    RudolfReuter
   5 # Copyright: GPL2
   6 # Source:    http://www.rudiswiki.de
   7 # Version:   2013-01-09
   8 # Version:   2013-05-30, catch "string to float" error in line 98
   9 # Features:
  10 # - computer: Dockstar-Debian
  11 # - Email and Text Alerts for High and Low Temperatures
  12 # - Data Logging and Analysis using RRDTool
  13 # - Custom Configuration parameter
  14 #------------------------------------------
  15 # Import Libaries
  16 import datetime
  17 from smtplib import SMTP
  18 import time
  19 import ConfigParser        # to read cfg file
  20 from email.MIMEText import MIMEText
  21 import sys                 # for exit
  22 import os                  # for file delete, cd
  23 
  24 #----- Functions -------------------
  25 
  26 def email_alert(subject, message):
  27     msg = MIMEText(message + ", see http://www.peterswiki.dd-dns.de")
  28     msg['From']    = myEmailFrom
  29     msg['To']      = myEmailTo
  30     msg['Subject'] = subject
  31     server = SMTP(myEmailSMTPsrv)
  32     server.login(myEmailFrom, myEmailPwd)
  33     try:
  34         print "EmailAlert"
  35         server.sendmail(myEmailFrom, myEmailTo, msg.as_string())
  36     finally:
  37         server.quit()
  38     try:
  39         fpw = open("temp_alert.dat","w")
  40     except:
  41         print ("Error: File open AlertSend.dat failed")
  42         sys.exit(0)
  43     fpw.write(message)
  44     fpw.close()
  45 
  46 # Begin main program --------------------
  47 
  48 # If called from crontab, change directory is needed.
  49 try:
  50     home = os.environ['HOME']
  51     os.chdir(home + "/Install/rrdtool")
  52     print(os.getcwd())
  53 except:
  54     print ("Error: cd HOME does not work")
  55     sys.exit(0)
  56 
  57 # Read in configuration file using Config Parser
  58 aConfig = ConfigParser.RawConfigParser()
  59 aConfig.read('temp_alarm.cfg')
  60 
  61 myAlertLabel = aConfig.get('cfg', 'alertlabel')
  62 
  63 # Alert level for Temperature
  64 alertHighTemp = aConfig.getfloat('cfg', 'alerthightemp')
  65 alertLowTemp  = aConfig.getfloat('cfg', 'alertlowtemp')
  66 alertHyst     = aConfig.getfloat('cfg', 'alerthyst')
  67 
  68 # Email parameters
  69 myEmailFrom    = aConfig.get('cfg', 'emailfrom')
  70 myEmailPwd     = aConfig.get('cfg', 'emailpwd')
  71 myEmailSMTPsrv = aConfig.get('cfg', 'emailsmtpsrv')
  72 myEmailTo      = aConfig.get('cfg', 'emailto')
  73 
  74 DateNow = time.strftime('%Y-%m-%d %H:%M:%S', (time.localtime(time.time())))
  75 
  76 # Read the actual indoor temperature
  77 try:
  78     fpr = open("temp5.dat")
  79 except:
  80     print ("Error at open temp5.dat")
  81     sys.exit(0)
  82 temp5 = fpr.readline()
  83 fpr.close()
  84 try:
  85     ftemp5 = float(temp5)
  86 except ValueError,e:
  87     # fake temperature in case of receive error
  88     ftemp5 = 20.0
  89 print("Temperature: " + temp5)
  90 
  91 # Check for Alert recovery
  92 try:
  93     fp = open("temp_alert.dat")
  94     # Check for temperature recovery with hysteris
  95     if ftemp5 > (alertLowTemp + alertHyst):
  96         if ftemp5 < (alertHighTemp - alertHyst):
  97             try:
  98                 os.remove("temp_alert.dat")
  99             except:
 100                 print("Error: File AlertSend.dat delete failed")
 101                 sys.exit(0)
 102 except:
 103     # Alerts checking for high and low temp
 104     if ftemp5 < 50.0:
 105         if ftemp5 < alertLowTemp:
 106             print(DateNow + " Alert: Low Temp °C " + temp5)
 107             email_alert(myAlertLabel, DateNow + " Low Temperature: " + temp5)
 108         if ftemp5 > alertHighTemp:
 109             print(DateNow + " Alert: High Temp °C " + temp5)
 110             email_alert(myAlertLabel, DateNow + " High Temperature: " + temp5)

The configuration file is used to change quickly parameters, with no need to touch the program.

[cfg]
alertlabel = Temperatur Alarm Vogelhaus
alerthightemp = 30.0
alertlowtemp = 15.0
alerthyst = 1.0
emailto = reuterx@xxxx.de
emailfrom = reuterx@xxxx.de
emailpwd = xxx
emailsmtpsrv = mail.xxx.de

Web Interface

The web interface is realized with the wiki software moinmoin, see MoinSetup.

Test WDE1 Data Reception

Connect the WDE1 to the USB plug and check if it is recognized:

# see if the USB device is seen
$ ls -ls /dev/ttyU*
0 crw-rw---- 1 root dialout 188, 0 Dez 26  2012 /dev/ttyUSB0

# check if your user is in the group "dialout"
$ groups

# if the user is not in the group "dialout", add it
$ sudo adduser user dialout

# logout and login to have the group attached to the user
$ groups

Check if you have reception of the sensors:

# if utility "socat" is not installed, do so
$ sudo apt-get install socat

# check for the data telegram
# 2 sensors at address 5 and 6
$ socat /dev/ttyUSB0,b9600 stdout
$1;1;;;;;;20,5;6,8;;;;;;;57;84;;;;;;;;0

# inside temperature = 20,5 °C, outside temperature =  6,8 °C
# inside humidity    = 57 %,    outside humidity   =  84 %

# some times I will receive also a foreign "Kombi sensor" from the neighborhood:
$1;1;;;;;;20,4;7,3;;;;;;;57;84;;;6,1;73;0,0;550;0;0

# Kombi Temperature = 6,1 °C, Kombi Humidity = 73 %
# Kombi Wind speed 0.0 kmph, Kombi Rain volume = 550 rocker beats
# Kombi Rain = 0 = no

Troubleshooting WDE1

If the USB device WDE1 is not seen:

# check for needed kernel modules
$ lsmod
...
cp210x                 17514  0
usbserial              37173  1 cp210x
...

# check for USB device
$ lsusb
...
Bus 002 Device 004: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP210x Composite Device

Copy Graphs per scp

In order to show the graphs also on a remote computer, you can copy the .png graphs to the remote computer with scp. In order to avoid the password question in a script file you can use an ssh ident file (key):

# generate public key
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/peter/.ssh/id_rsa): ENTER
Enter passphrase (empty for no passphrase): ENTER
Enter same passphrase again: ENTER
Your identification has been saved in /home/peter/.ssh/id_rsa.
Your public key has been saved in /home/peter/.ssh/id_rsa.pub.
The key fingerprint is:
df:0e:12:85:c6:ac:c2:21:e4:7c:18:12:31:74:e4:93 peter@FADS11
The key's randomart image is:
+--[ RSA 2048]----+
...

# on the remote computer, a folder /home/rudi/.ssh should already exist, otherwise make one with:
$ ssh rudi@192.168.17.72 mkdir .ssh/
# enter password

# transfer public key, do NOT overwrite an already existing file "authorized_keys"
$ ssh-copy-id -i $HOME/.ssh/id_rsa.pub rudi@192.168.17.72
# enter password
or
$ cat $HOME/.ssh/id_rsa.pub | ssh rudi@192.168.17.72 "cat >>.ssh/authorized_keys"
# enter password

# copy .png files to /home/rudi/Install/rrdtool/
$ cd Install/rrdtool
# Test with internal network
$ scp -i $HOME/.ssh/id_rsa *.png rudi@192.168.17.72:Install/rrdtool

# Test with internet access
$ scp -i $HOME/.ssh/id_rsa *.png rudi@www.rudiswiki.de:Install/rrdtool

# append the previous line to the file "rdd_graph_all.sh" to get an automatic refresh

Problem solving, see at Re: SSH key not working for 1 account but working for another:

# If you get still asked the password of the remote server, have a look at remote /var/log/auth.log
# If you see: "Authentication refused: bad ownership or modes for directory /home/user"
# you will have TOO much permissions on your home directory -> it must be 755
# change with:
$ chmod 755 /home/user

Start methods at boot

The start of the bash script wde1_read2.sh at boot time, running in an endless loop was not reliable. But I will describe two easy methods to start the script at boot time, just for reference. The start|stop method in /etc/init.d was too much effort for this job.

  • Start method: by /etc/rc.local

The shell script is started at boot time with a line in file /etc/rc.local:

start-stop-daemon --background --chuid user:group --start  --exec /home/user/Install/rrdtool/wde1_read2.sh --chdir /home/user/Install/rrdtool

# for a manual start do
$ sudo /etc/rc.local

Note: The shell script should be started with user rights (--chuid) in a specified folder (--chdir) and must run in the background (--background).

  • Start method: by crontab

$ crontab -e

# append
# start once at boot time
@reboot $HOME/Install/rrdtool/wde1_read2.sh > /dev/null

Note: Take care to make a change directory in the script to the working directory, or use an absolute path.

Maintenance

Installed fail2ban because of break in trials, see setup of fail2ban.

  • check regular /var/volatile/fail2ban.log

  • because the busybox-syslogd normally has a RAM circular buffer (-C128) fail2ban can not check the log. This has to be changed to a file logging in a RAM-disk, in order to reduce the access to the USB flash.

# edit /etc/default/busybox-syslogd
SYSLOG_OPTS="-O /var/volatile/messages"

# append to file /etc/fstab
tmpfs          /var/volatile   tmpfs   defaults,size=10%,mode=755 0       0
# after a restart the RAM-disk is working.
  • Unfortunately the busybox-syslogd was compiled without the possible logrotate feature, the package logrotate must be installed. The configuration file for the messages logging looks like:

# edit file /etc/logrotate.d/messages
/var/volatile/messages {
    daily
    rotate 1
    copytruncate
    missingok
    sharedscripts
    postrotate
        day=$(date +%Y-%m-%d)
        # Path "~" will give "/root" instead of "/home/rudi"!
        mv /var/volatile/messages.1 /home/peter/log/messages-$day.log
    endscript

}

So, a daily copy of the messages is kept in the user folder log.

  • The fail2ban.log is also rotated, but only the last copy is kept. The configuration file looks like:

# edit /etc/logrotate.d/fail2ban
/var/volatile/fail2ban.log {

    weekly
    rotate 1
    compress

    delaycompress
    missingok
    postrotate
        fail2ban-client set logtarget /var/volatile/fail2ban.log >/dev/null
    endscript

    # If fail2ban runs as non-root it still needs to have write access
    # to logfiles.
    # create 640 fail2ban adm
    create 640 root adm
}

Network setup

If the local network IP segment address differs between development environment and usage environment, you must set it up in the file /etc/resolv.conf, in order to get in both cases a name server access. In my case the file looks like :

search fritz box
nameserver 192.168.178.1
nameserver 192.168.17.1

Dynamic DNS

For access via Internet, a Dynamic DNS Server was needed. In order to have a .de domain the provider twodns.de was selected. The login data are stored in KeePass. The domain name in the Fritz!Box parameter window must be peterswiki.dd-dns. For security reason, the token is used. Parameters:

# IP-address
$ sudo ifconfig
  inet address: 192.168.178.29

# port mapping:
  22 SSH (remote shell)
  80 http (web server, wiki, TempMess)
  8080 http (web server, volkszaehler)

Replication of the Graphs

I wanted to show the actual Graphs also on another moin wiki. The setup was done with:

  • Place a symbolic link of the Graph file (source) into the moin web static folder. It is easiest done with the mc (Midnight Commander).

e.g. moin197/MoinMoin/web/static/htdocs/temp1d.png (symbolic link)
  • Embed in the local wiki the remote static Graph file, please see HelpOnConfiguration (url_prefix_static) and HelpOnLinking#Embedding. Unfortunately the link to the source is moin version dependent, so keep in mind.

e.g. {{http://www.peterswiki.dd-dns.de/moin_static197/temp1d.png}}

  1. Bash parameter examples - good examples

  2. Pass on Passwords with scp

  3. deactivate syslog pam cron_unix session

  4. Dynamic DNS Provider twodns.de

List of pages in this category:

-- RudolfReuter 2013-01-06 18:55:50


Go back to CategoryDockStar or FrontPage ; KontaktEmail (ContactEmail)

TempMess (last edited 2019-07-31 20:03:04 by RudolfReuter)