Redirecting Weather Station Data from ObserverIP
Last Updated 2/26/2017
I have an Ambient Weather WS-1200-IP weather station. It’s a great unit and very easy to use and setup! There is an outdoor unit which does temperature, humidity, wind speed, wind direction, rainfall, UV and solar radiation. There’s also an indoor unit which has a barometer sensor. The 2 sensors talk to an ObserverIP unit. This unit plugs directly into your router and has a little webpage that you configure your wunderground.com username with. Once you click save, it instantly starts uploading data to wunderground.com. Couldn’t be easier to start sharing your weather station data!
Except. I want that data. I want to archive it on my own system. I want to be able to report off it. The little web interface on the ObserverIP does have a live data page – but that’s it. No way to change the server name, nothing! You can telnet to the ObserverIP, but the options don’t work. Everything is locked in the firmware.
Note: this method should apply to any weather station that uses the ObserverIP module.
Table of Contents:
- Method 1: Configure the ObserverIP destination hostname using telnet
- Method 2: Redirect traffic using a router and iptables
- Accepting the data on your server
- Creating a socket for use with weewx
- Undoing your changes and returning back to defaults
- How I use this data to run my own website!
Method 1: Configure the ObserverIP destination hostname using telnet
This method may or may not work (see my note in Method 2) depending on your ObserverIP type. Let’s hope it works, since it’s the easiest way to redirect the weather data. This method will redirect the data right from the ObserverIP to your web server. Your web server can be on your LAN (in your home) or a server on the internet. Using a program like PuTTY, you need to telnet to the ObserverIP, then run these commands, setdsthn yourserver_ip_or_hostname, saveconfig, reboot. Remember to change yourserver_ip_or_hostname to a valid value (like 192.168.0.100 or yourserver.com).
Connected to 192.168.0.50.
Escape character is '^]'.
username: admin
password: admin
Successful login through telnet
telnet> setdsthn yourserver_ip_or_hostname
Ok
telnet> saveconfig
Ok
telnet> reboot
Connection closed by foreign host.
Continue to the next section titled Accepting the data on your server
Method 2: Redirect traffic using a router and iptables
When I initially wrote this blog in January 2016, the setdsthn didn’t work for me. I don’t know why – maybe I didn’t run saveconfig? When I revisited this in June 2016, the setdsthn worked for me, so that’s the solution I’m using now. Either way, I originally used this solution below, and it worked great!
I found myself poking around the weewx user group, just aimlessly browsing, when I came across this question that Sergei had. It doesn’t have anything to do with the ObserverIP, but it contained 2 pieces of information that I could use. I knew instantly I had my solution. Now to “translate” it to my needs. Two steps: redirect ObserverIP’s traffic, and have it go to a socket server.
All done without modifying any the ObserverIP’s core settings. No modifications are made to how the ObserverIP works. No warranty voiding or anything like that. This is all external to the weather station’s operation.
What you will need is a router capable of running dd-wrt. dd-wrt is custom firmware that unlocks more potential for your router. Think of it like a little Linux server running on your router. You can also accomplish this with a device that runs OpenWRT (such as the TP-LINK MR3020, however this write up will only cover the dd-wrt firmware).
I had recently upgraded my Linksys WRT54GL to a newer Asus wifi router, so the Linksys was a spare device collecting dust. It already had DD-WRT firmware loaded on it, this was a perfect fit for this project! I plugged the Linksys into my primary Asus router. Then I plugged my ObserverIP into the Linksys. The Linksys becomes the middleman and does the heavy lifting to redirect the data.
Here’s an awesome picture I made that describes what I mean.
Once you have it wired, and you’re able to log into the dd-wrt web interface, you need to give the ObserverIP a static DHCP reservation from the dd-wrt web interface. This is so every time the Linksys or the ObserverIP reboots, it gets the same IP.
Note: Unless you’re doing advanced static routing for multiple subnets, you will need to plug a laptop or PC into the Linksys to complete the steps below.
- From within the dd-wrt web interface, go to Services menu > Services sub menu > DHCP server > Static Leases. Just enter in the mac address of the ObserverIP, a name and IP you want to use. Click apply settings at the bottom and reboot the ObserverIP. It should come back up as the IP you just gave it.
Once you’ve done that, we need to apply some firewall rules:
- From within the dd-wrt web interface go to the Administration menu > Commands sub menu. Enter this into the box and click Save Firewall. Note; you will need to edit this to match your setup
iptables -t nat -A PREROUTING -s 192.168.1.55 -p tcp --dport 80 -j DNAT --to-destination x.x.x.x:80
iptables -t nat -A POSTROUTING -j MASQUERADE
- Replace 192.168.1.55 with the IP of your ObserverIP, and replace x.x.x.x:80 with the IP of your web server and port you want the traffic to be redirected to. This can be a public web server, or a private one on your LAN.
The first line says that ANY TCP TRAFFIC ON PORT 80 from 192.168.1.55, redirect it to x.x.x.x:80. Since we know that the wunderground API says to use port 80 to upload sensor readings, we want to catch and redirect traffic on port 80 from the ObserverIP.
Once you click Save Firewall, it takes about a minute or so, but you should start to see traffic coming in on your webserver (check the access and/or error logs to verify)! Note: this traffic coming in contains your wunderground username & password. Be sure the IP you’re redirecting to is yours!
On your webserver open your web logs, you should some new traffic coming in looking like
GET /weatherstation/updateweatherstation.php?ID=
Accepting the data on your server
Now that the traffic is coming in (from Method 1 or Method 2), we need to do something with it! You’ll see that it begins with getting a specific directory and file. We need to emulate this in order for the ObserverIP to be happy. On your webserver, make a new folder at the root level called weatherstation, and create a new file called updateweatherstation.php. Inside that PHP file, place this as the first couple of lines:
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);
echo $wunderground;
What is happening here is when the ObserverIP traffic is redirected, it’ll access this PHP file on your webserver. This PHP line is forwarding the entire request right to wunderground straight away so you don’t disrupt your wunderground readings. The result is being echoed, and it should say “success”. You will know if this is working because the “Server” light on your ObserverIP should come on. This indicates everything is being forwarded correctly.
Note: this PHP snippet does not have any security applied. See the below snippet on how to apply security using the ID & PASSWORD from the ObserverIP update string.
If you’ve gotten this far and you see data in your web server logs, and the ObserverIP has the “Server” light on, then that’s it! You’ve successfully redirected the weather data! It’s now yours to do whatever you want with it (e.g. send to a website, log to a file, send to MySQL), all while not disrupting your wunderground setup.
If you want to import this data into weewx, continue to the next section: Creating a socket for use with weewx
Creating a socket for use with weewx
Now that I finally have my weather data available to me, it’s time to get it into weewx. I had mentioned that question Sergei asked contained 2 pieces of information I needed. The first was the iptables redirect, and the second was from mwall talking about the hackulink driver. I forked a copy of that hackulink driver and created my own. I named my weewx driver weewx-socketlogger.
There’s some prerequisites to getting it running:
Step 1: Install a socket server. There’s a bunch out there, and even Python can create one for you. To keep it simple I used ncat. So, install ncat. I use CentOS, so yum install nmap. On Ubuntu you can run sudo apt-get install nmap to get ncat.
As a test, run ncat with these options:
ncat -l 2999 --keep-open --broker
Then connect to it using telnet to see if it works: telnet localhost 2999 If it connects, you’re in good shape!
This has ncat start a socket server on port 2999, keep it open even if clients disconnect, and broker the data so that all clients can see what the other clients are saying.
Now we need ncat to run on startup. To do this, I just ran it from my local user’s crontab. You can use /etc/rc.local if you’d like, but I found this to be easier.
Run crontab -e and add
@reboot /usr/bin/ncat -l 2999 --keep-open --broker >/dev/null 2>&1
This tells the server that on reboot, to start ncat with the same options as above, but run it in the background. That’s it!
To verify if it’s working you can run the command sudo netstat -tulpn | grep 2999 and you should see output similar to this:
tcp 0 0 0.0.0.0:2999 0.0.0.0:* LISTEN 1166/ncat
tcp6 0 0 :::2999 :::* LISTEN 1166/ncat
Step 2: Edit the updateweatherstation.php file so that on every ObserverIP update, it not only sends data to wunderground, but also to the socket that weewx will connect to. My full updateweatherstation.php file looks like this. Be sure to change the YOUR_WUNDERGROUND_USERNAME and YOUR_WUNDERGROUND_PASSWORD to match what you saw in the access logs from earlier. This security method is minor, but it helps to prevent any bots out there from submitting false weather data to your archives.
<?php
// Parse the wunderground packets sent from the ObserverIP
// Pat O'Brien, 1/14/2016
// The ObserverIP does not provide heat index data in its packet, this will calculate it
// Formula from NOAA http://www.wpc.ncep.noaa.gov/html/heatindex_equation.shtml
function heatindex($T, $RH) {
$HI = -42.379 + 2.04901523*$T + 10.14333127*$RH - .22475541*$T*$RH - .00683783*$T*$T - .05481717*$RH*$RH + .00122874*$T*$T*$RH + .00085282*$T*$RH*$RH - .00000199*$T*$T*$RH*$RH;
return round( $HI, 1 );
}
// Make sure the ID and PASSWORD fields match what we think it should match. If it does, this update is valid and continue.
if ( ($_GET["ID"] == "YOUR_WUNDERGROUND_USERNAME") && ($_GET["PASSWORD"] == "YOUR_WUNDERGROUND_PASSWORD") ) {
//============================================//
// Auto send data query right to wunderground //
//============================================//
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);
echo $wunderground;
//==============================================//
// Write data to ncat socket for use with weewx //
//==============================================//
$heatindex = heatindex($_GET['tempf'], $_GET['humidity']);
$socket = "outTemp=".$_GET['tempf'].",";
$socket .= "outHumidity=".$_GET['humidity'].",";
$socket .= "dewpoint=".$_GET['dewptf'].",";
$socket .= "windchill=".$_GET['windchillf'].",";
//$socket .= "heatindex=".$heatindex.","; // Enable heat index if you need it. WeeWx and other systems will provide this for you.
$socket .= "windDir=".$_GET['winddir'].",";
$socket .= "windSpeed=".$_GET['windspeedmph'].",";
$socket .= "windGust=".$_GET['windgustmph'].",";
//$socket .= "rain=".$_GET['rainin'].","; // This is really rain rate
$socket .= "dailyrain=".$_GET['dailyrainin'].",";
$socket .= "radiation=".$_GET['solarradiation'].",";
$socket .= "UV=".$_GET['UV'].",";
$socket .= "inTemp=".$_GET['indoortempf'].",";
$socket .= "inHumidity=".$_GET['indoorhumidity'].",";
$socket .= "barometer=".$_GET['baromin'].",";
$socket .= "epoch=".time().",";
$socket .= "txBatteryStatus=".$_GET['lowbatt']; // Last item, no comma
$socket .= PHP_EOL;
// Send data to the socket middle man for weewx to archive
$fp = fsockopen("localhost", 2999);
if ($fp) {
fwrite($fp, $socket);
fclose ($fp);
}
} else {
header('HTTP/1.0 401 Unauthorized');
echo 'failure';
die();
}
?>
With this added to our PHP file, you can telnet to your socket on port 2999 and watch the data come in. This is the same data that weewx will use.
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.1,windDir=255,windSpeed=4.03,windGust=7.38,dailyrain=0.00,radiation=448.98,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=336,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=451.75,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.7,windDir=249,windSpeed=3.58,windGust=4.92,dailyrain=0.00,radiation=461.08,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=339,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=462.55,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=319,windSpeed=2.46,windGust=7.38,dailyrain=0.00,radiation=460.68,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=15.3,windDir=316,windSpeed=9.84,windGust=14.99,dailyrain=0.00,radiation=469.42,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=14.7,windDir=304,windSpeed=10.74,windGust=12.30,dailyrain=0.00,radiation=478.19,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=17.4,windDir=334,windSpeed=6.71,windGust=7.38,dailyrain=0.00,radiation=471.64,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
If your telnet connection looks like the above, then you’re in good shape! PHP is getting the data from the ObserverIP, and sending it to the socket. Now, you need to install the socketlogger weewx driver from the instructions on GitHub, restart weewx and weewx should connect to the socket and start logging things for you!
Note, there is no weewx loop interval like that’s found with other weewx drivers. That’s because every time a new update comes in from ObserverIP, PHP will accept it and send it to the socket. Weewx is connected and listening to the socket. When it sees new data on the socket, it will grab it and parse it then submit it as a packet to the archive system. Then weewx can generate its reports and/or update other weather services like CWOP, PWSWeather, your own website and more!
I changed my mind, I don’t want to do this. How do I undo it?
Method 1:
Easy! Just telnet back into your ObserverIP and run setdsthn rtupdate.wunderground.com, saveconfig, reboot. That will send the data back to wunderground’s update servers.
Method 2:
Easy! Remember, we haven’t changed anything on the ObserverIP! Unplug the Linksys and put it back in the box. Plug your ObserverIP back into your primary WiFi router (or whatever you have). That’s it! You’re back to the normal way it should be setup. You can use the IPTools software again to find your ObserverIP and manage it that way.
How I use this data to run my own website.
So now that we have control of the data, you can do anything you want! In addition to hooking this into weewx for archiving, I also have it write to a simple JSON file on every update from the station. I use this file to run some AJAX background updates on my weather website. The website checks for updates to the JSON file every 10 seconds and updates the weather data. This means that the website can show the most up-to-date information without having to refresh the page!
You can see that website here: https://belchertownweather.com
I also use Raspberry Pi model 3, with the PiTFT Plus from Adafruit, and have created a 3.5″ version of my weather website specific for the Pi. So now when the Pi boots up, it opens Chrome automatically and loads the special Pi console webpage. I can have very inexpensive display consoles throughout my house, or even on my desk at work!
The Pi console webpage has 2 major changes, the first one is the obvious size constriction. So I had to cut out some details to make everything fit. The second change is that the normal website offers background updates for only 30 minutes, then stops. This is to prevent any visitors who come to the site from eating up a lot of bandwidth. I may change that one day. The Pi console – since it’s only me for now – doesn’t have that restriction.
Here’s a screenshot of the website console on the Pi:
And a photo of the Pi itself. Once I get a case for it, I’ll update the photo.
If you have questions or want me to elaborate on a specific step, let me know in the comments!