Photo from Amazon
Last Updated 2/26/2017
I have an Ambient Weather WS-1200-IP weather station. It’s a great unit and very easy to use and setup! There is an outdoor unit which does temperature, humidity, wind speed, wind direction, rainfall, UV and solar radiation. There’s also an indoor unit which has a barometer sensor. The 2 sensors talk to an ObserverIP unit. This unit plugs directly into your router and has a little webpage that you configure your wunderground.com username with. Once you click save, it instantly starts uploading data to wunderground.com. Couldn’t be easier to start sharing your weather station data!
Except. I want that data. I want to archive it on my own system. I want to be able to report off it. The little web interface on the ObserverIP does have a live data page -- but that’s it. No way to change the server name, nothing! You can telnet to the ObserverIP, but the options don’t work. Everything is locked in the firmware.
Note: this method should apply to any weather station that uses the ObserverIP module.
Table of Contents:
- Method 1: Configure the ObserverIP destination hostname using telnet
- Method 2: Redirect traffic using a router and iptables
- Accepting the data on your server
- Creating a socket for use with weewx
- Undoing your changes and returning back to defaults
- How I use this data to run my own website!
Method 1: Configure the ObserverIP destination hostname using telnet
This method may or may not work (see my note in Method 2) depending on your ObserverIP type. Let’s hope it works, since it’s the easiest way to redirect the weather data. This method will redirect the data right from the ObserverIP to your web server. Your web server can be on your LAN (in your home) or a server on the internet. Using a program like PuTTY, you need to telnet to the ObserverIP, then run these commands, setdsthn yourserver_ip_or_hostname, saveconfig, reboot. Remember to change yourserver_ip_or_hostname to a valid value (like 192.168.0.100 or yourserver.com).
Connected to 192.168.0.50. Escape character is '^]'. username: admin password: admin Successful login through telnet telnet> setdsthn yourserver_ip_or_hostname Ok telnet> saveconfig Ok telnet> reboot Connection closed by foreign host.
Continue to the next section titled Accepting the data on your server
Method 2: Redirect traffic using a router and iptables
When I initially wrote this blog in January 2016, the setdsthn didn’t work for me. I don’t know why -- maybe I didn’t run saveconfig? When I revisited this in June 2016, the setdsthn worked for me, so that’s the solution I’m using now. Either way, I originally used this solution below, and it worked great!
I found myself poking around the weewx user group, just aimlessly browsing, when I came across this question that Sergei had. It doesn’t have anything to do with the ObserverIP, but it contained 2 pieces of information that I could use. I knew instantly I had my solution. Now to “translate” it to my needs. Two steps: redirect ObserverIP’s traffic, and have it go to a socket server.
All done without modifying any the ObserverIP’s core settings. No modifications are made to how the ObserverIP works. No warranty voiding or anything like that. This is all external to the weather station’s operation.
What you will need is a router capable of running dd-wrt. dd-wrt is custom firmware that unlocks more potential for your router. Think of it like a little Linux server running on your router. You can also accomplish this with a device that runs OpenWRT (such as the TP-LINK MR3020, however this write up will only cover the dd-wrt firmware).
I had recently upgraded my Linksys WRT54GL to a newer Asus wifi router, so the Linksys was a spare device collecting dust. It already had DD-WRT firmware loaded on it, this was a perfect fit for this project! I plugged the Linksys into my primary Asus router. Then I plugged my ObserverIP into the Linksys. The Linksys becomes the middleman and does the heavy lifting to redirect the data.
Here’s an awesome picture I made that describes what I mean.
Once you have it wired, and you’re able to log into the dd-wrt web interface, you need to give the ObserverIP a static DHCP reservation from the dd-wrt web interface. This is so every time the Linksys or the ObserverIP reboots, it gets the same IP.
Note: Unless you’re doing advanced static routing for multiple subnets, you will need to plug a laptop or PC into the Linksys to complete the steps below.
- From within the dd-wrt web interface, go to Services menu > Services sub menu > DHCP server > Static Leases. Just enter in the mac address of the ObserverIP, a name and IP you want to use. Click apply settings at the bottom and reboot the ObserverIP. It should come back up as the IP you just gave it.
Once you’ve done that, we need to apply some firewall rules:
- From within the dd-wrt web interface go to the Administration menu > Commands sub menu. Enter this into the box and click Save Firewall. Note; you will need to edit this to match your setup
iptables -t nat -A PREROUTING -s 192.168.1.55 -p tcp --dport 80 -j DNAT --to-destination x.x.x.x:80 iptables -t nat -A POSTROUTING -j MASQUERADE
- Replace 192.168.1.55 with the IP of your ObserverIP, and replace x.x.x.x:80 with the IP of your web server and port you want the traffic to be redirected to. This can be a public web server, or a private one on your LAN.
The first line says that ANY TCP TRAFFIC ON PORT 80 from 192.168.1.55, redirect it to x.x.x.x:80. Since we know that the wunderground API says to use port 80 to upload sensor readings, we want to catch and redirect traffic on port 80 from the ObserverIP.
Once you click Save Firewall, it takes about a minute or so, but you should start to see traffic coming in on your webserver (check the access and/or error logs to verify)! Note: this traffic coming in contains your wunderground username & password. Be sure the IP you’re redirecting to is yours!
On your webserver open your web logs, you should some new traffic coming in looking like
GET /weatherstation/updateweatherstation.php?ID=
Accepting the data on your server
Now that the traffic is coming in (from Method 1 or Method 2), we need to do something with it! You’ll see that it begins with getting a specific directory and file. We need to emulate this in order for the ObserverIP to be happy. On your webserver, make a new folder at the root level called weatherstation, and create a new file called updateweatherstation.php. Inside that PHP file, place this as the first couple of lines:
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);
echo $wunderground;
What is happening here is when the ObserverIP traffic is redirected, it’ll access this PHP file on your webserver. This PHP line is forwarding the entire request right to wunderground straight away so you don’t disrupt your wunderground readings. The result is being echoed, and it should say “success”. You will know if this is working because the “Server” light on your ObserverIP should come on. This indicates everything is being forwarded correctly.
Note: this PHP snippet does not have any security applied. See the below snippet on how to apply security using the ID & PASSWORD from the ObserverIP update string.
If you’ve gotten this far and you see data in your web server logs, and the ObserverIP has the “Server” light on, then that’s it! You’ve successfully redirected the weather data! It’s now yours to do whatever you want with it (e.g. send to a website, log to a file, send to MySQL), all while not disrupting your wunderground setup.
If you want to import this data into weewx, continue to the next section: Creating a socket for use with weewx
Creating a socket for use with weewx
Now that I finally have my weather data available to me, it’s time to get it into weewx. I had mentioned that question Sergei asked contained 2 pieces of information I needed. The first was the iptables redirect, and the second was from mwall talking about the hackulink driver. I forked a copy of that hackulink driver and created my own. I named my weewx driver weewx-socketlogger.
There’s some prerequisites to getting it running:
Step 1: Install a socket server. There’s a bunch out there, and even Python can create one for you. To keep it simple I used ncat. So, install ncat. I use CentOS, so yum install nmap. On Ubuntu you can run sudo apt-get install nmap to get ncat.
As a test, run ncat with these options:
ncat -l 2999 --keep-open --broker
Then connect to it using telnet to see if it works: telnet localhost 2999 If it connects, you’re in good shape!
This has ncat start a socket server on port 2999, keep it open even if clients disconnect, and broker the data so that all clients can see what the other clients are saying.
Now we need ncat to run on startup. To do this, I just ran it from my local user’s crontab. You can use /etc/rc.local if you’d like, but I found this to be easier.
Run crontab -e and add
@reboot /usr/bin/ncat -l 2999 --keep-open --broker >/dev/null 2>&1
This tells the server that on reboot, to start ncat with the same options as above, but run it in the background. That’s it!
To verify if it’s working you can run the command sudo netstat -tulpn | grep 2999 and you should see output similar to this:
tcp 0 0 0.0.0.0:2999 0.0.0.0:* LISTEN 1166/ncat tcp6 0 0 :::2999 :::* LISTEN 1166/ncat
Step 2: Edit the updateweatherstation.php file so that on every ObserverIP update, it not only sends data to wunderground, but also to the socket that weewx will connect to. My full updateweatherstation.php file looks like this. Be sure to change the YOUR_WUNDERGROUND_USERNAME and YOUR_WUNDERGROUND_PASSWORD to match what you saw in the access logs from earlier. This security method is minor, but it helps to prevent any bots out there from submitting false weather data to your archives.
This file receives the data from the Ambient Weather WS-1200-IP ObserverIP and sends the data to wunderground, as well as to weewx.
For more details, read the blog write up here: http://obrienlabs.net/redirecting-weather-station-data-from-observerip/
With this added to our PHP file, you can telnet to your socket on port 2999 and watch the data come in. This is the same data that weewx will use.
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.1,windDir=255,windSpeed=4.03,windGust=7.38,dailyrain=0.00,radiation=448.98,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=336,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=451.75,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.7,windDir=249,windSpeed=3.58,windGust=4.92,dailyrain=0.00,radiation=461.08,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=339,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=462.55,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=319,windSpeed=2.46,windGust=7.38,dailyrain=0.00,radiation=460.68,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=15.3,windDir=316,windSpeed=9.84,windGust=14.99,dailyrain=0.00,radiation=469.42,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=14.7,windDir=304,windSpeed=10.74,windGust=12.30,dailyrain=0.00,radiation=478.19,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=17.4,windDir=334,windSpeed=6.71,windGust=7.38,dailyrain=0.00,radiation=471.64,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
If your telnet connection looks like the above, then you’re in good shape! PHP is getting the data from the ObserverIP, and sending it to the socket. Now, you need to install the socketlogger weewx driver from the instructions on GitHub, restart weewx and weewx should connect to the socket and start logging things for you!
Note, there is no weewx loop interval like that’s found with other weewx drivers. That’s because every time a new update comes in from ObserverIP, PHP will accept it and send it to the socket. Weewx is connected and listening to the socket. When it sees new data on the socket, it will grab it and parse it then submit it as a packet to the archive system. Then weewx can generate its reports and/or update other weather services like CWOP, PWSWeather, your own website and more!
I changed my mind, I don’t want to do this. How do I undo it?
Method 1:
Easy! Just telnet back into your ObserverIP and run setdsthn rtupdate.wunderground.com, saveconfig, reboot. That will send the data back to wunderground’s update servers.
Method 2:
Easy! Remember, we haven’t changed anything on the ObserverIP! Unplug the Linksys and put it back in the box. Plug your ObserverIP back into your primary WiFi router (or whatever you have). That’s it! You’re back to the normal way it should be setup. You can use the IPTools software again to find your ObserverIP and manage it that way.
How I use this data to run my own website.
So now that we have control of the data, you can do anything you want! In addition to hooking this into weewx for archiving, I also have it write to a simple JSON file on every update from the station. I use this file to run some AJAX background updates on my weather website. The website checks for updates to the JSON file every 10 seconds and updates the weather data. This means that the website can show the most up-to-date information without having to refresh the page!
You can see that website here: https://belchertownweather.com
I also use Raspberry Pi model 3, with the PiTFT Plus from Adafruit, and have created a 3.5″ version of my weather website specific for the Pi. So now when the Pi boots up, it opens Chrome automatically and loads the special Pi console webpage. I can have very inexpensive display consoles throughout my house, or even on my desk at work!
The Pi console webpage has 2 major changes, the first one is the obvious size constriction. So I had to cut out some details to make everything fit. The second change is that the normal website offers background updates for only 30 minutes, then stops. This is to prevent any visitors who come to the site from eating up a lot of bandwidth. I may change that one day. The Pi console -- since it’s only me for now -- doesn’t have that restriction.
Here’s a screenshot of the website console on the Pi:
And a photo of the Pi itself. Once I get a case for it, I’ll update the photo.
If you have questions or want me to elaborate on a specific step, let me know in the comments!
The purpose of this site is for me to share my DIY technical experiences as a place to document them publicly. Hopefully somewhere along the way they are helpful for you.
First of all THANK YOU for writing a great article. I am little new at this however and have a question: you wrote:
So on your webserver, make a new folder at the root level called weatherstation, and create a new file called updateweatherstation.php. Inside that PHP file, place this as the first couple of lines:
I dont have my own webserver. I just run a ws-1400-ip and a have a desktop. I as well am looking for that data though.
If I dont have my own webserver am I probably out of luck with your article?
Thanks
Jeff
Hey Jeff, sorry for the delayed response here. You could control your data still if you wanted to run a web server on your home PC. You could Google like XAMP or WAMP. This will give you a web server (Apache), PHP and a database server. I would suggest giving your PC a static IP on your network. You could then point the WS-1400-IP to your PC and log to data locally. Just need to keep your PC on 100% for it to track. Hopefully this helps!
Thanks for the detailed post. I spent a lot of time spon theinning my wheels not exactly realizing that I should _downgrade_ the ObserverIP firmware so I could put in my own server’s IP address. After trying the IP nat/masq tricks (on my Ubiquiti EdgeRouter [which runs Linux]), which did work, but was ugly for other reasons, I realized the need to downgrade. Eventually I found the trick to find old firmware versions on the Ambient Weather web site (which was also in the comments here). Now everything is working nominally with updates to Wunderground being made and making use of the SocketLogger functionality in WeeWx
I was wondering if I might have missed a post that describe the web page you built for your Pi-based display? I saw the “how to setup a kiosk” post, but I was wondering if there was a post somewhere or a github page for your mini-Pi-Display.
Thanks!
Hey Christopher – Glad this worked and glad socketlogger is working for you too! Nope you haven’t missed the Raspberry pi kiosk post. I have it on my spare-time-list of things to do but haven’t gotten to it yet.
It would require a little bit of setup, but nothing too hard. I’ll plan to get that going soon!
Pat,
Great article with clear directions.
I previously used Meteobridge to capture data from my ObserverIP, but it suffers from the same freeze-up you describe above.
Since then I managed to get the traffic redirection method working on a FreeBSD system. Your directions are clear and easy to follow. But like any similar project avery step of the way was a learning process, from flashing dd-wrt on my router, to getting ncat to work, and installing and configuring weewx with your socketlogger driver.
All is working now and appears to be running well – thank you.
Louis, I’m glad it was able to help you out!
Hi, just spent the weekend getting this all up and running. First off, thanks for the post and detailed instructions. They were spot on. I did come across some struggles as I was following directions and not thinking…specifically around setting up the router.
For the router, you need to connect the Observer IP to the LAN port and have your main network be connected to the WAN port. You also need to ensure that your LAN Network is different. In the example and my implementation, I used 192.168.x for my LAN network and my main network is 192.168.1.x
I am using Ubuntu server (16.04) and for the NCAT section, I found a really good set of directions and template https://github.com/fhd/init-script-template for adding a program to the start up. When using this template, ensure you remove the “&” (Background) character from the command line in order to get it working.
Once that’s set up, I simply used: sudo update-rc.d enable
Now, I can start/stop the service via the command line and it starts automatically on boot.
One question I did have was around a log entry that says the weather station doesn’t support date/time. Is there a reason why the dateutc field wasn’t implemented?
Glad you got it working! In regards to dateutc, I know the ObserverIP sends it with its update packet, I don’t know why I ignored it. Good catch. I updated my the PHP snippet with the dateutc.
$socket .= "dateutc=".$_GET['dateutc'].",";. Hope this helps!Oh now I remember. I think weewx needs the epoch, and not the format that ObserverIP sends dateutc. So I implemented this instead:
$socket .= "epoch=".time().",";. I changed the code snippet above to have the epoch instead of dateutc.For the record, it appears that the ObserverIP is using HTTP/1.0, without a HOST: header, so a named virtual host doesn’t work. I.E. the site you are sending to needs to be the default site for that IP address.
Great info Larry, thanks! If you need to send it to a VirtualHost on the IP, you could try the
iptablesmethod I explained.I just added another IP from my block as (yet another) alias on that ethernet, and made the changes to apache. Just wanted to get the info out there.
So, it’s actually sending a HOST: header, but with the IP not the damn name:
Hos
0x0210: 743a 2031 3932 2e31 3437 2e32 352e 3730 t:.192.147.25.70
Damn it.
Another quirk to add to the pile
RE: WS 1400IP to weewx
I downloaded and flashed the aercus firmwear to my 1400 IP
http://www.aercusinstruments.com/content/downloads/WH2600_V2.1.9.bin
which gives me the option to use a customized destination for data.
I sent this data to my server (at 192.168.1.100) on an open port and then used the WEEWX Interceptor driver with the observer option to capture the data on that port and used weewx to forward it to wunderground and my weather server at http://www.toxaway.tk
Since your work was used to make this possible, THANK YOU!!
I believe at about $160 for a WS1400 IP on Amazon that this is the best option for weather enthusiasts on a budget.
I’m glad I was able to help! So you have an Ambient WS1400, and the Aercus firmware worked on it? That was one thing I was wondering about – but I wasn’t brave enough to try it on my Ambient!
I did the same thing (following the posts on http://www.wxforum.net). The Aercus firmware has the same Host: header issue I report above, but I do like the ability to customize the destination in the UI.
For some reason I get an 400 error in the file_get_contents function in my local updateweatherstation.php script … So I did the following:
$wunderground = 'http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?' . $_SERVER['QUERY_STRING'];$wunderground = str_replace(' ', '%20', $wunderground);
$wunderground = file_get_contents($wunderground);
echo $wunderground;
and now it works.
Just in case someone else is struggling with this issue. I spent some weeks 🙂
Cheers,
Hanno
Thanks Hanno! This could be helpful for others who may run into it as well.
Hi Larry, can you elaborate on the host/header issue? Are you saying that a regular .com on a shared hosting account won’t work as the destination server?
I have the Aercus Weather Sleuth, but have been unable to get it to send data to my website (www.ozstorms.com). I have a weatherstation/updateweatherstation.php file setup on my server ready to write the content it recieves to a json file. But, I never even see the server light come back on after updating the destination server and doing a reboot of the device. Very frustrating!
Does it need to pointing it to a dedicated IP address?
This instructional is great and works well – it’s helped me out a lot! I love how your instructions actually teach what’s happening as you go along so when things don’t work it’s easy to troubleshoot.
I’ve had this implemented for about a week now but unfortunately my outside temperature sensor has decided to consistently read high. The manufacturer has said that I need to buy a complete outdoor unit, but as I am in New Zealand it’s $100 plus $100 shipping. I’m thinking the obvious thing to do here is grab the outside temperature and correct it, or try and find a replacement sensor and solder it in. Since the latter seems to be a bit hard I thought some extra code would work.
There doesn’t seem to be any obvious way to do it before sending the traffic to Weather Underground, but I’d be able to at least have my WeeWX correct and then pass it to Weather Underground that way.
I’m not that familiar with PHP as I am learning Python at the moment but I do know that I need to alter this line here:
$socket = "outTemp=".$_GET['tempf'].",";To say something like:
$socket = "outTemp="(.$_GET['tempf']) - 20.",";Are you able to give some advice?
Thanks Dene! Some of it is so I don’t forget what I hacked together in a couple years time 🙂 and at the same time hoping it helps!
To be honest, what you put looks close, but I think it should be below. Give it a try – telnet to the socket port and watch the data as it comes through to see if it’s adjusted correctly.
$socket = "outTemp=".($_GET['tempf'] - 20).",";Come to think of it, PHP thinks that’s a string so it may not work. Might need to do a type conversion first. You could try this. Add a new line above the
$socketblock, something like:$adjusted = (float)$_GET['tempf'] - 20;Then change the
outTempline with this one:$socket = "outTemp=".$adjusted.",";It’s two more steps (so it’s not optimized), but it should work. Telnetting and watching for data to come in will tell the story.
Also, you don’t have to use the
$_SERVER['QUERY_STRING']to send to wunderground. You can manually create the wunderground URL submission yourself with the adjustedoutTempvalue. That way wunderground will be in sync. To see what the URL parameters should look like, you’ll find that in your web server logs as well from when the ObserverIP submits info to this PHP script. It’ll all be there, then you can mirror it with PHP easily.Thanks for your help. If this doesn’t give accurate readings I’ll get an additional sensor with a Raspberry Pi for outside and then I can substitute this reading for the external temp.
Thanks this is great, I have the redirect working but I don’t have a box to run weewx so I was wondering if there was a way to output the variables as a string in a txt file using file_put_contents() that cumulus could read .
I’m not familiar with Cumulus but sure you can output the variables using
file_put_contents();. Remove the$fpsection from lines 47-51 and replace with something like$file = 'output.txt'; file_put_contents($file, $socket);. Tailor the output to your needs for Cumulus to read it.Excellent article! We are linking to this particularly
great article on our website. Keep up the good writing.
Thanks!
Thanks for the write up. I have a more fundamental question. Why not allow the data to be posted to Weather Underground? Did you not want it published? I recently did a project called the Weather Widget where I grab my backyard weather data off the WU server using their free API? It spits out JSON or XML data and as long as you only update the weather data about once every 10 minutes it will remain free.
I figure I want the data publish all my local weather first. I then built a little hardware IOT device using an ESP8266 Node MCU board. It talks to the API and displays the weather data locally on a small screen. See here: https://mobile.twitter.com/JP_AustinTx/status/824097414697074688/video/1
Similar to this: http://www.instructables.com/id/ESP8266-Weather-Widget/
This way all my data still goes to WU and I can retrieve it as well without a lot of hassle with my network. Just a second idea to try.
Hi JP, I am publishing my data to wunderground. It’s the first line in the PHP code 🙂
Ok, it’s line 18 after doing some security checks, and a function.
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);echo $wunderground;
Anyways, after WU responds to my rtupdate, I continue on to archive data to a socket server, and to my live update website using JSON.
Sure, I could scrape Wunderground for this data, but why? I already have it in this script as virtual data… submitting to weewx and updating my website is only milliseconds away, and saves me from having to scrape wunderground.
I want to stress that it’s important to me that I own my data. I don’t want to hit an API rate limiter to have my own custom website. Doing it this way allows my website to have unlimited background updates everytime the station checks in.
I like your micro screen, I may look into doing something like that too with my own JSON file and get live updates every time the station checks in! 🙂
Just came across this page, looks very useful, thanks!
To give another reason, at my place, when the weather is bad, my internet can go down and I lose all data while it’s down since it can’t be posted to wunderground. Also, with some of the recent changes they’ve been making, they showed my station wasn’t reporting, which ended up being I needed a new firmware upgrade to deal with their changes. I’ve been looking for a way to log data locally for these types of situations, and this sounds perfect. Thanks again!
Glad you found your way back! Hope it helps out!
Hi Pat,
Have you considered making the code that generates your website public? I may have missed it, but I didn’t see it on your site, and it’s far better than a lot of the other skins I’ve used with weeWX!
Hey Steve, are you referring to my https://belchertownweather.com website? If so, thanks! That’s a custom WordPress website I created using a mix of live weather data from this post, as well as the weewx report information updated every 5 mins. There’s other custom scripts to get the earthquake data, and wunderground forecast data too. I have thought about putting something together on it, but it’s so custom that it would be either a long post, or need to be tailored for public consumption.
Well, Ambient has updated the firmware to 4.0.1, and setdsthn no longer works (this FW also allows data to go to AmbientWeather.net, their Dashboard for personal use).
I think Aercus has picked up the latest stuff from Ambient. With WU’s move of stuff to AWS, you really need the later firmware. 🙁
I’m getting “attitude” from Ambient when I mention the telnet stuff.
I wish I knew what Boris (MeteoBridge Author) is doing . to get the data from the observerip. I can’t seem to snoop the packets. 🙁
Ideas?
MeteoBridge is just screen scraping the Observer’s web interface – which I’ve had very bad luck with doing.
I’m still on version 3.1.3 (which was the first fix to the AWS & indoor barometric problems). You could probably downgrade back? Here’s their firmware listing: http://site.ambientweatherstore.com/iptools/
Looking at their changelog, I’m guessing I won’t upgrade past 3.1.8 since I don’t need to have my ObserverIP connected to their “cloud dashboard service”
Thanks for the info that he’s just screen scraping. Since I have the meteobridge, I’ve set it up to “mimic” the WUnderground GET to my server that wanted the info. And I have the ObserverIP sending to WUnderground.
The newer firmware seems(!) to have less memory leaks, I.E. it doesn’t get slower over time, needing a reboot.
Ah yeah the memory leaks are a problem I have. I ended up putting my ObserverIP onto a power outlet I can turn off/turn on from my phone. So if I notice my site isn’t getting weather info, I can quickly reboot it.
I may upgrade to 3.1.8, but I still don’t think I’ll go to 4.0 – especially after you shared your experience!
Should be possible to binary patch FW to point own address instead setdsthn?
I’m not sure if that’s possible – never tried to reverse engineer the FW.
setdsthnseems work just fine in version < 4.0Yes Pat, you’re right, version 4.0 and setdsthn does not work.
I had a look, it seems to be an ASIX module and the address for update is clearly written on the binary. Is there a checksum to modify also?
I would have to assume there is a checksum, but I don’t have any experience with it
Hi Pat,
First of all, thanks for writing a very thorough article.
I have a Observer IP (433Mhz version). Tried your way and can see the data passing through the 2999 port; only thing left is to make my installed weewx (on ubuntu running on an Android TV stick, MK802) accept the data and display it correctly.
My question is, is there any way to steal or fork data being sent to wunderground (by ObserverIP) to my server?
The reason I ask for this because I do not want my wunderground update relying on my tiny server; the ObserverIP is reliable enough for me; just want to get data to my Weewx for private use.
Hey Chote, I understand what you’re saying and I don’t think there is a way to have the ObserverIP send to dual destinations. That’s why I have it come to my server, then I “fork” it from there.
Hi Pat,
Thanks. I copied the socketlogger.py to /usr/share/weewx/user, copied your snippet in weewx.conf to the end of the original weewx.conf. Stop and restart weewx. I still see data coming thru port 2999; Wunderground still get updated, but my local weewx webpage and my pwsweather station do not get updated. Not sure what I did wrong.
FYI, apache, php, weewx are all in the same server.
sorry Pat. It’s ok now. I did not change 2 things in the weewx.conf:
localhost –> 192.168….
station_type = from simulator —> SocketLogger
Working great now. Thanks a lot.
Great! Glad you were able to get it working
I’ve just started using a PWS, specifically the WS-1200-IP. At the moment I’m very happy with how everything is tied together, but since I like to tinker at times and wondered about the telnet interface on the oIP I stumbled across your blog here. I admit that I’ve just skimmed the surface of what you are doing here, but it just seems you are redirecting (technically proxying) the observer module’s posting to port 80 to your own server instead, which then allows you to do whatever you want with said data.
If setdsthn is broken, another method to do this is to lie to the observer what the DNS IP is for wunderground. Since I run my own internal DNS servers at home, this is relatively easy, but I understand it can be confusing and not as ideal as what you have done. Basically, if you are running a version of BIND, and you have the observer pointing to it for DNS, you can create a custom “view” in DNS that just equates wunderground’s DNS name to your own server. If you are running your own webserver you can easily install BIND and just set it up for use by the oIP.
Great idea Karl
Found out the hard way that I’m going to have top find a different way to do this. I updated to 4.2 from 3.1.2 to try and solve inconsistent updates to Wunderground and reading about the move to AWS. That immediately broke setdsthn. I tried downgrading because I really only want my data and don’t care about wunderground but never got a reliable connection after that.
Back on 4.2, I thought that I would try the DNS hack. I’m running PiHole and tried using it to redirect server requests to rtupdate.wunderground.com following the instruction here: https://discourse.pi-hole.net/t/domain-redirect/1826/4
It sort-of works, but Wunderground goes 3-5 minutes between updates, while updates sent the socket burst 10-15 updates every ~30 seconds, and they never get picked up by the socketlogger.
Can anyone confirm that the php get URL is the same with IPobserver 4.x firmware?
I never should have updated 🙁 To make matters worse, as I was doing all this Tropical Storm Emily was raging outside – think of all that good data I lost!!
Oh no that sucks!! I’ve played with Pi-Hole before but I don’t run it anymore since I am using some DNS Blacklisting on PFSense (admittedly I’ve been thinking of running Pi-Hole again).
I haven’t upgraded to 4.x in fear of breaking this setup so I can’t confirm what the PHP GET vars are. My guess is that they’ve changed because of the way Ambient is doing their weather pages now.
If downgrading doesn’t work you could try Method 2 of this article and buy a cheap TP-Link MR3020 router for $30 and flash it with DD-WRT and let it be the middle man. It would run in bridge mode, the ObserverIP would be hard wired to the MR3020 and then the MR3020 would connect to your WiFi network. The IPTABLES firewall on dd-wrt would be the redirecting method to your server (and not DNS). When I ran Method 2 on an old Linksys it was pretty reliable!
Running Method 2 can also give you a sense of what the PHP GET VARs would be since the ObserverIP would then be talking to your web server and you could check your web server logs to see the formatting of the URL.
Well, well, well… I just happened to have a spare Cisco e3000 lying around running Tomato router and presto – I’m back in business.
I did slightly modify your iptables command to:
iptables -t nat -A PREROUTING -d rtupdate.wunderground.com -p tcp –dport 80 -j DNAT –to-destination 192.168.your-dst.ip:8122
– since the command only reroutes traffic intended for Wunderground, the new ambient weather service works as well.
– also, this way you don’t have to worry about a static IP for the IPobserver
Also, while troubleshooting some connectivity issues I was having (thanks Synology Firewall!!!), I did get a chance to packet sniff the outgoing GET request. Nothing has changed, so 4.X interfaces with Wunderground the same as 3.X.
Interesting, good to know. I thought in 4.x they were routing the URL to Ambient’s URL. Maybe they kept the URL VARs the same, but different hostname?
4.X does both Wunderground and Ambient.
Or another way to do the same thing, since the firmware is not encrypted, is just use a hex editor to modify the “rtupdate.wunderground.com” to something inside your house. Putting an IP in there may work…
Pat,
GREAT Article. Thank you. I have been mucking around with Pi’s for years. I am starting work right now to incorporate this information into my “whole house dashboard” – Nest Data, Sense data, SMASUNG SH status et al.
My one question, is this a full redirect? or is it a fork. Does the Factory provided display continue to work in near real time? or is it an either or decision? I’ve been running Oregon Scientific weather station for about a year but most of the OSx components have proven unreliable – with no good API between collector and cloud.
GREAT Article. Talk to the Sense guys down the road – need an API for whole home power monitoring!
Hi Steve, the factory display continues to work in conjunction with this setup. Best I can tell the outdoor unit and the indoor unit sends their data over RF and the ObserverIP and the indoor unit both “listen” for that RF data and update their independent information (the screen or wunderground).
I’ve been thinking of replacing my factory display with the Raspberry Pi screen I show here entirely. I think my layout of information makes more sense than theirs does!
Appologies. I read the article too fast scanning the code. You explained it very well here:
“What is happening here is when the ObserverIP traffic is redirected, it’ll access this PHP file on your webserver. This PHP line is forwarding the entire request right to wunderground straight away so you don’t disrupt your wunderground readings. The result is being echoed, and it should say “success”. You will know if this is working because the “Server” light on your ObserverIP should come on. This indicates everything is being forwarded correctly.”
Thanks. sorry to bother.
No bother at all!
Pat, thank you for this great article. This gives me the method to add soil temp wx data I send to WU. In my case I’m running the Android app Bit Web Server ($3) on an old cracked screen Galaxy S3. This old phone also sends cloud pics to WU and is weatherproof mounted just below the WS-1400 sensor. The modified updateweatherstation.php (code snippet below) runs on the web server on the phone. With WS-1400 firmware version 3.1.2 and additionally adding the telnet command “setdstport 8080” to accommodate Bit Web Server’s port this combination works well. Thanks again for the idea! Oh, apparently Amazon fixed their server issue, which prompted 3.1.3+ updates, a couple months ago. I’ve run 3.1.2 for months, observed some baro pressure data dropouts for a few weeks, and now normal behavior for the last couple months – again 3.1.2 firmware. Maybe my trouble with 3.1.8 was some intermediate config mismatch but just wanted to confirm success with 3.1.2 this post date. Code snippet:
$wuget = “http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?” . $_SERVER[‘QUERY_STRING’];
if ( $soiltempf > -55.0 && $_SERVER[‘REQUEST_TIME’] – $updatetime < 2*60*60) {
$wuget .= "&soiltempf=" . $soiltempf;
}
$wunderground = file_get_contents($wuget);
echo $wunderground;
Hey Mike this is great! I’m glad this was helpful. Also – I’d really love to see a photo of your Galaxy phone mounted beneath the WS-1400! That sounds cool. I would love to be able to send cloud pics to wunderground but haven’t found a solid way to do it. My outdoor sensor isn’t near utility power, so I’d need something that can be solar charged and an old cell phone is not a bad idea!
If you want to share a photo, I’m not sure if you can do photos in comments, but you can upload to to Imgur and then share the link here. Would be cool to see it!
Edit: Also, is your cell phone running the soil temperature too?
I put the pics and code on github (link below). I checked the current usage and it seems to average about 120mA – may be tough for solar? If you can get / config a phone with less manufacturer/google bloatware you could likely save some power. ~120ma is stock Galaxy S3 4.3 but that seems high?? I’ve got mine wired to a wall adapter supply. A big potential problem is these phone chargers shutdown at temp extremes so I direct wired current limited 4v to the battery. My soil sensor is an ESP8266/DS18B20 which will periodically send updates to Bit Web Server on the S3. It will run off of solar power (uses ~15mA) in the garden – waiting for a low dropout regulator. I’ll get that code up to github at some point too. Just buying a camera would have been a lot easier 🙂 but this solution captions my WS-1400 data on the pics. My WU site: https://www.wunderground.com/personal-weather-station/dashboard?ID=KKSSHAWN60
https://github.com/mrcodegh/WU-Cloud-Cam-Pics-via-Android
This is awesome – I’m going to have to think about how to upgrade my setup with an old cell phone or something!
Great article
Can I ask where did you get those icons think they look really cool , currently building my own website based on your article .
Dan
Good question, I don’t remember. Let me see if I can find if I saved where I got them
Hello,
Thanks for your weather stations and weewx page, without which I would never have got started.
I now have a WeatherSleuth and weewx up and running. 🙂
I did encounter a few problems along the way, and I also expanded on what you’ve done, to add in a few more values to the schema – absolute pressure, ultraviolet (not the index) and also all the WeatherSleuth’s rain values. I used curl in PHP, to retrieve two of them from the WeatherSleuth’s web interface. Just to see what they do. 🙂 Though I’ve no idea what unit the ultraviolet value is measured in.
I found that the driver’s socket is timing out, which then caused weewx to hang or exit, with gaps in the data. Changing the timeout setting didn’t fix it, so I rewrote the socket part of the driver. I’m not sure why the sockets are timing out, it happens a couple of times a day, but weewx is now able to restart the socket-logger driver and carry on from there. That was also happening when rebooting the machine, with the weewx daemon failing to start up, but now the socket connects for me on the 2nd attempt.
Anyway if you’re interested in seeing that code for comparison, I’d be happy to send it to you. I should probably mention that it was my first foray into python, and I didn’t bother learning it, but I’m C programmer with some network programming experience.
I still haven’t figured out how the hourly rain rate from the WeatherSleuth produces the WeatherSleuth’s daily rate. I hope there isn’t some averaging going on, but I guess the only way to find out, is to put a jar in the garden!?
Cheerio,
Chris.
Glad this was able to help! I’ve experienced the socketlogger timeout as well and even mentioned it on the weewx developer forum and they basically said it’s expected behavior if weewx doesn’t receive data within a set period of time. It’ll give up, go to sleep for 60 seconds and then try again. I usually only get the timeouts when I am rebooting the unit, or have a power outage or something.
I am curious to check out the code though! Any way to make it better is always welcomed!
Hello,
I did try to post the code a while ago, but I think it didn’t like one long post.
So I’ll try breaking it up.
Seeing as I set up an extended database, you might want to ignore any bits for the extra fields.
However, I’ll include my schema anyway, in case that’s of interest to any one.
I have it running on a virtual host.
The extra fields are:
wsHourlyRain, wsDailyRain, wsWeeklyRain, wsMonthlyRain, wsYearlyRain.
Ultraviolet.
(pressure was already there).
I’m not sure if the white space will display ok, but here goes…
Chris.
All cool stuff! I’ll see if I can make it format better on the site, but thanks for sharing it!
Here’s the schema for my extended database (wee_database –create):
schema = [(‘dateTime’, ‘INTEGER NOT NULL UNIQUE PRIMARY KEY’),
(‘usUnits’, ‘INTEGER NOT NULL’),
(‘interval’, ‘INTEGER NOT NULL’),
(‘barometer’, ‘REAL’),
(‘pressure’, ‘REAL’),
(‘altimeter’, ‘REAL’),
(‘inTemp’, ‘REAL’),
(‘outTemp’, ‘REAL’),
(‘inHumidity’, ‘REAL’),
(‘outHumidity’, ‘REAL’),
(‘windSpeed’, ‘REAL’),
(‘windDir’, ‘REAL’),
(‘windGust’, ‘REAL’),
(‘windGustDir’, ‘REAL’),
(‘rainRate’, ‘REAL’),
(‘rain’, ‘REAL’),
(‘wsHourlyRain’, ‘REAL’),
(‘wsDailyRain’, ‘REAL’),
(‘wsWeeklyRain’, ‘REAL’),
(‘wsMonthlyRain’, ‘REAL’),
(‘wsYearlyRain’, ‘REAL’),
(‘dewpoint’, ‘REAL’),
(‘windchill’, ‘REAL’),
(‘heatindex’, ‘REAL’),
(‘ET’, ‘REAL’),
(‘radiation’, ‘REAL’),
(‘Ultraviolet’, ‘REAL’),
(‘UV’, ‘REAL’),
(‘extraTemp1’, ‘REAL’),
(‘extraTemp2’, ‘REAL’),
(‘extraTemp3’, ‘REAL’),
(‘soilTemp1’, ‘REAL’),
(‘soilTemp2’, ‘REAL’),
(‘soilTemp3’, ‘REAL’),
(‘soilTemp4’, ‘REAL’),
(‘leafTemp1’, ‘REAL’),
(‘leafTemp2’, ‘REAL’),
(‘extraHumid1’, ‘REAL’),
(‘extraHumid2’, ‘REAL’),
(‘soilMoist1’, ‘REAL’),
(‘soilMoist2’, ‘REAL’),
(‘soilMoist3’, ‘REAL’),
(‘soilMoist4’, ‘REAL’),
(‘leafWet1’, ‘REAL’),
(‘leafWet2’, ‘REAL’),
(‘rxCheckPercent’, ‘REAL’),
(‘txBatteryStatus’, ‘REAL’),
(‘consBatteryVoltage’, ‘REAL’),
(‘hail’, ‘REAL’),
(‘hailRate’, ‘REAL’),
(‘heatingTemp’, ‘REAL’),
(‘heatingVoltage’, ‘REAL’),
(‘supplyVoltage’, ‘REAL’),
(‘referenceVoltage’, ‘REAL’),
(‘windBatteryStatus’, ‘REAL’),
(‘rainBatteryStatus’, ‘REAL’),
(‘outTempBatteryStatus’, ‘REAL’),
(‘inTempBatteryStatus’, ‘REAL’)]
Here’s what I’ve got in extensions.py for my extended database schema:
import locale
locale.setlocale(locale.LC_ALL, ”)
import weewx.units
weewx.units.obs_group_dict[“wsHourlyRain”] = “group_wsrain”
weewx.units.obs_group_dict[“wsDailyRain”] = “group_wsrain”
weewx.units.obs_group_dict[“wsWeeklyRain”] = “group_wsrain”
weewx.units.obs_group_dict[“wsMonthlyRain”] = “group_wsrain”
weewx.units.obs_group_dict[“wsYearlyRain”] = “group_wsrain”
weewx.units.obs_group_dict[“Ultraviolet”] = “group_ultraviolet”
weewx.units.USUnits[“group_wsrain”] = “inch”
weewx.units.MetricUnits[“group_wsrain”] = “cm”
weewx.units.MetricWXUnits[“group_wsrain”] = “mm”
weewx.units.USUnits[“group_ultraviolet”] = “Ultraviolet”
weewx.units.MetricUnits[“group_ultraviolet”] = “Ultraviolet”
weewx.units.MetricWXUnits[“group_ultraviolet”] = “Ultraviolet”
weewx.units.default_unit_format_dict[“Ultraviolet”] = “%.1f”
weewx.units.default_unit_label_dict[“Ultraviolet”] = ” UV”
My php retrieves two extra values using curl, for absolute pressure and ultraviolet (not UVI).
It makes some extra logs.
It also saves the battery status to a file, to be checked by a shell script under cron.
You might want to ignore those extra bits too.
$value )
{
if ( ($key != “ID”) && ($key != “PASSWORD”) && ($key != “softwaretype”) && ($key != “action”) )
{
$weatherdata .= “,”;
$weatherdata .= $value;
}
}
$weatherdata .= “,” . $uv;
$weatherdata .= “,” . $absPress;
$weatherdata .= “\n”;
fwrite( $fd, $weatherdata );
fclose( $fd );
}
}
function logweewx( $wxdata )
{
global $weewxlog;
if ( ($fd = fopen( $weewxlog, “a” )) != FALSE )
{
fwrite( $fd, $wxdata );
fclose( $fd );
}
}
/*
function debugdata( $uv, $absPress )
{
global $errorlog;
if ( ($fd = fopen( $errorlog, “a” )) != FALSE )
{
$weatherdata = date( “d/m/Y H:i:s” );
foreach( $_GET as $key => $value )
$weatherdata .= $key . ” : ” . $value . “\n”;
$weatherdata .= “UV” . ” : ” . $uv;
$weatherdata .= “Absolute Pressure” . ” : ” . $absPress;
fwrite( $fd, $weatherdata );
fclose( $fd );
}
}
*/
function logerror( $msg )
{
global $errorlog;
$timestamp = date( “d/m/Y H:i:s” );
if ( ($fd = fopen( $errorlog, “a” )) != FALSE )
{
$errormsg = $timestamp . “,” . $msg . “\n”;
fwrite( $fd, $errormsg );
fclose( $fd );
}
}
function stationid()
{
// Make sure the ID and PASSWORD fields match what we think it should match. If it does, this update is valid and continue.
if ( ($_GET[“ID”] != “WeatherSleuth”) || ($_GET[“PASSWORD”] != “qwerty1234”) )
{
logerror( “Fail on stationid()” );
header( “HTTP/1.0 401 Unauthorized” );
echo “Error 401”;
die();
}
}
function formatchange()
{
$numkeys = count( $_GET );
// Check if the number of key/value pairs changed
if ( $numkeys != 25 )
{
logerror( “25 keys expected, got : ” . $numkeys );
if ( array_key_exists( “softwaretype”, $_GET ) )
{
if ( $_GET[“softwaretype”] != “WH2600GEN_V2.2.7” )
logerror( “Firmware update” );
}
if ( array_key_exists( “action”, $_GET ) )
{
if ( $_GET[“action”] != “updateraw” )
logerror( “Action/Update raw changed” );
}
}
}
function battery_status( $battery )
{
if ( ($fd = fopen( “/vhosts/ws.lan/weatherstation/logs/battery.status”, “w” )) != FALSE )
{
fwrite( $fd, $battery );
fclose( $fd );
}
}
//
// Main
//
$uv = 0;
$absPress = 0;
stationid();
formatchange();
// Retrieve data from Weather Sleuth Receiver’s web interface
getMissingData( $uv, $absPress );
logWSdata( $uv, $absPress );
send2ncat( $uv, $absPress ); //Send to ncat, to be forwarded to weewx
// Log the battery status
battery_status( $_GET[‘lowbatt’] );
//phpinfo();
//debugdata( $uv, $absPress ); // Whole function commented out
?>
Example shell script for the battery status, which emails if the battery is low:
#! /bin/sh
function f_email_alert()
{
mailx -s”Weather Sleuth – Low Battery Alert” -A myaccount myemailuser < 7 days ago
if [ “$age” -gt “604800” ]
then
f_email_alert
fi
fi
}
battery=”/vhosts/ws.lan/weatherstation/logs/battery.status”
alerted=”/vhosts/ws.lan/weatherstation/battery/low_battery.alerted”
batterystatus=`cat $battery`
if [ “$batterystatus” == “1” ]
then
# 0 = ok, 1 = low battery
f_alert
fi
================
Example shell script for gzipping the extra logs:
#!/bin/bash
wslogs=/vhosts/ws.lan/weatherstation/logs/*.log
for logfile in $wslogs
do
logtime=`date +%s -r $logfile`
timenow=`date +%s`
age=`expr $timenow – $logtime`
# Check logfile is > 1 day old
if [ “$age” -gt “86400” ]
then
gzipfile=”$logfile.gz”
# Rename any logs that already have a gzip file (shouldn’t happen)
if [ -e $gzipfile ]
then
mv $logfile “$logfile.$timenow”
logfile=”$logfile.$timenow”
fi
# gzip file (no need to tar a single file, or to remove log file)
gzip $logfile
fi
done
Finally here’s the socketlogger (which also includes those extra fields for rain, ultraviolet & pressure, and I’d also renamed UV to UVI):
# This driver opens a socket and waits for data, it processes any data and sends it to weewx.
import socket
import time
import syslog
import weedb
import weewx.drivers
import weeutil.weeutil
import weewx.wxformulas
def logmsg( dst, msg ):
syslog.syslog( dst, ‘SocketLogger: %s’ % msg )
def loginf( msg ):
logmsg( syslog.LOG_INFO, msg )
def logerror( msg ):
logmsg( syslog.LOG_ERR, msg )
def logdebug( msg ):
logmsg( syslog.LOG_DEBUG, msg )
def loader( config_dict, engine ):
station = SocketLogger( **config_dict[‘SocketLogger’] )
return station
class SocketLogger( weewx.drivers.AbstractDevice ):
# Driver for the SocketLogger station.
def __init__( self, **stn_dict ) :
# Initialize an object of type SocketLogger.
self.connected = 0
self.sockretry = 1
self.host_ip = stn_dict.get( ‘host_ip’ )
self.host_port = int( stn_dict.get( ‘host_port’ ) )
self.timeout = float( stn_dict.get( ‘timeout’ ) ) # Default is 20 secs
#self.port = None
self.openPort()
loginf( “Connected to socket on %s port %s” % ( self.host_ip, self.host_port ) )
self.connected = 1
self.station_hardware = stn_dict.get( ‘hardware’ )
self.rain_accumulation = None
def hardware_name( self ):
return self.station_hardware
def openPort( self ):
try:
loginf( “Connecting to socket on %s port %s, attempt %s” % ( self.host_ip, self.host_port, self.sockretry ) )
self.socket = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
self.socket.settimeout( self.timeout )
self.socket.connect( (self.host_ip, self.host_port) )
except ( socket.error, socket.timeout, socket.herror ), ex:
# Seems to timeout on reboot, wait for ncat to start and retry (10 times)
if ( self.sockretry self.rain_accumulation ):
# Get the increment
rain_increment = float( ws_rain ) – float( self.rain_accumulation )
elif ( ws_rain < self.rain_accumulation ):
# Assumes ws_rain has reset (and restarted from 0)
rain_increment = ws_rain
else:
rain_increment = 0.0
self.rain_accumulation = ws_rain
#loginf( rain_increment )
return rain_increment
def assign_value( self, sensor_data ):
# Handle sensor failure values
if sensor_data == -9999:
return None
else:
return float( sensor_data )
#===============================================================================
# LOOP record decoding functions
#===============================================================================
def genLoopPackets( self ):
# Generator function that continuously returns loop packets
for _packet in self.genPackets( ):
yield _packet
def genPackets( self ):
prevbuf = ""
# Generate measurement packets from the socket.
while True:
gotline = 0
try:
#_line = self.port.readline( 4096 )
buffer = self.socket.recv( 4096 )
except ( socket.error, socket.timeout ), ex:
logerror( "Socket error while reading from %s on port %s – %s." % ( self.host_ip, self.host_port, ex ) )
if self.connected == 1:
self.closePort()
raise weewx.WeeWxIOError( ex )
break
else:
if self.sockretry 16384 ):
# Reject huge strings (16k), just in case no “\n” ever received
prevbuf = “”
else:
prevbuf = buffer
def _process_message( self, message ):
_packet = {}
# Separate line into a dict
message = message.rstrip( ) # Strip any newline
line = message.split( ‘,’ ) # Split by comma
data = dict( [ i.split( ‘=’ ) for i in line] ) # Create dictionary of the values
_packet[‘dateTime’] = int( time.time( ) )
_packet[‘usUnits’] = weewx.US
_packet[‘outTemp’] = self.assign_value( data[“outTemp”] )
_packet[‘outHumidity’] = self.assign_value( data[“outHumidity”] )
_packet[‘dewpoint’] = self.assign_value( data[“dewpoint”] )
_packet[‘windchill’] = self.assign_value( data[“windchill”] )
_packet[‘windDir’] = self.assign_value( data[“windDir”] )
_packet[‘windSpeed’] = self.assign_value( data[“windSpeed”] )
_packet[‘windGust’] = self.assign_value( data[“windGust”] )
_packet[‘windGustDir’] = self.assign_value( data[“windDir”] )
# _packet[‘rain’] = self.assign_rain( data[“hourlyrain”] )
_packet[‘rain’] = self.assign_rain( data[“dailyrain”] )
# Adding Weather Sleuth rain rates anyway for comparison
_packet[‘wsHourlyRain’] = self.assign_value( data[“hourlyrain”] )
_packet[‘wsDailyRain’] = self.assign_value( data[“dailyrain”] )
_packet[‘wsWeeklyRain’] = self.assign_value( data[“weeklyrain”] )
_packet[‘wsMonthlyRain’] = self.assign_value( data[“monthlyrain”] )
_packet[‘wsYearlyRain’] = self.assign_value( data[“yearlyrain”] )
_packet[‘radiation’] = self.assign_value( data[“radiation”] )
_packet[‘Ultraviolet’] = self.assign_value( data[“Ultraviolet”] )
_packet[‘UV’] = self.assign_value( data[“UVI”] )
_packet[‘inTemp’] = self.assign_value( data[“inTemp”] )
_packet[‘inHumidity’] = self.assign_value( data[“inHumidity”] )
_packet[‘barometer’] = self.assign_value( data[“barometer”] )
_packet[‘pressure’] = self.assign_value( data[“pressure”] )
_packet[‘txBatteryStatus’] = self.assign_value( data[“txBatteryStatus”] )
# To view logs – journalctl
#loginf( _packet )
return _packet
================
All done. Phew!
🙂
You seem to be missing the first half of the php, let me know if you’d like me to retry posting it, or emailing it or something.
Chris.
Note: the 4.3.1 firmware out today (September 26, 2017) *REMOVES* the telnet server altogether from the observerIP.
Yikes
Hey Pat. Thanks for the great post. Regarding method 2 and proxying traffic from the ObserverIP to your custom server using iptables on your router… did you ever get that working for an external web server? That is, a server outside of your LAN (a VPS for example… something on the WAN)?
For whatever reason this doesn’t work for me. It does work however, if my webserver is running on a LAN.
Thanks!
I have had it working for an external server, like a VPS at AWS. Is your firewall open on that VPS for port 80 (or whatever port you’re forwarding to?) That’s the only thing I can think of off hand..
Hi Pat! Very nice work!
I have a question, If I just want to keep the data without sending it to weather undergound, can this be done?
Could it work by changing the WU IP in my DNS to that of my local server?
Does communication with WU use HTTPS or only HTTP?
Any comments is welcome!
Thanks!
If you’re setup just like how I am, where weewx is archiving everything then you could do this by removing the
file_get_contents("http://rtupdate.wunderground.com.......section of the updateweatherstation.php file.Hi Pat, I’ve been lurking on wxforum and your site the past few years hoping to get an ‘easy’ solution to get the indoor temp from my Ambient WX-1400, since wunderground removed it from the stationdatalookup–I guess due to privacy concerns. So, I attempted your method, but got stuck at where the data is going once I’ve redirected it to my WAMP (windows10) server. I really don’t have the background to setup socket servers and python, etc. I’m just hoping to monitor the temp from the indoor sensor that sends data to the ObserverIP. I was actually able to telnet into my ObserverIP and redirect it away from wunderground, but wasn’t sure where it was going and/or what to do with it. 🙂 Any help would be much appreciated as I am germinating some south african plants and need to keep a close eye on the temp and humidity. thanks.
Hey Tom, it all depends on what you want to do. What’s your end goal? A webpage that shows only the indoor temp? Push notifications to your phone if a threshold is met? This is advanced but doable… Once the data is coming into your webserver (WAMP should be fine), then you have all kinds of options. PHP would run the logic for the conditions you’re looking for (and if you want push notifications then I would use IFTTT Maker Webhooks and the IFTTT app on your phone) First I would make sure your WAMP server is seeing the data. Check the WAMP access log files for the incoming traffic.
Hi I’m using PHP 7
I get this ? (i’m not a dev/php type person..)
PHP Notice: Undefined index: ID in /weatherstation/updateweatherstation.php on line 14
Cheers
Mark
Sorry for the delay! Not sure what’s causing that issue. What version of the ObserverIP are you using? Could be a newer version that isn’t sending all the values that the PHP script is looking for. Anything in the Apache access logs that would indicate ID is there or not?
Sorry if I missed this somewhere in the thread. Is there a way to both capture your data locally and have it forwarded to a site like WeatherUnderground? Essentially, fork a copy of the data to send somewhere.
thanks!
That’s what the PHP script is doing. Line 18 sends it right to wunderground. The rest of it sends it to weewx which is a local weather data archive system.
I just got a WS-900-IP, and the Observer firmware that came with it (4.2.1) accepts telnet connections, but doesn’t let you login: it just eats whatever you send it. I was going to just use setdsthn to redirect to a Linux box, but that ship has sailed, and sunk, unless I downgrade. The firmware filenames on http://site.ambientweatherstore.com/iptools/ are ambiguous: there are version 3.x files prefixed with “WS0900” and “WS1400”, but the 4.x files are prefixed with “ambient”. I’m guessing that 4.0 added an ability to figure out what kind of outdoor unit was present, so there’s only one type needed. And I should download a WS0900 one if I decide to downgrade. Is that correct?
Since I have a few WRT54G’s available (usually only $5 at Goodwill!), I think I’ll follow your excellent instructions to reflash one and use it as a redirector, instead of messing with the Observer.
Thanks for taking the time to figure this stuff out, and for sharing the results.
Sounds good! I hope the redirector works for you. It’s what I used for a few months and had no problem.
Alas, it’s not going great so far.
One problem is that the 0900 reports are a little different from the 1200’s. I had some errors in the PHP when some parameters didn’t show up in the query string. I’ve kluged around that for now, but I’m going to add some input validation code, and I’ll send you a copy when I’m done with that.
It looks like another problem may be that sometimes parameters come in with a null value, which is causing an exception in the driver:
Dec 14 09:41:07 delve weewx[26231]: **** File “/usr/share/weewx/user/socketlogger.py”, line 126, in _process_mes
sage
Dec 14 09:41:07 delve weewx[26231]: **** data = dict( [ i.split( ‘=’ ) for i in line] ) # Create dictionary of
the values
Dec 14 09:41:07 delve weewx[26231]: **** ValueError: dictionary update sequence element #12 has length 1; 2 is req
uired
Dec 14 09:41:07 delve weewx[26231]: **** Exiting.
Adding some input checking to the redirector will probably fix that. But I may also wind up adding some to the driver. Again, I’ll send you a copy if I make changes.
I’m also seeing dozens of connections to ncat that are stuck in CLOSE_WAIT or FIN_WAIT1, and a few stuck in SYN_RECV, with no process on the other side. It looks like PHP may not be closing connections properly when the redirector is done. I saw some errors in the Apache log when I had the parameter problems, but not with these. If you have any tips on finding the source, or a reassurance that this is okay, I’d appreciate it.
Thanks again, and I’ll update you as things progress.
Hmm, on your CLOSE_WAIT and FIN_WAIT, not sure about that. The null values and weewx waiting could be causing some of that? If you know how to use GitHub, feel free to fork my weewx driver and submit the changes as a pull request! https://github.com/poblabs/weewx-socketlogger
I added a bunch of input validation to the PHP and Python code, and everything has been running well for several days. It looks like the “zombie sockets” problem was a product of the Python code throwing exceptions on input it didn’t like.
The one remaining problem is that the receiver occasionally loses connection with the wind unit (it’s on the roof of a metal building), and reports wind speed and direction as “-9999”. The WeeWx graph of wind speed over time becomes useless, because it auto-scales to have -10000 as its minimum value. From your experience with WeeWx, would it be better to replace the “-9999” with something like “-9” (which would show the error without trashing the graph completely), or to leave those readings out of the report?
Would you prefer that I email you the updated source? Or just post it here?
Thanks again,
Ran
I’ve noticed the -9999 in the reports when connection is lost too. I haven’t updated the code to exclude those but probably should. Pasting the code here may get sloppy with formatting. Is your email valid with your comment account? if so I’ll email you first so that way neither of us gives out our email on the site.
I’m able to telnet using admin/admin for login —v4.21
Thanks for mentioning that, Philip: mine wasn’t accepting telnet logins when it had a public IP address (maybe it was getting hammered by script kiddies, and had disabled logins due to too many failed attempts?), but it works now that it’s behind the router.
And it happened to be a lifesaver: an Ethernet switch upstream from the router went flaky, and most updates were getting lost. I was baffled because everything seemed fine when I connected to the router’s LAN, but the Observer’s ping command told me it wasn’t reaching my server. So I started checking cables, and found the bad switch.
Pat: I’m a trusting soul, so I do use my real address to post. 😉
Thanks for this great write up. I just purchased a LaCrosse Technology Professional Remote Monitoring Weather Station ( Model #: S84060) from Sam’s Club for $79. This procedure worked perfectly for the LaCrosse. This weather station uses port 443 to talk to the LaCrosse servers and port 80 for wunderground. Using dd wrt and PHP to intercept port 80 works great on the S84060 from Sam’s.
Hi Pat,
What would be a good way to display local weather conditions on a map of other local users? Also, do you have an example of the JSON file and AJAX scripting used on the back-end to auto-update conditions on your website?
Thank you!
Best,
Jason
Can you not use a Meteo template ?? e.g. http://www.meteo.capetown/pages/other/aboutPage.php
I’m sure you could, but those templates look….. like a template. Not to mention the number of menus and subitems within the menus is overwhelming and confusing.
Since web design and web developing is a hobby of mine, I wanted to make my own custom site tailored to what I wanted, including a simple UI and direct updates from the wx station without needing to reload the page.
As well as the ability to make my own Raspberry Pi weather station console with just some CSS updates (and not a whole new template or site)
Mostly I just want to know how the weather station hardware is still doing. 🙂 i’ve been thinking about buying one of those but i’m unsure how well they all hold up, especially in the strange Texas weather.
Hardware is still holding up! The Ambient station and IP updates work great. I do have a smart power outlet that reboots the ObserverIP unit every night as a fail-safe, but other than that I haven’t had much of a problem. I do have a confession to make… I’m replacing it with a Davis Vantage Pro 2. Just working out the weewx driver details since I live feed data to my weather website, the Davis has a whole new set of challenges to keep the same data sets updated.
But the Ambient is a great station to get started with!
Hello,
Im hoping someone could help me out (noob alert). I am using a WS2902 and a DD WRT router. I am stuck at seeing traffic in my web logs in method 2. I installed Apache2 and I see the “It works!” screen when I type in my address. Do I need to do anything else to my web server in order to see this traffic? I used the iptables commands listed here with no luck so I tried some that I found over at the weeWX user group on Google…still nothing.
My goal is to receive my sensor values to my RPi without the internet, at any point does this router need to have access to the internet? It is mentioned to be careful because my wunderground username/password is being transmitted, but if its never reaching out to the web, it doesn’t really need to be my real password, right? Maybe I have misinterpreted what the goal of this walkthrough is doing.
I have some maybe silly questions, does my DD WRT need to be in router or gateway mode? Do I absolutely need weeWX working? Ive had no luck with it.
If there are any other surefire ways to capture my sensor values without the internet I would be more than happy to give it a shot.
Thank you very much for any assistance!
Hi Pat,
Thanks for this article! I’m seeing the correct entries in my Apache logs every 16 seconds on my web server but no content is being uploaded to weather underground, and the server light is not turning on. If I manually construct the weather underground URL from the PHP file and the log file on the web server and use it through the web server web browser it uploads and returns “success” in the web browser. Are there any diagnostics I can do to see where it’s falling down? I’m not seeing any data reaching my firewall from the web server PHP file. Not sure if there are any ports apart from possibly 80 and 443 outgoing I need to have open?
Cheers
Hi Pat,
Thanks for this article!
I’m pointing my IP device to a Raspberry Pi LAMP server (my IP device allows me to choose the IP/DNS name), and am getting the following example in my apache logs, but WeatherUnderground is not being update and my “Server” light is not on, hoping you can point me in the right direction.
pi@raspberrypi:/var/www/html/weatherstation $ sudo tail -1 /var/log/apache2/access.log
192.168.10.200 – – [30/Oct/2018:20:31:28 +1100] “GET /weatherstation/updateweatherstation.php?ID=XXXXXXXX&PASSWORD=XXXXXXXX&tempf=68.4&humidity=74&dewptf=59.7&windchillf=68.4&winddir=33&windspeedmph=2.91&windgustmph=4.92&rainin=0.00&dailyrainin=0.00&weeklyrainin=0.00&monthlyrainin=8.57&yearlyrainin=9.77&solarradiation=0.00&UV=0&indoortempf=75.6&indoorhumidity=56&baromin=29.83&lowbatt=0&dateutc=2018-10-30%209:31:13&softwaretype=Weather%20logger%20V2.2.2&action=updateraw&realtime=1&rtfreq=5 HTTP/1.0” 200 359 “-” “-”
When I do a wget via the console by manually composing the string by appending the string from the logs to the WeatherUnderground site it works.
Cheers.