Photo from Amazon
Last Updated 2/26/2017
Table of Contents:
- Method 1: Configure the ObserverIP destination hostname using telnet
- Method 2: Redirect traffic using a router and iptables
- Accepting the data on your server
- Creating a socket for use with weewx
- Undoing your changes and returning back to defaults
- How I use this data to run my own website!
Recently I received an Ambient Weather WS-1200-IP weather station. It’s a great unit and very easy to use and setup! There is an outdoor unit which does temperature, humidity, wind speed, wind direction, rainfall, UV and solar radiation. There’s also an indoor unit which has a barometer sensor. The 2 sensors talk to an ObserverIP unit. This unit plugs directly into your router and has a little webpage that you configure your wunderground.com username with. Once you click save, it instantly starts uploading data to wunderground.com. Couldn’t be easier to start sharing your weather station data!
Except. I want that data. I want to archive it on my own system. I want to be able to report off it. The little web interface on the ObserverIP does have a live data page -- but that’s it. No way to change the server name, nothing! You can telnet to the ObserverIP, but the options don’t work. Everything is locked in the firmware.
Note: this method should apply to any weather station that uses the ObserverIP module.
Method 1: Configure the ObserverIP destination hostname using telnet
This method may or may not work (see my note in Method 2) depending on your ObserverIP type. Let’s hope it works, since it’s the easiest way to redirect the weather data. This method will redirect the data right from the ObserverIP to your web server. Your web server can be on your LAN (in your home) or a server on the internet. Using a program like PuTTY, you need to telnet to the ObserverIP, then run these commands, setdsthn yourserver_ip_or_hostname, saveconfig, reboot. Remember to change yourserver_ip_or_hostname to a valid value (like 192.168.0.100 or yourserver.com).
Connected to 192.168.0.50. Escape character is '^]'. username: admin password: admin Successful login through telnet telnet> setdsthn yourserver_ip_or_hostname Ok telnet> saveconfig Ok telnet> reboot Connection closed by foreign host.
Continue to the next section titled Accepting the data on your server
Method 2: Redirect traffic using a router and iptables
When I initially wrote this blog in January 2016, the setdsthn didn’t work for me. I don’t know why -- maybe I didn’t run saveconfig? When I revisited this in June 2016, the setdsthn worked for me, so that’s the solution I’m using now. Either way, I originally used this solution below, and it worked great!
I found myself poking around the weewx user group, just aimlessly browsing, when I came across this question that Sergei had. It doesn’t have anything to do with the ObserverIP, but it contained 2 pieces of information that I could use. I knew instantly I had my solution. Now to “translate” it to my needs. Two steps: redirect ObserverIP’s traffic, and have it go to a socket server.
All done without modifying any the ObserverIP’s core settings. No modifications are made to how the ObserverIP works. No warranty voiding or anything like that. This is all external to the weather station’s operation.
What you will need is a router capable of running dd-wrt. dd-wrt is custom firmware that unlocks more potential for your router. Think of it like a little Linux server running on your router. You can also accomplish this with a device that runs OpenWRT (such as the TP-LINK MR3020, however this write up will only cover the dd-wrt firmware).
I had recently upgraded my Linksys WRT54GL to a newer Asus wifi router, so the Linksys was a spare device collecting dust. It already had DD-WRT firmware loaded on it, this was a perfect fit for this project! I plugged the Linksys into my primary Asus router. Then I plugged my ObserverIP into the Linksys. The Linksys becomes the middleman and does the heavy lifting to redirect the data.
Here’s an awesome picture I made that describes what I mean.
Once you have it wired, and you’re able to log into the dd-wrt web interface, you need to give the ObserverIP a static DHCP reservation from the dd-wrt web interface. This is so every time the Linksys or the ObserverIP reboots, it gets the same IP.
Note: Unless you’re doing advanced static routing for multiple subnets, you will need to plug a laptop or PC into the Linksys to complete the steps below.
- From within the dd-wrt web interface, go to Services menu > Services sub menu > DHCP server > Static Leases. Just enter in the mac address of the ObserverIP, a name and IP you want to use. Click apply settings at the bottom and reboot the ObserverIP. It should come back up as the IP you just gave it.
Once you’ve done that, we need to apply some firewall rules:
- From within the dd-wrt web interface go to the Administration menu > Commands sub menu. Enter this into the box and click Save Firewall. Note; you will need to edit this to match your setup
iptables -t nat -A PREROUTING -s 192.168.1.55 -p tcp --dport 80 -j DNAT --to-destination x.x.x.x:80 iptables -t nat -A POSTROUTING -j MASQUERADE
- Replace 192.168.1.55 with the IP of your ObserverIP, and replace x.x.x.x:80 with the IP of your web server and port you want the traffic to be redirected to. This can be a public web server, or a private one on your LAN.
The first line says that ANY TCP TRAFFIC ON PORT 80 from 192.168.1.55, redirect it to x.x.x.x:80. Since we know that the wunderground API says to use port 80 to upload sensor readings, we want to catch and redirect traffic on port 80 from the ObserverIP.
Once you click Save Firewall, it takes about a minute or so, but you should start to see traffic coming in on your webserver (check the access and/or error logs to verify)! Note: this traffic coming in contains your wunderground username & password. Be sure the IP you’re redirecting to is yours!
On your webserver open your web logs, you should some new traffic coming in looking like
GET /weatherstation/updateweatherstation.php?ID=
Accepting the data on your server
Now that the traffic is coming in (from Method 1 or Method 2), we need to do something with it! You’ll see that it begins with getting a specific directory and file. We need to emulate this in order for the ObserverIP to be happy. On your webserver, make a new folder at the root level called weatherstation, and create a new file called updateweatherstation.php. Inside that PHP file, place this as the first couple of lines:
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);
echo $wunderground;
What is happening here is when the ObserverIP traffic is redirected, it’ll access this PHP file on your webserver. This PHP line is forwarding the entire request right to wunderground straight away so you don’t disrupt your wunderground readings. The result is being echoed, and it should say “success”. You will know if this is working because the “Server” light on your ObserverIP should come on. This indicates everything is being forwarded correctly.
Note: this PHP snippet does not have any security applied. See the below snippet on how to apply security using the ID & PASSWORD from the ObserverIP update string.
If you’ve gotten this far and you see data in your web server logs, and the ObserverIP has the “Server” light on, then that’s it! You’ve successfully redirected the weather data! It’s now yours to do whatever you want with it (e.g. send to a website, log to a file, send to MySQL), all while not disrupting your wunderground setup.
If you want to import this data into weewx, continue to the next section: Creating a socket for use with weewx
Creating a socket for use with weewx
Now that I finally have my weather data available to me, it’s time to get it into weewx. I had mentioned that question Sergei asked contained 2 pieces of information I needed. The first was the iptables redirect, and the second was from mwall talking about the hackulink driver. I forked a copy of that hackulink driver and created my own. I named my weewx driver weewx-socketlogger.
There’s some prerequisites to getting it running:
Step 1: Install a socket server. There’s a bunch out there, and even Python can create one for you. To keep it simple I used ncat. So, install ncat. I use CentOS, so yum install nmap. On Ubuntu you can run sudo apt-get install nmap to get ncat.
As a test, run ncat with these options:
ncat -l 2999 --keep-open --broker
Then connect to it using telnet to see if it works: telnet localhost 2999 If it connects, you’re in good shape!
This has ncat start a socket server on port 2999, keep it open even if clients disconnect, and broker the data so that all clients can see what the other clients are saying.
Now we need ncat to run on startup. To do this, I just ran it from my local user’s crontab. You can use /etc/rc.local if you’d like, but I found this to be easier.
Run crontab -e and add
@reboot /usr/bin/ncat -l 2999 --keep-open --broker >/dev/null 2>&1
This tells the server that on reboot, to start ncat with the same options as above, but run it in the background. That’s it!
To verify if it’s working you can run the command sudo netstat -tulpn | grep 2999 and you should see output similar to this:
tcp 0 0 0.0.0.0:2999 0.0.0.0:* LISTEN 1166/ncat tcp6 0 0 :::2999 :::* LISTEN 1166/ncat
Step 2: Edit the updateweatherstation.php file so that on every ObserverIP update, it not only sends data to wunderground, but also to the socket that weewx will connect to. My full updateweatherstation.php file looks like this. Be sure to change the YOUR_WUNDERGROUND_USERNAME and YOUR_WUNDERGROUND_PASSWORD to match what you saw in the access logs from earlier. This security method is minor, but it helps to prevent any bots out there from submitting false weather data to your archives.
With this added to our PHP file, you can telnet to your socket on port 2999 and watch the data come in. This is the same data that weewx will use.
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.1,windDir=255,windSpeed=4.03,windGust=7.38,dailyrain=0.00,radiation=448.98,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=201,windSpeed=0.00,windGust=0.00,dailyrain=0.00,radiation=458.40,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=336,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=451.75,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=20.7,windDir=249,windSpeed=3.58,windGust=4.92,dailyrain=0.00,radiation=461.08,UV=1,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=339,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=462.55,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=21.0,windDir=253,windSpeed=3.36,windGust=4.92,dailyrain=0.00,radiation=438.02,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=25.2,windDir=319,windSpeed=2.46,windGust=7.38,dailyrain=0.00,radiation=460.68,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=15.3,windDir=316,windSpeed=9.84,windGust=14.99,dailyrain=0.00,radiation=469.42,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=14.7,windDir=304,windSpeed=10.74,windGust=12.30,dailyrain=0.00,radiation=478.19,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.45,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=19.6,windDir=316,windSpeed=4.47,windGust=7.38,dailyrain=0.00,radiation=461.51,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0 outTemp=25.2,outHumidity=29,dewpoint=-2.7,windchill=17.4,windDir=334,windSpeed=6.71,windGust=7.38,dailyrain=0.00,radiation=471.64,UV=2,inTemp=68.2,inHumidity=48,barometer=29.44,txBatteryStatus=0
If your telnet connection looks like the above, then you’re in good shape! PHP is getting the data from the ObserverIP, and sending it to the socket. Now, you need to install the socketlogger weewx driver from the instructions on GitHub, restart weewx and weewx should connect to the socket and start logging things for you!
Note, there is no weewx loop interval like that’s found with other weewx drivers. That’s because every time a new update comes in from ObserverIP, PHP will accept it and send it to the socket. Weewx is connected and listening to the socket. When it sees new data on the socket, it will grab it and parse it then submit it as a packet to the archive system. Then weewx can generate its reports and/or update other weather services like CWOP, PWSWeather, your own website and more!
I changed my mind, I don’t want to do this. How do I undo it?
Method 1:
Easy! Just telnet back into your ObserverIP and run setdsthn rtupdate.wunderground.com, saveconfig, reboot. That will send the data back to wunderground’s update servers.
Method 2:
Easy! Remember, we haven’t changed anything on the ObserverIP! Unplug the Linksys and put it back in the box. Plug your ObserverIP back into your primary WiFi router (or whatever you have). That’s it! You’re back to the normal way it should be setup. You can use the IPTools software again to find your ObserverIP and manage it that way.
How I use this data to run my own website.
So now that we have control of the data, you can do anything you want! In addition to hooking this into weewx for archiving, I also have it write to a simple JSON file on every update from the station. I use this file to run some AJAX background updates on my weather website. The website checks for updates to the JSON file every 10 seconds and updates the weather data. This means that the website can show the most up-to-date information without having to refresh the page!
You can see that website here: https://belchertownweather.com
I also use Raspberry Pi model 3, with the PiTFT Plus from Adafruit, and have created a 3.5″ version of my weather website specific for the Pi. So now when the Pi boots up, it opens Chrome automatically and loads the special Pi console webpage. I can have very inexpensive display consoles throughout my house, or even on my desk at work!
The Pi console webpage has 2 major changes, the first one is the obvious size constriction. So I had to cut out some details to make everything fit. The second change is that the normal website offers background updates for only 30 minutes, then stops. This is to prevent any visitors who come to the site from eating up a lot of bandwidth. I may change that one day. The Pi console -- since it’s only me for now -- doesn’t have that restriction.
Here’s a screenshot of the website console on the Pi:
And a photo of the Pi itself. Once I get a case for it, I’ll update the photo.
If you have questions or want me to elaborate on a specific step, let me know in the comments!
The purpose of this site is for me to share my DIY technical experiences as a place to document them publicly. Hopefully somewhere along the way they are helpful for you.
First of all THANK YOU for writing a great article. I am little new at this however and have a question: you wrote:
So on your webserver, make a new folder at the root level called weatherstation, and create a new file called updateweatherstation.php. Inside that PHP file, place this as the first couple of lines:
I dont have my own webserver. I just run a ws-1400-ip and a have a desktop. I as well am looking for that data though.
If I dont have my own webserver am I probably out of luck with your article?
Thanks
Jeff
Hey Jeff, sorry for the delayed response here. You could control your data still if you wanted to run a web server on your home PC. You could Google like XAMP or WAMP. This will give you a web server (Apache), PHP and a database server. I would suggest giving your PC a static IP on your network. You could then point the WS-1400-IP to your PC and log to data locally. Just need to keep your PC on 100% for it to track. Hopefully this helps!
Thanks for the detailed post. I spent a lot of time spon theinning my wheels not exactly realizing that I should _downgrade_ the ObserverIP firmware so I could put in my own server’s IP address. After trying the IP nat/masq tricks (on my Ubiquiti EdgeRouter [which runs Linux]), which did work, but was ugly for other reasons, I realized the need to downgrade. Eventually I found the trick to find old firmware versions on the Ambient Weather web site (which was also in the comments here). Now everything is working nominally with updates to Wunderground being made and making use of the SocketLogger functionality in WeeWx
I was wondering if I might have missed a post that describe the web page you built for your Pi-based display? I saw the “how to setup a kiosk” post, but I was wondering if there was a post somewhere or a github page for your mini-Pi-Display.
Thanks!
Hey Christopher – Glad this worked and glad socketlogger is working for you too! Nope you haven’t missed the Raspberry pi kiosk post. I have it on my spare-time-list of things to do but haven’t gotten to it yet.
It would require a little bit of setup, but nothing too hard. I’ll plan to get that going soon!
Pat,
Great article with clear directions.
I previously used Meteobridge to capture data from my ObserverIP, but it suffers from the same freeze-up you describe above.
Since then I managed to get the traffic redirection method working on a FreeBSD system. Your directions are clear and easy to follow. But like any similar project avery step of the way was a learning process, from flashing dd-wrt on my router, to getting ncat to work, and installing and configuring weewx with your socketlogger driver.
All is working now and appears to be running well – thank you.
Louis, I’m glad it was able to help you out!
Hi, just spent the weekend getting this all up and running. First off, thanks for the post and detailed instructions. They were spot on. I did come across some struggles as I was following directions and not thinking…specifically around setting up the router.
For the router, you need to connect the Observer IP to the LAN port and have your main network be connected to the WAN port. You also need to ensure that your LAN Network is different. In the example and my implementation, I used 192.168.x for my LAN network and my main network is 192.168.1.x
I am using Ubuntu server (16.04) and for the NCAT section, I found a really good set of directions and template https://github.com/fhd/init-script-template for adding a program to the start up. When using this template, ensure you remove the “&” (Background) character from the command line in order to get it working.
Once that’s set up, I simply used: sudo update-rc.d enable
Now, I can start/stop the service via the command line and it starts automatically on boot.
One question I did have was around a log entry that says the weather station doesn’t support date/time. Is there a reason why the dateutc field wasn’t implemented?
Glad you got it working! In regards to dateutc, I know the ObserverIP sends it with its update packet, I don’t know why I ignored it. Good catch. I updated my the PHP snippet with the dateutc.
$socket .= "dateutc=".$_GET['dateutc'].",";. Hope this helps!Oh now I remember. I think weewx needs the epoch, and not the format that ObserverIP sends dateutc. So I implemented this instead:
$socket .= "epoch=".time().",";. I changed the code snippet above to have the epoch instead of dateutc.For the record, it appears that the ObserverIP is using HTTP/1.0, without a HOST: header, so a named virtual host doesn’t work. I.E. the site you are sending to needs to be the default site for that IP address.
Great info Larry, thanks! If you need to send it to a VirtualHost on the IP, you could try the
iptablesmethod I explained.I just added another IP from my block as (yet another) alias on that ethernet, and made the changes to apache. Just wanted to get the info out there.
So, it’s actually sending a HOST: header, but with the IP not the damn name:
Hos
0x0210: 743a 2031 3932 2e31 3437 2e32 352e 3730 t:.192.147.25.70
Damn it.
Another quirk to add to the pile
RE: WS 1400IP to weewx
I downloaded and flashed the aercus firmwear to my 1400 IP
http://www.aercusinstruments.com/content/downloads/WH2600_V2.1.9.bin
which gives me the option to use a customized destination for data.
I sent this data to my server (at 192.168.1.100) on an open port and then used the WEEWX Interceptor driver with the observer option to capture the data on that port and used weewx to forward it to wunderground and my weather server at http://www.toxaway.tk
Since your work was used to make this possible, THANK YOU!!
I believe at about $160 for a WS1400 IP on Amazon that this is the best option for weather enthusiasts on a budget.
I’m glad I was able to help! So you have an Ambient WS1400, and the Aercus firmware worked on it? That was one thing I was wondering about – but I wasn’t brave enough to try it on my Ambient!
I did the same thing (following the posts on http://www.wxforum.net). The Aercus firmware has the same Host: header issue I report above, but I do like the ability to customize the destination in the UI.
For some reason I get an 400 error in the file_get_contents function in my local updateweatherstation.php script … So I did the following:
$wunderground = 'http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?' . $_SERVER['QUERY_STRING'];$wunderground = str_replace(' ', '%20', $wunderground);
$wunderground = file_get_contents($wunderground);
echo $wunderground;
and now it works.
Just in case someone else is struggling with this issue. I spent some weeks 🙂
Cheers,
Hanno
Thanks Hanno! This could be helpful for others who may run into it as well.
Hi Larry, can you elaborate on the host/header issue? Are you saying that a regular .com on a shared hosting account won’t work as the destination server?
I have the Aercus Weather Sleuth, but have been unable to get it to send data to my website (www.ozstorms.com). I have a weatherstation/updateweatherstation.php file setup on my server ready to write the content it recieves to a json file. But, I never even see the server light come back on after updating the destination server and doing a reboot of the device. Very frustrating!
Does it need to pointing it to a dedicated IP address?
This instructional is great and works well – it’s helped me out a lot! I love how your instructions actually teach what’s happening as you go along so when things don’t work it’s easy to troubleshoot.
I’ve had this implemented for about a week now but unfortunately my outside temperature sensor has decided to consistently read high. The manufacturer has said that I need to buy a complete outdoor unit, but as I am in New Zealand it’s $100 plus $100 shipping. I’m thinking the obvious thing to do here is grab the outside temperature and correct it, or try and find a replacement sensor and solder it in. Since the latter seems to be a bit hard I thought some extra code would work.
There doesn’t seem to be any obvious way to do it before sending the traffic to Weather Underground, but I’d be able to at least have my WeeWX correct and then pass it to Weather Underground that way.
I’m not that familiar with PHP as I am learning Python at the moment but I do know that I need to alter this line here:
$socket = "outTemp=".$_GET['tempf'].",";To say something like:
$socket = "outTemp="(.$_GET['tempf']) - 20.",";Are you able to give some advice?
Thanks Dene! Some of it is so I don’t forget what I hacked together in a couple years time 🙂 and at the same time hoping it helps!
To be honest, what you put looks close, but I think it should be below. Give it a try – telnet to the socket port and watch the data as it comes through to see if it’s adjusted correctly.
$socket = "outTemp=".($_GET['tempf'] - 20).",";Come to think of it, PHP thinks that’s a string so it may not work. Might need to do a type conversion first. You could try this. Add a new line above the
$socketblock, something like:$adjusted = (float)$_GET['tempf'] - 20;Then change the
outTempline with this one:$socket = "outTemp=".$adjusted.",";It’s two more steps (so it’s not optimized), but it should work. Telnetting and watching for data to come in will tell the story.
Also, you don’t have to use the
$_SERVER['QUERY_STRING']to send to wunderground. You can manually create the wunderground URL submission yourself with the adjustedoutTempvalue. That way wunderground will be in sync. To see what the URL parameters should look like, you’ll find that in your web server logs as well from when the ObserverIP submits info to this PHP script. It’ll all be there, then you can mirror it with PHP easily.Thanks for your help. If this doesn’t give accurate readings I’ll get an additional sensor with a Raspberry Pi for outside and then I can substitute this reading for the external temp.
Thanks this is great, I have the redirect working but I don’t have a box to run weewx so I was wondering if there was a way to output the variables as a string in a txt file using file_put_contents() that cumulus could read .
I’m not familiar with Cumulus but sure you can output the variables using
file_put_contents();. Remove the$fpsection from lines 47-51 and replace with something like$file = 'output.txt'; file_put_contents($file, $socket);. Tailor the output to your needs for Cumulus to read it.Excellent article! We are linking to this particularly
great article on our website. Keep up the good writing.
Thanks!
Thanks for the write up. I have a more fundamental question. Why not allow the data to be posted to Weather Underground? Did you not want it published? I recently did a project called the Weather Widget where I grab my backyard weather data off the WU server using their free API? It spits out JSON or XML data and as long as you only update the weather data about once every 10 minutes it will remain free.
I figure I want the data publish all my local weather first. I then built a little hardware IOT device using an ESP8266 Node MCU board. It talks to the API and displays the weather data locally on a small screen. See here: https://mobile.twitter.com/JP_AustinTx/status/824097414697074688/video/1
Similar to this: http://www.instructables.com/id/ESP8266-Weather-Widget/
This way all my data still goes to WU and I can retrieve it as well without a lot of hassle with my network. Just a second idea to try.
Hi JP, I am publishing my data to wunderground. It’s the first line in the PHP code 🙂
Ok, it’s line 18 after doing some security checks, and a function.
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);echo $wunderground;
Anyways, after WU responds to my rtupdate, I continue on to archive data to a socket server, and to my live update website using JSON.
Sure, I could scrape Wunderground for this data, but why? I already have it in this script as virtual data… submitting to weewx and updating my website is only milliseconds away, and saves me from having to scrape wunderground.
I want to stress that it’s important to me that I own my data. I don’t want to hit an API rate limiter to have my own custom website. Doing it this way allows my website to have unlimited background updates everytime the station checks in.
I like your micro screen, I may look into doing something like that too with my own JSON file and get live updates every time the station checks in! 🙂
Just came across this page, looks very useful, thanks!
To give another reason, at my place, when the weather is bad, my internet can go down and I lose all data while it’s down since it can’t be posted to wunderground. Also, with some of the recent changes they’ve been making, they showed my station wasn’t reporting, which ended up being I needed a new firmware upgrade to deal with their changes. I’ve been looking for a way to log data locally for these types of situations, and this sounds perfect. Thanks again!
Glad you found your way back! Hope it helps out!
Hi Pat,
Have you considered making the code that generates your website public? I may have missed it, but I didn’t see it on your site, and it’s far better than a lot of the other skins I’ve used with weeWX!
Hey Steve, are you referring to my https://belchertownweather.com website? If so, thanks! That’s a custom WordPress website I created using a mix of live weather data from this post, as well as the weewx report information updated every 5 mins. There’s other custom scripts to get the earthquake data, and wunderground forecast data too. I have thought about putting something together on it, but it’s so custom that it would be either a long post, or need to be tailored for public consumption.
Well, Ambient has updated the firmware to 4.0.1, and setdsthn no longer works (this FW also allows data to go to AmbientWeather.net, their Dashboard for personal use).
I think Aercus has picked up the latest stuff from Ambient. With WU’s move of stuff to AWS, you really need the later firmware. 🙁
I’m getting “attitude” from Ambient when I mention the telnet stuff.
I wish I knew what Boris (MeteoBridge Author) is doing . to get the data from the observerip. I can’t seem to snoop the packets. 🙁
Ideas?
MeteoBridge is just screen scraping the Observer’s web interface – which I’ve had very bad luck with doing.
I’m still on version 3.1.3 (which was the first fix to the AWS & indoor barometric problems). You could probably downgrade back? Here’s their firmware listing: http://site.ambientweatherstore.com/iptools/
Looking at their changelog, I’m guessing I won’t upgrade past 3.1.8 since I don’t need to have my ObserverIP connected to their “cloud dashboard service”
Thanks for the info that he’s just screen scraping. Since I have the meteobridge, I’ve set it up to “mimic” the WUnderground GET to my server that wanted the info. And I have the ObserverIP sending to WUnderground.
The newer firmware seems(!) to have less memory leaks, I.E. it doesn’t get slower over time, needing a reboot.
Ah yeah the memory leaks are a problem I have. I ended up putting my ObserverIP onto a power outlet I can turn off/turn on from my phone. So if I notice my site isn’t getting weather info, I can quickly reboot it.
I may upgrade to 3.1.8, but I still don’t think I’ll go to 4.0 – especially after you shared your experience!
Should be possible to binary patch FW to point own address instead setdsthn?
I’m not sure if that’s possible – never tried to reverse engineer the FW.
setdsthnseems work just fine in version < 4.0Yes Pat, you’re right, version 4.0 and setdsthn does not work.
I had a look, it seems to be an ASIX module and the address for update is clearly written on the binary. Is there a checksum to modify also?
I would have to assume there is a checksum, but I don’t have any experience with it
Hi Pat,
First of all, thanks for writing a very thorough article.
I have a Observer IP (433Mhz version). Tried your way and can see the data passing through the 2999 port; only thing left is to make my installed weewx (on ubuntu running on an Android TV stick, MK802) accept the data and display it correctly.
My question is, is there any way to steal or fork data being sent to wunderground (by ObserverIP) to my server?
The reason I ask for this because I do not want my wunderground update relying on my tiny server; the ObserverIP is reliable enough for me; just want to get data to my Weewx for private use.
Hey Chote, I understand what you’re saying and I don’t think there is a way to have the ObserverIP send to dual destinations. That’s why I have it come to my server, then I “fork” it from there.
Hi Pat,
Thanks. I copied the socketlogger.py to /usr/share/weewx/user, copied your snippet in weewx.conf to the end of the original weewx.conf. Stop and restart weewx. I still see data coming thru port 2999; Wunderground still get updated, but my local weewx webpage and my pwsweather station do not get updated. Not sure what I did wrong.
FYI, apache, php, weewx are all in the same server.
sorry Pat. It’s ok now. I did not change 2 things in the weewx.conf:
localhost –> 192.168….
station_type = from simulator —> SocketLogger
Working great now. Thanks a lot.
Great! Glad you were able to get it working
I’ve just started using a PWS, specifically the WS-1200-IP. At the moment I’m very happy with how everything is tied together, but since I like to tinker at times and wondered about the telnet interface on the oIP I stumbled across your blog here. I admit that I’ve just skimmed the surface of what you are doing here, but it just seems you are redirecting (technically proxying) the observer module’s posting to port 80 to your own server instead, which then allows you to do whatever you want with said data.
If setdsthn is broken, another method to do this is to lie to the observer what the DNS IP is for wunderground. Since I run my own internal DNS servers at home, this is relatively easy, but I understand it can be confusing and not as ideal as what you have done. Basically, if you are running a version of BIND, and you have the observer pointing to it for DNS, you can create a custom “view” in DNS that just equates wunderground’s DNS name to your own server. If you are running your own webserver you can easily install BIND and just set it up for use by the oIP.
Great idea Karl
Found out the hard way that I’m going to have top find a different way to do this. I updated to 4.2 from 3.1.2 to try and solve inconsistent updates to Wunderground and reading about the move to AWS. That immediately broke setdsthn. I tried downgrading because I really only want my data and don’t care about wunderground but never got a reliable connection after that.
Back on 4.2, I thought that I would try the DNS hack. I’m running PiHole and tried using it to redirect server requests to rtupdate.wunderground.com following the instruction here: https://discourse.pi-hole.net/t/domain-redirect/1826/4
It sort-of works, but Wunderground goes 3-5 minutes between updates, while updates sent the socket burst 10-15 updates every ~30 seconds, and they never get picked up by the socketlogger.
Can anyone confirm that the php get URL is the same with IPobserver 4.x firmware?
I never should have updated 🙁 To make matters worse, as I was doing all this Tropical Storm Emily was raging outside – think of all that good data I lost!!
Oh no that sucks!! I’ve played with Pi-Hole before but I don’t run it anymore since I am using some DNS Blacklisting on PFSense (admittedly I’ve been thinking of running Pi-Hole again).
I haven’t upgraded to 4.x in fear of breaking this setup so I can’t confirm what the PHP GET vars are. My guess is that they’ve changed because of the way Ambient is doing their weather pages now.
If downgrading doesn’t work you could try Method 2 of this article and buy a cheap TP-Link MR3020 router for $30 and flash it with DD-WRT and let it be the middle man. It would run in bridge mode, the ObserverIP would be hard wired to the MR3020 and then the MR3020 would connect to your WiFi network. The IPTABLES firewall on dd-wrt would be the redirecting method to your server (and not DNS). When I ran Method 2 on an old Linksys it was pretty reliable!
Running Method 2 can also give you a sense of what the PHP GET VARs would be since the ObserverIP would then be talking to your web server and you could check your web server logs to see the formatting of the URL.
Well, well, well… I just happened to have a spare Cisco e3000 lying around running Tomato router and presto – I’m back in business.
I did slightly modify your iptables command to:
iptables -t nat -A PREROUTING -d rtupdate.wunderground.com -p tcp –dport 80 -j DNAT –to-destination 192.168.your-dst.ip:8122
– since the command only reroutes traffic intended for Wunderground, the new ambient weather service works as well.
– also, this way you don’t have to worry about a static IP for the IPobserver
Also, while troubleshooting some connectivity issues I was having (thanks Synology Firewall!!!), I did get a chance to packet sniff the outgoing GET request. Nothing has changed, so 4.X interfaces with Wunderground the same as 3.X.
Interesting, good to know. I thought in 4.x they were routing the URL to Ambient’s URL. Maybe they kept the URL VARs the same, but different hostname?
4.X does both Wunderground and Ambient.
Or another way to do the same thing, since the firmware is not encrypted, is just use a hex editor to modify the “rtupdate.wunderground.com” to something inside your house. Putting an IP in there may work…
Pat,
GREAT Article. Thank you. I have been mucking around with Pi’s for years. I am starting work right now to incorporate this information into my “whole house dashboard” – Nest Data, Sense data, SMASUNG SH status et al.
My one question, is this a full redirect? or is it a fork. Does the Factory provided display continue to work in near real time? or is it an either or decision? I’ve been running Oregon Scientific weather station for about a year but most of the OSx components have proven unreliable – with no good API between collector and cloud.
GREAT Article. Talk to the Sense guys down the road – need an API for whole home power monitoring!
Hi Steve, the factory display continues to work in conjunction with this setup. Best I can tell the outdoor unit and the indoor unit sends their data over RF and the ObserverIP and the indoor unit both “listen” for that RF data and update their independent information (the screen or wunderground).
I’ve been thinking of replacing my factory display with the Raspberry Pi screen I show here entirely. I think my layout of information makes more sense than theirs does!
Appologies. I read the article too fast scanning the code. You explained it very well here:
“What is happening here is when the ObserverIP traffic is redirected, it’ll access this PHP file on your webserver. This PHP line is forwarding the entire request right to wunderground straight away so you don’t disrupt your wunderground readings. The result is being echoed, and it should say “success”. You will know if this is working because the “Server” light on your ObserverIP should come on. This indicates everything is being forwarded correctly.”
Thanks. sorry to bother.
No bother at all!
Pat, thank you for this great article. This gives me the method to add soil temp wx data I send to WU. In my case I’m running the Android app Bit Web Server ($3) on an old cracked screen Galaxy S3. This old phone also sends cloud pics to WU and is weatherproof mounted just below the WS-1400 sensor. The modified updateweatherstation.php (code snippet below) runs on the web server on the phone. With WS-1400 firmware version 3.1.2 and additionally adding the telnet command “setdstport 8080” to accommodate Bit Web Server’s port this combination works well. Thanks again for the idea! Oh, apparently Amazon fixed their server issue, which prompted 3.1.3+ updates, a couple months ago. I’ve run 3.1.2 for months, observed some baro pressure data dropouts for a few weeks, and now normal behavior for the last couple months – again 3.1.2 firmware. Maybe my trouble with 3.1.8 was some intermediate config mismatch but just wanted to confirm success with 3.1.2 this post date. Code snippet:
$wuget = “http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?” . $_SERVER[‘QUERY_STRING’];
if ( $soiltempf > -55.0 && $_SERVER[‘REQUEST_TIME’] – $updatetime < 2*60*60) {
$wuget .= "&soiltempf=" . $soiltempf;
}
$wunderground = file_get_contents($wuget);
echo $wunderground;
Hey Mike this is great! I’m glad this was helpful. Also – I’d really love to see a photo of your Galaxy phone mounted beneath the WS-1400! That sounds cool. I would love to be able to send cloud pics to wunderground but haven’t found a solid way to do it. My outdoor sensor isn’t near utility power, so I’d need something that can be solar charged and an old cell phone is not a bad idea!
If you want to share a photo, I’m not sure if you can do photos in comments, but you can upload to to Imgur and then share the link here. Would be cool to see it!
Edit: Also, is your cell phone running the soil temperature too?