Wireless weather station - part 3

After parking this project for a long time, I have decided finally to give it a go and complete this time.

The latest approach is:

1)      having approximately 20 sensors across the house/outside.
2)      records stored on Raspberry Pi running MySQL. 
3)      data presented on a tablet as a web-based dashboard (haven’t decided it yet).

1)    I believe 20 sensor nodes should be enough to have them in various places around the house as well as around it. The design supports up to 32 sensors due to having 5 DIP switches allowing selecting 2^5 possible combinations. Each node will measure temperature and humidity using SHT21 sensor, crude light level using LDR and battery voltage (2x AA). Some of the nodes will be equipped with an e-ink display to present the measures.

2)    I’m currently testing RPi 3B+ but I may pick an earlier edition as the current one runs very hot. I may not need such performance just to read serial data, insert it into a MySQL table and run a webserver that will show just few pages in total.

3)    I’m thinking about Apache + PHP currently but if you have any suggestions -> please let me know.

Let me explain the above in details.

1)    I have started developing it using ATmega328P with Arduino bootloader. As this will be battery operated the MCU needs to be the “P” (Picopower) version. And of course, forget about using Arduino development board as there are plenty of power hungry devices (LEDs, USB communication, voltage regulator) that will eat 2 AA batteries in a couple of days.

ATmega328P on a breadboard with internal oscillator while in a deep sleep mode consumes less energy than a battery self-discharge is. Here is a great article of power saving techniques: https://www.gammon.com.au/power

The node will sleep for most of the time, waking up every minute, taking up few measurements, sending it wirelessly, updating the display and going back to sleep).

As the RFM69H radio as well as the WaveShare 1.54” e-ink display were new to me I have decided to test them separately to see if I can talk to them successfully rather than connecting everything together and wondering if for example the SHT21 is not responding because of display’s IRQ pin is assigned incorrectly in the program…

For the radio part, I have used the LowPowerLab’s RFM69 library (https://github.com/LowPowerLab/RFM69)

I have built a simple RFM69 transmitter and receiver to see if I can communicate between them.

Once the communication was confirmed I have added some complexity to the program. I’m planning to use a struct with all the data I need to send:

// packet structure

typedef struct {

       uint8_t       nodeId;

       uint32_t      uptime;

       int16_t       temp;

       uint8_t       humid;

       uint16_t      voltage;

       uint16_t      light;

} Payload;

I have also utilised transmission encryption (not that the temperature readings are sensitive data but having this working may be a very useful feature in the future).

The next step was to mock-up a fake sensor node that transmits a ‘real’ packet that I will use in the future but with some random data. The receiver got all the measurements as transmitted so at this point I decided that I have all I need to transfer the data and the RFM69W radio works along with my requirements.

Now, let’s try the e-ink.

First, I tried the code provided by the manufacturer (https://www.waveshare.com/wiki/1.54inch_e-Paper_Module) just to see if I can get it to work. Once proved, I have downloaded a great library (https://github.com/ZinggJM/GxEPD) that supports various models of displays. Having this connected in slightly different way and selecting the correct display I got the demo program working as well. The downside of e-inks is that they are very slow to refresh (which means long power draw). The good thing is that the library supports partial refresh, so only a small section, i.e. a digit that changed, can be redrawn – much quicker which means less power needed for the process.

As ATmega328P has very small amount of memory I couldn’t use one of the available font packs for ESP32. I decided to make my own font as I could trim it to my needs and I could make it to fit best on the screen.

As the screen is 200 x 200 px - I need to fit the longest string (i.e. 20.0*) in one line. I plan to put displays only on the nodes around the house and I don’t expect the temperature to fall below 0 (I hope not!) so I omitted negative temperatures/characters to be displayed.

A thing worth keeping in mind is that the partial refresh is 1 byte wide (so it refreshes 8 pixels in a row). This means that having a digit 9 pixels wide will ‘overlay’ the next 7 pixels on refresh. Ideally the digit’s width would need be in 8 bit increments (so 8, 16, 24… bits wide).

I have created an Excel spreadsheet where I designed my own digits. It helped me create the array of 1s and 0s to build a digit.

I use the same font for digits 0-9 for temperature and humidity, a smaller font to represent battery percentage. Then I have created some characters: a degree sign, decimal point, big percentage (humidity), small percentage (battery) and a small battery icon. The layout is as follow:

 

 

I can now read SHT21 and send it over RFM69 and (separately) read SHT21 and display the values on e-ink.

Let’s combine these 2 tasks, shall we?

First bump: once I initialised RFM69 radio, the display stopped responding. When I unplugged the radio from my breadboard the display came back to life. Uhm… pins conflict, SPI maybe? I had to redefine the RFM_IRQ_PIN to a different one (digital 10 in my case) to get the radio and display working together.

Second bump: once I displayed the 3^rd digit the program crashed. I was able to display max. 2 digits on the screen, doesn’t matter which ones, it could be tens from humidity and a decimal place from temperature… all worked fine. Adding more the to the screen (battery %, an icon, 3^rd digit or a degree symbol) caused the same behaviour… When I enabled detailed information after compilation (in Visual Micro) I noticed that I’m using 97% available RAM and 90% of available flash memory. Not good. I haven’t even implemented power saving functions or properly utilised reading light and supplied voltage level. I probably could save few bytes here and there by optimising the code slightly and/or using smaller fonts but making the numbers less visible from a distance and still balancing on the edge of memory capacity wasn’t good for me. I have decided to go for a bigger micro.

I went for ATmega1284P-PU as: 

      -          P - PicoPower 

      -          PU - DIP package (the main size factor is 2x AA batteries so going into SMD won’t help much) 

      -          128kB flash (vs. 32kB in ATmega328) 

      -          16kB RAM (vs. 2kB in ATmega328) 

      -          has an Optiboot bootloader (https://github.com/maniacbug/mighty-1284p) and can be easily programmed by serial connection

I had to change fuses slightly as I needed to run it on internal 8 MHz oscillator, have the BOD disabled (saves power consumption and also, the levels aren’t good in this application, the 1.8V is too low for the SHT21 and RFM69 to operate anyway and the next one – 2.7V would be reached after a while of running the node on 2xAA batteries (3V in total)) and change the boot up time to 0ms (from 65ms by default). The fuses I use are:

   low_fuses = 0xD2
   high_fuses = 0xDE
   extended_fuses = 0xFF

      Here is a schematic:

 

2)    The latest available Raspberry Pi (3B+, 2018) is slightly faster than the previous one but I may not gain much using it as the project is simple and doesn’t require a massive amount of computation power.

I have installed raspbian-jessie, configured the standard way (SSH, expand size, lower GPU memory, etc.), removed the default pi user for security reasons, installed python (receiving data over USB), installed MySQL and created an empty database.

From this point I can use HeidiSQL to connect to the RPi  from my PC and start the database development comfortably (rather than using a command window :)).

I expect to have 20 sensors, a ‘Live’ dashboard that will display the current measurements as well as a 2-day history (or maybe a weekly as well) to compare the values across days. My approach is:

• A python script will find an active USB connection (using USB2Serial board’s VID and/or PID), open the port and will be waiting for any incoming transmission. It will then check if the received string (i.e.  1,234,56,222,1023 -> SensorID (1), temperature (23.4C), humidity (56%), battery voltage (2.22V), light level (0-1023 from analogue pin) has all the values, are in correct ranges, etc. it will issue then an Insert statement to the MySQL database. Each sensor will send 1 message per minute.

• In the database I’m going to create a small table which will store only the latest record (once a new value is received it will replace the existing one) – this will allow me reading all the records without finding the latest record received along with thousands of received records (a sensor may hang or get transmission interrupted, etc. so the latest entry could be hours (or days) old. Also, I’m planning to archive this table periodically so if the sensor won’t start sending details again it may be archived completely and the table won’t store its entry. In the ‘latest record’ table however, the record will always be there.

• The 2^nd table will store all the entries from last 2 (or 7 days), this will be used to draw a daily dashboard (comparing yesterday’s day vs. today’s (from midnight up to now).

• Then, the archive table. I’m going to aggregate data from the above table by SensorID, record date and time (rounded to 15 minutes). I will average records split by 15-minute chunks. I don’t need to know what happened in the kitchen 20 years ago at 8:02 in the morning :)   An average of temperatures between 8:00 and 8:15 will be more than enough. And this will cut the number of records from 1,440 (24 hours * 60 minutes (if sending once per minute)) per sensor to 96 (4 averages per hour * 24 hours). Having 20 sensors, the annual record count will be reduced from 10M to just 700k. I’m not worried about the storage used, I’m going to use external 120GB SSD. The problem is MySQL performance on RPi. I found that it struggles where a table gets to 50M+ records. I tried index on date and time for reporting purposes as well as a table partitioning on year (2018-2100). I have loaded millions of dummy records, it created 10GB file on the SSD and any more complex query was taking noticeable longer.

 

3)    At the beginning I was thinking about an Android app that would pull data over the network and display the results. Seems having a webpage running full screen is much simpler solution. I’m going to create one main dashboard where all the temperature/humidity values will be displayed (maybe in some sort of a drawing of the house), like a top level summary. I may change the font colour depending on a temperature, have a battery indicator (just 3 colours: GREEN if the battery is >80%, YELLOW if between 80 and 30 and RED if <30%) and maybe transmission indicator (GREEN if the most recent entry for the sensor came within 10 minutes, YELLOW if less than 1 hour (to indicate that there is some interference, transmission loss, etc.), RED if the sensor didn’t send (or rather the receiver didn’t receive) a packet for more than an hour).  Then, additional pages could show more details, graphs, averages, history, etc.

I’m open to suggestions if you have any or you may have better solution to the above.

If you like my project and/or you would like to support it, please use the Donate button ☺ 
Follow me on Twitter, Google+, YouTube, Thingiverse or by email to receive the latest updates.