Friday, 11 January 2019

Yesterday was a good day

I've been fiddling with my B.A.T.M.A.N.-inspired mesh network code over the last few weeks and finally reached the point where I couldn't put off turning it into an Arduino library any more.

Sometime I'll learn from my past mistakes and start from the position of writing a library, but this project was not that time.

Now I'm in a position where an Arduino sketch needs just four lines of code to be a functional mesh network node that routes traffic.

  • Import the library eg. #include <EspNowMesh.h>
  • Declare an object eg. EspNowMesh mesh;
  • In setup() start the mesh eg. mesh.initEspNow();
  • In loop() keep it ticking over eg. mesh.meshHousekeeping();
As my goal is to make this simple usable code other people can work with I'm happy with this. My last mesh network code was too byzantine to be usable, even once wrapped up as a library.

I still need to refactor things to match common Arduino style conventions.

For example "initEspNow()" should really be "begin()" and so on. Also the functions I have for putting data into packets and retrieving it are very much single purpose kludgey things rather than something I'd want to offer for real use.

Maybe I'll also change the name of the class completely, I'm not sure I like EspNowMesh as a name

I've written Arduino libraries before but this was harder work. The big battle was around callback functions, which are sprinkled through the ESP8266 WiFi and ESP-Now libraries my library relies on.

Using class member functions as callbacks, or even worse, using class member functions as callbacks for C libraries is a real roadblock if your C++ skills are beginner level. When it's all a monolithic sketch this stuff 'just works'. Make it a class in a library and it's suddenly very broken. This is why I'm prone to just writing a flat Arduino sketch as a proof of concept and worrying about making it re-usable later.

I may do a couple of blog topics with the workarounds I did as other people may find them useful. Simple solutions did not jump off the page of a Google search.

Tuesday, 8 January 2019


Recently I treated myself to a Xiaomi Wowstick 1F+ as Banggood were doing big discounts around Black Friday.

These are a bit of a silly thing, a powered 'precision' screwdriver that's like a big fat pen.

Mostly they're very cute and come packaged like some kind of Apple product, all in separate little white boxes. They're the kind of thing that makes a cool gift for somebody who tinkers and even come with a carrying case that's like something from a sci-fi movie.

Notably the set I bought comes with many many high quality precision bits, perfect for taking modern consumer electronics to bits. I just used this to have the internal cover off my smartphone so I could replace the failing battery and it's already saved me from having to go and hunt a suitably small set of tools down.

There are a few variants available, I thought I was getting the one with a charging base but didn't realise this model was different, so check before order.

Friday, 4 January 2019

Squeezing a quart into a pint pot

The first mainstream appearance of the ESP8266 was the ESP-01 board and it's a breadboard unfriendly horror.

It has eight pins and only two of those are nominally GPIO, but even these are compromised as they have to be pulled high to enable normal bootup. You pull GPIO0 low to program the board.

I've got twenty ESP-01s I want to use. Mostly because I've already got them but also because barring some weird ESP8255 packages they're about as small as these things get.

For my application I need a GPS module, Infrared receiver, pushbutton, status LED and piezo buzzer and I've achieved this with a little care over pin choice.

First with the GPS module the ESP only needs to receive data. In principle you can send commands to the module but I don't need to and that saves the TX pin, GPIO1. It's not often you see this referred to but it's just a variant on the usual Arduino Serial configuration...
Serial.begin(9600, SERIAL_8N1, SERIAL_RX_ONLY);
Luckily the IR receiver is a pullup device, inverting the IR signal so that can be pretty safely connected to GPIO0. In the unlikely event you're unlucky enough to receive an IR pulse at the very moment you power the board on you'll notice as it has a startup sequence involving the buzzer and LED.

For the pushbutton I've combined it with the LED on GPIO2. The GPIO has a 10K pullup resistor, the button connects the GPIO to ground and I check for a low pin state. This is a very conventional setup for a button.

To double this up with the LED, it's connected in parallel with its cathode (normally connected to ground) connected to the GPIO and the anode to Vcc. If you push the switch then it connects the cathode to ground and the LED lights. However if you reconfigure the GPIO as an output and drive it low the LED also lights.

You can't use the button while the LED is lit but some simple logic swapping it from an input to output as needed works around this.

Which leaves the piezo buzzer but now that's enough free GPIOs to connect everything.

There's a lot of understandable negativity around ESP-01s in the community and I'm not sure I'd choose an ESP-01 afresh but you can use them in more than single use applications with some care.

Monday, 24 December 2018

ESP8266 brownout again

Having tried battery life with some Poundland NiMH AA cells I decided to give a LiPo a go.

I've a big pile of 18650 cells recovered from laptop batteries and in anticipation of trying this I bought a few of those ubiquitous TP4056 charger & protection boards as these cells don't have protection built in. Running down a LiPo below about 2.5v is a death sentence for it.

Running the ESP8266 straight off the cell wasn't an option this time either. The 4.2v from a fully charged LiPo is in principle enough to kill it. Having managed to run microcontrollers completely out of spec with no trouble before I expect 90% will survive just fine, at least for a while but if I'm ever to turn this into a completed project I wouldn't want to rely on that across a large number of devices. So I also stuck a little 3.3v regulator in line.

This made for a slightly untidy test rig, but it's functional.

As expected the LiPo lasted much better. I lost track of uptime as I reprogrammed it a few times while working on my code, but I got over 30 hours of constant running.

So I reckon 18650s are a great option for the wearable parts of the project I have in mind, even if they do add complication.

Update: I did this test again and got twenty seven hours runtime.

Monday, 17 December 2018

M5stack camera

I have been impressed with the projects out there to make a webcam out of an ESP32 and generic camera module.

Thinking of the faff in putting this together tidily I've not bought the bits, which is good as M5stack released a complete module for a very sensible £10.

It really is very small. More when I can fiddle with it.

ESP8266 brownout

As power consumption and battery life are on my mind I did an unscientific brownout/rundown experiment with an ESP8266.

I took an ESP-01S module and a couple of part used AA batteries and left it to run sat on my mesh network until it stopped responding.

This won't be the first time somebody has tried this kind of thing but I was impressed it worked down until almost 1.8v. It ran for ten hours on a pair of batteries that weren't great to begin with, coming in at 2.7v when I started.

It would have been sending packets every few seconds all this time, so it's not like it was sat there doing nothing.  I know my code currently causes quite heavy power use, averaging at about 80mA.

This is telling me that my aspiration to run a wearable mesh node 'all day' on normal alkaline batteries is almost certainly achievable. A twin AA battery box is not egregiously large, I can probably desolder the onboard LEDs in a final version and maybe improvements in power management in the code will have some effect.

Of course I'm currently ignoring that I want to connect a GPS module to the wearable and this will eat a consequential amount of power, but I'll worry about that later.

A LiPo battery is what most people would go with but I prefer the wearables to have field replaceable batteries. This is because they will literally be used in a field/forest, the sort of place where you worry about being able to charge your phone as the evening draws in. Having a few AAs to hand is very easy to manage if something goes flat.

For my next experiment along these lines I'll try the same with a 18650 cell and 3.3v regulator. If AAs won't cut it 18650 cells are the sane removable LiPo option in my opinion. There are also 14500 cells which are AA sized so very convenient but these have a much lower capacity.

Sunday, 16 December 2018

Obscure Arduino tips #1

Want to know exactly which ESP8266 board your sketch is compiled for in the Arduino IDE and act on that in your sketch?

Why do I need this? I'm building for a couple of different flavours of ESP8266 boards and wanted to know which board is the target so I can change a value to match the board automatically.

I'm using the ESP8266 core as an example but this should also apply to other boards supported in the Arduino IDE with some tinkering.

  • Find the file 'boards.txt' in ESP8266 Arduino core. On Windows this will be somewhere like "C:\Users\YOUR USERNAME\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.2\boards.txt"
  • Search for the name of your board as shown in the Arduino IDE, for example "LOLIN(WEMOS) D1 mini Pro". You should find a line that looks like " D1 mini Pro"
  • Immediately below this there should be line similar to "build.board=ESP8266_WEMOS_D1MINIPRO".
  • The compiler passes the build.board value on as a #define but it prepends "ARDUINO_".
  • So if "ARDUINO_ESP8266_WEMOS_D1MINIPRO" is defined in your code you know it has been compiled for a Wemos D1 mini pro.
I am using this so I can do a readVcc() and get an accurate value for Vcc across a couple of different flavours of module. Here's a usable code fragment...

float vcc()
  uint16_t v = ESP.getVcc();
      #ifdef ARDUINO_ESP8266_GENERIC

The slightly different values are because the boards have a slightly different set of resistors on the Vcc monitoring connection and these are the values I've found generate realistic readings verified by a meter.

The IDE also sets the environment variable ARDUINO_BOARD, which you can use, for example...


You could use this to avoid wading through boards.txt, or code the above example differently.

Saturday, 15 December 2018

ESP-Now BATMAN test rig

Unusually, I posted one of my videos publicly and I got a good swathe of comments so I think I'll do this more often in the future.

One of the questions was how big does the mesh scale?

Frankly I don't know.

My aim is to support about 40 nodes actively sending data every few seconds because location tracking is my primary goal. This feels achievable.

However to prove this I'll either have to build it or use some network modelling software. I like building stuff.

I have over time bought quite a few ESP8266 modules for various projects, a lot of them speculatively or for things that are over and done with.

  • 12x Wemos D1 Mini
  • 12x Wemos D1 Mini pro
  • 10x ESP-01S 1MB flash
  • 10x ESP-01S 512KB flash
So when I allow for a few I've lost, given away or killed that's about forty.

Plugging all this in at the same time would be a pain in the neck. So I've built a little test rig that allows me to get fourteen ESP-01S running off my bench power supply.

This circuit is simply a bare minimum of suplying power and a pullup resistor to CH_PD so it boots. In principle you need pullups on GPIO0 and GPIO2 as well but in my experience they boot fine with these pins left floating, at least reliably enough for a little testing.

This took a few leisurely hours to build as there was quite a chunk of soldering, especially as I added individual on/off switches.

Running off my bench PSU it draws 1.1-1.2A which is quite a lot. I need to work on my power efficiency, but given my desired runtime is 'all day' I reckon I can get there. The outdoor nodes lasted about eight hours, which is about in line with this power draw given the rubbish batteries I used. 

Compared to a sensor that sleeps almost all the time and draws microAmps when doing so this power usage is awful, but my kit just can't sleep as it has other jobs to do, one of which is always being there to relay traffic for other nodes.

If the nodes were connecting to APs, the various radio sleep modes would reduce consumption massively as the DTIM table means they know when to wake up. Without an AP to manage this I'd have to create my own scheduling algorithm equivalent to DTIM. This is a job for later if I can't make improvements in other ways. I've already got a time sync protocol it might just need to be more accurate.

I did a little video of this test rig running...

Tuesday, 11 December 2018

ESP-Now BATMAN time sync

One of the things I really liked about PainlessMesh was it had a built in time protocol that synced across all the nodes. If you're building a mesh of interacting things then having them share a common clock is useful.

The PainlessMesh developers have gone the whole hog and implemented an NTP inspired protocol taking into account latency and jitter of communication between nodes. While I don't have the enthusiasm to do this, I have come up with a simple clock syncing option that seems to work fine at low mesh sizes.

I was already sharing uptime information across the mesh so my very simple scheme is as follows...

  • The node with the highest uptime is considered the time server.
  • All other nodes work out their offset from this figure as NHS packets come in.
  • There's then a function that returns this calculated 'mesh time'.
  • If the clocks drift then every NHS packet from the time server tweaks it back into line.
  • If the time server goes away, the node with the next highest uptime takes over, faking its own uptime to be what it understands 'mesh time' to be.
  • If the previous time server comes back the current time server stops.
This simplistic approach seems to be working just fine so far and the sum of all clock drift corrections over several hours is in the tens of milliseconds. Which means it really doesn't need to sync very often.

The nodes aren't perfectly in sync down to the millisecond (mostly because ESP-Now iterates through sending packets to its peers) but nobody is going to notice in real use. This is entirely about making events happen in sequence on human timeframes. It doesn't need to be more accurate to achieve this.

I've done a little video demo of it syncing up.

Sunday, 9 December 2018

ESP-Now BATMAN first field test

It was our end of year LARP social event this weekend which includes a little bit of shooting at each other in the woods with Lasertag guns, so I took advantage of this to do a field test of my code and hardware.

With the current sketch loaded onto the six nodes I originally built for testing PainlessMesh I headed off to Banbury. It's been a real success, with a couple of provisos.

First the good news. The range of five of the nodes was very solid, I could see direct ELP neighbour discovery packets from them using an ESP-01 plugged into my laptop inside a large wooden hut. This was just with the trace antenna. I think the sixth node has a fault where I've moved the link to connect the external antenna. This node had crappy connectivity despite being no further way from its neighbours than the others.

Secondly propagation of OGM packets works perfectly, I was getting packets forwarded from every node and the ESP-01 connected to my laptop looked to be making good routing decisions.

Finally I got almost eight hours runtime out of at least one of the nodes with nasty cheap pound shop NiMH AA batteries in the freezing cold and pouring rain. This bodes well for better batteries and my target is only to have 'all day' life from a set, fitting fresh at the start of each day. When I got home and opened the nodes they were all bone dry, despite my vague worry they'd leak at the antenna.

The big proviso is this was only a small site (42 acres) so I simply couldn't expect much in the way of a range test. I think I could have achieved coverage of this site with PainlessMesh.

The other proviso is I've now got eight hours of logs to wade through before I'm sure what I've just said is true. Mostly I was there to socialise so peering at a screen for hours wasn't going to happen.

I now have until March for the next field testing opportunity, unless I make a special trip somewhere.

In the meantime I need to make a device that actively uses the mesh so this isn't all just peering at MAC addresses and routing tables. Given my end goal is GPS tracking and status reporting I think it's time to build a minimal GPS tracker that reports back to a specific node.