ESP8266 Custom Firmware

If you would like to go beyond the basic features in the ‘default’ ESP8266  firmware, it is possible (and relatively easy) to create your own ESP8266 custom firmware.

ESP8266 Custom FirmwareWith ESP8266 custom firmware, your possibilities for interaction with connected devices are almost endless.

The reason why is that you can now connect pins on the ESP8266 to other devices and control those devices ‘programatically’. This is possible due to the custom firmware using the LUA programming language (specifically embedded LUA or eLUA). The team at nodemcu.com have developed an ESP8266 based IOT module called NodeMcu and open source firmware for it. You can see examples of what can be done with LUA on their main WEB page. Their firmware (available on Github HERE) can be FLASHed on any ESP8266 Wifi SOC.

I am using an ESP8266-01 module (as seen in the above picture). It only has two GPIO pins available, however, that is sufficient for what I need. Other ESP8266 models (like the NodeMcu) have a lot more pins broken out for use.

The documentation for the NodeMCU firmware is HERE. There is a ‘cloud’ (i.e. WEB) based service to build a custom firmware image HERE. However, I prefer to experiment on my own, so I installed the development tools locally on my Fedora Linux system. Here is how I did it:

First, make sure all the Fedora Linux prerequisites are installed:

Next, make sure that the flash tool (esptool.py) is installed (or if installed, is up to date):

Next install the development toolchain for the ESP8266:

That will take a while and let you know the PATH to export so the next step can invoke the toolchain correctly. Next, install the NodeMcu firmware:

Next, read the build documentation to see how you want to customize the options and what ESP8266 function modules you want to include. After you customize:

This will take a while and then produce two bin images in the bin directory:

0x00000.bin and 0x10000.bin

Now you are ready to flash those files into your ESP8266 module. Make sure the ESP8266 is in flash mode via GPIO00 pulled low (normal GPIO00 is high or floating). Here is how it should be wired:

ESP8266 Wired for Flashing(The Fritzing diagram source is HERE.) Then flash the ESP8266 with these commands:

An alternative is to combine (aggregate) the binary images into one file. To make the aggregated file, here is a python script to automate it:

To flash that aggregated file use these commands:

The ESP8266 can then be accessed by a serial connection. I recommend getting the ESPlorer developer IDE found HERE. It makes coding and uploading your LUA code very easy. Here is a screenshot of ESPlorer:

ESPlorer Screen Shot (ESP8266 IDE tool)There are sample LUA scripts in the nodemcu-firmware-master directory for you to experiment with or exercise your LUA skills and develop your own.

For example, here are the commands to setup the ESP8266 as an Access Point and set its SSID and Password:

HERE is an example LUA WEB server script to toggle PIN 4 (GPIO2) from a WEB page. It is called init.lua so that when the ESP8266 is powered on it will run that script automatically. Of course just toggling a PIN is not so useful 🙄 so you will want to attach it to something like a relay to turn something on or off.

For the latest information on this whole process, consult the NodeMcu documentation HERE.

Best Kept Secret

Yes, it is an ambiguous topic. No, I don’t think it should be hidden, however, the fact that this topic does not seem to garner much press could mean it has been a best kept secret.

The “best kept secret” is the programming language Forth. For reference, take a look at “A  Beginner’s Guide to Forth” from J.V. Noble.

Forth is specially useful on microcontrollers due to its ‘small’ size. ‘Mecrisp Stellaris’ from Matthias Koch is a case in point. I have been experimenting with it on my STM32F103C8T6 boards and am amazed at what it can do.

Forth is quite different than other programming languages. When loaded into a microcontroller, It interacts via a serial/usb ‘console’ and, to me, is like an Operating System but better!  It’s an ‘applications programmer’ dream environment uncluttered by a ‘systems programmer’ idea of what should be there. If you read the history of Forth and thus the history of Chuck Moore (the inventor of Forth) you will better see what I mean.

I think Jean-Claude Wippler (jeelabs.org)  understates the usefulness in his article titled ‘F103 + USB = Swiss Army Knife’.  Jean-Claude has enhanced (Mecrisp Stellaris) Forth with lots of code/examples. Take a look at his artcile about it HERE. He also has a communications program (written in ‘GO‘) called Folie which makes it easier to interact with Forth.

To install it on my Fedora Linux system, I ran this:

Then run it via:

and it will prompt you for the serial port.

I have also updated my Python serial program script to communicate with Forth as an alternative to Folie. Here is the code:

Inspired by the built in commands that Folie has to interact with Mecrisp Stellaris Forth I added added some to  my script. Use: !help to list them.

If you are new to Forth it may take some getting used to. It relies on the stack to store everything and thus uses ‘reverse polish notation‘. This way of thinking is different that ‘traditional’ languages such as C or Python, etc.  and is perhaps why some programmers get frustrated and give up.

So, is Forth ‘hidden’ due to frustration and lack of programmers using it or is it just not widely marketed? In any case, why not give it a try? If you need more to convince you, take a look at Elliot Williams’ article called Forth: The Hacker’s Language on Hackaday.com.

How to convert an STM32F103C8T6 into a Black Magic Probe

There are several tutorials on the internet showing how to convert an STM32F103C8T6 into a Black Magic Probe. Unfortunately, none of them worked for me. So, I decided to write this post in case others run into the same problem as I did. The problem I ran into was the memory size of the STM32F103C8T6 chip I am using. It reports that it only has 64K of flash. The Black Magic Probe software is a bit larger than 64K so the utilities report there is not enough memory and stop.  Decoding ‘STM32F103C8T6’

STM32F103C8T6
STM STMicroelectronics – Vendor
32 32-bit, ARM series
F103 model, ‘F1’ = ARM Cortex M3
C 48-pins
8 64 KB flash memory
T LQFP package type
6 temperature range -40..+85 °C

we see that it is only supposed to have 64K, however, it probably has 128K. Mine does!  So, with a bit of ‘trickery’, we can make it work.

There are possibly three different ways to connect and interface with the STM32F103C8T6 board:

  1. Serial port
  2. USB
  3. JTAG / SWD

The utility software (listed below) varies as to what interface it uses. To get my STM32F103C8T6 module to work with the utility I need, I used an ST-LINK programmer which attaches to the SWDCLK/SWDIO/PWR/GND pins.

How to convert an STM32F103C8T6 into a Black Magic Probe Here is a list and some information about the software tools (utilities) you will run across when reading articles about programming the STM32F103C8T6. Incidentally,  I run a Linux distribution called ‘Fedora‘. If you are running a different Linux distribution, you will need to run your appropriate package installer rather than the one (dnf) Fedora uses.

For installing and compiling some of these utilites, you will need git, cmake and the ‘normal’ set of development tools installed. If you do any development work these tools should already be installed.

Here is the command to install all the needed development (C compiler) tools for the ARM processor:

Many of the useful utilities are coded in Python and need the Python serial module. Here is how to install it:

Here is a list of the utilities:

stm32flash
Open source cross platform flash program for the STM32 ARM microcontrollers using the built-in ST serial bootloader over UART or I2C

stm32loader.py
Python script which talks to the STM32 bootloader (via the serial port) to upload and download firmware.

stlink
Open source version of the STMicroelectronics Stlink Tools

dfu-util
DFU is intended to download and upload firmware to/from devices connected over USB.

OpenOCD
Open On-Chip Debugger

Black Magic Probe
In-application debugging tool for embedded microprocessors.

I eventually installed all of these utilities, however to create a “Black Magic Probe Clone” I really only needed stlink, OpenOCD and the Black Magic Probe software. So, here is how to convert an STM32F103C8T6 into a Black Magic Probe:

I installed OpenOCD via:

This step (using OpenOCD to unlock the code protection of the board that will become your new Black Magic Probe) may not be needed but just in case, here it is:

Next, I installed the stlink utilities globally via:

Then, to create the Black Magic Probe Software:

Make sure you are in the black magic probe src directory. Then erase memory and flash the black magic DFU and software ‘bin’ files via:

Note: Here is where the ‘trickery’ comes in. Using the parameter –flash=0x20000 we can override the reported 64K and specify 128K. Now you can unplug the STLINK programmer and plug in the usb port. You should then see two enumerated ports (probably /dev/ttyACM0 and /dev/ttyACM1).

The Black Magic Probe supports both JTAG/SWD and Serial/UART debugging simultaneously.  The 1st port (/dev/ttyACM0) is for JTAG/SWD (GDB server) and the second port (/dev/ttyACM1) is a serial UART.  Here are the Pin references for the Target and the Debugger (i.e.  the Black Magic Probe):

To make it easier, next time I wanted to do this, I created a Python script which runs those needed commands. Additionally, it combines blackmagic_dfu.bin and blackmagic.bin into one bin file called bmp.bin. The script allso runs stlink to dump the memory (after flashing) to compare it with the original to make sure the flash worked ok.

Here is the Python script (it needs to run from the blackmagic probe src directory):

If you don’t have an STLINK programmer, there is an alternate method to create and flash the bmp.bin file. Create the bmp.bin file with this Python script:

STM32FLASH can be used with a USB to serial converter to flash the bmp.bin file via the following command:

Make sure to replace the /dev/ttyUSB0 with your specific port.

To test your new ‘Black Magic Probe clone’, attach it to a target:

STM32F103C8T6 as a BlackMagic ProbeThen run:

Further documentation on GDB (The GNU Project Debugger) can be found HERE.

SSD1306 OLED Displays

For many Arduino projects, there is a need to display information graphically or display more text than what fits on a 16×2 LCD. For that need, consider using SSD1306 OLED Displays.

SSD1306 refers to the controller or driver chip for the display. OLED or  Organic Light Emitting Diodes refers to the dot-matrix graphic display system. These displays attach using SPI or I2C thus using less connecting pins versus an LCD.

Here is how to hook-up a I2C SSD1306 OLED display to an Arduino UNO:

SSD1306 OLED Displays (attached to an Arduino UNO)

The Fritzing diagram source is here. The OLED fritzing part is here.

To write Arduino code for SSD1306 OLED displays, we need to find the I2C address. The MiniPirate sketch I wrote about here can be used to find it. Also there is an I2C scanner sketch here. There are also two libraries needed to control the OLED display:

Adafruit SSD1306 Library   Adafruit GFX Library

Once those libraries are added and you have the I2C address, try the following sketch:

The I2C address for my OLED Display is 0x3C so the display.begin code is:

   display.begin(SSD1306_SWITCHCAPVCC, 0x3C);

You will need to change the 0x3C to the address of your display. Also, the library does not detect the resolution of the display. The sample sketches that come with the SSD1306 library mention how to change the library to match the resolution of your display. Basically, look at the code in Adafruit_SSD1306.h and find the lines which define the resolution. Uncomment the one you need and comment the rest. Here is what that code looks like for the OLED display I have:

Here is what the OLED Display looks like when displaying text:

OLED Display attached to an Arduino NanoOLED Display attached to an Arduino Nano

ARM Cortex M3 STM32F103C8T6

It’s amazing what you can get in a microcontroller board these days for just over $2. For example, just search ebay.com or alibaba.com for the key word ‘STM32F103C8T6’.  You will see the links to an ARM Cortex M3 STM32F103C8T6 development board for just over $2 US including shipping (usually from China, so it may take a bit to arrive). If you want it faster (if you live in the U.S.), you can order from US suppliers but expect to pay more.

The board, sometimes called the ‘blue pill‘ (a reference to the 2016 film “The Matrix”) looks like this:

ARM Cortex M3 STM32F103C8T6 ARM processors are based on the RISC (reduced instruction set computer) architecture developed by Advanced RISC Machines (ARM). ARM licenses the technology to manufacturers such as STMicroelectronics which makes the processor on this particular board. The processor data sheet can be downloaded HERE. The reference manual is HERE. The programming manual is HERE. The system memory boot mode application note is HERE.

Briefly, here are the specifications:

  • 32-bit ARM Cortex-M3 processor 72 MHz
  • 64 kB flash memory, 20 kB SRAM
  • USB and CAN controllers
  • 32 kHz crystal for RTC
  • dual 1 us A/D converter, DMA controller
  • SPI, UART, I2C
  • Serial wire debug (SWD) & JTAG interfaces
  • 7 timers

You might also see this ARM Cortex M3 STM32F103C8T6 board advertised as an ‘Arduino’ board. Although it is not really an Arduino, the guys at stm32duino.com have developed a means to integrate it into the Arduino IDE. They have also ported some of the Arudino libraries to support this board. However, do not expect every library supported for an Arduino to work on this processor. For more information on what works and what might not, check out the stm32duino forum HERE.

This board is easily programmed with a ‘standard’ USB to Serial converter from FTDI or SILABS or an ST-LINK V2 in-circuit debugger and programmer from STMicroelectronics. These devices communicate with the bootloader which is stored in the internal boot ROM memory (system memory) of STM32 devices. This bootloader is programmed by STMicroelectronics during production and its main purpose is to download the application program to internal FLASH memory.

STLINK V2 Programmer / Debugger

My preference is the ST-LINK V2 since it also functions as an in-circuit debugger. OPENOCD.org fully supports the ST-LINK V2 and the STM32F103C8T6 board. OCD (On Chip Debugging) comes in quite handy to diagnose those ‘complexities’ 😎 in your code.

 

 

To program it, you might need to change the BOOT0 and/or BOOT1 jumpers as those pins select how the STM32F103C8T6 starts.

 

BOOT1 BOOT0 Boot mode
X 0 User Flash memory
0 1 System memory (bootloader)
1 1 Embedded SRAM

 

Here is what the STM32F103C8T6 looks like attached to an ST-LINK V2 programmer and a SILABS CP2102 USB – Serial converter:

STM32F103C8T6 with ST-LINK V2STM32F103C8T6 with CP2102

Hacking your Car with an Arduino and a CAN BUS Module

Your car is full of electronic devices. In fact, General Motors is one of the largest computer manufacturers due to its need of microcomputers in cars that it makes.  If you have an interest in what goes on with your cars’ electronics, you might want to invest some time hacking your car with an Arduino and a CAN BUS Module.

Quoting from the CAN BUS Wikipedia article:

‘A Controller Area Network (CAN bus) is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host computer. It is a message-based protocol, designed originally for multiplex electrical wiring within automobiles, but is also used in many other contexts.’

A modern car has many ECUs (or Electronic Control Units). These ‘computers’ send and receive information over the cars’ CAN BUS. De-coding that information can give you a wealth of information about the workings of your car.

The CAN BUS on your vehicle can be accessed from the OBD II connector.  I wrote about OBD II and diagnosing car problems in an earlier post here. If you are wondering if your car has an OBD II port, it probably does. All cars and light trucks built and sold in the United States after January 1, 1996 were required to be OBD II equipped.

The OBD II connector looks like this:

OBD II Connector

 
PIN DESCRIPTION PIN DESCRIPTION
1 Vendor Option 9 Vendor Option
2 J1850 Bus + 10 J1850 Bus
3 Vendor Option 11 Vendor Option
4 Chassis Ground 12 Vendor Option
5 Signal Ground 13 Vendor Option
6 CAN (J-2234) High 14 CAN (J-2234) Low
7 ISO 9141-2 K-Line 15 ISO 9141-2 Low
8 Vendor Option 16 Battery Power

 

Note: Pins 6 and 14 are specifically for the CAN BUS.

The CAN BUS module that I am using looks like this:

CAN BUS Module

The module has a seven pin connector for attachment to the Arduino . Two pins are used for +5V (power) and GND. The other five (MISO, MOSI, CS, SCK, INT) are for the SPI interface.. The J1 connector is a jumper for 120Ohm termination of the bus. (I have that jumper connected on both modules in my test environment shown below.) The CAN BUS is connected to the module either by the screw terminals or the two pin header behind the screw terminals. Note that one is labeled H for High and the other is L for Low.

Before hacking your car with an Arduino and a CAN BUS module, you may want to do what I did. I used two modules connected together to form a two node test CAN BUS network. That setup looked like this:

Hacking your Car with an Arduino and a CAN BUS Module

Basically, a double wire connects the CAN BUS High/Low connections on the module which is wired (SPI interface) to the Arduino. Each DIY Arduino has an FTDI USB to Serial connection to the Laptop/Phone/Computer. This setup allows you to make sure the modules are working properly and gain understanding of how the CAN BUS works.

Here is how the Arduino is wired to the CAN BUS module:

Get the Fritzing code for that diagram HERE.

There are several libraries for the Arduino to interface with the CAN BUS modules. I used the one from here. When you attach your module to the CAN BUS you will need to know its speed and set that value accordingly in the sketch. Also the crystal speed that your module uses (mine is 8Mhz)  needs to be specified. Make sure the library you choose (if you don’t use the one I linked to) supports your crystal speed.

For my two module CAN BUS test network, I loaded the send example sketch into one Arduino and the receive example sketch into the other Arduino. I set the crystal speed to 8Mhz (MCP_8MHZ) and the rate to 500Kbps (CAN_500KBPS) in each sketch. The send sketch sends a message every 100ms which the receive sketch then prints to the serial console.

Once  you get the modules working, you are ready to attach one of the modules to your cars’ CAN BUS.  The only possible change that will need to be made on the Arduino receive sketch is the CAN BUS rate. It needs to match the CAN BUS rate for your specific car. A bit of googling will hopefully get you more information on that.

Hacking your Car with an Arduino and a CAN BUS Module will send a lot of data. The ‘fun’ part is sifting through it to find what electronic components are sending what. Again, on that, GOOGLE is your friend! 🙂

PIC Assembler Coding

I became re-interested in Microchip PIC assembler coding recently. I hadn’t really used it for a number of years (since before 2009). The ‘C’ language is usually efficient enough and easier to code for most projects I do.

I used one of my DIY PIC18F4550 Pinguinos for experimentation. As you might guess, I decided to try the standard “Hello World!” program (i.e. blinking an LED). I wanted to create example PIC18F4550 assembler programs to blink an LED at intervals with a delay routine and with an interrupt timer. Also, I wanted both of them to work stand alone and with the Pinguino bootloader. You might also guess,  and be correct, that the internet is full of tutorials and code on how to blink an LED. Even the Pinguino WIKI shows 6 ways to blink an LED here, however all examples there are in ‘C’.

I did not find much on what I specifically wanted, especially using interrupts with the Pinguino bootloader. However, I did some experimentation and found solutions to make it all work.

Before I show the code, let’s first talk about the tools I am using (under Linux). To assemble the source code, I use GPASM which is part of the GPUTILS package. You can download it here.  I am using a PICKit2 programmer clone from tepuyelectronics.com to burn stand alone programs and the Pinguino bootloader. It looks like this:

PICKit2 Clone Programmer PIC Assembler CodingI use the Microchip pk2cmd software to run it. It can be downloaded from here.This is a command line example to run pk2cmd and load the Pinguino bootloader:

Once the bootloader is burned in, ‘C’ programs can be compiled and uploaded via the Pinguino IDE. The output from the GPASM (PIC assembler) produces a hex format file which can be uploaded (to the bootloader) via the ‘stand alone’ uploader8.py Python script. For example:

The Pinguino bootloader(s) for various chips and clock speeds can be downloaded from here. That file also includes the uploader8.py script used to upload hex files to the bootloader.  Those same hex format files can of course be burned in stand alone (no bootloader) using pk2cmd.

Probably the best development tool for PIC microcontrollers is the Microchip MPLAB IDE which you can download from here. You can use it to develop ‘C’ and assembler programs.

Here is an example ‘C’ program for the PIC18F4550 to blink an LED used in an MPLAB IDE project:

Here is basically that same program in assembler (use GPASM to assemble it):

Here is basically the same assembler program but this time it is set up to be loaded into a Pinguino with a bootloader. (Notice the org 0x0c00 statement to locate the code beyond the bootloader.) It is a bit different as it uses TIMER0 in 16 bit mode.

The included file pic18f4550.inc is a standard header file which comes with GPASM. The included config1.inc is something I coded which sets up the CONFIG directives. Here is what it contains:

If you don’t set up the PIC fuse (configuration) settings correctly (especially for the clock speed) the timing will not work correctly.

Here is the stand alone assembler program to blink an LED using a delay routine:

To run that same program in a Pinguino with a bootloader, change the org 0x0000 to org 0x0C00 so it loads beyond the bootloader.  I used a script (found here) to generate the delay routine. Note: You may need to change the code it generates for the specific processor being used. I had to change the relative offsets it generated (the $+ statements) to match the  lengths of opcodes for the PIC18F4550.

The delay routine is interesting to look at for the sake of learning the code involved, however, the interrupt version is ultimately more practical.

If you are interested in PIC assembler coding there are only a few instructions to learn. View the documentation here. Also of interest would be the assembler user guide here.

DIY Arduino Shield

An Arduino shield is basically another circuit board that plugs into the Arduino and extends its capabilities in some way. So, of course, if you create your own DIY Arduino, you have to create a DIY Arduino shield.

I needed a way to attach an HD44780 LCD to my DIY Arduino so I created a shield for it. Here is how my DIY Arduino shield turned out:

DIY Arduino ShieldAnd here are pictures of the front and back:

DIY LCD Shield Front ViewDIY LCD Shield Back View

 

 

 

 

 

 

 

It has a 16 pin female header to plug in the LCD, a 10K potentiometer to adjust the contrast and a jumper to turn on the backlight. The backlight can also be turned on (if the jumper is off) by pin 10 which is connected to a transistor.

The shield itself is basically a 5×7 cm fiberglass board with two rows of 14 pin male headers. The 14 pin male headers plug into the female headers on the DIY Arduino and thus extend all the Atmega 328 pins onto the shield. This then provides an whole new surface area to work with. That is how a shield works as a means to enhance the functionality of the base Arduino,

The wiring for this shield is nothing special (it is the standard way to hook up an LCD). Here is a Fritzing graphic diagram:

LCD Fritzing Hookup Diagram

You can get the Fritzing diagram source file HERE. When the DIY LCD shield is attached and running, this is what it looks like:

 

DIY LCD Shield Connected

Updated DIY Arduino Uno

I have a new version of my DIY Arduino UNO. It is somewhat different than the old one that I have written about before (HERE).  I moved some components around on the board, added the user LED and moved all the wiring to the bottom of the board.

Here is a picture of what it looks like:

DIY Arduino [Top View]

Updated DIY Arduino Uno

If you’ve followed along in my blog posts, you will have seen other pictures of it in use.  Additionally, I have added an optional 40 (10 x 4) pin breadboard which makes it quite handy to test any add on circuitry. I also added jumpers to connect / disconnect the LEDs (to conserver power). I printed ‘stickers’ for the Atmega328 and used silicon to stick them on. Get them here.  The sticker makes it easy to tell which pin is which.

Here are pictures of the top and bottom views of one:

 

DIY Arduino [Top View]DIY Arduino [Bottom View]

 

 

 

 

 

 

 

 

The bottom view was taken before all the wires (necessary to complete the connections)  were soldered on. This updated DIY Arduino UNO features:

The parts list for my updated DIY Arduino UNO is very basic:

Even though the layout has changed, how things are connected has remained the same. View the larger graphic by clicking the image below which shows the basic connections. (Get the fritzing source .fzz HERE)

Arduino on a Breadboard

Experimenting with Arduino-Builder

Lately, I have been experimenting with Arduino-Builder. It was been available as a command line interface to compile sketches since version 1.6.6 of the Arduino IDE.  It is actually what the Arduino IDE invokes behind the scenes to compile the sketch.

It is written in the Go language. The source code is available HERE on github.com.

Arduino-Builder is easily invoked to compile all of your sketches with a script like this:

To see a list of all the ways it can be customized, invoke it without any parameters.

I am not a fan of the Arduino IDE due to lack of an easy to use editor.  I’d prefer to use VI to edit Arduino sketches. However, the IDE does have an easy way to compile the sketches and load them via AVRDUDE.

Arduino IDE

 

 

 

 

 

Now that I’ve been experimenting with Arduino-Builder, I came up with a bash script to both compile and load a sketch.

The bash script:

Experimenting with Arduino-BuilderNow, I have the best of all worlds (so to speak). I can edit a sketch using VI and run the script containing Arduino-Builder to compile and load it.

Load more