I recently coded a Python script for Decoding an ESP8266 Firmware Image. It’s the first part of my quest to create a Linux tool for creating a single flash image.
More specifically, the script will decode a single binary image or a ‘combined’ image. A combined image (single flash image) can be created with the flash download tool from Expressif. (Exressif is the company which makes the ESP8266.) However, that download tool only runs on Windows. Here is what the Expressif Flash Tool looks like:
Notice, the ‘CombineBin’ button. It is used to create a single binary image from the multiple files (‘segments’). In that image you can see seven files (the current SDK 2.1.0 files as of this post date) listed and the addresses to load them . I used the ‘CombineBin’ button and created a single binary image called sdk-2.1.0.bin. (The default file name it creates is called target.bin.)
I run Linux. and the recommend ESP8266 tool for it is esptool.py. The esptool script, however, does not have a option to create a single combined image. It may have that feature as an enhancement at some future time. Hence, my quest, to develop a Linux tool to create a single flash image.
I like to keep my ESP8266 firmware current. To flash the latest SDK, I use this script:
esptool.py --baud 115200 --port /dev/ttyUSB0 write_flash -fm dio -ff 40m -fs detect \ 0x00000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/boot_v1.7.bin \ 0x01000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/at/512+512/user1.1024.new.2.bin \ 0x7E000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin \ 0x81000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/at/512+512/user2.1024.new.2.bin \ 0xFB000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin \ 0xFC000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/esp_init_data_default_v08.bin \ 0xFE000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin
While that works fine, I think it would be more efficient to create a single combined image to flash. Especially since I have a number of ESP8266 modules to flash. The single image (sdk-2.1.0.bin created from the windows tool) can be flashed as follows:
esptool.py --baud 115200 --port /dev/ttyUSB0 write_flash -fm dio -ff 40m -fs detect 0x00 sdk-2.1.0.bin
To create a tool to combine all the ‘single’ images into one, one needs to know the format of the single images and the format of the ‘combined’ image.
Espressif has a brief document (HERE), describing the firmware image format. However, that document does not fully cover what is need to decode a combined image. Also, it does not cover all of the single image ‘formats’. Esptool.py has an image_info command, however, it only works on single modules. It does not work on combined modules.
In my ‘reverse engineering’ attempt at Decoding an ESP8266 Firmware Image, I created this Python script called esp8266_parse_bin.py. Here is the code:
#!/usr/bin/env python
#
# esp8266_parse_bin.py Version 1.1 1/22/2018
# (C)opyright 2018 Email: earl@microcontrollerelectronics.com
# Parses (decodes) ESP8266 single or combined binary image files
#
#
# Partial document on the ESP8266 Image Format
# https://github.com/espressif/esptool/wiki/Firmware-Image-Format
#
# 0 1 2 3 4-7 8
# 0xE9 segments SPI mem/speed entry ...
# 0xEA segments SPI mem/speed entry ...
#
# --SEGMENTS--
#
# 0-3 4-7 8-n
# Offset size data
import os, sys, string, struct
fsize = 0
chk = 0
htype = ''
filename = sys.argv[1]
f = open(filename, "rb")
afsize = os.stat(filename).st_size
baddr = ""
bsize = 0
useg = ""
uaddr = ""
usize = 0
def blank_header():
global baddr,bsize
print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("","","Size/Hex","Size/Bytes","File/Offset","File/Offset")
print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Blank(s)","0xff",'0x'+hex(bsize)[2:].zfill(8),bsize,'0x'+hex(baddr)[2:].zfill(8),baddr)
baddr = ""
bsize = 0
def unknown_header():
global uaddr,usize,useg
print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("","","Size/Hex","Size/Bytes","File/Offset","File/Offset")
print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Unknown","",'0x'+hex(usize)[2:].zfill(8),usize,'0x'+hex(uaddr)[2:].zfill(8),uaddr)
for b in bytearray(useg): print(hex(b)),
print
uaddr = ""
useg = ""
usize = 0
print("Parsing: %s Size: %s/%s") % (filename,hex(afsize),afsize)
while(1):
t = f.read(8)
if not t: break
l = len(t)
sh = ord(t[0])
if (sh != 0xff):
if (baddr != ""): blank_header()
if ((sh == 0xea) or (sh == 0xe9) or (sh == 0xff)):
if (uaddr != ""): unknown_header()
if l < 8:
print("Extra Data [no-header] of Length: %s -> ") % (len(t)),
for b in bytearray(t): print(hex(b)),
print
fsize += l
break
h = struct.Struct("<BBBBI").unpack(t)
if (h[0] == 0xea):
segments = 1
htype = 0xea
else:
if (h[0] == 0xe9):
segments = int(h[1])
htype = 0xe9
chk = 0
else:
if (h[0] == 0xff):
if (baddr == ""): baddr = fsize
bsize += l
fsize += l
continue
else:
if (uaddr == ""): uaddr = fsize
useg += t
usize += l
fsize += l
continue
fsize += l
print "Header: ",hex(h[0]),int(h[1]),hex(h[2]),hex(h[3]),hex(h[4])
print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Segment","Offset","Size/Hex","Size/Bytes","File/Offset","File/Offset")
for x in range(0,segments):
data = f.read(8)
if not data: break
s = struct.Struct("<II").unpack(data)
offset = s[0]
size = s[1]
print("%-8s %-12s %-12s %-12s %-12s %-12s") % (x+1,'0x'+hex(offset)[2:].zfill(8),'0x'+hex(size)[2:].zfill(8),size,'0x'+hex(fsize)[2:].zfill(8),fsize),
if (htype != 0xea):
if (offset > 0x40200000): print(" <-- Offset > 0x40200000 "),
if (offset < 0x3ffe0000): print(" <-- Offset < 0x3ffe0000 "),
if (size > 65536): print(" <-- Size > 65536 "),
print
l = int(s[1])
fsize += 8 + l
data = f.read(l)
for b in bytearray(data): chk ^= int(b)
if (htype == 0xea): continue
pad = 16 - (fsize % 16)
print("Padding: %s bytes -> ") % (pad),
data = f.read(pad)
for b in bytearray(data): print(hex(b)),
print
chk = chk ^ 0xEF
print("Calculated Checksum: %s") % (hex(chk)),
if (ord(data[-1]) == chk): print " [matches]",
print
fsize += pad
continue
if (uaddr != ""): unknown_header()
if (baddr != ""): blank_header()
print("File Size: %s Calculated: %s") % (afsize,fsize),
if (afsize == fsize): print(" [matches]"),
print
Esp8266_parse_bin.py can be used to decode any of the single modules that comprise the combined sdk-2.1.0.bin image or it can be used to decode the ‘combined’ image.
When it is used to decode the combined sdk-2.1.0.bin image via:
./esp8266_parse_bin.py sdk-2.1.0.bin
Here is what the output looks like:
Parsing: sdk-2.1.0.bin Size: 0xff000/1044480
Header: 0xe9 3 0x0 0x20 0x4010057c
Segment Offset Size/Hex Size/Bytes File/Offset File/Offset
1 0x40100000 0x00000a20 2592 0x00000008 8
2 0x3ffe8000 0x000002fc 764 0x00000a30 2608
3 0x3ffe82fc 0x000002a4 676 0x00000d34 3380
Padding: 16 bytes -> 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x22
Calculated Checksum: 0x22 [matches]
Size/Hex Size/Bytes File/Offset File/Offset
Blank 0xff 0x00000010 16 0x00000ff0 4080
Header: 0xea 4 0x0 0x1 0x40100004
Segment Offset Size/Hex Size/Bytes File/Offset File/Offset
1 0x00000000 0x0005e0f0 385264 0x00001008 4104
Header: 0xe9 3 0x0 0x20 0x40100004
Segment Offset Size/Hex Size/Bytes File/Offset File/Offset
1 0x40100000 0x000076dc 30428 0x0005f108 389384
2 0x3ffe8000 0x00000874 2164 0x000667ec 419820
3 0x3ffe8880 0x000023bc 9148 0x00067068 421992
Padding: 4 bytes -> 0x0 0x0 0x0 0x69
Calculated Checksum: 0x69 [matches]
Size/Hex Size/Bytes File/Offset File/Offset
Unknown 0x00000008 8 0x00069430 431152
0x42 0xd 0x2b 0x65 0xff 0xff 0xff 0xff
Size/Hex Size/Bytes File/Offset File/Offset
Blank 0xff 0x00017bc8 97224 0x00069438 431160
Header: 0xea 4 0x0 0x2 0x40100004
Segment Offset Size/Hex Size/Bytes File/Offset File/Offset
1 0x00000000 0x0005e0f0 385264 0x00081008 528392
Header: 0xe9 3 0x0 0x20 0x40100004
Segment Offset Size/Hex Size/Bytes File/Offset File/Offset
1 0x40100000 0x000076dc 30428 0x000df108 913672
2 0x3ffe8000 0x00000874 2164 0x000e67ec 944108
3 0x3ffe8880 0x000023bc 9148 0x000e7068 946280
Padding: 4 bytes -> 0x0 0x0 0x0 0x68
Calculated Checksum: 0x68 [matches]
Size/Hex Size/Bytes File/Offset File/Offset
Unknown 0x00000008 8 0x000e9430 955440
0x98 0x88 0x89 0x38 0xff 0xff 0xff 0xff
Size/Hex Size/Bytes File/Offset File/Offset
Blank 0xff 0x00012bc8 76744 0x000e9438 955448
Size/Hex Size/Bytes File/Offset File/Offset
Unknown 0x00000080 128 0x000fc000 1032192
0x5 0x8 0x4 0x2 0x5 0x5 0x5 0x2 0x5 0x0 0x4 0x5 0x5 0x4 0x5 0x5 0x4 0xfe 0xfd 0xff 0xf0 0xf0 0xf0 0xe0 0xe0 0xe0 0xe1 0xa 0xff 0xff 0xf8 0x0 0xf8 0xf8 0x4e 0x4a 0x46 0x40 0x3c 0x38 0x0 0x0 0x1 0x1 0x2 0x3 0x4 0x5 0x1 0x0 0x0 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xe1 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x93 0x43 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
Size/Hex Size/Bytes File/Offset File/Offset
Blank 0xff 0x00002f80 12160 0x000fc080 1032320
File Size: 1044480 Calculated: 1044480 [matches]
From that output, one can see where (address) and how each of the single images are placed in the ‘combined’ image. Now that the combined image format is known, a script can be coded to create it from the individual files.
Recent Comments