I recently coded a Python script for Decoding an ESP8266 Firmware Image. It’s the first part of my quest to create a Linux tool for creating a single flash image.
More specifically, the script will decode a single binary image or a ‘combined’ image. A combined image (single flash image) can be created with the flash download tool from Expressif. (Exressif is the company which makes the ESP8266.) However, that download tool only runs on Windows. Here is what the Expressif Flash Tool looks like:
Notice, the ‘CombineBin’ button. It is used to create a single binary image from the multiple files (‘segments’). In that image you can see seven files (the current SDK 2.1.0 files as of this post date) listed and the addresses to load them . I used the ‘CombineBin’ button and created a single binary image called sdk-2.1.0.bin. (The default file name it creates is called target.bin.)
I run Linux. and the recommend ESP8266 tool for it is esptool.py. The esptool script, however, does not have a option to create a single combined image. It may have that feature as an enhancement at some future time. Hence, my quest, to develop a Linux tool to create a single flash image.
I like to keep my ESP8266 firmware current. To flash the latest SDK, I use this script:
|
1 2 3 4 5 6 7 8 |
esptool.py --baud 115200 --port /dev/ttyUSB0 write_flash -fm dio -ff 40m -fs detect \ 0x00000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/boot_v1.7.bin \ 0x01000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/at/512+512/user1.1024.new.2.bin \ 0x7E000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin \ 0x81000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/at/512+512/user2.1024.new.2.bin \ 0xFB000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin \ 0xFC000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/esp_init_data_default_v08.bin \ 0xFE000 /opt/arduino/esp8266/ESP8266_NONOS_SDK-master/bin/blank.bin |
While that works fine, I think it would be more efficient to create a single combined image to flash. Especially since I have a number of ESP8266 modules to flash. The single image (sdk-2.1.0.bin created from the windows tool) can be flashed as follows:
|
1 |
esptool.py --baud 115200 --port /dev/ttyUSB0 write_flash -fm dio -ff 40m -fs detect 0x00 sdk-2.1.0.bin |
To create a tool to combine all the ‘single’ images into one, one needs to know the format of the single images and the format of the ‘combined’ image.
Espressif has a brief document (HERE), describing the firmware image format. However, that document does not fully cover what is need to decode a combined image. Also, it does not cover all of the single image ‘formats’. Esptool.py has an image_info command, however, it only works on single modules. It does not work on combined modules.
In my ‘reverse engineering’ attempt at Decoding an ESP8266 Firmware Image, I created this Python script called esp8266_parse_bin.py. Here is the code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 |
#!/usr/bin/env python # # esp8266_parse_bin.py Version 1.1 1/22/2018 # (C)opyright 2018 Email: earl@microcontrollerelectronics.com # Parses (decodes) ESP8266 single or combined binary image files # # # Partial document on the ESP8266 Image Format # https://github.com/espressif/esptool/wiki/Firmware-Image-Format # # 0 1 2 3 4-7 8 # 0xE9 segments SPI mem/speed entry ... # 0xEA segments SPI mem/speed entry ... # # --SEGMENTS-- # # 0-3 4-7 8-n # Offset size data import os, sys, string, struct fsize = 0 chk = 0 htype = '' filename = sys.argv[1] f = open(filename, "rb") afsize = os.stat(filename).st_size baddr = "" bsize = 0 useg = "" uaddr = "" usize = 0 def blank_header(): global baddr,bsize print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("","","Size/Hex","Size/Bytes","File/Offset","File/Offset") print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Blank(s)","0xff",'0x'+hex(bsize)[2:].zfill(8),bsize,'0x'+hex(baddr)[2:].zfill(8),baddr) baddr = "" bsize = 0 def unknown_header(): global uaddr,usize,useg print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("","","Size/Hex","Size/Bytes","File/Offset","File/Offset") print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Unknown","",'0x'+hex(usize)[2:].zfill(8),usize,'0x'+hex(uaddr)[2:].zfill(8),uaddr) for b in bytearray(useg): print(hex(b)), print uaddr = "" useg = "" usize = 0 print("Parsing: %s Size: %s/%s") % (filename,hex(afsize),afsize) while(1): t = f.read(8) if not t: break l = len(t) sh = ord(t[0]) if (sh != 0xff): if (baddr != ""): blank_header() if ((sh == 0xea) or (sh == 0xe9) or (sh == 0xff)): if (uaddr != ""): unknown_header() if l < 8: print("Extra Data [no-header] of Length: %s -> ") % (len(t)), for b in bytearray(t): print(hex(b)), print fsize += l break h = struct.Struct("<BBBBI").unpack(t) if (h[0] == 0xea): segments = 1 htype = 0xea else: if (h[0] == 0xe9): segments = int(h[1]) htype = 0xe9 chk = 0 else: if (h[0] == 0xff): if (baddr == ""): baddr = fsize bsize += l fsize += l continue else: if (uaddr == ""): uaddr = fsize useg += t usize += l fsize += l continue fsize += l print "Header: ",hex(h[0]),int(h[1]),hex(h[2]),hex(h[3]),hex(h[4]) print("%-8s %-12s %-12s %-12s %-12s %-12s") % ("Segment","Offset","Size/Hex","Size/Bytes","File/Offset","File/Offset") for x in range(0,segments): data = f.read(8) if not data: break s = struct.Struct("<II").unpack(data) offset = s[0] size = s[1] print("%-8s %-12s %-12s %-12s %-12s %-12s") % (x+1,'0x'+hex(offset)[2:].zfill(8),'0x'+hex(size)[2:].zfill(8),size,'0x'+hex(fsize)[2:].zfill(8),fsize), if (htype != 0xea): if (offset > 0x40200000): print(" <-- Offset > 0x40200000 "), if (offset < 0x3ffe0000): print(" <-- Offset < 0x3ffe0000 "), if (size > 65536): print(" <-- Size > 65536 "), print l = int(s[1]) fsize += 8 + l data = f.read(l) for b in bytearray(data): chk ^= int(b) if (htype == 0xea): continue pad = 16 - (fsize % 16) print("Padding: %s bytes -> ") % (pad), data = f.read(pad) for b in bytearray(data): print(hex(b)), print chk = chk ^ 0xEF print("Calculated Checksum: %s") % (hex(chk)), if (ord(data[-1]) == chk): print " [matches]", print fsize += pad continue if (uaddr != ""): unknown_header() if (baddr != ""): blank_header() print("File Size: %s Calculated: %s") % (afsize,fsize), if (afsize == fsize): print(" [matches]"), print |
Esp8266_parse_bin.py can be used to decode any of the single modules that comprise the combined sdk-2.1.0.bin image or it can be used to decode the ‘combined’ image.
When it is used to decode the combined sdk-2.1.0.bin image via:
|
1 |
./esp8266_parse_bin.py sdk-2.1.0.bin |
Here is what the output looks like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
Parsing: sdk-2.1.0.bin Size: 0xff000/1044480 Header: 0xe9 3 0x0 0x20 0x4010057c Segment Offset Size/Hex Size/Bytes File/Offset File/Offset 1 0x40100000 0x00000a20 2592 0x00000008 8 2 0x3ffe8000 0x000002fc 764 0x00000a30 2608 3 0x3ffe82fc 0x000002a4 676 0x00000d34 3380 Padding: 16 bytes -> 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x22 Calculated Checksum: 0x22 [matches] Size/Hex Size/Bytes File/Offset File/Offset Blank 0xff 0x00000010 16 0x00000ff0 4080 Header: 0xea 4 0x0 0x1 0x40100004 Segment Offset Size/Hex Size/Bytes File/Offset File/Offset 1 0x00000000 0x0005e0f0 385264 0x00001008 4104 Header: 0xe9 3 0x0 0x20 0x40100004 Segment Offset Size/Hex Size/Bytes File/Offset File/Offset 1 0x40100000 0x000076dc 30428 0x0005f108 389384 2 0x3ffe8000 0x00000874 2164 0x000667ec 419820 3 0x3ffe8880 0x000023bc 9148 0x00067068 421992 Padding: 4 bytes -> 0x0 0x0 0x0 0x69 Calculated Checksum: 0x69 [matches] Size/Hex Size/Bytes File/Offset File/Offset Unknown 0x00000008 8 0x00069430 431152 0x42 0xd 0x2b 0x65 0xff 0xff 0xff 0xff Size/Hex Size/Bytes File/Offset File/Offset Blank 0xff 0x00017bc8 97224 0x00069438 431160 Header: 0xea 4 0x0 0x2 0x40100004 Segment Offset Size/Hex Size/Bytes File/Offset File/Offset 1 0x00000000 0x0005e0f0 385264 0x00081008 528392 Header: 0xe9 3 0x0 0x20 0x40100004 Segment Offset Size/Hex Size/Bytes File/Offset File/Offset 1 0x40100000 0x000076dc 30428 0x000df108 913672 2 0x3ffe8000 0x00000874 2164 0x000e67ec 944108 3 0x3ffe8880 0x000023bc 9148 0x000e7068 946280 Padding: 4 bytes -> 0x0 0x0 0x0 0x68 Calculated Checksum: 0x68 [matches] Size/Hex Size/Bytes File/Offset File/Offset Unknown 0x00000008 8 0x000e9430 955440 0x98 0x88 0x89 0x38 0xff 0xff 0xff 0xff Size/Hex Size/Bytes File/Offset File/Offset Blank 0xff 0x00012bc8 76744 0x000e9438 955448 Size/Hex Size/Bytes File/Offset File/Offset Unknown 0x00000080 128 0x000fc000 1032192 0x5 0x8 0x4 0x2 0x5 0x5 0x5 0x2 0x5 0x0 0x4 0x5 0x5 0x4 0x5 0x5 0x4 0xfe 0xfd 0xff 0xf0 0xf0 0xf0 0xe0 0xe0 0xe0 0xe1 0xa 0xff 0xff 0xf8 0x0 0xf8 0xf8 0x4e 0x4a 0x46 0x40 0x3c 0x38 0x0 0x0 0x1 0x1 0x2 0x3 0x4 0x5 0x1 0x0 0x0 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xe1 0xa 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x93 0x43 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 Size/Hex Size/Bytes File/Offset File/Offset Blank 0xff 0x00002f80 12160 0x000fc080 1032320 File Size: 1044480 Calculated: 1044480 [matches] |
From that output, one can see where (address) and how each of the single images are placed in the ‘combined’ image. Now that the combined image format is known, a script can be coded to create it from the individual files.
Recent Comments