Best Kept Secret

Yes, it is an ambiguous topic. No, I don’t think it should be hidden, however, the fact that this topic does not seem to garner much press could mean it has been a best kept secret.

The “best kept secret” is the programming language Forth. For reference, take a look at “A  Beginner’s Guide to Forth” from J.V. Noble.

Forth is specially useful on microcontrollers due to its ‘small’ size. ‘Mecrisp Stellaris’ from Matthias Koch is a case in point. I have been experimenting with it on my STM32F103C8T6 boards and am amazed at what it can do.

Forth is quite different than other programming languages. When loaded into a microcontroller, It interacts via a serial/usb ‘console’ and, to me, is like an Operating System but better!  It’s an ‘applications programmer’ dream environment uncluttered by a ‘systems programmer’ idea of what should be there. If you read the history of Forth and thus the history of Chuck Moore (the inventor of Forth) you will better see what I mean.

I think Jean-Claude Wippler (jeelabs.org)  understates the usefulness in his article titled ‘F103 + USB = Swiss Army Knife’.  Jean-Claude has enhanced (Mecrisp Stellaris) Forth with lots of code/examples. Take a look at his artcile about it HERE. He also has a communications program (written in ‘GO‘) called Folie which makes it easier to interact with Forth.

To install it on my Fedora Linux system, I ran this:

dnf install go
GOPATH=/root/gocode
go get github.com/jeelabs/folie

Then run it via:

/root/gocode/bin/folie -r

and it will prompt you for the serial port.

I have also updated my Python serial program script to communicate with Forth as an alternative to Folie. Here is the code:

#!/bin/env python
from __future__ import print_function

#
# From: earl@microcontrollerelectronics.com
# Serial Communications in python via pyserial
# to interact with Mecrisp Stellaris Forth
#  
# !help for help
#

import serial,sys,glob,select,re,os

incpat = "^include\s+([\.\w/-]+)"

def help():
  print("\n--Function----|-----Result----------------------")
  print("!help          help function")
  print("! command      run 'command' in linux shell")
  print("!ls            list files in current directory")
  print("!cd dir        change to specified dir")
  print("!s file        read file and send to serial port")
  print("!hexdump file  hexdump to file")
  print()
 
def hexdump(fname):
  global ser
  hex = []
  ser.write("hexdump\n")
  while True:
    sline = ser.readline()
    if (sline[0:4] == " ok."): break
    hex.append(sline)
  print(sline)
  hfile = open(fname,'w')
  for x in hex: hfile.write(x)
  hfile.close()

def p_cmd(cmd):
  handle = os.popen(cmd,'r',1)
  for line in handle:
    print(line[:-1])
  handle.close()

def s_read(fname):
  global ser
  try:
    file = open(fname)
    for rline in file:
      inc = re.match(incpat, rline)
      if inc:
        incfile = inc.group(1)
        s_read(os.path.join(os.path.dirname(fname), incfile))
        continue
      rline = rline.replace("\r\n","\n")
      print(rline,end='')
      ser.write(rline)
      while True:
        sline = ser.readline()
        if sline:
          print (sline,end='')
          if (sline.find(" ok.") != -1): break
  except:
    print ("Unable to open file: ",fname)
    return

def init_ser():
  global ser
  rate = "115200"
  dev  = "/dev/ttyACM*"
  scan = glob.glob(dev)

  if (len(scan) == 0):
    dev  = '/dev/ttyUSB*'
    scan = glob.glob(dev)
    if (len(scan) == 0):
      print ("Unable to find any ports scanning for /dev/[ttyACM*|ttyUSB*]")
      sys.exit()

  serport = scan[0]

  if (len(sys.argv) > 1):
   l = len(sys.argv) - 1
   while(l>0):
     if (sys.argv[l][0] == '/'): serport = sys.argv[l]
     else:                       rate    = sys.argv[l]
     l = l - 1

  try:
    ser = serial.Serial(port=serport,baudrate=rate,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)
    print("connected to: ",ser.portstr," at ",rate," BAUD")
    ser.write("hello\n")
  except:
    print("Unable to open: ",serport)
    sys.exit()

init_ser()

while True:
  try:
    tr = ser.inWaiting()
    if tr > 0:
      line = ser.read(tr)
#     for x in line: print (x.encode('hex'),end='')
      print (line,end='')
  except KeyboardInterrupt:
    sys.exit()
  except:
    pass
  while sys.stdin in select.select([sys.stdin], [], [], 0)[0]:
    line = sys.stdin.readline()
#    for x in line: print (x.encode('hex'),end='')
    if (line.find("!hexdump") == 0):
      hexdump(line[9:].strip())
      continue
    if (line.find("! ") == 0):
      p_cmd(line[2:].strip())
      continue
    if (line.find("!s ") == 0):
      s_read(line[3:].strip())
      continue
    if (line.find("!ls") == 0):
      print(os.getcwd())
      for file in os.listdir(os.getcwd()):
        print(file)
      continue
    if (line.find("!cd ") == 0):
      os.chdir(line[4:].strip())
      continue
    if (line.find("!help") == 0):
      help()
      continue
    try:
      ser.write(line)
    except:
      init_ser()

ser.close()
sys.exit()

Inspired by the built in commands that Folie has to interact with Mecrisp Stellaris Forth I added added some to  my script. Use: !help to list them.

If you are new to Forth it may take some getting used to. It relies on the stack to store everything and thus uses ‘reverse polish notation‘. This way of thinking is different that ‘traditional’ languages such as C or Python, etc.  and is perhaps why some programmers get frustrated and give up.

So, is Forth ‘hidden’ due to frustration and lack of programmers using it or is it just not widely marketed? In any case, why not give it a try? If you need more to convince you, take a look at Elliot Williams’ article called Forth: The Hacker’s Language on Hackaday.com.