Glasgow does not support Espressif esptool - possible to convert to applet?

[victorhooi]

This is related to the Discord chat from here:

https://discord.com/channels/613131135903596547/613461005392936961/1239057227055239269

tl;dr - glasgow does not work in UART mode with ESP devices. @whitequark suggested easiest fix may be to simply implement the esptool functionality as an applet.

I was trying to use the glasgow UART support with esptool, which is the official Espressif tool for interacting with ESPXX/ESPXXXX bootloaders.

This was how I started glasgow in UART mode:

victorhooi@Victors-MacBook-Pro-3 ~/c/g/firmware (main)> glasgow run uart -V 3.3 --pin-tx 1 --pin-rx 0 -b 9600 pty
I: g.device.hardware: device already has bitstream ID 75401762f86c41aae9f5996a07c0b632
I: g.cli: running handler for applet 'uart'
I: g.applet.interface.uart: port(s) A, B voltage set to 3.3 V
/dev/ttys012
W: g.applet.interface.uart: 24 frame errors detected

This is the error I got with esptool:

victorhooi@Victors-MacBook-Pro-3 ~/c/g/firmware (main)> esptool.py -p /dev/ttys012 flash_id
esptool.py v4.7.0
Serial port /dev/ttys012
Connecting...
Failed to get PID of a device on /dev/ttys012, using standard reset sequence.

Traceback (most recent call last):
  File "/opt/homebrew/bin/esptool.py", line 37, in <module>
    esptool._main()
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/__init__.py", line 1139, in _main
    main()
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/__init__.py", line 712, in main
    esp = esp or get_default_connected_device(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/__init__.py", line 1018, in get_default_connected_device
    _esp = detect_chip(
           ^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/__init__.py", line 1018, in get_default_connected_device
    _esp = detect_chip(
           ^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/cmds.py", line 97, in detect_chip
    detect_port.connect(connect_mode, connect_attempts, detecting=True)
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/loader.py", line 678, in connect
    last_error = self._connect_attempt(reset_strategy, mode)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/loader.py", line 563, in _connect_attempt
    reset_strategy()  # Reset the chip to bootloader (download mode)
    ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/reset.py", line 83, in __call__
    self._setDTRandRTS(False, False)
  File "/opt/homebrew/Cellar/esptool/4.7.0_1/libexec/lib/python3.12/site-packages/esptool/reset.py", line 48, in _setDTRandRTS
    "I", fcntl.ioctl(self.port.fileno(), TIOCMGET, struct.pack("I", 0))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 25] Inappropriate ioctl for device

And for comparison - here is esptool working using a tigard as the UART device:

victorhooi@Victors-MacBook-Pro-3 ~/c/g/firmware (main) [2]> esptool.py -p /dev/tty.usbserial-TG1001d00 flash_id
esptool.py v4.7.0
Serial port /dev/tty.usbserial-TG1001d00
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 48:e7:29:49:1b:19
Uploading stub...
Running stub...
Stub running...
Manufacturer: ef
Device: 4014
Detected flash size: 1MB
Hard resetting via RTS pin...

@whitequark suggested this might be due to some tricks esptool is doing with DTR/RST, and the fact that Glasgow is using a pty (versus the Tigard which uses a real tty at the hardware level).

[bburky]

Yes, by default esptool resets your device into the bootloader using RTS and DTR which is unsupported by glasgow's pty mode, but you can disable this in esptool however, which should allow you to use esptool with a Glasgow today (you'll have to manually put the device into bootloader mode first, perhaps by holding a BOOT button during reset or tying GPIO0 to ground). Similarly, you can use a socket instead of pty which doesn't support RTS and DTR anyway.

# first, put the esp device into bootloader mode manually. Then:

# untested, should work
glasgow run uart -V 3.3 --port A --pin-tx 1 --pin-rx 0 -b 115200 pty
esptool --before no_reset -p /dev/ttys012 flash_id
# you may also need --after no_reset

# works, even on windows which doesn't support pty
glasgow run uart -V 3.3 --port A --pin-tx 0 --pin-rx 1 -b 115200 socket tcp:127.0.0.1:4000
esptool -p socket://127.0.0.1:4000 flash_id

I'm not sure this is the best option (I've never used it, and CUSE was discussed as another alternative in Discord) but if you want to add support for esptool reset via glagow uart... esptool seems to support RFC 2217 (telnet) remote serial ports:
https://docs.espressif.com/projects/esptool/en/latest/esp8266/esptool/remote-serial-ports.html

There's an example in their source code, it supports detecting RTS and DTR over the serial port and doing the bootloader reset. Though they do not actually pass through RTS and DTR directly to the serial port and instead trigger the esptool specific reset process in the serial server code (those docs say this is to handle network latency, see espressif/esptool#383. Maybe you could skip this if the localhost rfc2217 server is on the same machine as esptool... but it sounds like they had issues in a fully local system too)
https://github.com/espressif/esptool/blob/v4.7.0/esp_rfc2217_server.py#L75

You could probably add support for an RFC 2217 server (similarly to socket) in the uart applet? Unless you can pass through RTS and DTR directly to output pins, the applet would need to do implement the esptool specific reset modes. I'm unsure if RFC 2217 is a widely used protocol, but apparently pyserial supports it and that's where esptool's support comes from.

If you had an RFC 2217 server applet, you could use it with esptool like the docs above describe. Like: esptool.py --port rfc2217://127.0.0.1:4000?ign_set_control flash_id

[whitequark]

We could add an uart-rs232 applet which does have the additional control lines.