Minicom woes

Home Forums Z80 Playground Early-Adopters Minicom woes

Viewing 15 posts - 1 through 15 (of 19 total)
  • Author
    Posts
  • #862
    johnw
    Participant

    I have several Pis knocking around and have no experience of Arduino sketch programming for the VGA32 board.

    So I had this idea to run the Z80 PG through a Raspberry Pi and use a 7″ screen and usb wireless keyboard to make the whole system truly portable and even work off a battery pack.

    I am using minicom on the Pi and it all works BUT I can not get minicom to use the console font in the Pi. It seems to use its own font. This means that the 8 bit characters are different, with the line characters completely missing, and there is no chance of correcting them so programs like John’s Sargon78 and 2048 look very strange. Also, the colours are wrong compared to those on Teraterm on the PC. I don’t know if this is a bug in the Raspbian version of minicom but I can find no mention of the subject on google searches.

    Here is the Pi console font:
    Pi console font

    Here is the font displayed by minicom:
    Minicom font

    I will keep searching.

    #867
    kevin
    Participant

    To the best of my knowledge, Minicom does no character set conversion, and includes no fonts. Any data it receives is passed directly to the pseudo-TTY it’s running on. You’ll certainly get different results if you run Minicom in a Linux kernel console, compared to an xterm or something graphical. So far as I know, there’s nothing in Minicom itself that can change this.

    My understanding is that CP/M never really had any general agreement on how to interpret 8-bit characters with values > 128. Software that uses these characters was usually designed for some specific hardware or terminal. Because of the continuity between CP/M -> MSDOS -> Windows, Windows-based terminal emulators seem to handle old CP/M programs better than Linux terminal emulators do by default — particular those that run in a graphical environment, and expect to use Unicode character sets.

    I find that if I run programs like 2048 in a raw Linux terminal (no graphical desktop of any kind) then I get reasonable results when running 2048, etc. That’s because this raw terminal uses the CP437 character set, not unicode. If I want to use Minicom in any kind of graphical desktop, then I have to fiddle about with the console settings (not Minicom settings, which has no interest in these things).

    I’m embarked on the same exercise as you, except that I’m using a Pi Zero rather than a “real” Pi. I have a custom Linux that boots directly to Minicom. This works reasonably well, but I confess that I still haven’t sorted out all the details of the console. I’ll post the results here when I’m completely happy with it.

    Kevin.

    #868
    johnw
    Participant

    Hi Kevin,
    Thanks for your reply.

    I am also using the Pi Zero but my results are obviously different to yours in that I have not yet found how to get a CP437 character set, despite working my way through the console and minicom setup screens. So I wonder if you would mind comparing our installations.

    I installed the latest raspbian, updated it, did a sudo apt install minicom, did the procedure for getting ttyS0 operational through the GPIO pins, connected the Z80 PG uart through level conversion to the GPIO, and set minicom to use /dev/serial0.

    I set raspbian to autostart to the console and run “minicom -c on” to get colour.

    As an example of what I get, when I run 2048, instead of solid block characters forming the edge of the board and dots in the empty squares, there are foreign language letters.

    Would you mind letting me know how you set up your system.

    John

    #869
    kevin
    Participant

    Hi

    This probably doesn’t help much, but…

    I use a completely custom Linux, which I build using shell and Perl scripts that are here:

    https://github.com/kevinboone/rasp-pi-minicom-build

    It’s all very ugly and badly-documented, because I use essentially the same scripts to build other Pi projects. There’s definitely stuff in the github repo that has bled over from other projects, and makes no sense.

    At the moment I’m struggling to find time to assemble all the Pi and Z80-PG bits into a case, so I can’t test anything because it’s half-finished. When I get it all working again, I’ll try to publish something that is in a state that other folks could use, if they wanted. Right now it’s working for me, but I’m not sure how to distribute something that can be used directly.

    Kevin.

    PS. I’m keeping a close eye on this:

    https://www.raspberrypi.org/forums/viewtopic.php?f=144&t=308154

    Use a Pi Pico is a much better approach, I think, than a Pi Zero. But I don’t want to write this mass of code myself — I’m waiting for someone else to do it, so I can steal it 😉

    • This reply was modified 7 months ago by kevin.
    #878
    kevin
    Participant

    Sorry — everything I wrote earlier was nonsense :/

    It seems that the Linux kernel console stopped using CP437 by default years ago, and I suspect the Raspberry Pi devices never used it. All modern Linux consoles now use a framebuffer-based console with Unicode support. This is better for almost everything — except running old DOS and CP/M code.

    Configuration of the framebuffer console to use different character sets and fonts is stupendously complex. It might actually be easier to use a terminal emulator other than minicom, that can actually translate CP437 to UTF-8 or something that the Linux console likes.

    Still, I’m sure I’ve had this working in the past, just using the basic “setfont” and “setupcon” utilities. I just can’t remember how, now.

    I’ll continue to work on this, but it’s a lot of work, just to get “2048” to look nice 😉

    Sorry.

    Kevin.

    #882
    johnw
    Participant

    It isn’t just 2048, there’s sargon78 as well, and any other program that might be written to use the cp437 “graphic” characters. I might even want to experiment myself in future!

    I have actually found an option in the minicom command line, “-R cp437”, that informs minicom that the remote system is using cp437, and this does work to give the correct character set, however….. there is a weird effect going on whereby extra spaces and incorrect characters are generated part way through a line and the line wraps.

    Using the print hex option it is clear that each byte is being converted into 3 bytes of code and it seems as if, when the total number of bytes (including the triple bytes) exceeds the line length the weird effect occurs, eg. if the line has say 40 spaces followed by 14 graphics the total number of bytes is 82 (40 + 14 x 3) and the corruption occurs.

    I haven’t yet sorted out whether it is possible to change the framebuffer size without altering the displayed raster, but that is my next step.

    #884
    kevin
    Participant

    When I use Minicom with “-R CP437″, in a Unicode kernel console, then it seems to work OK.

    But…

    With the default font settings, I get a console that is (I guess) 160 characters wide and perhaps 40 high. That’s no use for CP/M, and it’s unreadable on my 11” monitor, anyway.

    Problem is… I don’t know any way to change this text size without changing the overall screen resolution — which is very ugly — or using a differently-sized PSF font. I’ve been using the “Terminus” fonts until now, and they look very nice on a full-HD display (because they can be anti-aliased, I guess).

    But none of the PSF console fonts I know about contain a complete set of Unicode glyphs, so even if Minicom does the translation properly, I still don’t see useful characters. So using “-R CP437” I seem to have a choice between proper glyph rendering but text too small and ugly, or nice text of a useful size without the extended glyphs.

    I _think_ (but wouldn’t bet my house) that the way I had this working before was to install a CP437 PSF font, and not configure Minicom at all. However, where I got a CP437 PSF font from is not clear to me any more. As I said earlier, I sort-of thought that the console would be using CP437 anyway. Perhaps the Kernel or ARM firmware does have a setting to do CP437 automatically, as all Linux kernels did back in the day? Perhaps I just enabled it accidentally while messing around?

    This is most vexing.

    Kevin.

    #895
    kevin
    Participant

    More information which may, or may not, be useful…

    I think the Terminus fonts provided in the Raspbian repo are only a subset of the full thing. The full Terminus font set does seem to include fonts with CP437 glyphs. The problem is that a PSF font only allows for 512 glyphs in total, so there is no “Unicode” PSF font.

    The complete set is here:

    https://sourceforge.net/projects/terminus-font/

    I had to build the PSF fonts from source but, in fact, it was as easy as running “make”. This generates a huge pile of PSF fonts. It seems that the fonts with names of the form ter-iXXX contain CP437 glyphs.

    To use these in a unicode framebuffer console, you’ll need to specify a CP437-to-unicode mapping. This works for me:

    $ setfont -u cp437 ter-i32n.psf

    Then you can run “minicom -R CP437” to have Minicom convert the incoming CP437 characters to Unicode. Then the Unicode map converts them to offsets in the PSF font.

    The largest of the Terminus fonts still isn’t quite large enough to give a 25×80 display on my monitor, but it’s not far off.

    I think I’m sort-of getting there, but it’s still not quite right. And I’m sure there used to be an easier way.

    Kevin.

    #896
    johnw
    Participant

    One step forward and two steps back!

    I have managed to install a cp437 font and set it as the console font. Showconsolefont displays the correct characters but minicom does not use it. It replaces all the glyphs with query signs.

    This is an excerpt from the minicom help page:

    -t

    Terminal type. With this flag, you can override the environment TERM variable. This is handy for use in the MINICOM environment variable; one can create a special termcap entry for use with minicom on the console, that initializes the screen to raw mode so that in conjunction with the -l flag, the IBM line characters are displayed untranslated.

    This looks useful, but unfortunately I am not familiar enough with linux to know what it means!! It is one of those “explanations” that relies on previous knowledge to understand it.

    I feel like I am so close.

    #900
    johnw
    Participant

    Our last posts obviously crossed!

    I have tried what you suggested and I do get the glyphs displayed but I am still getting the effect I described before, ie. it seems that when the total of ordinary plus 3 byte unicode characters exceeds 80 bytes corruption occurs. It might be because the width of my screen is 80 bytes.

    #901
    kevin
    Participant

    I guess this could be a bug in Minicom. I’d bet that somewhere there’s a function that works out how many bytes are in a screen row, and mistakenly assumes that one byte = one glyph. I feel sure that I had a solution that did not rely on Minicom doing anything clever, but I just can’t remember what it was :/

    I’m going to try the Raspberry Pi forum (unless you already tried that).

    Kevin.

    #902
    kevin
    Participant

    OK… this works for me with a desktop Linux system, although I haven’t had time to try it on the Pi Zero yet.

    Run minicom without any translation. That is, use the “-8” switch so that 8-bit characters are passed straight through.

    Send the following character sequences to the console (e.g., using printf)

    ESC %@
    ESC (U

    The first of these turns of Unicode support; the second disables any character translation.

    Use setfont to install a terminal font that contains CP437 glyphs, e.g.,

    ter-i32n.psf

    Cross your fingers );

    Kevin.

    #920
    johnw
    Participant

    No luck. Maybe I am doing it wrong?

    On the pi I have a python script:

    string=chr(27)+”%@”+chr(27)+”(U”
    print string

    I run this, then
    setfont ter-i16b.psf, then I have tried various minicom options:

    minicom -8
    minicom -R ter-i6b
    minicom -R cp437

    None of them give the correct font.

    I don’t think the desktop gives the same results as the console, but have you had any success with it?

    John.

    #928
    kevin
    Participant

    Hi

    I’ve found another approach, that gives what I think is complete cp437 coverage. Unfortunately, the font I’m using is not large enough to fill the whole screen (without reducing resolution).

    setfont -u none -16 /usr/share/consolefonts/cp865-8×16.psf.gz
    printf ‘\x1B%%@\x1B(U`
    minicom … -8

    The font is in the Raspbian ‘consolefonts’ package (IIRC).

    Here’s a picture showing the results:

    https://kevinboone.me/temp/2048.jpg

    I’m using a trivial CP/M program to dump the character set; it’s here:

    https://github.com/kevinboone/cpmchars

    I checked the output against the listing for cp437 on Wikipedia, and it seems to be a match.

    Now… I got a much nicer display using the Terminus fonts, but I couldn’t get complete CP437 coverage. It was close, but not exact. I haven’t given up hope of making Terminus work properly, because the appearance is just so much nicer. The problem I have is that my Z80-PG is now in a case, so to make changes to the Pi Zero I have to dismantle the case, take the SD card out, move it to my PC, yadda, yadda, yadda. The Linux installation I’m using on the Zero is completely home-made, and read-only. So I have no way to change anything except by rewriting the SD card. That’s why it takes me such a long time to test this stuff.

    Kevin.

    #929
    johnw
    Participant

    I’m glad you have reached a successful conclusion, but I am as far away as ever!

    I tried your suggestions and this is what I got:
    My result

    Before entering minicom I can type
    printf ‘\xDB’
    and get the correct block character on the screen, proving that the console is displaying the correct characters before minicom starts.

    My only query now is that you say “minicom … -8”. If I type that I get “cannot open /dev/tty8 permission denied”, but I am using serial0 not tty8, so I take it that is a typo. If I type “minicom -8” it opens but I get the incorrect font.

    When minicom starts up it says it was compiled on Aug 13 2017 at 15:25:34. I wonder if you have a different version that works.

    John.

Viewing 15 posts - 1 through 15 (of 19 total)
  • You must be logged in to reply to this topic.