Code from Windows simulator does not load in DB48X on DM42.

[jthole]

The files written by the Windows simulator (latest version) are not compatible with DB48X 0.9.15 running on the DM42.

To test this, I created a trivial program and compared the two:

In both cases (DM42 and Win) the program was created using the 'x²' and 'Swap' keys (both with the 'CMD' setting):

«
        SQ SWAP SQ + √
»

After that, I STO-ed the code to an output file.

When comparing both files, the Windows simulator output x² instead of SQ. Besides, it writes "Swap" instead of "SWAP".

$ diff PYTH-DM42.48S PYTH-WIN.48S
1,2c1,2
< «
<      SQ SWAP SQ + √
---
> «
>      x² Swap x² + √

The Windows generated code cannot be loaded in the DM42 (RCL gives a syntax error).

After changing x² to SQ and Swap to SWAP, and running "dos2unix", the file loads correctly in the DM42.

[evgaster]

I will try to reproduce this ... not with Windows but with Linux.
Lets start with default settings (meaning Modes returns an empty program; i.e. CommandDisplayMode = LongForm).

Enter the program using the 'x²' and 'Swap' keys gives:

«
	x² Swap x² + √
»

If I store the program in a file the content is indeed the above.
I can move that to my DM42n and Recall the file.
The program then does what it is expected to do.

If I change the setting to CommandDisplayMode = UpperCase ('CMD') the display changes to:

«
	SQ SWAP SQ + √
»

If I store the program in a file the content is indeed the above.
I can move that to my DM42n and Recall the file.
Since I have my DM42n in CommandDisplayMode = LongForm, it comes up in that form.
The program then does what it is expected to do.

I suspect that there was no need for you to "changing x² to SQ and Swap to SWAP".
I guess the only issue is the end-of-line in Windows, which you resolved with dos2unix.

[jthole]

Thanks for looking into this!

Edit:

I am not sure why my export / import did not work last time. But I started from the same calculator state file on the DM42 and Windows now, and import works successful.

Note: import fails from windows when the files are directly copied, but succeeds after a 'dos2unix'.

Blame the above on user error!

[jthole]

Can be closed

[prismv]

IMHO this is indeed a valid issue report. Please see respective statement in MoHPC thread.

(Eventually too simple) fix is:

diff --git a/src/files.cc b/src/files.cc
index 792f4cc2..a1bd0536 100644
--- a/src/files.cc
+++ b/src/files.cc
@@ -373,9 +373,13 @@ object_p files::recall_source(text_p name) const
         for (unicode c = prog.get(); c; c = prog.get())
         {
             byte buffer[4];
-            size_t count = utf8_encode(c, buffer);
-            rt.insert(bytes, buffer, count);
-            bytes += count;
+            // skip carriage return
+            if (c != 0x0d)
+            {
+                size_t count = utf8_encode(c, buffer);
+                rt.insert(bytes, buffer, count);
+                bytes += count;
+            }
         }
     }

[jthole]

Reopened on request of @prismv

[evgaster]

@prismv your fix would prevent any, even intended, use of a carriage return character in a program.

[evgaster]

@jthole If you write a program with, say notepad on Windows, will the Windows DB48X simulator load such a file or does it also choke (on the carriage return character, which is part of the Windows end of line sequence)?

[prismv]

@prismv your fix would prevent any, even intended, use of a carriage return character in a program.

Indeed. But \r is of no use on a calculator. If at all ...

PS for clarification: So one may decide not to support CR chars. Also wondering, if it is planned to insert this 'character' by db48x menu.

[prismv]

@jthole If you write a program with, say notepad on Windows, will the Windows DB48X simulator load such a file or does it also choke (on the carriage return character, which is part of the Windows end of line sequence)?

Notepad was (in)famous to be a very simple editor. Now on recent Windows releases it even shows that it uses Windows CRLF. Still wonder how to change this in this editor.

PS: Notepad is still no nice code editor, but there are enough alternatives also for Windows. Though one could ask Copilot to change line endings ;-)

[jthole]

@prismv @evgaster Here is a file (attached as ZIP) with carriage returns:

$ od -c LOGBN.48S
0000000 302 253 \r \n \t D U P 2 L N S W A
0000020 P L N / \r \n 302 273
0000032

The file successfully loads in the Windows simulator.

As expected, it fails to load in the DM42.

It also fails to load in the iPhone DB48X app.

LOGBN.48S.zip

[prismv]

@prismv @evgaster Here is a file (attached as ZIP) with carriage returns:

$ od -c LOGBN.48S 0000000 302 253 \r \n \t D U P 2 L N S W A 0000020 P L N / \r \n 302 273 0000032

The file successfully loads in the Windows simulator.

[...]

On Linux such examples (alternatively just generated using unix2dos conversion ...) give same issue as @jthole reported for DM42.

Indeed it is interesting that it works on Windows simulator. Let's read e.g. on https://github.com/MicrosoftDocs/cpp-docs/blob/main/docs/c-runtime-library/reference/fopen-wfopen.md

In text mode, carriage return-line feed (CRLF) combinations are translated into single line feed (LF) characters on input, and LF characters are translated to CRLF combinations on output.

Since macOS X is also Posix compliant, it's the easiest way to remove all carriage returns on read. Alternatively it would only do strict CRLF to LF substitution. Though IMHO not needed, if CR character input is not going to happen by calculator CHAR menu.

[evgaster]

From the above my (preliminary) conclusion is that if you stay on Windows it works.

If you move between Windows and a "Posix compliant" system, like Linux, MacOS, iPhone, iPad, Android or ... DMCP, you will experience that a carriage return will be treated as "any other character".

The rpl syntax doesn't like it.

either:

1. the moving user deals with it (dos2unix / unix2dos); or
2. the application, DB48X / DB50X, deals with it ... to pamper Windows users :-)

I am Ok with filtering out carriage return, provided that behaviour and why is (user) documented.

[prismv]

Here is another, and probably better fix:

diff --git a/src/utf8.h b/src/utf8.h
index f17654e3..6baf118d 100644
--- a/src/utf8.h
+++ b/src/utf8.h
@@ -308,7 +308,7 @@ inline bool utf8_whitespace(unicode cp)
 //   Check if something is a whitespace
 // ----------------------------------------------------------------------------
 {
-    return cp == ' ' || cp == '\n' || cp == '\t';
+    return cp == ' ' || cp == '\n' || cp == '\r' || cp == '\t';
 }

From the above my (preliminary) conclusion is that if you stay on Windows it works.

If you move between Windows and a "Posix compliant" system, like Linux, MacOS, iPhone, iPad, Android or ... DMCP, you will experience that a carriage return will be treated as "any other character".

The rpl syntax doesn't like it.

either:

1. the moving user deals with it (dos2unix / unix2dos); or
2. the application, DB48X / DB50X, deals with it ... to pamper Windows users :-)

I am Ok with filtering out carriage return, provided that behaviour and why is (user) documented.

In fact the Windows simulator will not only remove CR during reads, but also reintroduce CR during writes - it's just the standard way for Windows text mode. It's not even needed anymore with current editors on Windows 11.

But it is complete mess, if you wants to transfer such files directly to other systems. Option 1. is IMO completely unnecessary overhead for usage.

Actually the tabular \t in previous example made me wonder, where the code already deals with it correctly.

@prismv @evgaster Here is a file (attached as ZIP) with carriage returns:

$ od -c LOGBN.48S 0000000 302 253 \r \n \t D U P 2 L N S W A 0000020 P L N / \r \n 302 273 0000032

[....]