AT+CIPSEND with TCP link having non-zero so_sndtimeo leads to loss of stream synchronization after failed attempt

[ArtemPisarenko]

Answers checklist.

• I have read the documentation ESP-AT Programming Guide and the issue is not addressed there.
• I have used the latest released firmware or have updated my ESP-AT branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

When TCP connection configured with AT+CIPTCPOPT so_sndtimeo non-default value, AT+CIPSEND expected to operate similar to non-blocking write()/send()/sendto() socket function. It may response with SEND FAIL, meaning that send buffer is full (remote peer didn't acknowledged enough data) and user should proceed with trying again this command. Am I right?

ESP doesn't report nothing in this case (except for Recv xxx bytes, which only means how many data it received from user via UART). It makes me (user) think that ESP failed to send whole chunk of data, because documentation doesn't describe such a behavior.

The problem is that it's not true. I found that ESP actually accepts (may accept, depending on how lucky you are at a given moment with sizes and buffer state) part of received data for transmission and will deliver it sooner or later. But user doesn't know size of that data chunk in order to skip it in next AT+CIPSEND attempt (to avoid sending same data twice). It breaks integrity of data stream guaranteed by TCP protocol.

I guess it's a known issue and it seems to be the reason for #908 submitted as a feature request. But I consider issue described above a serious limitation (or even bug) requiring attention.

I don't see any workaround for it (except for modifying ESP-AT firmware with custom commands).

1. AT+CIPSENDL seems to be possible solution (because it reports +CIPSENDL:<had sent len>,<port recv len> ?), but anyway it's unacceptable because of undesirable cancelling sequence +++ (I cannot guarantee continuous UART data flow).
2. Passthrough Mode isn't acceptable because of single-connection restriction and tricky +++ sequence.
3. Blocking mode (so_sndtimeo = 0) isn't acceptable because it blocks whole AT command interface in unpredictable manner (e.g. other links/connections can't be processed).

[ustccw]

@ArtemPisarenko May i ask what exact behavior you would expect in this case?

[ArtemPisarenko]

@ustccw , I don't expect anything in this specific case, because it's a drawback of ESP-AT protocol itself. One of drawbacks. Because now, after weeks of pain, I finally found this protocol too poorly designed to be usable as M2M interface in reliable multi-connection production-ready scenario, where this issue only makes sense.

Caution for new users

I would suggest to append DISCLAIMER ( https://docs.espressif.com/projects/esp-at/en/latest/esp32/disclaimer.html ) with something like:

Don't be fooled with rich functionality and flexibility of ESP-AT solution at first look! Don't expect that this "ready" solution will save your product design time! Generally, you should start with development of your own application and protocol from scratch based on ESP-IDF framework, even if you don't familiar with it - this still will be faster and much less risky.
Use ESP-AT only and only if you aim for simple application-level commands usage (such as downloading file via AT+HTTPCGET), or for simple TCP/IP single-connection module-state-unaware usage provided that either host transfers data at slow rate in high-throughput ideal network environment, or UART command interface used with hardware flow control enabled. 95% of TCP/IP commands protocol functionality have non-obvious limitations and pitfalls, you discover only after diving deeply into docs and/or putting serious effort to integrate it into your design.

Angry critic

This wrong design decision made at very start caused me to finally waste a lot of time, because each time I faced new quirk and made workaround for it (or discovered that already implemented workaround is naive and problems are too deeper), it was looking as "almost ready" to discard all work done in favor of custom firmware development. Main reason is that you can't simply adopt some generic AT command parser. Full-feature ESP-AT extension (even just for TCP/IP commands) makes proper and robust reception on host side very complex task (only on host side, because for ESP-side messages are designed more carefully and accurately }:-[ ). Low-level lexema parsing is very tightly coupled with high-level logic operating with different modes and configurations. Some notable quirks: varying parameters format for same message, +IPD variant with its end (critical for keeping protocol synchronization!) encoded in one of its parameters which position depends on mode, +IPD format specifics with starting comma instead of colon, +CIPSTA responses containing colon delimiter as part of name, etc. There are no strict rules! It's a nightmare! It looks like protocol designed (or later modified) by monkey. You have to either develop complex parsing code from scratch (only because of few quirk messages), or refuse from using some protocol features (which MAY look as non-important at beginning), or depend on some non-obvious consumptions and limitations (which may be found as unacceptable much LATER). And this is only to get sub-optimal and error-prone interface working.

Since I've already made too much effort in adopting ESP-AT for my usage, my attempt to address this issue hopefully was relatively easily succeeded by modifying AT firmware with custom commands (only thanks to hidden API function int32_t esp_at_get_socket_by_link_id(int32_t id)).
So my approach for addressing this specific issue is alternative AT+EIPSEND command with "one-phase" data transfer (without SEND OK/FAIL).
This command checks arguments and, if everything is ok, prompts with

>

, receives length bytes and outputs

+EIPSEND:<result>
OK

(If arguments check wasn't succeeded, then it immediately outputs ERROR and finishes, not even prompting for data input. There are no other reason for "fail" during command execution.)
<result> is integer parameter. Non-negative value indicates how much data commited to tcp/ip stack (it corresponds to either positive value returned by write() call, or zero in case when write() returns -1 and errno is equal to either one of EWOULDBLOCK / EAGAIN / EINPROGRESS ). Negative value indicates negated error code (-errno) in case when write() call returns -1 and errno equal to value different from mentioned above.

[ustccw]

Sorry for the inconvenience caused. Due to limitations of the current ESP-AT framework, it may not fully meet your requirements in this scenario. We appreciate your feedback and thank you for your understanding.