srt-file-transmit file corruption

[timeslice]

Hi,
We're using v1.3.2 srt file transmit and seem to be seeing corruption on sent files 95% of the time, it seems we're seeing some packets lost but not retransmitted so we've tried increasing the send buffer (and receive) but this hasn't helped, also, below is the output of a transfer resulting in corrupt file output.

It also seems the link bandwidth is reported incorrectly for some reason, eg, on a receiver that has 1Gb/s LAN connection srt-file reports over 2Gb/s, so we're not sure how those bw stats are generated as they don't seem correct.

Also for some reason on our 1Gb LAN we're seeing transfers reach ~500Mb/s with Linux sender and only 110Mb/s on windows, I think Maxlovic mentions this might due to different sleep behaviour across OS platforms so we're looking at patching this for windows at the moment.

Here's an example sender command we've been testing with (on LAN at the moment):

srt-file-transmit.exe "file://d/path/to/file.mov" "srt://192.168.10.12:4001?mode=listener&sndbuf=4250000&rcvbuf=8250000&fc=2000000" -s:100000

The above command seems to be more reliable under Linux than windows corruption wise.

======= SRT STATS: sid=396209979
PACKETS SENT: 0 RECEIVED: 1269
LOST PKT SENT: 0 RECEIVED: 642
REXMIT SENT: 0 RECEIVED: 0
DROP PKT SENT: 0 RECEIVED: 309
RATE SENDING: 0 RECEIVING: 98.9744
BELATED RECEIVED: 0 AVG TIME: 1.84467e+16
REORDER DISTANCE: 0
WINDOW FLOW: 8192 CONGESTION: 16 FLIGHT: 0
LINK RTT: 132.831ms BANDWIDTH: 2348.94Mb/s
BUFFERLEFT: SND: 12288000 RCV: 12286500
Source disconnected

Thanks

timeslice

[maxsharabayko]

Hi @timeslice
Is it a degradation in v.1.3.2 you are seeing, or a general observation? You using v.1.3.2 release version, right?
If it is just an observation, we are awared about it, and working on an improvement.

[maxsharabayko]

@timeslice Also fc=2000000 is too high. Current implementation might have problems because of that high value. Please refer to #700 to determine FC calculation. Twice the value, actually:
FC = 2 × bps × RTT_sec / 8 / (1500 - 28).

[timeslice]

Thanks Maxlovic for quick response, yes we're on 1.3.2 but haven't tested older releases much, one tweak we did change was in srtcore/common.h to change the max bw variable (const int64_t BW_INFINITE = 1000000000/8; //Infinite=> 30Mbps). Not sure if this could have had other effects?

We've tried calculating numbers for FC based on the above formula and using 1000Mbps for bps value we worked out an FC of 80 (our RTT was between 0.5 and 1.5ms) receive buffer numbers based on this formula would be 1472*40 we think but we did still see corruption with those values and on both windows and linux (ie. win>win and lin>lin) we saw the max bw for the transfer only get to 10% of the reported link speed from srt stats ( win>win link was 1Gbps and lin> lin was 10Gbps and transfers were ~100Mbps and ~500Mbps respectively.

One thing we're confused about is that if we supply no parameters (use fc and rcv buf at defaults) speed is about 1% of the link, but where we change values to eg. FC 80, rcv buf = 150000000 we get 10 times that speed. We're testing on LAN atm to make sure we get stable and fast file transfer but have also found similar results when testing over LAN.

Are you able to clarify how the bw stats are generated and why eg. we see large stats even where there's a 30Mbps wan link to the remote end?

[maxsharabayko]

@timeslice Could you please collect CSV stats by specifying the following command line arguments:
srt-file-transmit.exe "file://d/path/to/file.mov" "srt://192.168.10.12:4001?mode=listener&sndbuf=4250000&rcvbuf=8250000&fc=2000000" -s:100000 -statsout:srtstats.csv -pf:csv

Regarding the stats accuracy, bandwidth estimation is only a guess of the available bandwidth of a link, so don't expect it to be very accurate.

[maxsharabayko]

You can also take a look at develop/filecc branch. I've copyed some changes to file congestion control. It is still work in progress though. But it you can make use of it, it would be anyway good to have your feedback.
Just specify the modifyed CC by providing congestion=filev2 URI query option.
Like sender: srt-file-transmit ./file "srt://host:port?congestion=filev2"
and receiver:
srt-file-transmit "srt://:port?congestion=filev2" ./

[maxsharabayko]

Please build with -DUSE_BUSY_WAITING=ON
E.g.: cmake ../ -DUSE_BUSY_WAITING=ON. This should help on Windows.

[timeslice]

@maxlovic Thanks, we've been testing with both main and file-cc branch and with the congestion param but not seeing any improvement, results are quote inconclusive. Sometimes we see file corruption even though there's no packet drops reported at receiver, csv of stats from one file transfer are attached from sender and receiver.
We've been using your formula for FC calc (FC = 2 × bps × RTTsec / 8 / (1500 - 28), doing a test transfer and then using latency and rtt numbers from that for tests, this does seems to help but we're still seeing a lot of drops on congested links particularly we think, although sometimes it seems it might be the transfer causing the congestion, also we're unclear how the buffer should be calculated for file transfers?

maxfix-v8 stats.zip

[maxsharabayko]

The plots look weird. The number of packets received stays the same, but the receiving rate increases.

Try to conduct a bandwidth loop test described here.

[maxsharabayko]

Also for some reason on our 1Gb LAN we're seeing transfers reach ~500Mb/s with Linux sender and only 110Mb/s on windows.

v1.3.3 introduces USE_BUSY_WAITING build option (PR #711), that should solve this. The reason you are getting 110 Mbps on Windows is most probably because of the high fluctuations in the sending rate within 10 ms intervals, that causes network congestion.