FTPino - A FTP Client and Server for the Particle

I had some problems with ftpRush as well, before I configured single socket mode. Which mode is ftpRush set to (Passive / Active)?

If I remember correctly, it only worked in Passive? If you already set it to Passive, try setting it to active, what result would that produce?

Yes Iā€™m in passive mode.
Active mode dont seems to work at all right now. Iā€™ll debug this tonight.

Woah. Where did you get the sources from? I see the lib in Particle is not the latest / I donā€™t know if itā€™s supposed to sync to github?

Anyhow, the code from github differs slightly from what you wrote above. You should be able to get a one-shot working read-write as described by others in this thread.

Indeed, it works in Passive Mode.

I retested it and could upload and then download a 76kB file :

Nope, there is no auto resync with github. If you have a new stable version, it would be good to push that onto the Build lib repo again.

@ScruffR thanks for the info!

@Suprazz Iā€™m uploading my Wireshark trace and the file in question (itā€™s randomly generated, so the text doesnā€™t make any senseā€¦ ;))

File : http://expirebox.com/download/a59d91413350dee743f28dfc76d4dae2.html
Wireshark Pcap: http://expirebox.com/download/eeee543ffde651a311202f6ec843cf0d.html

My Wireshark version is Version 2.0.3 (v2.0.3-0-geed34f0 from master-2.0).

Hope this helps,
Mihai

Here is what I get with your lastest version and without any modifications to you files using FTPRush in passive mode, single socket.

150 Opening BINARY mode data connection for file transfer.
[2] 226 Transfer complete. 45568 bytes written.
[2] MDTM 20160116110000 sett123.bmp
[2] 502 Command not implemented: MDTM 20160116110000 sett123.bmp.
[i] sett123.bmp 48,1 kbytes/0,45(s)/108,26Kbps
[i]  1 fichier(s) transfere(s) - Total: 48,1 k byte(s) en 0,45 (108,26 KBps)

In active mode, Iā€™m not able to list the files (FTPRush again, active mode, single socket.)

ā€¦ does it transfer all the info? if it only writes 45568 bytes it didnā€™t transfer the whold 48,1 kB?!
if so, at least itā€™s more than 128 bytes :slight_smile:

no itā€™s the same issue. there is always bytes missing at the end of the file. Same issue as beforeā€¦ but the issue look like to be in the TCPclient which is disconnecting without reading all the data received

ok, as a starting point, you could look at the pcap mentioned above. Can you please post your test file as well? I will only have access to the hw and sw again on Tuesday, the 27th.

EDIT ; When building the sources, I linked against SDFAT 0.0.3 from Particle directly, which SD Driver are you using. Did you link to the library or are you compiling it yourself?

Iā€™m compiling the sd fat library that we can get on github for particle photon.

The problem is not related to the writing part but with the TCPClient itself or ftp program. I have exactly the same issue with SPIFFS and the SPI flashā€¦ so itā€™s not related to the Sd card library.

If you look at the debug info below itā€™s leaving the while loop and if according to wireshark, all the data is sent correctly so it look like the TCPClient is dropping the connection while there is still bytes available.

[2] 220 Welcome to FTPino FTP Server.
[2] USER sam
[2] 331 Password required for user sam.
[2] PASS (hidden)
[2] 230 User successfuly logged in.
[2] SYST
[2] 215 UNIX emulated by FTPino.
[2] TYPE A
[2] 200 Type set to BINARY.
[2] REST 1
[2] 350 Restarting at 1. !!! Not implemented
[2] REST 0
[2] 350 Restarting at 0. !!! Not implemented
[2] FEAT
[2] 211 Extensions supported SIZE MDTM XCRC.
[2] PWD
[2] 257 "/" is current directory.
[2] CWD /
[2] 250 "//" is current directory.
[2] PWD
[2] 257 "/" is current directory.
[2] PASV
[2] 227 Entering Passive Mode (192,168,2,155,4,0)
[2] Ouverture de la connexion IP: 192.168.2.155 PORT: 1024
[2] LIST
[2] 150 Opening ASCII mode data connection for transfer.
[2] 226 Transfer complete.
[2] Liste Complete: 572 bytes en 0,00s (0,57KB/s)
[2] TYPE I
[2] 200 Type set to BINARY.
[2] PASV
[2] 227 Entering Passive Mode (192,168,2,155,4,0)
[2] Ouverture de la connexion IP: 192.168.2.155 PORT: 1024
[2] REST 0
[2] 350 Restarting at 0. !!! Not implemented
[2] STOR oil-rtestsam.bmp
[2] 150 Opening BINARY mode data connection for file transfer.
[2] 226 Transfer complete. 4736 bytes written.
[2] MDTM 20160714194300 oil-rtestsam.bmp
[2] 502 Command not implemented: MDTM 20160714194300 oil-rtestsam.bmp.
[i] oil-rtestsam.bmp 8,5 kbytes/0,25(s)/32,77Kbps
[i]  1 fichier(s) transfere(s) - Total: 8,5 k byte(s) en 0,25 (32,77 KBps)

This is the code:

         int bytesRead =0;
         do
         {
            bytesRead = dclient.read(readBuffer, sizeof(readBuffer));
            if(bytesRead > 0)
            {
                dbg("Bytes read:", String(bytesRead));
                dbg("Client available:", String(dclient.available()));
                maxIterations = 2000;
                fh->writeFile(file, reinterpret_cast<char*>(readBuffer), bytesRead, isAppend);
                totalBytes+= bytesRead, bytesRead = 0; 
            }
            Particle.process();
            maxIterations--;
            dbg("Client connected:", String(dclient.connected()));
        }while(( dclient.connected() || dclient.available()) ||
                (isFTPinoClient && (maxIterations > 0)));

        dbg("Client connected:", String(dclient.connected()));
        dbg("Client available:", String(dclient.available()));
        dbg("IsFTPinoClient:", String(isFTPinoClient ? 1 : 0));
        dbg("Iterations:", String(maxIterations));

And here is the serial debug trace:

Bytes read:->128
Client available:->128
Client connected:->1
Bytes read:->128
Client available:->128
Client connected:->1
Bytes read:->128
Client available:->0
Client connected:->0
Client connected:->0
Client available:->0
IsFTPinoClient:->0
Iterations:->1999

@ScruffR do you have an idea what is happening. Only receiving data in pack of 128 bytes (which is normal) but there is missing data at the end and there is no ā€œpartialā€ buffer at the end (to complete all the data sent).

Same issue with WINSCPā€¦

Is calling Particle.process is enough to let all the tcpclient thing do itā€™s job because we are in a loop and not doing anything else?

No, sorry!
Iā€™d have to dive into the library for which I havenā€™t got the time now.

But Particle.process() wouldnā€™t do anything about not closing the TCP connection prematurely which might be at least one of possible reasons for your symptoms.
On the other hand, looking at your while() condition, you seem to expect that dclient.connected() could be false while dclient.available() might be !=0.
In that case, if you wanted to give the device some more time for a final transfer you could do this instead of the single Particle.process()

  for(uint32_t ms=millis(); millis()-ms <= (dclient.connected() ? 0 : 20); Particle.process());  // 20ms for final call, otherwise at least one Particle.process()

BTW, just to make sure, what your intention is with this line totalBytes+= bytesRead, bytesRead = 0;

// do you meant to do this
  totalBytes+= (bytesRead = 0);
// or this
  totalBytes+= bytesRead;
  bytesRead = 0;

Iā€™d assume you want the latter one, so writing it this way might be clearer.

You might also try to print bytesRead irrespective of it being >0 or not.

The line is valid C++ and defaults.to a ā€˜;ā€™

Now concerning the missing bytes, I could not reproduce the behavior. @shm45 also transferred a 466kB *.wav single-shot without any problems.

Please just link against the Sdfat library from Particle (libraries view in the browser), do not compile it yourself. That way we can at least rule this out.

Other than this, I am really at a loss as to what could cause the symptoms. Please post a demo file which doesnā€™t work and Iā€™m going to look at it Tuesday.

The reason behind you not transferring everything is that either the Ftp Client (pc) or the server (Ftp Server From FTPino) is closing the socket prematurely, before all data is transferred.

You should, as far as I can tell, not call System.process() as there is no system stuff in need of being handled before the transfer is finished.

Mihai

This might be valid C++ but it's not good readable style especially in respect of execution order IMHO.

A matter of taste IMHO, same as opening brackets for blocks on the same line as an instruction (i.e. ā€œif(ā€¦){ā€, or on the next line ā€œif(ā€¦)\n{ā€). I also use comma quite extensively.

According to @shm45, he also spent a lot of time trying to adapt the sdfat. So I guess you should just link against the library when ypuā€™re compiling your sources.

"Hello Mihaiā€“I got it working!

The trick was using the SdFat-Particle library (I spent way too much time trying to adapt the Arduino SdFat library).

I transferred a 466 kByte .wav file using a python client to upload the file to a Photon running FTPServerā€“worked like a charm"

I think this might be the issue. The available fonction will return 0 as soon as the _sock is closed.

I would like to test that locally but Iā€™m not sure how to figure out if my local build is really working and not overwritten by OTA. Is there a place where I can specify a custom version so I know that the code running is really mine?
I used the command: make clean all program-dfu PLATFORM=photon in the firmware\modules folder.

In the function TCPClient::available() in spark_wiring_tcpclient.cpp
int TCPClient::available()
{
int avail = 0;

// At EOB => Flush it
if (_total && (_offset == _total))
{
    flush_buffer();
}

if(Network.from(nif).ready() && isOpen(_sock))
{
    // Have room
    if ( _total < arraySize(_buffer))
    {
        int ret = socket_receive(_sock, _buffer + _total , arraySize(_buffer)-_total, 0);
        if (ret > 0)
        {
            DEBUG("recv(=%d)",ret);
            if (_total == 0) _offset = 0;
            _total += ret;
        }
    } // Have Space
} // WiFi.ready() && isOpen(_sock)
avail = bufferCount();
return avail;

}

Hey @mihaigalos thanks for putting together this useful library! We're currently working on revamping the Libraries feature within our development tools to make them compatible with Arduino's library structure and make it easier to port libraries to Particle's library system.

We're running an open beta now, if you're interested in checking it out! Let me know :slight_smile:

Hi @KyleG, sure. Let me know how I can help.

Iā€™m still having the issue and still searching for a solutionā€¦

It really look like an issue with the TCPClient because I have the same problem with the softap applicationā€¦ Half of the file is sent. TCPClient read function is returning 0 but there is still data leftā€¦ I dont know how to debug thatā€¦