Working on new FTP backend - various issues

Hi All,

I’m trying to implement an FTP backend for rclone (cfr. issue #540) but I’m having a few issues. I admit i am a go newbie, so I might be doing something really really stupid…

However, what happens is that when I try to list a directory the program seems stuck. The List() function terminates (as far as I can see), but then the code seems stuck in some sort of deadlock (strace says it’s stuck in a futex() call).

I’ve also tried with delve but I couldn’t really see what happens after the List() is called.

If you want to check the code it’s in my github fork at

Uhm, turns out it works if I call Finished(). Strange, swift backend is not calling it…

Exciting :smile:

swift calls defer out.Finished() and you’ll need to do that too.

I had a quick look at your code - I suggest you move your mu into the Fs if it protects the ftp.ServeConn from concurrent use. Leaving it as a global var will mean trouble if the user decides to make two ftp remotes!

The concurrency aspects will need some thought…

I hope you saw the outline of how to make a new remote here:

Thank you.

I think I have fixed most of the concurrency issues, but I cannot run tests. It seems that the “make test” target assumes there is a configuration somewhere, how can I run the tests?

Follow the instructions here:

For the “unit tests” section, then the “integration tests” section.

make test is used to do a complete integration test before release so probably isn’t what you want.

Thank you, it should now be fixed. I’ll issue a new pull request

I have issues with the tests. The main issue I guess is that I don’t know how to add a dependency from
How can I add the dependency?

You need to use godep:

Then commit the changes to the vendor directory in a separate commit.

I just had a quick look over your code - looks like you are getting there - well done!

How are you doing with the integration tests?

Integration tests work. I’m also using it to complete the sync of 300TB (1000-genomes project), so far it seems to work.

About godep: I’ve tried, but I’m not sure I’m doing it right. In case, would you mind giving me more detailed instructions? (again: go newbie here :))

That is great news

I have to say I don’t find godep easy to use!

There must be a better way than this but…

Edit Godep/Godeps.json and add this to the end (and also an extra comma to the line before)

	                "ImportPath": ""
			"Comment": "",
			"Rev": ""


go get -u
godep update

Then commit vendor/ and Godeps/Godeps.json in a single commit.

Is this working OK? I think I’m going to try it. :slight_smile:

I’m having some issues but didn’t have much time to work on it.

When dealing with a very big tree, apparently not all files are processed.

Thanks for the reply, will give feedback.

Off topic but how does one get access to that data. I was trying to locate the 10000 genomes project but can't actually find anything but publications.