STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.
What is the problem you are having with rclone?
The 1st part is, I am trying to transfer some files from an ftp server to an s3 bucket. The files are gziped (eg file001.xml.gz). After copying the directory, I find that many of the files are corrupted and the md5 checksum is not the same as the one provided by the server.
I believe this might have something to do with the transfer mode. The server mentions it should happen in binary mode. The package used by rclone to implement the ftp backend has an option for binary mode, but I can't seem to find something relevant in the source code or the documentation of Rclone itself.
The 2nd part, I am trying to change the logging level from NOTICE to INFO. As already mentioned, I am trying to do this from inside Go. Since I'm not super familiar with the language, I'm finding it difficult to do. Is there any way to directly access and edit --log-level and --log-file from inside Go?
Run the command 'rclone version' and share the full output of the command.
I am using Rclone as part of a Go package I'm writing, so I'm getting it from github instead. According to go.mod, I am using github.com/rclone/rclone v1.62.2
Which cloud storage system are you using? (eg Google Drive)
S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
ctx := context.Background()
config.SetConfigPath(rcloneConfigPath)
configfile.Install()
var filterOpts = filter.DefaultOpt
filterOpts.RulesOpt = filter.RulesOpt{
IncludeRule: []string{filterString},
}
firmFilter, err := filter.NewFilter(&filterOpts)
if err != nil {
return nil, err
}
filteredCtx := filter.ReplaceConfig(ctx, firmFilter)
fsource, err := fs.NewFs(filteredCtx, sourceRemote)
if err != nil {
return nil, fmt.Errorf("could not create source remote '%s': %w", sourceRemote, err)
}
fdest, err := fs.NewFs(filteredCtx, targetRemote)
if err != nil {
return nil, fmt.Errorf("could not create target remote '%s': %w", targetRemote, err)
}
entries, err := walkSourceRemote(filteredCtx, fsource) // result entries are not filtered
if err != nil {
return nil, fmt.Errorf("could not list source remote '%s': %w", sourceRemote, err)
}
log.Printf("Found %d files to copy.\n", len(entries))
err = sync.CopyDir(filteredCtx, fdest, fsource, true)
if err != nil {
return nil, fmt.Errorf("could not copy source remote '%s' to target remote '%s': %w", sourceRemote, targetRemote, err)
}
log.Printf("Done copying files.\n")
return entries, nil
}```
#### The rclone config contents with secrets removed.
<!-- You should use 3 backticks to begin and end your paste to make it readable. -->
Paste config here
#### A log from the command with the `-vv` flag
<!-- You should use 3 backticks to begin and end your paste to make it readable. Or use a service such as https://pastebin.com or https://gist.github.com/ -->
Paste log here