Tuesday, March 4, 2014

Upload files to FTP server

Hi Readers,

We got a requirement to upload files to ftp from an application server that will drop files to particular folder.

So we have created a nice shell script to do this job & scheduled it via windows task manager.

Script does the following:

  1. It will check if the source folders have files to process , if not than it will exit

  2. Upload those files to FTP

  3. Files that have been uploaded are moved to processed folder (folder name is time stamped)

  4. Error checking--> if there is error while processing it wll email.

  5. Detailed logging has been added.


Download & extract the script, below are the variables that I have used , you can change them accordingly

http://gallery.technet.microsoft.com/scriptcenter/Upload-files-to-FTP-server-8373e632

###########################Variables######################################
$smtpServer = "smtp.lab.com" # Change

$fromadd = "DoNotReplyftp@lab.com" # Change

$email1 = "Vikas.Sukhija@lab.com"  # Change

$ftp = "ftp://127.0.0.1/" # Change

$user = "vikas"  # Change$pass = "password" # Change

$uploadfpath = "C:\Uploadftp\ftpfiles" # Define the Folder from where files will be uploaded
###########################################################################

here is what it will look like, after changing the variables, you just have to run the batch file


PowerShell

##################################################################################### 
##           Script will Upload files to FTP    
##           Author:  Vikas Sukhija                                                            
##                                                                     
##           Date: 02-24-2013 
##           Modified Date:- 02-26-2013 (included loging & monitoring) 
##################################################################################### 
#############################Define Log Files######################################## 

$date = get-date -format d 
$date = $date.ToString().Replace(“/”, “-”) 
$time = get-date -format t 
$month = get-date  
$month1 = $month.month 
$year1 = $month.year 
$time = $time.ToString().Replace(":", "-") 
$time = $time.ToString().Replace(" ", "") 

$log1 = ".\Logs" + "\" + "FTP_" + $date + "_.log" 
$log2 = ".\Logs" + "\" + "FTP_" + $month1 +"_" + $year1 +"_.log" 
$log3 = ".\Logs" + "\" + "FTP_" + $date + $time + "_.log" 

$logs = ".\Logs" + "\" + "Powershell" + $date + "_" + $time + "_.txt" 

#Start-Transcript -Path $logs  
$dt = Get-Date 
Add-Content $log3 "$dt : Script Started" 

###########################Variables###################################### 

$smtpServer = "smtp.lab.com" # Change  
$fromadd = "DoNotReplyftp@lab.com" # Change  
$email1 = "Vikas.Sukhija@lab.com"  # Change  
$ftp = "ftp://127.0.0.1/" # Change 
$user = "vikas"  # Change 
$pass = "password" # Change 
$uploadfpath = "C:\Uploadftp\ftpfiles" # Define the Folder from where files will be uploaded  

########################################################################### 

$checkitems = Get-ChildItem $uploadfpath 
$countitems = $checkitems.count 
if ($countitems -eq 0) 

Write-Host "No items to process" -ForegroundColor Green 
$dt = Get-Date 
Add-Content $log3 "$dt : No items to process, script will exit" 
exit 

$dt = Get-Date 
Add-Content $log3 "$dt : Total number of items to process $countitems" 

$processed = ".\processed\$date" + "-" + $time 
if((test-path $processed) -like $false) 

New-Item -Path "$processed" -type directory 


##################################################################################### 

if ($error -ne $null) 
      { 
#SMTP Relay address 
$msg = new-object Net.Mail.MailMessage 
$smtp = new-object Net.Mail.SmtpClient($smtpServer) 

#Mail sender 
$msg.From = $fromadd 
#mail recipient 
$msg.To.Add($email1) 
$msg.Subject = "FTP Script error" 
$msg.Body = $error 
$smtp.Send($msg) 
$dt = Get-Date 
Add-Content $log3 "$dt : Script Terminated because of error: $error" 
$error.clear() 
exit 

       } 
  else 

      { 
    Write-host "no errors till now" 
      } 

$webclient = New-Object System.Net.WebClient  

$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)   

#Upload each file in upload directory... 

foreach($item in (dir $uploadfpath "*.*")){  
    Write-host  "Uploading $item..."  -ForegroundColor Green 
    $dt = Get-Date 
    Add-Content $log3 "$dt : Uploading $item..." 
    $uri = New-Object System.Uri($ftp+$item.Name)  
    $webclient.UploadFile($uri, $item.FullName) 
      if($error -ne $null) 
        { 
    Write-Host "Items will not be moved" -ForegroundColor Red 
        } 
        else 
        { 
    Write-Host "Moving $item to processed" -ForegroundColor green 
    Move-Item "$uploadfpath\$item" $processed 
    $dt = Get-Date 
    Add-Content $log3 "$dt : Moving $item to processed" 
        } 
 }  

if ($error -ne $null) 
      { 
#SMTP Relay address 
$msg = new-object Net.Mail.MailMessage 
$smtp = new-object Net.Mail.SmtpClient($smtpServer) 

#Mail sender 
$msg.From = $fromadd 
#mail recipient 
$msg.To.Add($email1) 
$msg.Subject = "FTP Script error" 
$msg.Body = $error 
$smtp.Send($msg) 
$dt = Get-Date 
Add-Content $log3 "$dt : Script encountered error: $error" 
$error.clear() 
       } 
  else 

      { 
    Write-host "no errors till now" 
      } 
$dt = Get-Date 
Add-Content $log3 "$dt : Script Processing finished" 

#Stop-Transcript 
##################################################################

Regards

Sukhija Vikas

 

22 comments:

  1. Excellent script, works like a charm ;-) However, for my particular situation I need to download files from FTP server onto my workstation. After successful downloads, files needs to be deleted from the FTP folder. Could you please advice on what needs to be change on your original script to accomplish my task? Thanks in advance

    ReplyDelete
  2. you have to use delete method, see if below link helps
    https://social.technet.microsoft.com/Forums/windowsserver/en-US/2e07df8a-c7cd-47be-8e13-35de9b9189a8/remove-files-on-ftp-site-with-powershell

    ReplyDelete
  3. Nice Script. I have Windows 2012 Server R2 installed on my local machine.

    I need to download specific folder included files from a Remote Server via FTP. Then a special Windows Software will work with these files and convert and export into other file formats in a other folder. From there i need to upload the folder included the new created CSV/XML Files (from the Windows Software) to a other Remote Server (where i can have FTP or SFTP access).

    ReplyDelete
  4. Hi Vikas, upon uploading the files to ftp site, I don't want original files to be deleted from source. How do you do that? Thanks

    ReplyDelete
  5. Its not deleting original files, it moving them to processed folder.

    ReplyDelete
  6. Hello Vikas,

    I try the script and get this error message :

    Exception calling "UploadFile" with "2" argument(s): "An exception occurred during a WebClient request."
    It is a general error message with linking to this line:

    $webclient.UploadFile($uri, $item.FullName)

    Is this something I can easily check? I know we have a proxy within our network. Can this be the reason?

    Thanks a lot for your reply in advance

    Dimi

    ReplyDelete
  7. Yes, what you are thinking should be the correct reason, As because of proxy direct connection to ftp might not be working. first check if you are able to open ftp & can read /write to it with proxy & than without proxy.

    ReplyDelete
  8. Helo Vikas,
    thanx for this usefull script.
    how can i implement a remote upload directory (not root)?
    thanks, Agis

    ReplyDelete
  9. can you provide with example so that I can understand what you are trying to accomplish, if you want to upload it to some ftp path than change the $ftp variable accordingly..

    ReplyDelete
  10. Hello Vikas,
    Thank you so much for this script - I am using as part of my automation process after a video project finishes encoding.
    Just thought I'd let you and maybe others who might read this comment know that Move-Item doesn't work well with filenames containing special characters, -LiteralPath must be specified as well for this.

    For the email part, if the user uses gmail, they need to specify using SSL as well as use login credentials somehow. You might want to add that.

    Thanks again!
    /Choke

    ReplyDelete
  11. Hello Vikas,
    Thank you for the script, but I keep getting The remote server returned an error: 227 Entering Passive Mode, are you aware if this is a firewall issue?

    ReplyDelete
  12. Powershell method that has been used in the script needs to be changed to use passive mode.

    ReplyDelete
  13. Hello!
    I have a problem with a file name, wich has a '#' sign in it. The URI method does not take it as a string, so on a remote server is only one file with the begining if its name, no ending.
    Thank You for help!

    ReplyDelete
  14. Thank you for your response, I have the script working on my local pc. When I run it on the server It need to be on I get a webclient error. I can connect to the FTP site via command line from the server.

    ReplyDelete
  15. Just guessing have you checked IE settings on the servers , IE --> options --> advanced there are options for passive ftp.. match those with the machine from which it works.

    ReplyDelete
  16. This script will work for Windows 2008 R2?
    Please respond immediately.

    ReplyDelete
  17. have not tested but it should work.

    ReplyDelete
  18. Hello Vikas

    is it there a way to validate if the file already exists in the server prior to try to upload de file?

    i mean to check it, before executing this part:

    $webclient.UploadFile($uri, $item.FullName)
    if($error -ne $null)
    {
    Write-Host "Items will not be moved" -ForegroundColor Red
    }

    I ask this beacuase my webhostgin provider is saying that its servers virus scan the file even if the upload is not performed beacause the file already exist in the server, so i want to avoid this scan of all the files if they are not going to be uploaded because already exists.

    ReplyDelete
  19. yes you can do it, refer the link https://syscloudpro.com/2014/11/21/monitoring-secure-ftp-file-uploads/
    to get the idea.

    ReplyDelete
  20. thanks for great script, will use it to load files to an linux ftp server.
    get the error The remote server returned an error: (553) File name not allowed.
    testet also with double // in the part of config $ftp = "ftp://127.0.0.1//"
    but same 553 error
    can you help?

    ReplyDelete
  21. Check the commands in the script, run one by one..

    ReplyDelete
  22. Hi,
    Does anybody converted this script to use psftp instead of ftp?
    Best Regards,
    Hicham

    ReplyDelete