New Post: Delete Symbolic Link

Another thought,

As far as deleting directories,

You need to use DeleteDirectory method since Delete can delete only files, this is due to SFTP specification

As far as why it can give you an error here possible explanations from specification:

An error will be returned if no directory
   with the specified path exists, or if the specified directory is not
   empty, or if the path specified a file system object other than a
   directory.

So I suspect because this is link and not directory object it fail. So it seems to me it should always fail then.