New Post: [SOLVED] Retrieving a LOT of output from a command...

Experts,

OK... I'd like to ask you for feedback/opinions on the following, seeking a better way to do this. You know the requirement: Capture a TON of output from a command...

The standard Execute (synchronous) truncated output at the buffer limit of the shell. In this self-contained example, there is no ShellStream():

[Top level Sub()]

Sub TopBatch()
    Using client As SshClient = New SshClient("IPOrHostname", "userName", "userPassword")
        Try
            client.Connect()
            If client.IsConnected Then
                Dim result1 As String = __DoCommand__(client, "enable" & vbLf & "enablePass" & vbLf & "term pager 0" & vbLf & "sh run" & vbLf)
                Console.WriteLine("-1-" & vbCrLf & result1 & vbCrLf & "-1-")
                Console.WriteLine(vbCrLf & "result 1 is " & result1.Length & " characters long..")
            End If
            client.Disconnect()
        Catch ex As Exception
            Console.WriteLine("EXCEPTION: " & ex.ToString)
        End Try
    End Using
End Sub

That sub calls this routine... At the time of this test, I didn't know if I'd be calling the following once or multiple times... Come to find out that with this approach, you need to stack the commands like I did above:

Dim result1 As String = DoCommand(client, "enable" & vbLf & "enablePass" & vbLf & "term pager 0" & vbLf & "sh run" & vbLf)

This is the DoCommand function:

Function DoCommand(ByRef client As SshClient, TheCommand As String) As String
    Dim tsAsyncTimeout As TimeSpan = TimeSpan.FromSeconds(8)
    Dim swAsyncTimeout As New Stopwatch
    Dim sbResult As New StringBuilder
    Try
        Dim cmd As SshCommand = client.CreateCommand(TheCommand)
        Dim asynch As System.IAsyncResult = cmd.BeginExecute()
        Using reader As StreamReader = New StreamReader(cmd.OutputStream)
            Dim res As String = ""
            swAsyncTimeout.Start()
            Do While Not asynch.IsCompleted
                res = reader.ReadToEnd()
                If String.IsNullOrEmpty(res) Then
                    If swAsyncTimeout.Elapsed.TotalSeconds >= tsAsyncTimeout.TotalSeconds Then
                        Console.WriteLine(String.Format("Cancelling async execution.. idle for {0} seconds [{1}]...", swAsyncTimeout.Elapsed.TotalSeconds.ToString("0.00"), tsAsyncTimeout.ToString))
                        cmd.CancelAsync() ' turns asynch.IsCompleted to True; same as doing:
                        ' Exit Do
                    End If
                Continue Do
                Else
                    sbResult.Append(res)
                    swAsyncTimeout.Restart() ' every time we get something, we re-start the timer...
                End If
            Loop
        End Using
        ' cmd.EndExecute(asynch)
    Catch ex As Exception
        Console.WriteLine("EXCEPTION: " & ex.ToString)
    End Try
    Return sbResult.ToString
End Function

I could have passed the timeout to the Function, but I'm lazy & it's just a test... The line where you define the timeout is here:

Dim tsAsyncTimeout As TimeSpan = TimeSpan.FromSeconds(8)

Since I couldn't get the async loop to terminate properly (asynch.IsCompleted was never true (unless I manually set it with cmd.CancelAsync(), so after all text returned, the loop was just run even though it clearly was done with the command. Probably not a bug, just my ignorance.

So I used:

If String.IsNullOrEmpty(res) Then

...to tell me that output has stopped/paused. If this was not the case, I would ReStart() the stopwatch (timer), so any time the command returned even one line, I would reset the timeout stopwatch (swAsyncTimeout)... With a lot of output, it would be silly to try to estimate how long it would take to download it all and then try to create a timeout based on the largest file. I think this approach is a little more robust, and easily adjustable in code at runtime. I'm interested to see how you guys are handling timeouts, though, as I'm not relying on anything internal.

I tried to use cmd.Timeout() but it was ignored in the loop of the example in the help chm. Also, I found the ExtendedOutputStream() hung on ReadToEnd(), where using OutputStream() got what it could & continued (which is desired).

OK. So on to the output. The program ran, getting the output from the router for about 1 minute, then spit it out to console (I wanted the test/proof to be easy for you to run, so I didn't use any fancy loggers):

In the code, you can see that the last line of output asks the program how many characters were returned:

[...]
result 1 is 2887590 characters long..
End...

Before this exercise, I was left with just the first 1/2 of whatever the buffer in my ShellStream was set to, the rest was tossed/truncated. So, I'd say this is a pretty darned good proof! I could show you my 'before' code, but you wouldn't be impressed.

I will tell you this, though. This has caused me to completely re-work my programs using SSH. Unless I'm wrong, here is the difference:

• If you use ShellStream(), and define a terminal, you can interact with the shell, command by command, and then shut the shell down when done. This is the way I'm doing things today.
• If you do it the way earlier in this post (via IAsyncResult()), though, like this:

Dim cmd As SshCommand = client.CreateCommand(TheCommand)
Dim asynch As System.IAsyncResult = cmd.BeginExecute()
Using reader As StreamReader = New StreamReader(cmd.OutputStream)
    [...]

...you have to line up all your commands, and fire them all off for processing.

Which one to use?

As usual, "It Depends", is what I found out so far.

• If you want to enter commands based on the output of the previous command, and they are short, brief back-n-forths, then ShellStream() is the way to go.
• If, on the other hand, you need to do everything in one session (lifetime: Client.Connect() to Client.Disconnect()), and you expect a BLAST of output, then you can use the method shown here, as it works (for me).
  

-> All I want to ask the community for are ways to make this better, or your examples (C# is fine, I use both) of how you are doing it.

Sound fair? :)

Thanks!

pat
:)

P.S. Thanks to the authors & developers for keeping the community happy, but more importantly engaged... :)