I recall reading an anecdote (it was definitely in Civil War: a History by Shelby Foote, but I don't remember exactly which volume; I believe that it occurred during the Vicksburg campaign in the third volume, but I might be misremembering) about a Navy officer who had gotten his boat sunk as a result of an encounter with an underwater mine (then called a torpedo). He made it back safely to shore and, believing that he would likely be punished, nervously asked the commanding Admiral for an inquest. He responded something along the lines of "Nonsense! I don't punish people for getting their ship close to the enemy!" and gave him a new boat. However, I've unfortunately been unable to locate the quote. Can someone help me to identify the two officers involved in this exchange? (I would be interested in locating further biographical details of the officers in question as well, particularly the Admiral, but that's perhaps a different question).
Though I cannot find the quote as remembered by the OP, researching the conditions of the event leads me to believe this was in reference to the sinking of the USS Cairo by an electrically detonated 'torpedo' (see sketch below). Her captain, a Lt. Commander Selfridge, was on several vessels which were sunk in one way or another. After the Cairo was sunk his commander, Admiral Porter said (emphasis mine):
“My own opinion is that due caution was not observed.” The admiral, however, apparently impressed with Selfridge’s aggressiveness, later withdrew his criticism. “I can see in it nothing more than one of the accidents of war arising from a zealous disposition on the part of the commanding officer to perform his duty.”
Which seems to fit the meaning of the quote, if not the words. Selfridge was then placed in command of the USS Conestoga, matching the conditions of the OPs quote again. This would make the admiral in question David Dixon Porter.
You can follow up on Porter in his Incidents and Anecdotes of the Civil War.
The Civil War : a narrative. [Vol.] 3, Red River to Appomattox, by Shelby Foote can be borrowed at archive.org. I did not, however, find the quote within that text.
Another source, Official Records of the Union and Confederate Navies in the War of the Rebellion United States. Navy Department, contains the actual orders and communications concerning this incident. It contains an interesting sketch of what these 'torpedoes' (or infernal machine as mentioned in a couple of letters) actually looked like (from pg 549):
These were definitely much more mine-like then the spar torpedoes mentioned in comments above.
The same book has the report Selfridge himself filed (pgs. 548-550), and the contrite attitude from the OPs quote seems to be apparent towards the end of his report (pg. 550):
Having accomplishd all that was in our power and destroyed what vestige of the unfortunate Cairo that remained above water it was with deep regret and melancholy that I felt obliged to return down the river I have nothing to add in justification of myself that does not appear in this report
Note that these are official reports and orders, so the conversational quote remembered by the OP would not occur in this books context.