In terms of the 5 6-siders, there are 6^5 possible combinations. That's 7776.
There are a number of ways to determine the number of favorable combinations. The minumum sum is 5 and the maximum is 30, of course. Since dice like this are normally distributed, that puts the average at 17.5 (vs. 16.5 for 3d10).
One way to solve this kind of problem is to determine the standard deviation (sigma) of the normal curve for both cases, and figure out how many sigmas away from the center 18 is. There are numerous places that can help you with this kind of a method, just google for "standard deviation dice". One particularly interesting page is:
http://www.rpg.net/news+reviews/columns ... sep05.html
If you'd rather calculate the answer exactly, the you're correct that you need to figure out the number of favorable results. This can be quite complicated in general, though I'm sure there's a solution out there to be found. I'm not sure what the best way to do it is for 5d6.
Let's see... how about inductively...
When the first die is a 1, the remaining 4 dice need to sum to 17. Similarly when the first 2 dice are 1, the remaining 3 need to sum to 16. 3d6 is a tractable number of combinations to figure out by hand...
If you made a table of the possible values of the first 2 dice, it would only have 36 rows, with corresponding required values for the remaining 3d6 (and the number of favorable outcomes for those). If you summed these favorable outcomes for the "leftover" 3d6, you'd have the number of favorable outcomes for 5d6.
This wouldn't of course, be practical for larger numbers of dice, though if you think about it some more, you may be able to see how to reduce this to an equation... Let us know if that isn't a big enough hint...