One of my bikes had a small gouge in the fork tube, filled it in with JB Weld. Never leaked.
I've also heard of nail polish as a filler, your choice of color is optional, 

It really depends how bad the rust is and where it is.
My RT3 forks had rust minor rust where the sliders travel which caused the forks to leak badly.
I ended up dismantling the forks and getting the tubes hard chromed by a company (here in Australia) that specialises in it.
They came out better than new.

The following is displayed on Rece Tech's web site and is offered a through their headquarters location in Corona, CA. While I have not used this service I would expect it to be top quality as with all their products and services. 

 

Pretty good video on repairing chrome tubes:  

Bad-azz, man!

I would guess that, as with painting, surface prep and cleaning is everything.

Thanks for the good ideas. I will try a little scraping with a blade, then some 0000 steel wool and WD-40.
I certainly appreciate all of the comments. I will take some before and afters when I get a moment to work on it.
 

Ummm...  I may be wrong but I don't think you want to scrape with a blade too much... 

Dragging a blade across the surface is a technique used to identify any burrs and imperfections.   Bad ones will catch the blade...  on more minor ones you can hear the sound change.