Going to full frame is frightening the Shiites out of me!!

I'm having to look at ditching my fav lens (EF-s 10-22mm) in my bag for "hopefully" an upgrade to a "L" series equivalent for the full frame 5d mII.

Has anyone gone through the similar process?
What lens did you end up settling on? the 16-35 or the 17-40?

My biggest fear is that if I choose the cheaper 17-40mm that the optics will be less than the 10-22mm. To avoid this I'm leading towards the "twice the price" 16-35L because of this reason.

With the 17-40, I wouldn't miss the 1mm difference at the wide end, nor the faster f stop compared to the 16-35 but I'm not willing to compromise any image quality compared to my beloved 10-22mm ef-s (which is similar in price to the 17-40mm L)

is the $800 difference really noticeable in the image quality when shooting above f/4 on between these lenses?

Another pro for the 17-40mm is there will be no need to get new filters, as I can't imagine how expensive a ND Grad filter set or slim UV would be for the 16-35mm let alone a 400 ND

Anyone had any real world experience for the dilemma I'm going through?

Your thoughts on choice?

Cheers

Tim