My boyfriend said he feels for me with his more than anything else :( ?

We were talking and I said I loved him, and he said I love you too, but I feel for you more here (and then groped his crutch) then I said "More than anything else?" and he said "Well, yeah". I'm starting to think that he doesn't find me physically attractive one day, he'll just leave. What do you think?