If you love someone, is there any legitimate reason NOT to say it?

Some people might not really know what love is until they figure it out themselves. It might be that he might not feel as obligated to you, but with the information you gave it doesn't seem to be the case. I would recommend talking about it, but make sure he knows that you're just curious.