I will never understand this about myself...

Today, Joe calls me and tells me all about the great time he had visiting his brother in Destin, FL. He goes on and on about the bar and the restaurant and the fact that he is going to a Braves game next weekend. Shouldn't I be happy that he is not lonely and is having a great time? I instead found myself getting a little angry about it. I felt so sad and lonely while he was marveling at the dolphins that he saw while at the beach.
I feel like I have always felt like this when Joe does something awesome without me. I know he misses me and wishes I was with him, so why do I get angry? While he lived in GA and I in NY the 2 years before we got married, I found myself reacting the same way when he would tell me about weekend getaways with his friends and parties/get-togethers I was missing out on. I really think it is my way of just missing him. It must be the way it comes out, I get upset. I wonder if anyone else has had similar feelings.
I don't like feeling like that nor do I like making Joe feel horrible about having a good time. He should have a good time. We had a good talk about this today. This is why I love him so much, he loves me even for my flaws. He made me explain to him what I was feeling and we both realized I must be feeling left out because this was the summer I pictured doing "summery" things as a family....just him, me and Adrianna. I wanted to take Adrianna to the beach for the first time with him, explore Rhode Island, Maine and Cape Cod now that she could handle the heat better, and now we can't because he is gone basically the whole summer. For the last 6 months, we have not taken much time to enjoy being together, as a family without anyone else or doing something for anyone else. We have been busy doing things with our families and always talked about having the summer all to ourselves. Once we realized that this was the problem we decided we needed to get away just the three of us. He called me a little while ago and said that on our drive back up from Alabama (Adrianna and I are flying to Alabama to join him on his drive back up to MA, making a pit stop on our old stomping grounds Warner Robins, GA) we are going to spend 2 days in Myrtle Beach just the three of us....exploring, going to the beach and enjoying the last bits of summer together. I love that he understood just what I needed and made the effort to really listen to my feelings. I do have to say he has come a long way from our early dating days and has learned to actively listen and pay attention to the details. He is not perfect, but I don't want perfect I want perfect for me....and that is what he is. Adrianna and I miss him so much....only 18 more days until we can have our "first kiss" moment.