Would like an opinion on child custody.....?

I have physical custody of my son. I live a long way away from the other parent. My child wanted to try living with the other parent for awhile, so I let him go although he was still quite young. Because I did NOT want to keep my child from the other parent and wanted to encourage the relationship, I loved him enough to let him go. It hurt like crazy, but I adjusted to life. However, while there, the other parent never "stepped up to the plate" although they had good intentions of doing so. My child ended up with grandparents all week long for 5 months. The other parent would get him on the weekends (probably) 2 or 3 times per month and would visit him at the grandparents house a couple of times per week. There were times that my child didn't even see his other parent because of a bad work schedule even on the weekends. I am not sure that my son's other parent got him up for school - even once in a 5 month period. All contact has been through the grandparents. I have not any contact with the other parent nor has he updated me on my son's school progress, illnesses, happy moments or sad moments. Now my son is visiting me for the summer and I have not heard ANYTHING from the other parent until today when he stated what time he will be flying in to pick up our son and take him back "home" . I am able to provide full-time parenting to him and will be able to do so even when school starts again. If he goes back to live with his other parent it will be the grandparents doing the "parenting". Although my son likes this arrangement, it is not in his best interest to do so for lots of reasons. I have not discussed with my son that I am thinking about not allowing him to return. He is still quite young (10 years old) and does not know what is best for him. But if he totally contests to living here, there is a part of me that just says, "go" --- rather than deal with the heartache it will cause if I "force" him to stay. There's no abuse or neglect or anything else going on at either house and he is well cared for in both places. But I just wondered what other parents would do in the same situation. He will be "safe" in both places (here or there) but it's just the other parent isn't as available as I am and just doesn't seem overly interested in doing what it takes to be available for our child. Emotionally, I feel our son would be better off just staying here. What would you do in a similar situation --- especially if both places were safe places without neglect or other issues?