You don’t have to ship Destiel, but you do have to admit, Dean and Cas’ relationship… it’s different.
The writers, the producers, the directors… they all had a part of this. They easily could have portrayed Dean and Cas’ relationship differently than what they have. They easily could have had Cas have the same exact relationship with Sam as he does with Dean. But he doesn’t. Dean and Cas share a more profound bond. Enough so that an angel, fresh to human emotions and concepts, was able to grasp the fact that his relationship with the older Winchester was different than his relationship with the younger one. It’s always Dean seen alone with Cas. Think about the times you’ve seen the two alone versus the times you’ve seen Cas alone with Sam. And the people working on the show chose to do this. Dean and Cas are the ones they focus on, and they’ve focused on it time and time again. They share their doubts and fears and failures and hopes. They had Cas always come back to DEAN, and vice versa. Dean was the one that argued with Sam and Bobby when they thought Cas was working with Crowley. Dean was the one that fought tooth and nail defending Cas, because there’s no way he could ever believe that Cas could betray him. Dean is the one that held on to Cas’ trench coat after he died, just hoping, praying that his angel would come back to him. Dean spent a year in Purgatory searching for Cas, when he easily could have left with Benny, only to find that the reason Cas fled was to protect HIM. Dean was the one that admitted that the WInchesters needed Cas, only to then emphasize that he specifically needed Cas. Cas was able to break through mind control for DEAN. Even though he had practiced killing duplicates of Dean a million times over, when it came down to the real Dean, he couldn’t do it. Cas chose Dean over an entire army. Cas chose Dean over heaven. Cas chose Dean over everything else, so often that he was even called out for it. Cas admitted that he loved all of the Winchesters, but whose facial expression did the camera focus on? Dean’s. Just like when Cas came back in 13x05, it’s DEAN’S reaction the camera focuses on the moment they make eye contact.
Things are just DIFFERENT between Dean and Cas, even Sam knows it. Sam asks DEAN what they’re going to do with Cas’ body, because he feels like he doesn’t have a say so. He knows that his brother’s relationship with Cas is completely different that his. Sam, without a doubt, loves Cas too, but he knew it was different. It’s the reason why Cas always comes when Dean calls. It’s the reason why Cas threw away everything for the older brother. It’s the reason that Dean throws up Cas whenever Sam can’t focus on losing anyone but their mom. It’s the reason why Dean can’t give Jack a chance.
So if Sam Winchester, the person that knows Dean better than anyone, sees that there’s something DIFFERENT between his brother and Cas, why wouldn’t we?