The way he starts the scene so intently looking at her with not admiration but almost gloating, in a sense, because he’s told her that he missed her and her letters, so of course Penelope being Penelope she’ll just faun and look bashful and maybe mumble a line out to him, but it shifts to a 180 when she starts saying she’s the laughing stock of the ton and that she knows he’d never court her no matter what she ‘changes’ about herself to fit the ton’s needs and wants. You can see he avoids her gaze and looks anywhere but directly into her eyes, because she’s no longer praising him and awarding his own thoughts by amusing him. I think here he realizes that what she’s saying is true and he’s embarrassed/maybe even a little ashamed to admit that he’s also thought that or thought something similar. But I think he also realizes, once she says he’s cruel and just like the rest of the ton, that he’s taken Penelope for granted. She’s always indulged in his adventures and whims and travels, all of his family troubles, all of his internal-self issues, etc. and that she’s always been nothing but “kind” to him, when in reality she wasn’t revealing everything she probably wanted to say or felt at the time, because how could she ever hurt Colin or do something to take the good-natured smile and easy going wiles away from him? His Penelope would never do that! But she’s becoming her own person for once and entering into her own agency, and he’s not sure how to maneuver through it, let alone with her through it. So I feel as though he looks at her with a sense of shame and regret as she walks away- so much he wants to deny and refute but not being able to because he believes some of it to be true as she believes as well.