Yall tell me if I yall think I'm reaching but I actually think the track is about his relationship with Rap/Hip-Hop and Fame and not a woman.
If you sit with the lyrics I think you will be able to make the connections."Being together" is being the number 1 guy. The "kids" are his peers, more than likely Drake and K.Dot.
I think he is reflecting, reminiscing and finally coming to terms or being more comfortable about his status in the game. He is no longer caught up w wanting that title.
I dont think its far fetched at all since Cole has done this a few times in the past... most recently the "TKO remix" and "May the Bitter Man win" both of those verses you would think are about a girl but they are not.