Hulk and Falcon believe that Steve is only going to be leaving for a moment, returning the Infinity Stones to their proper places and then coming back… but when you rewatch the scene, Bucky seems to know something that nobody else does.
One important argument not taken into consideration in what’s been covered above is the interaction that is featured between Captain America and Bucky right before the former goes on his time travel adventure.
In addition to the two men sharing the same exchange they had before Bucky went off to war, the man formerly known as the Winter Soldier tells Steve that he is going to miss him.