Granted, future books tells very different stories than what happens in Drums of Autumn, and each book takes Jamie and Claire in some very different directions, so there are undoubtedly limits to how much seasons of Outlander can borrow from future books.
Still, we may no longer be able to count on seasons corresponding with the books as numbered, and that’s very Game of Thrones.
Season 4 will be based on the fourth book in the Outlander series, called Drums of Autumn.
Future seasons of Outlander may take liberties with adapting the source material in the order it was written, which is Game of Thrones fans who have also read the A Song of Ice and Fire books know well.