The characters of Roman and Tej were introduced in the second Fast and Furious movie and then vanished until Fast Five.
With the third Fast and Furious movie not really being directly connected to the previous two (until after the fact), a spinoff could have worked around the same time period.
On the one hand, a spinoff that took place between those two movies could have made some sense.
If the spinoff was offered post-Fast Five, the desire to turn it down might make more sense, though one wonders if the opportunity for such a film may have passed by then.