Are you entering the frame number set in Time Manager into the Frame Window under Animation Options in the advanced settings (gear icon enabled)?
Time Manager
Frame Window
Animation Options
I tested it and did not find any particular issues.