Move some more time related stuff to controller_time
Add code to handle skipped samples.