Sunday, December 10, 2017

Scheduling machine learning jobs to run in sequence

This might save you a few minutes of research time: I sometimes need to set up a number of Keras (or TensorFlow) runs to occur in sequence to run overnight, while I am away from work, etc. I don't want the processing to stop if any single job fails.

I use Task Spooler that is in Ubuntu and other Linux distros and can be installed on MacOS using "brew install task-spooler". Note, on Ubuntu, the command is tsp


You can schedule any shell command command to run by prepending "ts". Examples:

cd run1
ts python variational_auto.py
cd ../run2
ts python lstm_text_model.py
ts # get a list of all queued, running, and finished processes
ts -c 3 # get the stdout for process 3
ts -t 3 # get the tail output for process 3
ts -C # clear the list of finished jobs

This simple setup is not really appropriate for doing hyper parameter tuning but is useful to set up a series of runs.

Sunday, December 03, 2017

Forget the whole world and focus on your world

Some people dream of “making it big,” dreaming of starting the next Facebook or Google. Poor people fantasize about becoming very wealthy. I think this is misplaced focus.

I prefer to live in and think about a much smaller world:

  • Being of service to family and friends and enjoying their company 
  • Being of value to coworkers and customers
  • Providing value and entertainment to people who read my books
  • Getting a lot of exercise and eating great food, in small portions
  • Enjoy reading great books and watching great movies

Yes, that is enough for me. I leave changing the world for other people.