Multiprocessing Niceness in Python
Quick and dirty one that tripped me up. Recently I’ve been doing lots of multiprocessing and joblib-based parallel processing, with loooong simulation times. In an effort to make sure that my machine was still useable during these runs, I changed the ’niceness’ value of the spawned processes… or so I thought. import os ... def thread_mask(args): # Properly Parallel RNG #http://stackoverflow.com/questions/444591/convert-a-string-of-bytes-into-an-int-python myid=current_process()._identity[0] np.random.seed(myid^struct.unpack("<L",os.urandom(4))[0]) os.nice(5) return long_simulation(args) First part is a handy way to make sure that your subprocess simulations actually use different random numbers…. which for Monte Carlo style simulation is pretty damned important… ...