Replace the RPN with a machine learning network and train the model end to end. TODO, need to read up more on how to implement this.