The possibilities of machine learning and neural networks in particular are ever expanding. With increased opportunities to do good, however there are just as many opportunities to do harm and even in the case that good intentions are at the helm, evidence suggests that opportunities for good may eventually prove to be the opposite. The greatest threat to what machine learning is able to achieve and to us as humans, is machine learning that does not reflect the diversity of the users it is meant to serve. It is important that we are not so pre-occupied with advancing technology into the future that we have not taken the time to invest the energy into engineering the security measures this future requires. It is important to investigate now, as thoroughly as we investigate differing deep neural network architectures, the complex questions regarding the fact that humans and the society in which they operate is inherently biased and loaded with prejudice and that these traits find themselves in the machines we create (and increasingly allow to run our lives).