Deep neural networks (DNNs) are being increasingly applied in mission-critical situations, where high accuracy and robustness are essential. While deploying DNNs for computer vision and image processing tasks, the focus of adversarial analysis has been mainly on finding pixel-level changes that may impact the network’s robustness. We examine an alternate approach - generating adversarial images by inducing perturbations in the semantics of an image, yielding interesting scenes that are more likely to occur in the real world. One can then use the generated examples to improve the robustness of DNNs through using them in data augmentation and adversarial retraining. In this report, we provide and implement a pipeline that can generate such scenes on demand by combining a differentiable rendering framework with gradient-based attacks. We demonstrate that the semantic adversarial examples generated by the pipeline can fool an object classification or detection framework, that retraining on these counterexamples is effective in making the network more robust to such attacks, and that the semantic robustness achieved against one attack appears to help achieve semantic robustness against other gradient-based attacks.




Download Full History