Smartphones contain a large amount of highly personal data, much of it accessible to third-party applications. Much of this information is safeguarded by a permission model, which regulates access to this information. This work primarily focuses on improving the Android permission model, which is known to have notoriously large amounts of sensitive data leakage, but many of its findings can be analogously applied to other mobile operating systems. We evaluate the two currently employed Android permission models: ask at install-time and ask-on-first-use to determine if they fulfill user expectations of privacy, and find that this is not the case for either model. We analyze the different facets that comprise user expectations and recommend a better mechanism to satisfy these expectations without excess effort from the user. This mechanism incorporates the contextual nature of privacy into the permission-granting process through the use of a machine learning classifier. We contribute the most extensive instrumentation of the Android operating system targeting user behavior and related runtime states to our knowledge, spanning across nearly 40 classes in the Android platform. This instrumentation allows us to utilize user behavior and system-level features to determine context for permission requests. The data from this instrumentation is used to generate features for the classifier. We evaluate the classifier on a large labeled dataset we collect from over 200 users using our modified operating system, and recommend ways to employ such a system in the real world based on our analysis.