Description
However, analysts with a programming background have not adopted such tools as widely as their non-programmer colleagues have. The desire to avoid shuffling between tools and work in a single environment, as well as a need for the full, unbounded expressivity of programming-based analysis tools, are a few major reasons. This does not mean that programmers are immune to the specification burden; the expressivity of programming comes at the cost of complexity and steep learning curves. Novices have to spend much time learning these tools from books or fragmented resources online. Even experts report a loss in productivity from having to constantly look up documentation to get uninteresting details such as function names and argument values right.
Thus, there is a need for programming assistants that reconcile the need to reduce the specification burden for programmer analysts with their desire to work with code in their preferred development environments. These assistants should help programmer-analysts write code more efficiently by automatically generating human-readable and readily-integrable code from high-level specifications.
This dissertation introduces techniques and corresponding prototypical assistants that accept input-output examples, demonstrations, or natural language specifications and automatically generate suitable data processing and visualization code utilizing popular data science libraries such as pandas, matplotlib, seaborn, and scikit-learn. Automatic code generation has long faced the tradeoff barrier between expressivity and performance/accuracy: supporting a large number of analysis tasks makes the problem of generating the right code quickly that much more difficult. Accordingly, prior research in program synthesis and semantic parsing has largely sacrificed full expressivity to support efficient code generation for a small but useful subset of tasks. The code-as-text approach of modern natural language processing systems, including the use of large language models, promises unbounded expressivity, but their sub-optimal accuracy remains a concern.
This dissertation tries to push the boundaries in terms of breaking this tradeoff barrier --- can we build programming systems that are fully expressive while remaining fast and accurate? Specifically, this dissertation builds upon prior work and introduces novel code generation techniques that combine insights from synthesis, automated testing, program analysis, and machine learning. It contributes four core techniques and corresponding assistants, namely AUTOPANDAS, GAUSS, VIZSMITH, and DATANA. AUTOPANDAS and GAUSS constitute core advances in search space and algorithm design for example-based synthesis. VIZSMITH and DATANA introduce novel mining and auto-summarization techniques to automatically build aligned code and natural language corpora, which DATANA uses to greatly improve the code-generation capabilities of modern large language models. Compared to prior work, these assistants improve the expressivity of synthesis-based systems and the accuracy of machine-learning-based systems.