How AI can help programmers write better code ?
Waren Long, source{d} .
How AI can help programmers write
AI & Society - March 4, 2019
Waren Long
source{d}
About source{d}
35 software engineers
Offices in Madrid and San Francisco but 100% remote
Large scale code analysis and 2 main directions :
Code as Data to turn code into actionable insightsML on Code with assisted code review
sourced.tech
Plan
MLonCode, Origins & Motivation
Lookout, a Framework for Assisted Code Review
style-analyzer, an Adaptative Code Formatter
Software Development Workflow
5h/week, 13% of coding time, generally done by senior developers
72% of developers work with blocking code reviews
45% of developers lack time to review code
Stricter code reviews lead to more time delivering new features instead of fixing bugs.
codacy.com/blog
At Google
25,000 developers
20,000 code reviews per workday
4h of average latency for the entire review process
15% of comments indicate a possible defect
Modern Code Review: A Case Study at Google
PR Review Comments on GitHub by Topic
dataset extracted from gharchive.org , January 2015
The Machine Learning Perspective
The Alternative Hypothesis
"Programming languages are inherently harder to write and read... so programmers deliberately write code as unsurprising as possible."
"Code (in all languages) is more predicatble than natural language because it more technical and difficult to learn."
On the Naturalness of Software
The Machine Learning Perspective
Software is bimodal
"Source code is bimodal: it combines a formal algorithmic channel and a natural language channel of identifiers and comments. Because the two channels interact, [...] bimodality is a natural fit for machine learning."
RefiNym: Using Names to Refine Types
When to help ?
While you type = IDE
While you check = CI
While you review = PR
Periodically, asynchronously
Pros and Cons :
Part of the workflow
More time to run the models
Nice UI
High precision score required
Longer feedback loop
Goals
Assisted code review platform
Tight git/GitHub integration
Implementation language agnostic
Batteries included
Example of Lookout Comment on GitHub
Architecture
Push event
Review event
Approach
Parse to intermediate representation
Train Decision Tree Forest
Extract production rules
Representations of Source Code
Token-level models
Syntactic models
Babelfish, a Universal Code Parser
Aims to parse any file in any language
Extracts ASTs and converts them to uniform structures
Standard node properties
Client/server system that supports XPath queries
4 traversal orders
doc.bblf.sh/architecture
Classes Predicted by style-analyzer
␣ whitespace
→ tabulation
↲ newline
␣+/- whitespace indentation increase/decrease
→+/- tabulation indentation increase/decrease
'/" single/double quotes
∅ empty gaps between non-label nodes, NOOP
Mixed representation
AST-augmented token stream
a = b * 2Feature extraction :
Window size of 10 tokens
2 levels up in the AST hierarchy
Annotated Code Snippet with Style Mistakes
Explainability is key
Build trust with the users
Prefer interpretable output with human-readble decisions
Decision Tree Forest models
Optimization to limit the number and length of the rules
Generating Production Rules From Decision Trees
Rules
a≤5 Λ b≤1 Λ c ⇒ αa≤5 Λ 1<b<4 ⇒ β5<a<10 Λ c ⇒ γa>5 Λ c Λ b>2 ⇒ α
Machine Learning
Feature selection (univariate, ANOVA F-criterion)
Hyperparameter optimization (Bayesian)
Evaluation : 80% + 20% split
Evaluation
Evaluation improvements
Extend classes
Test the real behaviour of users
Random mutations
Extract from commits
