Skip to content

Commit

Permalink
Update use_cases.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ksrinivs64 authored Oct 17, 2023
1 parent c65bcb0 commit e507332
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/use_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ We also use this analysis for feature engineering from code. Within an enterpri

**SEMFORMS Paper**: [https://arxiv.org/abs/2111.00083](https://www.ijcai.org/proceedings/2023/0827.pdf)

**DataRinse Paper**: [https://dl.acm.org/doi/10.14778/3611540.3611628]
**DataRinse Paper**: [https://dl.acm.org/doi/10.14778/3611540.3611628](https://dl.acm.org/doi/10.14778/3611540.3611628)

### Buildng Better Language Models for Code Understanding<a name="lm"></a>
Code understanding is an increasingly important application of Artificial Intelligence. A fundamental aspect of understanding code is understanding text about code, e.g., documentation and forum discussions. Pre-trained language models (e.g., BERT) are a popular approach for various NLP tasks, and there are now a variety of benchmarks, such as GLUE, to help improve the development of such models for natural language understanding. However, little is known about how well such models work on textual artifacts about code, and we are unaware of any systematic set of downstream tasks for such an evaluation. In this paper, we derive a set of benchmarks (BLANCA - Benchmarks for LANguage models on Coding Artifacts) that assess code understanding based on tasks such as predicting the best answer to a question in a forum post, finding related forum posts, or predicting classes related in a hierarchy from class documentation. We evaluate the performance of current state-of-the-art language models on these tasks and show that there is a significant improvement on each task from fine tuning. We also show that multi-task training over BLANCA tasks helps build better language models for code understanding.
Expand Down

0 comments on commit e507332

Please sign in to comment.