We analyzed the GitHub Issues from the most popular Front-End frameworks, here’s what we found
Using unsupervised AI to discover insights
Disclamer: This article is based on intents discovered by an unsupervised algorithm. There was no human supervision that biased the results. However, the takeaways may be biased for the sake of sharing a practical (and fun) view.
As developers, we are used to join programming communities (like GitHub) to contribute or just to ask for help. We were wondering what developers were asking for, and what problems they found when using one of the most popular Front-End frameworks.
To answer these questions, we decided to analyze issue titles from GitHub with our unsupervised AI technology for language understanding. The aim was to discover the patterns and most relevant concepts being discussed in these active Open Source projects.
We uploaded 4 datasets into our Lang.ai console, one for each framework (Angular, React and Vue) and another one combining the 3 of them.
The whole process took less than one hour to run, and the outcome was a visualization with a list of intents, as well as the semantic clustering between intents to further understand how the data was related.
An intent is a set of words with a semantic meaning that represents, in this case, a group of issue titles
The results from the analysis is the following:
Now let’s go deeper and discuss over the main takeaways for each dataset.
Angular, React and Vue
Takeaways:
1. You should check for null values
The first intent we find is Uncaught TypeError, which is a well-known issue by the JavaScript community. We can see that most of the documents are talking about the same kind of errors, like: Uncaught TypeError: Cannot read property 'name' of null. Maybe Typescript is not such a bad idea after all.
2. Handling inputs is more difficult than expected
The pattern of asking how to correctly handle forms is something that we will also see repeated in all of these frameworks. Issues related with validation, events firing when they shouldn’t (or should), are on the list. In fact, each one of the frameworks have their own guide on how to work with forms: Angular, React, and Vue.
3. People definitely ask for features
A quick search for feature request on the Angular repo reveals that almost 5k from their total of 15k issues are relevant for that query. This is also present in React (~20%) but not so much in Vue (~10%).
4. No one said that importing modules was easy
There are hundreds of issues out there that begin in the same way: Cannot find module. You know things may go wrong when you are not able to find the https module.
5. Things don’t work as expected when changing the NODE_ENV
It’s kind of the Works on my machine for JavaScript. We have seen issues related with the development and production mode. There is also another bunch that mention the strict mode, which is not related with the Node environment, but is semantically close.
Takeaways:
1. Documentation is mentioned a lot
“AIO” is the name used to refer to the Angular.io site, and they even have a board to handle those issues. Around 2.3k from their 15k issues are related.
2. Feature Requests are prominent
Another pattern, specially relevant for a repo that started from scratch after years of including features in their previous Angular 1 version.
3. Routing may be complicated
With the Child Route intent we are getting issues like “Angular 2 child routes lost data”. There are other questions like “How to update parent’s variable from child’s component?”. Routing will give you some headaches, even with that kind of cross-browser issues (spoiler alert: turns out it was the OP fault, not the browser).
4. The Change Detection mechanism still have issues
The Angular change detection mechanism is a key part from the framework and, although is more transparent and easier to reason about than its equivalent in Angular 1, it still seems to cause many issues.
Takeaways:
1. It doesn’t take one day to handle input fields in React
As we already saw in the big dataset, handling inputs or building forms is more complicated than expected. We can see that is one of the main headaches when developing a React app. Maybe people forget to think in React?.
2. It’s all about setState()
The concept seems simple: this is the primary method you use to update the user interface in response to event handlers and server responses. The documentation already warns about the pitfalls and anti-patterns, but it seems that a little portion of developers have read about it.
3. Server Rendering has a cost
As you can read in this blog post, server side rendering a React app can provide a few different benefits, like performance and SEO. The problem is with those benefits comes a cost of additional complexity to your application. Simple things like displaying an image may need some additional work.
4. We want support for more attributes
We all want more attributes. It may go from adding attributes to the SVG element or even for iframes. How many here still use iframes?.
Takeaways:
1. Again, handling forms is still an issue
Despite the clear and detailed documentation on the topic, it seems that Vue and React developers usually have a bad time when handling input elements. Maybe they should add this to their comparison with other frameworks.
2. Be careful with Computed Properties
Computed properties can be used to do quick calculations of properties that are displayed in the view. It seems as simple as React’s setState(), but it can also be tricky. As we can see in the features detected in our analysis, there are issues related with caching, setters and dependencies.
3. The <Select> element may be a trickier one
We already have the intent Input Value that is related with forms, but here we can see a semantic relevance towards the specific <Select> element. Sometimes this element doesn’t select what was supposed to select.
4. It’s all about passing props
Props come in so we can pass data and actually make our components useful. It looks like we are also having some issues here.
How we did this
Getting the data
The first step was getting the data we wanted to analyze. Fortunately, GitHub provides an API and the packages to seamlessly integrate it into any script. For this purpose, we used the Node.js module @octokit/rest.
It’s important to note that the GitHub API treats pull requests as issues. Since we were only looking for real issues, we needed to filter out the PRs once we got the results from the API. The last step was to get the titles and save them.
The final dataset was composed of 27.4k issue titles from the following three repositories: 15k from Angular (not the AngularJS one), 6.4k from Vue and 6k from React.
Analyzing the data
Once we had the dataset ready, we created four projects in Lang.ai: one with all the issues and one per repository. Creating each project was as simple as uploading the dataset file and wait for the intent extraction process to finish. In less than an hour, the datasets were ready to be reviewed.
The analysis discovered a total of 145 intents in the dataset with all the issues. For each dataset that contained issues from a single repository, it discovered 153 intents for Angular, 86 for React, and 78 for Vue.
Keep in mind that once a dataset is analyzed, a classifier is also generated. This way we can also use the Lang.ai API to tag new issues with the detected intents that are available in the classifier (it can even learn once we feed it with new data), opening a new set of possibilities.
Next steps
We could have also included the comments and the conversation within the issues. That would have been another interesting experiment, but it was out of the scope for this article.
We believe that introducing a natural language processing technology (NLP) can help developers deliver a better product. Use cases go from helping to build and improve chatbots, to content moderation, document analysis, or just helping people discover patterns in a more efficient way.
If you want to try out Lang.ai, you can request an invite.
Check the other articles in our Building Lang.ai publication. We write about Machine Learning, Software Development, and our Company Culture.
