“Vision for a Secure Elixir Ecosystem: An Empirical Study of Vulnerabilities in Elixir Programs” was published by the ACM in April, 2022. From the paper:
Practitioners perceive Elixir to be a ‘safe’ language, as the language allows practitioners to write fast software programs without introducing vulnerabilities, unlike other languages, such as C[16]
The ACM is a reputable journal, and this paper may come up in many contexts, for example if a business is considering using Elixir. I do not believe the paper gives an accurate picture of Elixir from a security POV, have published my reply:
Elixir’s reputation is important, and I hope this article is useful in showing why the claims in the paper are misleading.
What the hell is that paper about, the quality of the content is worse than my laboratory work in the first year of the university, just repeating 100 times that we extracted commits from some random projects and found X vulnerabilities, without even showing a single example of a vulnerability.
With their methodology, I could hire 2 kids from school and search for words in commits with the same result:
And by the way, when vulnerabilities are fixed, they are not marked as CRITICAL VULNERABILITY FIXED, HERE IS AN EXAMPLE if you want to replicate it, as vulnerabilities are things that can happen in any language/library.
Having quickly read the article cited above, my opinion is the following.
The research question is narrow and does appear to be part of a broader, established research field. It is unclear which conclusions for Elixir, the field of computer science or for security research the answering of the research question yields.
The paper does not seem to follow established methodology. What are the established metrics for measuring code quality and security issues ? What prior research exist in that field ? No papers on established methodology are quoted. Normally, a paper does either apply established methodology to a new dataset, or it applies new methodology to an already studied dataset. This study does not explain, which one it does.
The whole paper is short. It does not explore the topic in its depth nor in its breadth. Additionally, the few pages written are bloated with unnecessary information. This paper is not research, this is undergraduate homework.
How can the authors address these issues? I see two ways. Either (i) broaden the scope and explore the security aspect of Elixir in a comparative study with other languages. Does Elixir code show statistically different levels of security issues than other code ? Are there differences in the types of security issues ? Do code bases with high levels of security issues have further differences compared to code bases with lower levels ? Are these differences also observable for Elixir code ? Or (ii) deepen the research and explore which security problems can be identified for the specific case of Elixir code. What type of issues can be identified ? How extensive are they ? What can these issue say about the language and tools used ? What suggestion for improved code can be derived ?
Relax, some people write papers because they are mandatory to finish the university to get a grade. Whole academia is more about getting a degree, not about science or anything really useful.
I don’t think that these students are wrong. In my opinion, they are 100% correct: nobody cares about academic papers or their quality, young people just need a degree to have more opportunities in the future. I think that there should be more papers like this. This would destroy authority of universities in terms of defining competence
