Security Pitfalls in AI-Generated Code: What Happens When Developers Skip Review

Which is very, very important nowadays. Since we use a lot of AI code and also a lot of AI tools to generate code, I want to raise the topic about security pitfalls in AI-generated code, a topic which more and more developers will explore. We know that the models are getting better and better, but do our code is secure in the end? And what is happening if the developer don't review this code later on? So today's agenda will be a little introduction about me, who I am, what I'm doing, and also the promise of AI code. How AI code is good for us, how we can use it, why we can use it, and some dangerous assumptions about this code and AI help.

Later on, we'll continue with what changes when the developer itself not just uses the tool, but also reviews the code later on. And here I will share with you five pitfalls, which I believe are really important in the world of security, and we should be aware. Later on, I want to share with you some information for where you can learn about security, also how you can be up-to-date, and of course, if you have questions, you can ask later on. So let's start.

About me, shortly, I'm a front-end developer with 11 years of experience, using AI daily, different tools of AI, like a cursor, like code, and also VS Code, extensions for AI, like a Copilot, Kiro also, so all of the varieties. My role in my company is a principle. So for that reason, I have the opportunity to use many tools, but also to see what are fitting, bad, or good for us, like company standards. Also, I'm doing tech walks, and I'm the author of some books, and I'm a Docker captain, this is a title given by Docker, to the community participants, which are really good at what they're doing, and also, I'm a speaker.

So, talking about Docker guides, all of these guides was authored by me, so if you're interested, for example, how to dockernize ReactJS application in best way, you can go to official documentation and check. That being said, I want to share with you something very important, and that is release of my new book, and this is Docker for ReactJS Developers. This book is a missing piece, which I try to fill in our world of technology. Just because many resources in Internet, it's written in a way that back-end developers should explore and understand Docker. And when we talk about front-end perspective is missing. So, I try to bring the information and fill this gap to the community. You can check the Docker for ReactJS Developers. If you love to go to Docker or React, this book is for you. The rest of the books are just related to interviews in React world, and also I have a book for TypeScript. Definitely go to my website, and if you're interested, review, write to me, or just share opinion.

Now, back to the presentation itself. Today, I want really to mention something very important, and that is the promise of AI. What is promise of AI? So, most of developers today is using AI to write code. But do the code itself is enough good, and do the speed which AI is really good at it, it's promising to deliver a good quality of code. And also, I saw many people, including me also, that we are using AI, but we stopped thinking of the whole process. We stopped thinking how we should develop the application in right way. And also, I saw myself and many of developers, I'm teaching that this comfort zone of AI generated code and speed itself, it's not meaningful. And for that reason, I want to say few uncomfortable reality reasons and what is really happening when we use AI without thinking what will cause later on in our application. What we assume.

So, when the code runs without errors, the verification process, it's shifted in a way to AI. What I mean here. So, imagine we generate some feature. This feature is really big. And after AI change, let's say thousands of files, what we do usually is blindly trust AI that the code itself is very good just because the model is very good. But often that is not reality. And I saw it in firsthand how our applications was compromised, was hacked just because what is happening is that the developer doesn't do what is need to be done in the end. And that is to review the code itself. And the review process is very hard just because AI can change a lot of files and usually people just watching them and don't try to spot the mistakes or to use AI to fix them. So, what is happening in my team nowadays is that people, senior developers, are saying that this is fault of AI. The code is shit, but let's blame AI.

And AI is too. So, when you blame AI, you're just not saying nothing. You're the person using AI. So, the ownership of the code is yours. And it's your responsibility to verify the code itself.

Some dangerous assumption for AI code. First of all, it looks clean. So, nothing suspicious. But is this really true? The application compiles. So, technically, this should work. The application is fine. My test passed. So, everything is covered. And also, the sprint is ending. We will fix this later on.

So, all these assumptions, I did it by myself and also people do them a lot. But the hard reality to understand is that AI really produces code fast. It's appear really nice. It's structurally good. It's able to understand your context of the application. But it's blindly inherited bad practices like code which is not verified enough, code which exposes security vulnerabilities to the public, code which is using, let's say, some of the environment variables, hard-coded in the application.

But it's blindly inherited bad practices like code which is not verified enough, code which exposes security vulnerabilities to the public, code which is using, let's say, some of the environment variables, hard-coded in the application. Or AI, just because it's very fast of generating code, it's not meaning that we should trust it. And we saw many, many attacks raising nowadays, including the official React.js library. It was compromised. The server side of it, it was compromised just because nowadays we try to produce a lot of code, but in this fast process, we don't have validity check.

Do this is the right approach? Do this is secure code? And, again, the reality is no share responsibility with AI. If someone tried to hack you and succeed, they will blame developers, not the AI tool behind it. So, keep in mind that each code which is going to production, it's your code. You're assigned to it. In GitHub story and also commits, your name will be put on, not the AI tool, which generate this code.

So what is changed when the developer itself reviewed the code before submitting it? Code is trusted rather than questioned. We saw that AI can generate code really fast, but this code can be compromised. And when you review the code, you're sure that the code is good to go to the production and everything will be fine. So also, edge cases are ignored. What is meaning that? Sometimes we use different applications communicating between each other, which is meaning the relation between applications can be compromised, can be exposing secrets, or we can communicate with the application in the wrong way, or many, many varieties. When we talk about security, we should follow the rules that as much possible security layers you have in your application to verify that your code is secure.

This is meaning in the end, your application will be really secure and you will be not compromised. So more security, it's equal to enough good application and good name in the community. So also, here it's very, very important to understand ownership. Again, I said that many times that your code is your responsibility, but this is very, very important nowadays, just because I saw many examples of, okay, AI can do that, AI can do a lot of things, but in the end, you are the person behind AI. So use it like a developer, but also not good enough to ship anything which is not verified.

As I said, let's watch some pitfalls, which AI often do and how to avoid them, of course. So first of all, what is happening when AI create applications is that it's used a lot of good HTML, it's used a lot of good, let's say, CSS, and it's very good of creating the application itself. But when we're talking about security attacks like injection attacks, for example, you can put a SQL injection in your input just because AI never sanitized the input itself. Or let's say, attacks which are really common nowadays, it's you forgot to remove some sanitization checks or you compromise the sanitization in net alone, what is leading, it's compromised application.

So here the rule, it's always to validate, when we're talking about forms, when we're talking about inputs, or any kind of submission or interaction in the user to database and backend to be validated and sanitized in proper way. And to be honest, here the example is very bad. AI will not do the validation and sanitization by himself. Tools will generate the code, but they are not so smart enough to think in direction that how this should be validated or sanitized in the best possible way. And here also I want to share that usually AI grab information for Internet. And we teach our models from Internet. In Internet you can find a lot of good examples, but most of the examples are really dangerous and you don't know who is the person behind them. Again, don't trust AI. Validate the code before going to production.

Second attack is very popular. So this is unsecure authentication logic. Imagine you have a form in your website. And this form is used by people to be able to log into your application. So imagine you are prompting and typing to AI, please generate me a login form. I want to be able to log in. And don't specify how the authentication process will happen. What authentication tools you will use. Usually what AI will do is to generate the login credentials, test users, or test 1, 2, 3, or something which is not ready for any environment like staging or production. This is for test purposes. So to be able AI to generate proper authorization and authentication process, you should exactly tell the AI what to follow and also what tools you will be using, like Auth0, for example, or JWT token or something else.

About authentication logic, each time you should validate, do anything left. Like, let's say user, which is a test user, use it in your application or test files, which you can securely go to your application, but you forgot to remove it, just because AI is very good of that to generate cost fast. So the second pitfall, it's basically broken authorization. Okay, so imagine that AI will generate the form. This login form is very good. It's looking nice, shiny, beautiful, but in the end, you're just telling AI, okay, this is my user. I want this user to be able to access only the dashboard. And I don't want to be able to access the, let's say system settings or any kind of settings, which will affect different users. So AI will generate the form, will put some validation, probably the validation itself will be stupid, just because AI don't know a lot of information, what you have in your database. Imagine that this test user is able to access the dashboard, but also the settings. And when the settings have changed, everything is changed into your application. Basically, it's compromised just because you never specify to AI, what is your authorization process, what permissions you should have for different roles, and do this specific user, it's editor role or admin role or what role itself.

Secrets in the code. Yes, realistic view. We have really good progress of AI tools and those models, which generate code. But still, I saw so many examples in code reviews, that AI blindly put secrets in code, rather than to put them in some files, which are never committed to Git, like dot NV files or any secret mechanism, or let's say GitHub secrets and etc. They blindly put something which will never be exposed, but it is right now exposed just because we use AI. And let me give you an example.

And let me give you an example. So I have a client, and I was working with AI to change, let's say, how the backend communicates with frontend. And we changed the API URL, and also the credentials to be able to use this backend. And I provide information to AI, like context itself, what is the backend, how we can connect and et cetera. And what I saw is rather than update the dot NV file, it's created directly the variable, it was const. So directly, it was put the secret in the code itself, which is making my application really exposed to the world. And please, never commit dot NV files, never trust AI blindly, and also the secrets should stay in specific places. For that reason, many, let's say, hosting providers, they specify specific place for the secrets. Your code is basically code, but never should include any kind of secrets or environment variables.

And the last one, it's also very, very good, and that is dependency and supply chain blindness. So, I saw one application, it was ReactJS application. So this ReactJS application, it was old, like one year and a half, and no one touched it. And of course, we decide to use AI to update our application to a new version of ReactJS, and also to update everything related to the changes in this version. What is happening is that React, it was really old, it was like, I think, 16.8, and it was not updated, so AI itself tried to update to new versions. But what is happening is that AI update to specific version, which one week before, React World and also Next.js and many people share around the world that this specific version, it is compromised, and we should update to the latest patch. Unfortunately, AI don't do that, but directly install the compromised version of ReactJS. And this is just a small example. Also, when I saw like example is that when AI update the NPM packages, also added some packages which I don't want, and later on, don't do like check, for example, NPM audit, but what is happening is just tell me, okay, it's done. Everything is working fine. The build is good. Everything is fine. But in simple audit check, I found like 10 or 20. I don't remember exactly, but it was a lot of security vulnerabilities just because, again, I blindly trust AI.

So many people ask me what they can use, like information to be secure and also stay up to date. This is my preferences. Of course, I highly advise you to check them, explore the internet. We have a lot of information about security, but what I trust is really CVA organization, which is website including all of the security vulnerabilities in this website and also give them specific identifier or ID, which later on can be used to reference the vulnerability to see also how serious is this vulnerability. For example, this was the place where our React community and React itself mentioned that they have vulnerability in server components. They provide the ID and this vulnerability, it was in the maximum possible range and it was high. So for that reason, everyone in the world using React understand that they have something in React which is compromised, and for that reason, we should update the budgets.

Another resource, very, very beautiful, very good to use, is Cloudflare, of course. They are a really good, let's say, provider, and also they have Radar, which is giving us information in the real world in right now time, in lifetime, basically, what is happening, how many security issues we have, and how the security risks are raising nowadays. Unfortunately, if you do a little research, you will find that before AI era, when we write the code, of course, we was compromised. The hacking itself will be involving, but nowadays, this number are raising a lot just because nowadays we can use AI not just to write the code, but also to try to compromise your application and to use bots to try to slow down or hack your application. To do DDoS attacks or any kind of vulnerabilities.

Another resource will be basically the official websites, like Meta, Apple, Google, Reddit, Next.js, or bloggers, YouTubers. When something happen really big, like the last security issues in Next.js and React.js, probably, many developers will be sharing this information around the world and trying to prevent themselves and also to help to the rest. But, okay, we know from where to stay up to date what is going on, but question it is, from where I should start learning security? Usually, security is very big topic. In this presentation, I basically cover the most common pitfalls I saw, like example AI is doing. Of course, the variety of AI security risks are really a lot and to be able to stay up to date and to learn also security and how to check and explore security in a way, I want to give you some examples of very good websites I personally use.

The first one is all apps or basically organization website, which you can go there and they explain in a very, very nice way each security vulnerability. They explain what it is, how it's affecting your application and also how you can protect yourself. Another good resource is called Flare Learning. They have a learning path. You can go and follow some courses. In the end, you will have a very good understanding of security itself. And last but not least, it's TryHackMe. This really beautiful website is created in a way that your purpose there is to try to hack the website itself. You have tasks and if you are smart enough, you will be able to successfully hack the website, which is really nice and a good resource to learn. Of course, explore books, explore YouTube, blog posts, and et cetera. The topic about security is very, very big. And I believe many people should talk about it just because what is happening nowadays is really applications are done in a way that AI help us, but AI not defend us.

All of these pitfalls are very nice. We understand them. We will try to stay secure, but also how we should prevent these security issues, what we can do is basically to create AI rules. For example, in Cursor, you can create specific rules for the tool Cursor and use these specific rules. So whenever you generate the code, these tools will be used, and later on, we are sure that we have a checklist and this checklist will validate that each step is validated, you are secure, and all of these pitfalls are not in your code.

I created a public repo about that. Of course, you can go explore, commit it if you want in your code itself or try to support the community or contribute to open source and create more rules. Personally, what I'm doing is I have rules for code, how to write a code, what structure to follow, what is best practices. Also, I'm including here AI security, performance and not in the end, testing.

Testing is very important. It's like a way to prevent yourself before anything worse happen. And if you change something, of course, the test will fail. So you should update your code. And not in the last point is probably the documentation. Don't forget to update the documentation, treat it like it's code, and also update your application NPM packages to the latest one. So in that way, you will be up to date and you will know that nothing bad will happen, at least for now. Because when we talk about security, everything is for today.