AI Can Build Your Software. Can You Trust It?
With guest, Rob Clyde, Software Development, AI & Cybersecurity Expert, and Dan Martin, Eleven Canterbury Program and Relationship Manager
Summary
AI can now generate software in minutes that once required entire development teams and months of work. But what happens when the code moves faster than human oversight?
Former Symantec CTO Rob Clyde has spent decades inside cybersecurity, enterprise software, and technology leadership. In this conversation with Dan Martin, he explains why AI-generated code may become one of the most consequential shifts in modern software development while also introducing security, governance, and intellectual property risks that many organizations still underestimate.
Rob shares real-world examples of organizations reducing software projects from six months to two weeks using AI coding tools, while also confronting authorization failures, security vulnerabilities, malicious code generation, spaghetti code, and the near-future reality that humans will no longer be able to review all the software AI creates.
The conversation explores:
- Why developers increasingly rely on AI-generated code despite declining trust in its accuracy
- How AI coding tools can unintentionally introduce vulnerabilities and malware
- The role of deterministic guardrails, testing frameworks, and NCAST in reducing risk
- Why memory-safe languages like Rust are becoming increasingly important
- How AI-generated code is reshaping software patents, copyrights, and intellectual property strategy
- What boards, technology leaders, and cybersecurity teams should be asking now
AI is no longer changing only how software is written. It is reshaping who controls it, how it is secured, what can be protected, and how much of the digital world humans can still realistically understand.
Transcript
Dan Martin: We’re talking today about software developed by AI. What could possibly go wrong?
I have a daughter who has a degree in liberal arts. She had some software written using the Vibe application. It’s really sophisticated software. She spent a long time detailing what she wanted done, and the whole thing cost $2. If she had a whole team of people, it would have cost a fortune. So, it seems promising. I’m really interested in learning about what’s going on.
I’m speaking with Rob Clyde. He’s the former CTO of Symantec and the former CEO of Adaptive Computing. He’s on the board of the ISACA. He’s the chairman of CryptoQuantique and the Executive Board Chairman of White Cloud Securities. He’s also on the board of Cybor. It’s really exciting to have someone who knows what they’re talking about on this really interesting subject.
AI software development. Are they high quality? Does it work? Are there some pitfalls? What do you think, Rob?
Rob Clyde: First of all, all of these companies that you’re mentioning do software development, so this is something we’re talking about all the time at each of the companies I’m on the boards of and with my clients.
But let me give you a couple of examples. One of my clients recently did a project taking on-premise software and lifting it to become SaaS, so it runs in the cloud and customers can access it in the cloud. I’ve done a lot of those lifts throughout my career, and it’s usually fairly heavy, takes a lot of developers, and a lot of time.
In this particular case, they estimated that working in a conventional way would take their team six months to do that lift. But using AI to generate code, not all the code, they had to do some of the work themselves, and they were able to get it done in two weeks. So, a six-month project in two weeks.
Similarly, they were able to transform their website, which they thought would take two weeks, into a weekend project. And so, another good example of moving very, very quickly. As we look across the industry, I do have a couple of stats. According to Stack Overflow, 84% of developers are using or planning to use AI to help them. That kind of matches up with all the companies I’m working with. They’re all using AI in some fashion. 15%, kind of like your daughter, are using AI in a vibe coding sense to completely build the application and are never actually manually writing any of the software. That’s often a little harder, particularly when you get to system software and those types of things.
We are already at the point where 22% of the code being produced worldwide appears to be generated by AI, and that number is just going to go up. So, we live in this world, but interestingly enough, the experts, the developers, if you ask them how they feel about AI tools, they like the help it gives them. 60% have a positive sentiment. But that’s actually down from 70% in 2024, so it’s decreased. So, the more they use it, the less positive they are about it, even as the tools get better. That’s kind of interesting. Only a third of them trust it, so they know it needs additional work, and only 3% have high trust in the accuracy of the code that it’s developing.
So, back to my client who did this. They found some real issues. One was the code that was being generated; the tools kept trying to bypass authorization. It wanted them to just give it full access to everything and not really use the access controls and authorizations that should have been in place. So, they had to really work at it to ask it to put in the authorization that was needed, and in the end, they often had to do that manually. They found it was often spaghetti code, code that would be very difficult to maintain. So, once it was produced using AI, you would probably have to use AI to continue to maintain it. You’re definitely going down the AI path when you start doing that. They found that it did, in fact, sometimes include vulnerabilities, actual security flaws. And in at least one instance, it actually produced malicious code. It essentially put a virus in the code that it generated, likely because it picked it up in some open source as it was learning to write certain types of code. There was a lot of code duplication, et cetera.
So, we have these downsides, and it’s one of those things where you just have to go in eyes wide open, realizing that AI is, by definition, probabilistic, which translates to it’s not always going to get it right.
People who actually use this on a daily basis will express frustration with how long it takes them to describe exactly what to do and actually get it to produce the code they want.
Dan Martin: I think about when we outsourced software development and sent it to a foreign country. You had to be so specific about what you wanted that it ended up taking more time to get it developed than it would’ve been to do it locally, where people had the knowledge of your business. It seems like this is a bit of a replication,
Rob Clyde: It is, but the production benefits are definitely there. So, the effort to get it right appears to be worth it, unlike the outsourcing thing, which I lived through as well. I can tell you all kinds of ways to have made it better: the effort to instruct AI correctly and learn how to use it. Most developers, the vast majority, are saying, “We use it.”
And even at low levels, one of my clients, which I’m on the board of, is actually using it to develop embedded code, the code that goes into the firmware of devices. And you would think that, no way, you could use it to develop code at that low a level. But they’ve indicated that it actually really helps them, and they’re moving faster as a result.
Dan Martin: What are some things you can do to increase the likelihood that it’ll be successful, or not screw up? As you said, it’s a probabilistic mechanism, which means it picks out the things that have mostly worked in the past.
Rob Clyde: Yeah. So, markdown files are important. Those are files that can include the rules of how you want AI to help generate your code. Your specifications, as you laid out, are very important, and you can reuse many of those specifications that describe how you want it to generate the code.
They get quite long in your conversation with AI. So, things that you build to be able to use AI, you can think of it as generating a vibe coding platform, which is essentially what you’re trying to put together as a development team that can be reused from project to project and iteration to iteration.
So, these types of controls become very important. And then there are also things you can add during the testing cycle. In particular, since AI is probabilistic, as development teams, you want to be able to inject as much determinism into the process as possible. That’s what you’re trying to do with those guardrails: inject a bit of that into it.
Examples might include instructing it that when you’re going to use math, please use certain types of mathematical algorithms. Make use of certain libraries, and make use of deterministic algorithms in the code so that not everything is probabilistic, because AI can’t do math. Or I should say, Generative AI can’t do math very well.
What you want it to do when it’s building the code is actually use deterministic math algorithms that we’ve had forever. There are libraries of great math algorithms that you can use, or chemical equation solving, or many other kinds of deterministic processes that are out there that we can tell it to use.
For communication, use SSL, don’t invent a new communication protocol. Use the standard one that everybody uses that’s out there. There are ways to do this that end up with much better code by injecting determinism into the process. And then, similarly, and we can talk a little bit more about this, there are things you can do on the testing side.
Dan Martin: That’s what I was thinking when you talked about how they sometimes develop spaghetti code. Part of quality control is a code review. Is it possible to review the code that they generate?
Rob Clyde: Great question, Dan. No. Here’s the reality. We’re not even good at doing the code reviews of human-written code. Probably less than 30% of code gets really reviewed. The fact is, developers don’t really love to review other people’s code. They find it irritating. Occasionally they’ll learn something, particularly if they’re a younger developer, but are they really adding much to the process by reviewing a senior developer’s code? It’s more of a learning experience.
If we’re going to generate more and more code using AI, and we can’t even keep up with code reviews for human-generated code, the chance of us being able to always review, have a human review every line of code that AI writes, is slim to none. And as we move forward, that will only get worse. We will rapidly hit the point where there’s no way a human can review all the code. So, take the whole code review idea and set it aside. It is doomed to failure. It will become a bottleneck, and organizations that find ways not to have to do a code review are going to outproduce organizations that require code reviews. I almost hate to say this because earlier in my career, I was a huge code review guy, which was all part of the agile process. But I’m just looking at the trend lines. That’s where we’re headed, and we’ve been headed this way of not doing much code review for a long time. So, we’ve got to find other testing mechanisms.
There is some good news. Many code-generating tools are getting better, including generating cleaner code. That’s the trend. They’re getting better at this. Less spaghetti code, cleaner code, that’s a little bit better. That’s good news.
Secondly, many code generation tools now include static application security testing as part of the cycle, and it’s deterministic. So that’s looking for common vulnerabilities like in C and C++, looking for buffer overflows. That’s pretty cool. You’ve got these tests we’ve been using forever that you can now add into the code generation part, and they’re like a deterministic guardrail.
Similarly, you can use tools like NCAST at test time. NCAST, Network Comparison Application Security Testing, is a category of tools. What it does is it can take version one, then, when you use AI to build version two, you run all the previous tests through it and compare the differences. You can essentially record it as you play version one at the network level, and then replay it on version two, and the only differences you should see are the intended ones. You’re going to have to use AI to tell you which ones are intended and which are unintended. But now, if it’s got unintended ones, you just feed it right back into the AI process for generating code. You feed it right back into the LLM and ask it to please generate the code and remove the unintended ones. And you get this iteration. Studies have already shown that this guardrail of NCAST, one of my clients did a great study on it, you get a 30% reduction in the use of tokens. So, it starts costing you less.
Tokens are the charging mechanism LLMs use to make money. Frankly, they’re quite happy for you to have to iterate over and over again and use all these tokens to try to get your code right. The trick is to add determinism, like NCAST, to the process.
One last thing I’ll mention: there’s a huge opportunity now, thanks to AI, to refactor our code. We have a tremendous amount of tech debt and many vulnerabilities, particularly in C and some C++ programs. And if we refactor that software that’s out there and has been in use into memory-safe languages like Rust and Python, both memory-safe languages, Rust in particular for C and C++, if you refactor it, then you immediately eliminate the buffer overflow and other types of memory problems, which are the vast majority of vulnerabilities that can be taken advantage of. And since AI can now automatically find and exploit vulnerabilities, it becomes really important.
Think of Claude Mythos, for example. It is really important that we can easily move code to memory-safe languages and eliminate these types of vulnerabilities. And you can just tell AI, “Please convert this program from C to Rust.” And then with something like NCAST, you can run the old version in C in the test and run the same test against the new version, and they should operate the same way. And you do the comparison, all the network behavior should be identical. And if they are, you’re good to go. You know, you’ve most likely now got a memory-safe version of your application.
So there’s a tremendous opportunity that AI can bring as well. I hope this helps illustrate how guardrails can be used.
Dan Martin: You’ve pointed out the way you can get improved efficiency; you can get assistance for human beings for doing this kind of development, the application work. There’s still a big place where people are deciding not how to build it, but what to build and why we’re building it.
Rob Clyde: The what has always been the bigger issue. What should I build? And there are some little tidbits as part of this whole process that boards need to be aware of.
For example, when I just refactored that code from C to Rust, since it was generated by AI rather than a human, it looks like it is not copyrightable so far. So, the whole intellectual property process is being thrown on its head. If code is completely generated by AI, say, the vibe coding that your daughter did, that code that was generated, that source code is not copyrightable. It’s not protectable by copyright. The specification that she put into AI might be. So, it kind of changes it, because she wrote it down. It changes the way you think about the markdown files.
So, it changes the way we think about it, and, as boards, my advice, and it’s really important, is to start asking about what’s patentable, because it doesn’t matter what the implementation is. In other words, the exact way the code is written doesn’t matter. You could still violate your patent. So, I think software patents are going to become more important with AI and trade secrets than they were before. Copyrights are going to be harder and harder to enforce. Even if AI wrote a good chunk of the software, it’s no longer all copyrightable. It gets very, very murky.
An intellectual property strategy for AI-written code should be high on the priority list for management and boards to ensure they understand this, because the world has shifted.
Dan Martin: There is so much going on. I’ve really appreciated this conversation. I learned a lot. Thank you for making the time, Rob.
Rob Clyde: Thank you. My pleasure.
