#554 Securing the AI Era: Alex Schlager on Why AI Agents Are the New Attack Surface

[00:00:00] 

Mehmet: Hello and welcome back to any opposite of the CTO show with me today. I'm very pleased to have with me Alex Schlager. He's founder of AIceberg, a company which is in the intersection of AI and [00:01:00] cybersecurity. Probably maybe from the name you guessed. I'm saying this to, to the folks who listen to us or are watching us.

Alex, the way I love to do it, I don't, you know, steal the spotlights from my guests. I keep it to them. Tell us more about you, your background, your journey, and what you're currently up to, and you know what exactly AI Asberg is all about, and then we can deep dive from there. 

Alex: Sounds good. Thank you for having MeMed.

Uh, Alex Schlager. I'm the founder and CEO of AIceberg, and, uh, we do detection and response for AI powered workflows. Um, workflow could be as simple as a LLM powered chat bot, uh, or as complex as a multi-step multi hierarchical agent workflow. Uh, our mission is to provide safety and security for enterprises adopting ai.

Um. And, um, the special sauce, if you like, is we do not use large language models. We rely solely on explainable machine learning models. We [00:02:00] are very adamant about never to monitor a black box, uh, with another black box. 

Mehmet: Cool, and thank you again, Alex, for he being here with me today. Now. Um, of course what I like to do always, I like to do my homework, getting prepared as much as I can.

Of course. Um, and I know like the main reason why you founded AI AIceberg because you saw AI becoming a news attack surface, like, um, long before people noticed so. Tell let's, I, I like to go into the problem first before talking about the solution. So what are the shortcomings you saw in the traditional Let's, and they are traditional legacy, probably now the cybersecurity tools like that.

Let you feel that this is not enough for this AI era? 

Alex: Well, to be honest, we started this as a research project on AI explainability. [00:03:00] Mm-hmm. Um, so it was a, a bit of a tangent to cybersecurity, if you like, at the time. Um, and, and the idea was to really try to reverse engineer deep learning, uh, you know, deep learning based models.

Um. So-called black boxes that are intransparent or incomprehensive to the human user. And with the rise of large language models, let's call them, you know, deep learning on, on steroids, um, we saw that this problem, this issue, this challenge is even, you know, becoming more severe. Um, so it started as a research on explainability.

Um, and it ultimately ended up in, to my earlier point in providing guardrail safeguards, uh, on the basis of explainable ai. So to use one type of ai which is transparent and accessible to humans, to safeguard and protect humans from, uh, from black boxes. Now, the irony is we actually do both, right? Mm-hmm.

We protect the user from the AI and the AI from the user. Depending on the [00:04:00] scenario and the context, 

Mehmet: you know, every person experience in this domain, like yourself, Alex, that they currently, they're telling me it's like kind of course, it was always a, uh, you know, chicken and egg problem, you know. Um, the companies who works in this domain, they try to, you know, think before the hackers what they can do.

But now with ai, like it became accelerated on, on both sides Now. In today's companies, where do you see the biggest blind spots when it comes? We, you talk about like, uh, guardrails. Where are like the blind spots when it comes to AI system vulnerabilities? 

Alex: I, I think it starts with shadow ai. The enthusiasm and the, the push on AI adoption is so strong that you have AI popping up all over organizations, right?

Whether it's marketing, using gen AI for image generation, um, legal departments [00:05:00] to, you know, using gen AI for summarization or, or content creation. And the CSOs being challenged by losing a little bit control, at least temporary in sense of where AI is being used, which data is being shared, um, and what the enterprise's exposure is to gen public gen AI services in particular.

Um, so this is where we see it starting. Uh, and it got, I wouldn't say worse, but the challenge got bigger with agentic ai. Um, where now it's not only using public services that, you know, like the image creation, um, use case, but actually business entities starting to build agents for the simple reason that you don't need it necessarily, or coders anymore to set up a agent.

Workflows. So the visibility is really the, the first challenge, uh, that we hear from CISOs as adoption really. Um. You know, is, is moving literally to every, through every business functioning unit. [00:06:00] 

Mehmet: Do you see Alex also this challenge where people, they think that, um, you know, they assume that AI models are safe because you know themselves, you know, they are sitting maybe in their offices or maybe someone is use even if they are working from home.

But hey, like we have the VPNs and we have these firewalls. All these internal tools. Uh, so do you see this misconception, um, you know, with, with the companies that at least you know, you have, uh, uh, talked to so far? 

Alex: Um, no, actually not. Um, well, the issue obviously is, or the risk is with in the transaction or interaction itself.

Um, so to your point, if the infrastructure part is secure, there is still the risk of, uh, you know, of. The model being poisoned, the data being used, being poisoned. Uh, a website that the model goes to, to scrape, uh, or content or to fetch content, uh, being poisoned. So there's still a lot of attack [00:07:00] vectors and, and risks remaining, um, let alone the safety aspect.

So the interesting part, or discussions we have with CSOs is now that they should care about safety as well as security because at the onset of an, uh, on, at the onset of an incident. You don't really know if it's a safety issue, a model error or downstream propagated error or the result, uh, of an adversary.

And so all of a sudden we have safety conversations with CSOs, um, that they're not necessarily saw coming in that context. 

Mehmet: Right now, this is, this is, uh, you know, kind of would let, I would believe some non-technical, um, uh, CEOs or non-technical, uh, leaders. Um. Think about like, what, what can go wrong? Like, again, like what I'm trying to do, I'm, I'm trying to simplify, you know, the, this [00:08:00] attack surface as much as it can be.

So if we want to kind of tell them this in, in, in a way that, uh, uh, we can, you know, resemble it to, to something else, like what would it be like, uh, if you want to kind of compare to a physical world, like how we can compare that. 

Alex: Well, think of it this way. Let's say you are, um, you're an enterprise operating a public facing chat bot that has been set up using a large language model for, you know, customer service, uh, oriented tasks.

Um, the, the, the, the bot or the, the, the large language model could hallucinate and produce a response to your customer, to your consumer that is factually incorrect. Uh, and the consumer might realize that. So there's a certain degree of reputational damage or reputational aspect in there, but it's not the end of the world, right?

The next level could be, uh, that the model has been manipulated or you know, to a certain degree that it [00:09:00] gives, you know, let's say vulgar or insulting answers to your customer. So I talked to a CISO who literally said, look. You know, our chatbot swearing at our customer is not great, but it's not the end of the world from a security perspective.

So in the chatbot realm, you know, or we have a media, me, media, um, customer who will release, um, a chatbot for children next year, right? So there you go into areas of unwanted speech, of toxic speech, of illegal speech that simply needs to be detected and avoided. Or you can imagine the fallout for this company if parents, for example.

Would feel uneasy, um, having their kids using the chatbot or visiting that website. So that's really on the chatbot side and mm-hmm. To your point, the, in the, the Blast radios is relatively contained there. As we move forward, uh, once, you know, rag re retrieval, augmented generation, meaning, uh, you know, active data retrieval from sources is involved, it gets a bit more tricky in sense of, you know, sensitive data [00:10:00] disclosure, whether it's P-A-I-P-C-I-P-H-I.

Or company confidential information, right? The whole topic of identity and access management and privilege, uh, is, is really not fully solved yet in sense of, um, which information under which context by which user should be retrievable, for example. Um, as we move into agent, this is where the risk surface really widens and broadens because we move legacy processes.

Into autonomous states with agents. Um, and there they access tools, they access data, they execute on tools, they go to external sources. So this is where the attack and risk surface significantly expands. Um, and, uh, to my earlier point on, on privilege, this is one of the challenges. You have an agent. That has certain privilege of accessing data, 

Mehmet: right?

Alex: But now we also need to think about on whose behalf the agent is accessing data. So MeMed runs the agent, and Alex [00:11:00] runs the agent. Both of us have different privilege settings, right? MeMed is allowed to see, uh, you know, all the confidential and sensitive information. I am not, but we need mechanisms so that the agent understands.

On whose behalf it is acting. So it's not only the agent's privilege of Axis, it's on whose behalf the agent is, is is performing that, for example. 

Mehmet: Great. And I think all what we are discussing, Alex falls under, you know, what we call, um, governance, risk compliance, GRC, you know, um, so now I would like to understand how you come and help, you know, customers.

In making sure that all what we discussed so far doesn't happen, right? Like it doesn't go curse the customers. It doesn't access sensitive data and then leak it somehow. So. If you can explain to us how do you do that on a high level? 

Alex: So we use what's called [00:12:00] semantic classification. We classify language.

We classify code, um, to determine, uh, the information being disclosed in an input or output, um, and the content being disclosed. So, to your point, whether we find. You know, private information, health information, payment information, uh, we can detect, detect that and redact that. So we don't need, necessarily need to block the entire input or output, but we can redact this kind of information on the fly.

Um, on the content side, for example, we look for toxic speech, hate speech, racism, sexism, uh, you know, et cetera. Um. In order to, to, to block such inputs or outputs the same, you know, holds true for, for illegality, for example. Um, moving beyond that, we classify the user intention, right? So often chatbots by design have a specific scope that, you know, ideally should not be, should not be, um.[00:13:00] 

For example, we have a, a, a banking client that wants to avoid people asking medical questions to ab bot, right? Because, 

Mehmet: right 

Alex: liability and simply of, of this being out of scope for the purpose of the chat bot. So by, by detecting the user intention, um, you can, you know, enforce policy and, uh, for example, provide an automatic response to the user.

We are, sorry, this is not a medical support bot. You know, we are here to provide. Financial advice or support for your account, whatever it is. Um, as you move into ag agentic, things become a little bit more complex because we need to look at the relationship between what a user is asking for and what the agents are doing.

So think of it this way, it's an example I use. It's not very realistic, but it's easy to, to follow. I ask an agent to book a, you know, a ticket to Los Angeles for next week. And I see in the agent interactions that the agents wants to call GitHub. That doesn't make sense, that doesn't align. There is no alignment [00:14:00] between what the user intention is and what the agents are actually executing.

And that applies to safety and security, right? So you could, to my earlier point, there could be a safety issue because the agent is simply, um, you know, mistaken based on the LLM instructions, or it could be an adversary involved, uh, in that tic chain. 

Mehmet: Would ba there, because you know, when we talk about agents and we talk about, you know, different systems talking to each others, um, is it like.

The same, I would say, uh, guardrails that are applied, for example, to the LLMs, uh, applied somehow to what these LMS might be talking to. So to your point now, you, because you just mentioned the gi mm-hmm. You know, GitHub and or so, right. Securing the APIs or access to the APIs, or maybe, you know, uh, maybe they have some SaaS products that, and, and nowadays we are seeing this a lot, [00:15:00] Alex, um.

A lot of these LLM, you know, companies or like companies behind lms, if, if, if I want to put it in the right way. So they are also kind of, you know, establishing these partnerships with like different providers, right? And if we talk about the B2B space, of course, which usually I focus more on, so we find like, let's say.

Uh, access to your OneDrive or Google Drive. And so, and they do this within the application. So how here you can also help, you know, when it comes to safeguarding the whole thing instead, just, you know, the LLM, how and how this is different, uh, from just securing the LLM. 

Alex: Yeah. Well, there's multiple use cases.

Um, so we differentiate normally three use cases. The first use case. Uh, CSOs who wanna, you know, have observability of how AI is being used in their organization. Right. Back to the shadow ai, we talked [00:16:00] briefly about mm-hmm. And control over data, data exposure. Um, in, in this case, we are integrated with the firewall, for example, right?

Mm-hmm. We act as a forward proxy. We support the ICAP protocol and specific traffic that is AI bound, uh, is inspected by us, so it's invisible to the end user. And the CSO gets the full observability and control over, you know, what public services are being used for, which services are being used, um, and, uh, the ability to control data, data exposure, data loss protection, if you like.

The second use case is companies that run their own, uh, l empowered application. Let's take a chat bot. Mm-hmm. Right there. We act, uh, you know, like a proxy or a firewall if you like, where the application calls us, we call the model. The model gives us the reply, we return the reply, uh, to the application, right?

So we basically sit in line. Of the traffic between application and let's say a large [00:17:00] language model. And then the third use case and final use case is AG agentic ai. All ag agentic frameworks today have so-called hooks and callback functions, right? Um, so we, we, we, we ship an SDK. Customers drop it into the config file.

And what now happens is that the framework lama index, hugging face, uh, AWS strands, it doesn't matter which of the frameworks will send a copy of each event to us. Let's say, you know, the user activating the agent, the agent talking to a large language model, the agent executing a tool. So for all of these events, after, before and after the event, we get a copy.

We tell the framework whether it's allowed to move to the next step or not. So we don't sit in line, but we kind of listen in and we tell the framework, this looks good. You're allowed to move to the next step. Right? Or if we find a safety or security issue, then we would instruct the framework to basically terminate the agent flow.

Mehmet: Cool. Now, I gonna ask you a question not [00:18:00] related to what you do really here, Alex, but I would like to hear your, your thoughts about this. As I was giving you the example a few minutes ago about someone, uh, maybe in the office or outside of the office because people work hybrid or mm-hmm. Completely remote sometimes.

Um. I started to hear about some incidents which are like, kind of, and I know it, again, it's not your area what you do, but you know, as I, as I was mentioning, it's uh, good to hear your thoughts where us as people who are working for these companies are going outside and feeding these LLMs the public LLMs with kind of private data, right?

So, yeah. Like, there's nothing that stops me today from. You know, and I'm talking about a private company, not public company. So maybe I would go and take, you know, a, um, [00:19:00] an excel file com containing maybe some financial, uh, information about the company, and I give it to Chad, GPT, Claude, whatever, Gemini and I start to this.

Now, when, when companies want also to, to harden these things, what's the best practice here? Especially like, it's, it's like the wild West, as they say, right? So you can't. Con you can control people sitting maybe in your perimeters, but when they are just outside, what have you seen working at least to decrease the amount of, uh, you know, uh, data leakage that can happen or, you know, uh, any other, uh, consequences that can, uh, occur.

Alex: I mean, I think that's one of the biggest challenges you're bringing up mimed because, uh, we had a client. We have a client that should rather say where in the early days they simply blocked any access to a large language model. Mm-hmm. So what employees started doing, they basically copy pasted corporate information into their private email, send it to their private account, then use the large language model, and then [00:20:00] sent the response back to their corporate email account.

Right. So to your point, there is no a hundred percent protection to a certain degree. Um, we have one client who are basically now applying scanning and DLP to all their emails as well. Which many companies do, um, to check for that use case, right? But in, in general, the short answer is data loss protection, right?

The ability to detect whether sensitive information is being shared. The challenge is many companies do not want to invest into the effort of, you know, sorting and organizing the data, labeling it, indexing it. Whatever the DLP um, solution calls for, it can be labor intensive, right? So we are working on a solution that's off the shelf that gives you a general indication of sensitivity.

Um, but of course it is not as precise and user specific as if you would start, you know, labeling your own data. The other thing which we experimented on, which, which I still like as an idea and we might actually end up doing it, is. Is [00:21:00] the reverse. Let's say you have a a highly sensitive document and you want to understand whether the large language model has ever seen that content.

Mehmet: Hmm. 

Alex: So you can. You can create questions out of this document without disclosing its information, and basically ask the model and based on the response you get, you can gauge, has this model ever seen this information or not? So it's not so much the prevention, but kind of the test and validation that a particular document or information has not been disclosed or leaked to a large language model yet.

Mehmet: Right. I, I, and you know, and the reason I ask you this, because you highlighted shadow it multiple times, and I think, you know, um, I started my career, you know, in, in, in the IT department, uh, and we were always like complaining to, to our managers about like people using Shadow it. And you know, I can bet today that people doing what you just mentioned, you know, coping and pasting and, and doing all these things, um, funny enough, [00:22:00] like I've heard from someone that even if.

They have a good DLP solution that can somehow don't allow you to copy and paste in, in, you know, between, uh, email boxes. So they take photos using their mobile phones. And AI is so good today to even, you know. Par the information on Yeah. 

Alex: Opr, et cetera. I agree. Yeah. 

Mehmet: Yeah. And 

Alex: plus the variety is the variety of access and, and you, you have to control is quite complex, right?

There's API calls to foundation model. I can use the webpage of the foundation model. Uh, and it gets more tricky with SaaS embedded ai, right? So you use Salesforce. Which has, you know, with agent force, um, generative and agent AI under the hood. So my point is many employees might not even know that they're using an AI powered service, uh, online in the public.

Mehmet: A hundred percent. Yeah. And I, I'm hearing these stories on daily basis almost Alex, now you [00:23:00] mentioned the CSOs couple of times and I'm sure like they are your personas and the people to go to, to bring this up because again, it's like part of, uh, the governance, it's part of the. Risk management and all this stuff.

Um, did you start to see a change in the way, you know, because we, we always hear about security audits, we always hear about, you know, um, getting the compliance done for majority of the companies. Have you start to see AI being like one. Part of the long lists that, uh, CSO, they have to go with, uh, their team on to make sure that, you know, everything is safeguarded, best of practices or, we didn't reach that yet, Alex.

Alex: Oh, it depends on the organization, right. So we work with organizations who just started their AI journey. So they're still in the. Um, you know, testing and learning mode if you like. Um, and equally work with [00:24:00] organizations who have already, you know, AI powered solutions in production. Uh, I would say the biggest difference is, is the degree of collaboration with cross functions required.

So unlike, uh, in, in know, for other security disciplines in ai. I mentioned the safety security aspect, uh, earlier today. Uh, but it's also the, the group of stakeholders that are normally aligned or need to be aligned when, you know, you start deploying treason trust and trust risk and security management for, for ai.

So normally we see close cooperation or collaboration between CSOs and application development teams, right? So when we get deployed. Application development teams will test us to make sure we don't add too much latency to the application. Mm-hmm. As a simple example, uh, you have other organizations where they already have a responsible AI body, right?

That needs to be involved in use cases or agen, um, use cases that are being deployed. Um, you might have CSOs, and I think I mentioned this, [00:25:00] who, you know, basically tell business departments, Hey, everybody can build agents as they like, but you have to drop this SDK. So I as a CSO organization, can see what's happening, 

Mehmet: what's happening, 

Alex: right?

So long story short, I think it's the degree of cross of, of cross-functional collaboration that kind of is different to other tech deployments or tech, uh, adaptations because we see sometimes, you know, four or five different business functions being aligned in the adoption, safety and security piece, 

Mehmet: right?

Uh, uh correct me if I'm wrong, Alex. So, because you come. On, you know, all the layers that you just mentioned, which is basically, um, observability, like understanding what's happening and, you know, kind of, um, also detecting these anomalies and make sure that everything is compliance now. Usually when we put such systems, right, or when, um, so we have this clash between like, uh, 2, 2, 2 sides the side.

Like they want to adopt [00:26:00] AI at no at any cost. Um, don't stop us guys. Like we, we need, you know, we have a, a mandate from the, you know, like from the board, all the executives who we have to put ai, right? While CISOs and, you know, other folks would say, guys. We understand, but we need to make sure. So do you see this clash happening, uh, you know, uh, in, in the enterprise today?

Uh, and in reality, is there like kind of a compromise that sometimes you see the CSOs need to do in order to also make sure that the company is not left behind adopting ai? 

Alex: Uh, yes. Although, you know, to your point, right, the CISO cannot and doesn't want to be the department of no. Um, so, so yes, we, we do see that.

And you know, to my early example, we started a, a pilot with a, with a large, uh, with a large retailer. Couple months back and the CSO told us, you first have to go to the application development [00:27:00] team, right? Mm-hmm. Once they have greenlighted, you, then we will look at it. So in this case, they were fully aware of, you know, of, of constraints that they have based on the tech they want to deploy or implement.

Uh, the other example that I mentioned is the agentic AI example. I, I can't tell you how many CISO conversations we have. Where the CSO tells us the CEO has declared, you know, AI is the new thing everybody should use agents, and every business department is starts starting to play with agents. Uh, and the CSOs basically says, I have no idea which agents are running, what they're doing, which data they're accessing, which external sources they're accessing, and so on and so forth.

Um, so. While there was this aggressive push towards AI adoption, we do see, um, some degree of, of, of maturity kicking in, in sense of governance. Mm-hmm. Um, to my earlier point, right. CISO's issuing or, or, you know, responsible AI groups issuing governance. That, you know, again, repeating myself [00:28:00] says, Hey, everybody can play with agents, but we have to be able to see them.

Right. Be able to, to observe them. So, um, and lastly, we have seen many cases where, where, you know, bots were designed and they never saw, you know, the light of day because ultimately the board stopped it, or a governance body stopped it in an enterprise. Mm-hmm. Um, I don't think the CSOs really can say no.

Uh, I think the topic is too hot and too much perceived as a competitive differentiator by senior executives. Um, so in many cases we do see the g the CISOs kind of chasing a bit behind the ball, um, when it comes to AI safety and security. 

Mehmet: Right. Uh, but, but the good thing, which I'm noticing, Alex in general, that people are aware of the balance that have to be put, you know, in this.

Because also, you know, the stories that are breaking out from, um. Misconfigurations and everything you, you just mentioned about [00:29:00] the problems and you know how you're solving it. So for example, like the other day there was a big, uh, uh, you know, big news about, uh, something that happened I think with Entropic, uh, about, you know, the LLM injection kind of, you know.

Actors were using this for trying to extract data, and I think this is like raised the, the, the flag for even non-technical executives to understand that this is important. And I think, by the way, you mentioned something interesting that the CS now, uh. Push people to go talk to the application owners, like, you know, to make sure that these guys are okay.

Because again, speed is important. No one want to put another tool causing latency. Um, alert fatigue, you know, and all the things that, that, you know, always people talk about. Now, we, we kept talking about agents and agent ai, Alex, um. Are we accelerating this [00:30:00] more furthermore, in 2026, you think and and beyond or, you know, still people are like, Hey, let's let, let's finish the LLMs and let's finish.

Also, you know, what we were doing before, you know, the, these LLM uh, came out to, to, to public. And then start to think how we can, we're gonna use agents like in general, how, how are you seeing the right, the adoption? 

Alex: You see a lot of news around, uh, the ROI is simply not there, and only 5% or 6% of companies actually, you know, produce cost savings or ATA improvements with AI that will generally improve.

Um, I mean, don't forget, agents are relatively dumb. They're nothing else but orchestrators, right? So they can't, most agents, unless they're very closely coupled to the large language model. Are dumb orchestrators that need a large language model for all their instructions, right, for everything that they're doing.

So we are basically, you know, completely relying on, on the large language model. [00:31:00] Uh. Capabilities in architecture. Now we know that LLMs, the way they're built and trained today, uh, have significant limitations, right? The notion was, Hey, the more data I provide to a large language model during training, the better the model will get.

Right? Uh, but we know that this is a not true, and b, we are, we are, we are reaching physical, uh, limitations to the capabilities of these models. So I do think you'll see further. Uh, adoption in particular ai. Um, but you know, you'll see a certain limit as to how complex the process is. Uh. Can be or should be that we can basically move into an autonomous state, uh, because LMS make mistakes.

And if you have three, four different models involved and you have the issue of error downstream propagation, then there will be certain processes which you simply, from a risk perspective, will not move into the autonomous space yet. Um, so I [00:32:00] think long, long story short, 26, you'll see a broader adaptation.

You'll see more companies. Having the lessons learned to use AI effectively. Um, but I think there is limitations as to which kind of processes we actually can move into an autonomous state based on the. Quality, maturity, evolution of large language models. I, 

Mehmet: I a hundred percent agree with you, Alex, and, you know, I'm not an expert by any mean in this field specifically.

You know, I'm, I call myself a generalist, but, uh, interacting with different tools that, you know, I have, uh, the chance sometimes to explore new things. Um. Which you touched on, you know, the, the, the hallucination of the LLMs. And again, agents are relying on the LLMs. So the other day, uh, I've seen like people like jumping, Hey, like this new thing came out, like, you should try it, it gonna like break the wall.

And you know, I go and I just ask a simple thing. Nothing [00:33:00] hard. And it start to hallucinate from the first prompt, uh oh, okay. What people are like doing this whole thing about. And again, you know, back to the point, uh, so for these agents to, to become a hundred percent reliable, it's possible. We're not saying no, we're not denying.

But yeah, I think there's some work in progress and even. All of the CEOs of, you know, the, the open ais and Microsoft and Googles of the world, like they still say, we still have work to do in enhancing, you know, what, what we have built so far. Yeah, 

Alex: that's an understatement. 

Mehmet: Yeah, of course I can understand the marketing partner.

You, you, of course you always founder yourself, Alex, so, yeah, I'm sure like, uh, I, I work in sales also as well, so I can understand all this now. About, you know, something a little bit non-technical on, on the business part. Um, you started a company in a complete new [00:34:00] category, um, which if we ask someone, or we tell someone five years ago about, you know, what you're doing, they say what?

Like, what's that like? But what was the hardest part, uh, building the product in this category, both from technical perspective and business perspective? 

Alex: On the technical perspective, it was the fact that we decided, to my earlier point, to only build models that are fully explainable. So we had to do data curation.

That was just insane, right? So if, if, if you lose a, use a large language model, so many of our competitors just take your input and throw it into another language model and say, Hey, this looks great, let's go. Um, as we. You know, wanted to use explainable models. We had to curate samples from scratch. So we now, by now have over 4 million samples that were curated.

They're labeled, they're verified. And that was an extremely labor intensive process. Mm-hmm. Yes, we automated large parts of that over time. [00:35:00] But that was one thing that, uh, underestimated. I mean, it's a well known that you can have the best machine learning model if the data is bad, you know, it won't help you or the model is only as good as your data.

But I think that was one of the. Harder parts to, to, you know, automate and devise an effective mechanism to curate that much of data, um, across very, you know, across a very broad field, from toxic language to illegal language, to jailbreaks, to prompt injections, you name it. Um, I think that was hard and we are still.

To your other part of the question, it's still a challenge to us sometimes talking to CISOs because the space to your point, is very new. Mm-hmm. So the concept that. You now do detection and response, not on network packets or code snippets or whatever, but on natural language, right? Where you have a benign state and a malicious state that is still a new concept to many people that you're now classifying [00:36:00] and detecting language and making decisions on that basis for safety and security.

Mehmet: Yeah. And I think like many other technologies like history showed us, like it's gonna take a couple of, um, unfortunately like bad things to happen before people, they will have the wake up calls, um, and say, yeah, like, we need a solution for this. But on the other side, the good news is like the acceleration of AI itself.

Um, Alex, I I, I've seen like, it also kind of accelerated even the slowest. People that we could think about adopting new technologies because, you know, AI kind of one of the good things that happened, it forced people to go and say, Hey, like, we can't wait. We need to do something like, uh, and yeah, of course when you do things, uh, in hurry, bad things can happen.

That's, that's, uh, unfortunately, um, unfortunately, maybe, uh. You know, [00:37:00] good things to happen, right? Yeah. So, um, because again, we need to learn from, from our mistakes. Now, the other thing I want to ask you about the, you know, the business, um, of course you have customs now, so, uh. What were like the early signals that you kind of, you know, convinced you, although like you said, it's hard to convince, but I mean, kind of now we have what we call a product market fit.

Like there is a validation mm-hmm. Uh, that what you are building, um, has, uh, a, a, a a, uh, possibility of becoming something that we can scale, uh, with time. 

Alex: You mean, what are the indications that Yes. 

Mehmet: Yeah. The early signals, I would call it the 

Alex: early signals. Well, uh, you know, the, the early adopters of AI to a large degree that we experienced also had a relatively mature and advanced understanding of the all, all the associated risks.

Um mm-hmm. There [00:38:00] was increasing media coverage, right by AI going wrong. Whether it's the, uh, Canada example or, or I think it was, was it Chrysler? Was this example of somebody buying a car online? So my point is, there was increasing media coverage of, you know, of, of the downside or negative effects of ai.

So that obviously helped. Uh, but the biggest push we have seen so far is, is really on the agent AI side, because the problem there starts with a very simple premise. I don't have observability. Right, so before we even go into safety and security and alignment, classification, all that stuff, it's the simple problem is I can't secure what I can't see.

Mehmet: Right. 

Alex: So yes. You know, with the chatbots, the discussion was about, you know, unwanted speech and, and, and utterances that, that are toxic and illegal, but in agent it's really that, Hey, I don't need anything fancy to start with. I just want to have observability. I just want a single pane of glass [00:39:00] where I can see which agents are lying around in the organization and what they're doing.

You're not even at security or safety there, right? It's just the visibility, the observability, and that's. Relatively, you know, low entry barrier and, and low hanging fruit because you don't have to talk about the tech vectors and risk vectors and what possibly can go wrong. You really start with the basics.

Central, you know, single pane of glass, agent observability. And then you can start working with, with, with your, with your peers, with your, with the CSOs or the security soc organization on particular risks, and you start dialing in those controls and those policies. 

Mehmet: Right. Um, I don't like futuristic questions much.

I'm not big fan, but, uh, you know, the discussion brought us to, to this point, I believe. Do you think, Alex, we're gonna see in any, at any time? I would not specify when, uh. [00:40:00] AI agents working alongside the, the, the security team in a sense, you know, maybe a director of risk and compliance, which is basically an AI agent that reports to, to the CO.

I'm not asking a an AI CO because I know like this is still like very futuristic, but in the near future, can we expect to have. AI agents that are part of the security team, not in a sense that they just show us dashboards that actually they can do some, some tasks like, 

Alex: oh, it's already. Exactly. Yeah. So you brought up the alert fatigue.

Right? So one very common, uh, thing we see, um, in the, in the SOC space, in the seam space, in the SOAR security orchestration, automated response space is vendors deploying agents to start sifting through logs. To start, you know, just elevating incidents and issues that, you know, that were done by triage, by a human in the past.

So, uh, it is early [00:41:00] days of course, but we already see. A high degree of automation of legacy security functions. And I say legacy because I don't necessarily mean, you know, securing, uh, gene AI and e agent ai, uh, relying and deploying, uh, uh, agents to, you know, basically automate the lower ha, the the lower level tasks that were performed by human.

So if there's one summary of agent ai, if you look across use cases, is. Low level, highly repetitive human driven tasks. Those are prime time for moving to agent ai because they don't have too much variance. They have a high degree of repe repeatability. Um. And therefore they're good candidates for agen, TKI.

So you can apply that logic to hr, cybersecurity, engineering, finance, you name it. Uh, so long story short, it's happening already. The interesting debate in many enterprises is who is in charge ultimately of the agent, right? Is it it, [00:42:00] is it hr? I mean, we have clients where HR. Technically owns the agents, right?

Mm-hmm. Because it's kind of a, a digitized employee, if you like, other organizations say, this is nonsense. It's ultimately, you know, a, a program to a certain degree, therefore it should belong to it. So I'm curious to see how that, you know, how that pans out over time in sense of how do we perceive an agent?

Is it an employee or just software? And, uh, who ultimately manages these entities? 

Mehmet: Time will, will reveal this, uh, this for sure to us. Uh, and you know, there is a debate, as you said, not only for the cybersecurity, like maybe an ai, CEO and AI board member and all this stuff. So of course I think we still in the early days, as you mentioned Alex, but um, we, we, I agree with you about, you know, like things gonna accelerate, uh, at least 26 and and beyond.

So. People are like talking about 26, 27 [00:43:00] as a transition phase. Mm-hmm. Uh, and where we gonna do a lot of, uh, uh, you know, trial and error until we can go to the next level of what's waiting for us. So we now, we don't know as we are coming close to. And Alex, finally thing, I always ask my guests, um, anything that, you know, you wanted to mention maybe I didn't ask, and how people can get in touch and learn more.

Alex: So, uh, very simple. You can find us under AIceberg.ai. Um, there's in particular a resource section where we publish all our papers and articles. We have our own podcast called How Hard Can it Be? Mm-hmm. Uh, you'll find the, you'll find the access to that podcast, and we talk about all things ai, right? So we have legal professionals there and talk about the implications of AI on, on, on, on the law.

In context of law, we have board members there that talk about how AI is covered in the boardroom. So that's a, a good resource, I would say, to, to have a look at if you're interesting and wanna dive, [00:44:00] uh, deeper into the topic. 

Mehmet: Great. I'll make sure that all the links are available in the show notes. Also, people are listening to us, uh, on, on their favorite podcasting app.

Or if they are watching this on YouTube, they will find it on in the description. Alex, I can't thank you enough for, you know, this insightful discussion today and, uh, eyeopening, I would say for a lot of, uh. Leaders as well as you know, developers and everyone who works with, uh, uh, with the LLMs and agents, because as you mentioned, at some point, every one of us, whether you are in tech or not, you will interact with agents.

So it's good to understand, you know, the attack surface, the risks, and at the same time how we can understand. To your point, like we have the observability, the explainability, and of course the, uh, guardrails to secure these systems. So thank you very much for, you know, this discussion and this is how usually I end my opposites.

This is for the audience. If you just [00:45:00] discovered us by luck, thank you for passing by. If you enjoyed, give me a favor, uh, subscribe and share it with your friends and colleagues. And if you are one of the people who keeps coming again and again, thank you very much. 2025 was fantastic. Um, listenership went very high.

We did it in the top 200 Apple Podcast charts in multiple countries all the years, so keep with changing countries. So thank you very much for everyone from wherever you listen to us or watch us. And as I say, always stay tuned for a new episode very soon. Thank you. Bye-bye.