Tariq Islam 0:07 We do intros first, right? Yes. Jamie Duncan 0:13 Thanks for joining us on the K files. Each week we take on a different topic in the Kubernetes community and the IT industry. Tariq Islam 0:19 Depending on the week we may find ourselves knee deep in kernel headers, magic quadrants or even start a pitch decks. John Osborne 0:26 Our goal is to close each file with a sense of completeness and satisfaction for us in anyone listening. Jamie Duncan 0:35 Thanks for joining us on k files today. My name is Jamie Duncan. I'm a staff architect with VMware. And like always I have with me Tarte Islam. Everyone. I'm Tarik Islam. Tariq Islam 0:44 I'm a specialist manager here at Google Cloud. Jamie Duncan 0:47 And also john Osborne. John Osborne 0:50 Hi everyone, john Osborne. I'm a chief architect at Red Hat. Jamie Duncan 0:52 Today's topic the file that we're going to try to crack open today is all around APA or opa which is a tool that has been around in some circles for a long time. That is really gaining a lot of steam and a lot of acceptance in the Kubernetes. Community we're wanting to at least I want to unpack that a little bit. Talk about what okay does or what opa, I'm trying my best to say opa, because that's what the documentation says you should call it. So let's talk a little bit about the problem specific to Kubernetes. If you want to get into a little bit about about what opa is in general, that's cool, too. But I'm really interested in the problem that it solves for Kubernetes because I think it's a really good answer. John Osborne 1:32 Well, I think before we dive into Kubernetes, I think it's important to note that it is a general purpose policy engine. There are other projects like Sef and Kafka that have done integrations with it. But in the context of Kubernetes there is a Kubernetes specific implementation called gatekeeper. And I think it can do a lot. So there's going back a little bit in the history of Kubernetes. There has been attempts before to implement policies, most commonly the pod Security Policy Object that's been in Kubernetes. I think it's beta still There was some talk on GitHub by Tim allclear, about about six months ago, where they were kind of saying that it probably will never hit GA, they're looking at other tools like opa, to kind of fill in this gap. And then I don't know if he specifically call it opa. But then when he went to coop con in November, it was pretty clear. That was the direction of cn CF. Because there was, I think, eight different opa talks. It was mentioned in the keynote, which was really good for me, because I've been using it in some capacity for about a year on one project. So of course, like getting mainstream adoption, it is a CNC f project already, but getting more mainstream adoption. I think it's helpful, but the ideas that you can use opa is pretty powerful, but to lock down the cluster and the workloads and basically take a lot of the policies that you'd have for your organization that might be in Confluence or a wiki or something like that and kind of put it as code run that policy in your cluster. Jamie Duncan 2:58 Yeah, exactly. There's an entire industry around security in Kubernetes. At this point, I mean, there are startups launching almost feels like almost daily. And they fall into, in particular really combat Kubernetes. Security, you fall into one of the handful of buckets. And there are tools to make sure that the code inside the container inside the pod is secure. Things like Sona buoy, things, it'll do like CVE, scanning things. It'll do code, code, best practices, scanning, all of that kind of stuff. And then you have products to make sure your pipelines, the things that deploy your container images, deploy your pods, are done properly. And there are tools like twistlock, and all those guys that do all those things. And then there are tools like the low level tools that actually live as part of Kubernetes to make sure that things don't go awry on the on the actual application hosts, right, you have like se Linux and C groups, and all of that stuff and some low level objects inside Kubernetes as well, like you mentioned like pod security policies, setting up what can I what When I deploy a pod, what can you do? What can you not do? What permissions can it have? What permissions does it not have? But even with that entire industry, there are still holes. There's one pretty good hole, pretty good sized hole that opa fills. And that one is not what can a pod do when it gets deployed? Like what permissions does it have? But should I deploy this pod under these circumstances? John Osborne 4:24 Yeah, I think that having that context, where it's like I have the data about the cluster becomes pretty powerful, because there are gaps like if you go back to the Kubernetes security audit, the first finding, I have it right in front of me policies may not be applied, leading to a false sense of security. And how many times have you seen that and there's a few open if you open it up, it's got lots of things that opa can kind of crack down on one of them is se comp, which is also a little bit loaded with a backstory, you can overwrite horspath persistent volumes. Even if you use pod security policies, there's places that can be overridden. There's a whole list of things like that, where it's like, that's something that opa could solve, or have a part in solving the the use case that that I've seen customers asking for. So when a request comes in to deploy a new object or request comes in to update an object into the Kubernetes API, you could use opa to write these policies is very declarative policies that says, Okay, if you are just the kind of universal example is setting up an Ingress object, and so it looks, it can look at like the annotations on the namespace you're trying to deploy to, and read the annotations to decide whether or not the Ingress that you're trying to stand up should be stood up in that namespace. Jamie Duncan 5:42 And it can stop you. So those things where someone you know, forgets to change a coop CTL context and deploys something into pod by accident, or deploys pod code pointing to the prod database in Dev and then start slamming it with a load test all of those scenarios You can use that metadata in the Kubernetes in exedy. And Octa can read it and base logic around it to decide whether or not that should happen. And that's a pretty big hole in security. It's not security, like, Is there a CVE against it? But it's security and that should this kind of object be deployed under the circumstances you're trying to deploy it. There was a really good talk John Osborne 6:23 at coop con from Yelp. And it's not the greatest talk to learn about opa itself, when we're going to put some chat notes or some notes in the document about how to learn about opa but it's a great session to learn about the story of how they ended up adopting something like opa they had their basically it talks about their a Python shop and they had they had implemented their own specific policy enforcement, Python library. And they kept getting asked to support more tools like I am policies and Cassandra in And then they wanted to support maybe some other languages. And, you know, one of the other big things that they said was a huge pain for them was, every time they updated that library to support more things, they had to get all their developers to update their dependencies for even a minor patch, which, like in an organization, like an enterprise, getting developers to update their stuff, because you've updated the policy enforcement point is really difficult. So then they started looking at sto and an envoy and kind of using the external logic there to be customized their homegrown solution for that. And then they're like, well, let's go let's just reevaluate and see what's open, open source and they kind of realized that, you know, if they use an open source tool like opa, you know, they're not the only one doing the development, they got the whole community reach back to, they've got the feedback on slack and all these other things. And I'll say like, for me, so I've been using opa for about a year. Like the community's is really good. I've been on slack a few times. I usually get answers right away. I talked to the people at coop con, they were like super helpful. They're like how Can I help you? Oh my gosh, wait, what? So it's a, it's a, it's one of the more helpful ones. And it actually kind of sounded like I don't know if you've ever listened to Dr. Forrest green from accelerate state of DevOps, where she kind of talks about speed versus stability. And the good organizations use open source to not have these homegrown solutions that they have to maintain. And there's all these kind of debt on and the Yelp story kind of followed that and that's kind of how they ended up with with opa. But if you like that story aspect, it was pretty good. There's other talks that actually followed, like the actual tech and how to learn about the bits and stuff that you can put in as well. Tariq Islam 8:35 So one of the things that I've noticed about opa and this is actually something so just building off of what you'd mentioned a few minutes ago, john and Jamie actually around the context right opa brings it's it's contextually aware of security policies and capabilities. I think I think there's you know, we did a whole session or an episode on multi tenancy, right or at least part of an episode was on run multi tenancy multi cluster Things like that. I think this has huge implications in that space. Because what what better context Can you have than the various environments? And I think I think opa, at least, you know, from from, from my perspective, and this is a little bit biased given where I'm at today and Google Cloud, you know, we see this as being a cornerstone piece of tech for for enterprises to be able to adopt Kubernetes as a platform, and the implications that this could have for security teams, and maybe this is a bit of a segue here, but the implication here is that how they adopt this is going to be fundamental right with rego and getting them to adopt something like that, versus you know, whether it's spreadsheets or you know, something else, some other tool that they're using. I think that's going to be the next the next step in this evolution of opa. John Osborne 9:49 It's funny you say that because I actually was whenever you're thinking about doing this episode, one of the points was around this whole discussion we had on a multi Time Episode around single model clusters are these kind of Very large heterogeneous clusters. And I can kind of feel like as someone who works in ops for a while rolling out a live policy to a cluster is still kind of scary to me. Because it does have the potential to brick production. One of the cool things that they added in three, version three, which is the gatekeeper release was this kind of dry run concept where you can actually test it with the live data in prod and, and then actually will just tell you if it will fail or not. And then once you put it into the cluster, you could actually set things as worn and kind of roll them out on a namespace almost like a canary deployment, which is which is pretty powerful as well, because I was kind of thinking about even Jamie, you're one time I asked you about live updating a Linux kernel, and you gave me the analogy of, it's kind of like pulling the tablecloth off the table and hoping the plates Don't move, but in some respects, like applying a new policy to an entire cluster can kind of be in the same regard. Right? And so you do need a you do need to roll it out very slowly. I kind of feel like it kind of does bleed into the multi tenant. See there? Because it is I don't want to roll it any policy to 10,000 node cluster, even if I have all these checks in place, right. So having, you know, multiple clusters, I think, can really help kind of limit blast radius, just in case. But there are, if you look at the adoption so far, it's a lot of regulated industry. It's a lot of risk averse organizations like big banks, and federal government and things like that. So there's definitely people doing a lot in prod that you wouldn't expect or are not known to be risk averse. Jamie Duncan 11:28 We've had several engagements and none of them were using gatekeeper. So I haven't done a lot of digging into that. And I'm really interested about those features you just talked about. Because you're exactly right. There is nothing to solve that fear of dropping a change on it, of anything to a large cluster like that. And there's no amount of testing that makes you comfortable there. I want to roll back a little bit on you. You're talking about you've been you've been working with opa for about a year. Can you talk a little bit about how you got up to speed with it, like how do you like what are In the tools, and we'll stick links in the show notes with some specifics, what sort of what's a good philosophy around getting to getting used to this tool so to learn how to use it, John Osborne 12:10 there's a couple talks from Khan that are pretty good. There was one by Rita Zhang, which I'll put in the show notes, which was kind of an introduction to open kind of the basic core concepts and so forth. It was a deep dive from the people, it's theory as well, which I think is worth watching. You can go to the rego playground, and just start playing around with the way opa works fundamentally is like it literally just accepts any JSON, runs a policy and then returns any JSON. So you can go into the rego playground, which is an online interpreter, you don't even have to install anything. And you can just kind of mess around with it. I put a working example in the show notes where it's a policy to block privileged containers. One thing you'll note about it is since it's since it actually works in the admission control phase, the pod that we use for the examples actually nested in this review with the object is called admission review. object. And so that's, that's just how Kubernetes works under the covers for admission controller. So if you see that, that's just part of it. But that's a good way to get started. And then just installing it locally, there's a VS code plug in. One of the cool things about opa is that you can work just on the COI. There is some developer focus tools out there. If you've seen Gareth thrush Grove on Twitter, he works at sneak, which is one of the container security companies that's out there. He built this kind of developer experience around it where you can plug in to things like tecton and GitHub actions and things like that. So we'll put that link in the show notes as well. But just running it locally starting off in the playground. I will say like since I started a year ago, like a year ago when I started this all the examples were that Ingress controller example where it's like you know, you need a globally unique hostname for your Ingress. But there's been a lot of work done to take out if you've seen coop sec, but it's this kind of best practices. Things you want to take care of the Kubernetes website coop sec.io. They took all those policies, they moved him over to opa and implemented those. And then there was another, I'll put the link in the show notes. But there was another effort. That revision was working on to take all of the policies from pod security policy. So anything you could do in a pod security policy, there's now an opa policy for and they donated that under the open policy aging GitHub repo, so we'll put the link in there for that as well. Tariq Islam 14:26 you'd mentioned gatekeeper Jamie Unknown Speaker 14:29 and Tariq Islam 14:30 I wanted to just go check here, john, since you've been using RPA for sorry opa for the last year, your plus is it safe to say that gatekeeper is the native ization of opa into Kubernetes? You know, there there are a couple of things that gatekeeper I think, and again, correct me if I'm wrong does where we're opa by itself, or at least the first iteration of gatekeeper as part of opa did not do things like certain auditing things. nationality being able to validate or verify what resources are, or in violation of a policy. So it's more than just the admission aspect. You know, we've mentioned that, that opas is seen as more of like an admission controller. But is it safe to say that gatekeeper kind of brings opa further into the Kubernetes ecosystem? And then before? John Osborne 15:22 I'd say so it, it brings, I think the biggest thing that brings is the ability to parameterize and template templatized, all of these rego policies. So if you have a policy, for instance, say, No, don't run privileged containers, well, I cannot run templates against that's to say, don't just run that policy for this namespace. So this these types of namespaces and you can add kind of context and things around that. So you can see at scale where they could get more, you would really need those types of things. Most of the people I run into are still kind of running the version one which is a cube management aspect to it like like Jamie said, where you actually load all the regular config maps. And in version three with a gatekeeper, it's all around you take the rego you actually load it into CR DS that way and you have these kind of constraint templatized that you can instantiate or not instantiate, but there are other features as well, around auditing and other things. But there's still even with opa there's still some gaps, but there's still some things nuances to Kubernetes itself, that that you need to kind of be aware of as well. Jamie Duncan 16:26 I don't know if you, is there an example. John Osborne 16:28 Yeah, so let's say one example. That is, that's a pretty classic example. It's not specific to opa, but specific to Kubernetes is SC comp. So SC comp is something that you can set in a deployment. But if you schedule, if that gets, you know that that's part of a replica set, Nick, it's scheduled to a node that doesn't support SC comp, well then not only does it not do it, but it doesn't tell you it's not doing it. So it's one of those things where there's still things implemented on the note that you need to be aware of that you can't just magically set it in the YAML. Another thing that a problem that I was trying to solve, which is really cool around one of the things I really like about attending coop con versus just watching the YouTube videos is you can kind of talk to people after, after their sessions. And Goldman Sachs is actually trying to solve the same problem I am, which is this provisioning problem where anytime a namespace is created, I want a network policy that breaks the namespace so that people actually have to be thoughtful about whitelisting certain network paths in certain network policies. But there's nothing since it's an admission controller. There's nothing that's going to create additional resource objects. So one of the things that that Goldman Sachs did was they created a controller, which is written in go and then it pulls the different opa policies and you have an opa policy around. Well, I'm going to create our back I'm going to create quotas. I'm going to create persistent volumes. I'm going to create network policies. When the namespace is created, they said they're working on open sourcing, it'd be nice, nice if there was something open sourced already, but it was kind of cool to see that somebody else had kind of solved that problem at least to tie Jamie Duncan 18:12 off. We're talking about SATCOM a node, not having set comp enabled, because set comp is actually a component in the Linux kernel. So it's not something that's part of Kubernetes. So you could if you forgot to do that, when you booted up the server, you'd have to add that to your kernel boom on. It just wouldn't be there. And everything would just fail silently. Hmm. So that's a that's a really that's a that's a really cool part to look at. And second, so we've mentioned Rico several times. JOHN, again, you have the most experience with Okay, between us, between the three of us, I think, can you go in a little bit like what is Rico and sort of pros and cons is, it doesn't come from sort of a normal DevOps world is coming from a different part of it. And it feels weird when you start using it. John Osborne 19:01 I think that a goal for rego or for opa, which I think they've gone a long way with, if you look at all the examples, which we'll put in the show notes, is that hopefully they get to the point where 99% of the use cases they'll have out of the box we go policies for. So you don't need to have people that are gonna go really deep into this stuff. I think the type of organization or enterprise that would go really deep into these policies where they are creating like multisection policies that have super context awareness to them, are probably writing something custom anyway. So Rico is not going to be a huge lift for them over writing their custom policies in Python, for instance, like like Yelp is doing, or Goldman Sachs had something written custom and go before they moved to opa as well. It's pretty easy to understand. You can write these simple policies for something three, four lines of code and a lot of cases. I think there are nuances around it. One of the ones that I brought up with the gospel stare at qcon was if you look a lot of these samples are out there. They apply To a pod, if you open up and this is I hit this problem was if you open up the Kubernetes documentation, there's, I think at the time I looked, there was 16, places for a pod could be instantiated. You look at daemon sets, replica sets, jobs, and all these things deployments. And so you kind of need to build your own helper functions for that. And you also kind of have to have somebody that's aware of how this is evolving, for instance, with I think, 117 that just came out, there's ephemeral containers. So if you had a policy that was set for 116, you actually have would have to know that there's actually a new place that you have to go lock down that was 117, to now handle ephemeral containers as well. So you have to have somebody that's knows Kubernetes well enough that they're going to evolve the policy with the API. I think that's important. Jamie Duncan 20:53 Yeah, I think that's a really good point that even though this is a really powerful language, even though it helps solve Some really sticky real world problems. Nothing replaces having someone that goes out and keep track of the change logs in is a Kubernetes. Expert. That requirement doesn't go away. John Osborne 21:11 Tarik, I think like Google's The only Kubernetes vendor that's productized. This, aside from their sthira has some management tools on top of opa, which do solve problems to actually one of my most advanced customers is actually procuring the sthira management tool on top because one of the things I talked about was you need cluster admin to deploy these policies, right. So it'd be nice if we had some more fine grain, our back around who can deploy the policies. And so that's I think that's one of the things that the sthira tools do on top of on top of opa but I know that Google had anthos. I saw that they were prioritizing it. Do you want to maybe talk about some of the policies that you see or customer adoption in this area? I mean, it's still early days, and Tariq Islam 21:51 I actually haven't admittedly have a significant finger on the pulse for for gatekeeper as part of anthem But yeah, we are we are essentially packaging it up with, with our as part of the platform. So to your point earlier around things like examples having, you know, repository, for example policies and things like that. You know, that's that's all, that's all fair game. But as far as privatization and you know, getting that enterprise support and things like that, in between steer and anthos, and I imagine other vendors will pick this up in short order as well. I don't think we're going to be unique in this regard for very long. But I think that we've been putting a lot of time and effort into making sure that we can, we can bring this into the fold. At least that's, you know, that's where that's how we're approaching it. But again, this is not us taking a specific opinion on it one way or the other. This is just us, including it as a way for security teams, operations teams to be able to define their policies in the standard way. So everything that we've been talking about so far, you know, applies the, you know, my hang up right now is usability right rigo regular however we want to pronounce it. Is it John Osborne 23:04 rigo? It's I think they say rainbow. But either way, either way. Tariq Islam 23:08 Anyway, I think I think that's probably going to be a major usability gap. And in my personal opinion, I think folks are still wrangling a lot of YAML right now and then having to learn yet another thing might be an inhibiting factor as far as gatekeeper adoption goes, but, you know, I think there's going to be a race in terms of who makes it more usable, more consumable? John Osborne 23:29 Yeah, I think that and that's where I hope that, you know, we get to the point where I think that the fact that they have the right policies for everything you would get in PSP closes a big gap. Because, you know, hopefully someone wouldn't have to write a radio just to get something that was in a pod security policy, right. And then if they do need more advanced use cases, well, then they're probably gonna have to code something anyway. Right. And so hopefully, that won't be a heavy lift over what they probably be less of a less of a heavy lift versus, you know, writing something and go, Jamie Duncan 23:56 if I think about the way the teams that are going to be leveraging This technology should be put together. It doesn't bother me. Because rigo is based on a format called data log, which has been around for 20 or 30 years. And is very commonly found in sort of the infosec circles. So for the security people rhaego is not a jarring thing. And for me as an old as as primarily an ops person, there are some places when you're writing a policy for Kubernetes and rigo, where things are counterintuitive, where you know, true really equals false. Sometimes, if I pass all these tests, then I'm because I'm writing a deny policy, I have to pass the test to deny the request kind of thing. And that just feels weird until you get used to it. But from the security teams, they've been writing these things for a long time using other tools, and firewalls and things that I'm nowhere near an expert in from the aspect of developers and ops having to learn yet another language I agree completely. But for the idea of meeting the security people where they are, and bringing them into the team for their expertise in the language, I think it's kind of a genius stroke, that if we bring a team that is that often finds themselves on the cold edges. And we adopted language that they're comfortable with, to bring them closer to the center with Dev and Ops and make dev sec Ops, a little more realistic. I like that person. I think John Osborne 25:29 a lot of the learning curve or gotchas are also based around Kubernetes and not necessarily regular how you're interested in policy, and I mentioned the SD comp, example. But then the classic example of Ingress, you know, you can set multiple Ingress objects to have the same hostname. I don't know if many people know that until they actually start looking at opa. And then if you look at things like sto to like the same problem exists in sto you can create multiple virtual services that have the same UI, right and so you need policy like that. And then you kind of look at, I don't know, if you look at the Kubernetes security policy or the Kubernetes security on it. I don't know if any people knew before that audit, that if you created a policy that said, Don't allow access to the host names on the host file system, that that can be overridden by using host path and a persistent volume. I don't know if many people knew that right. And so but that's another kind of gotcha where there's, there's a lot of nuances to how Kubernetes is secured, that you have to know and learn. Jamie Duncan 26:33 And I think that that is just part of the learning curve outside of just rego. For me that learning curve was kind of staring at it for a couple of days to get passable. And then I would say a couple of weeks of leveraging it to feel comfortable. And I'm looking at a very specific use case of just Riding, riding writing opa policies in rigo for Kubernetes clusters. So in that use Case took me a day or two to be able to do it. And then a week or two to feel comfortable doing is, you know, from a year plus ago, do you feel like your learning curve was similar? You're a lot smarter than I am. So it's probably an hour for you. Let's, John Osborne 27:14 let's not go too far. So I, I have a really low attention span. So I have trouble reading documentation. So like, normally, the way I learn is just by doing and then I realized that once I hit a certain wall, we're actually have to do learn something that I hope I forced myself to go back and read it. But I basically had started with existing policies, and then just kind of modified them to fit my own policies. And then once they hit a wall with certain things, we're actually to understand the underlying fundamental concepts, no, went back and read documentation and read about data log and things like that. But you could modify an existing policy and it would make sense, I think, in you know, just without any, without any studying, but you know, eventually if you're going to do this and create, especially more context aware things and yeah, you'd probably take you about a week or so. Tariq Islam 27:58 That's actually not that bad. If we're talking about a week or two to ramp up on something like really go for something as powerful as as opa gaycupid, right? That's a good deal. Right? That's a really good deal in terms of investment in time and the return that you get off of that. Jamie Duncan 28:18 No, I agree completely, it's well worth the effort. Because that is from 100 different angles, just that idea of, here's a tool that I can write interactive policies against, that is aware that's able to, that has all the information in my Kubernetes API. It knows the namespaces are better deployed. It has all of the annotations attached to that namespace. It has all of the information it needs to make a really intelligent decision on whether or not something should happen in my cluster. It has where it's positioned as an admission controller. It has the ability to decide whether or not that happens. I think it's a it's a really powerful, useful tool. I do agree Like riego, like abstracting away rigo. And God forbid, we have another abstraction but abstracting away Griego a little. And I think that was one of the goals of gatekeeper that template ization, that kind of thing. Maybe we go a little easier to consume. I think that's a big goal. And I think that's a good direction for the project to go in. But it's Yeah, I think chart and gear you're dead on. But it's it's ROI is massive. I think one of the John Osborne 29:27 No. So I work in I cover most customers I cover in federal government. So a lot of it is air gapped. And I know turkey cover a lot of big banks and stuff too, that also do this have air gapped environments where they literally can't touch anything, once it's moved over into the air gapped network. And so I think one of the things that opa can also help out with before we kind of close this file is the fact that you can run. We talked a little bit this with a kid ops section last week as you can run this. With unit testing. You can run this in a CI CD pipeline. I think the context part will be hard to match production in a CI CD pipeline. But you can test the policies as they move through. Once it does get to prod, you can dry run and things like that to limit that blast radius, you can set things to warn, so you don't break your cluster. But having that ability to kind of lightweight ability to add something like this to the CI CD aspect to deployment pipeline is is pretty powerful. Tariq Islam 30:27 I agree with that. I mean, it's it's actually it's interesting that you mentioned the CI CD aspect, that's that's something that I had not considered for for, as far as you know, for the purposes of automated sanity checks. You know, I've always considered open and go cue cake. goalkeeper gatekeeper is something that would be more for for a production level environment or set of environments. But you're right, it absolutely makes sense for something conceptually, like unit testing, where your gut checking, you know, maybe it's a configuration change, right. You know, the whole episode that we had on get ops that up totally makes sense. Jamie Duncan 31:01 Just to be just to be clear, goalkeeper is a completely different project. It's also completely different. It's more for the cattle mentality. Just to run that horrible analogy back to life. Sorry, sorry, I couldn't resist all good. Yeah, so it's hard. We're, we're probably a half hour plus or minus and do you want to kind of bring us home with a little call to action? Maybe? Tariq Islam 31:29 Yeah. So we've covered opa. gatekeeper. And opa stands for open policy agent, basically, I think the call to action here is that this is something that the community is is coming around. companies are starting to commercialize it, package it up and ship it out as part of their, you know, respective Kubernetes platforms one by one. It's still pretty early. JOHN has been working with it for a little over a year now. And that's a pretty long time. Given the how long OVA has been around, he might actually be one of the most experienced peoples and people in the world with John Osborne 32:07 with soccer too far. Tariq Islam 32:10 But now just getting away from being superlative here I think I think the main call to action here is really take a hard look at how how you're doing security and how you're defining your policies and how you're auditing those policies and how you want to be able to do that in a cloud native environment. As you look to adopt Kubernetes and containers you know, the the industry around that ecosystem is evolving and it's again, it's coming around with Oh pa is coming around with with gatekeeper and and rego and figuring out how you can adopt that type of thing and how you work today and how you want to work tomorrow, I think is going to be action item number one, the P zero here would be you know, getting your hands on it. As john mentioned, rightfully so it probably will take you about a week or two of it. investment to get fairly okay with with with rego and and the tool chain associated with opa, but I think for me anyone I defer to john on this and I'd like him to chime in. I think that would be the the upfront the upfront call to action. John Osborne 33:19 Yeah, thanks. So we put we're putting some resources in the show notes on coop con talks and then link to the rego playground we can actually just get started with our ego policy and an admission review object and I'm also providing some links to the pod security policies that have been rewritten in rego and some other good resources around service mesh and other things with regular as well. Jamie Duncan 33:45 I think that'll close this out. Okay, is is a pretty awesome tool. I think we're going to see it around in a year. And that's always my, my baseline of whether or not I need to know something. Absolutely. I think this is easily Yeah, I think this is easily one that's going to get continue to grow. So thanks a lot for joining us on the K files and we look forward to talking to you next time. Thanks, everyone.