Thank you for the nice video! In Percona we have 4 database operators, 2 are tested with a custom framework written in Bash, and 2 are tested with Kuttl. Since Kuttl is almost dead we are currently considering either moving to Chainsaw or writing something in python or golang.
@TomislavPlavcic I would suggest giving Chsinsaw a try. It might fit your needs or it might not. If it doesn't, the question is whether it's easier to add that something to the project or start from scratch. More often than not, it is the former.
Thanks for another great video! I don't particularly like the approach of writing more and more yaml but it's definitively good news having a new player in the scene for kubernetes testing
It seems Chainsaw could be very useful for rapidly writing operator tests... Declare things that you want to create (or patch, delete etc...) in the cluster, assert the operator creates the CRDs that it manages... Unfortunately for what I'm working on right now, it does not seem to fulfill my needs... I'm working on a Controller that watches over Services with some specifics Annotations and makes requests to an external API... So the Controller does not make any changes to the cluster... I was looking for something that would ease the pain of having to manually create mocks everywhere for me to test the Reconcile loop but chainsaw does not seem to solve that
Chainsaw does not solve all that. You still need to test that your code (doesn't matter whether it is running in a controller or elsewhere) makes correct API calls. You probably do not need even to run it in kubernetes for that.
I just started to study Kubernetes, and I just stuck with my on site cluster. I need to make a network among few pcs across different places. On all pcs instances are running on multipass instances. How to connect those instances and make 1 cluster form multiple nodes? I can see that control plane receives node IP’s that are locally assigned by multipass instances? Thank you
Do you know any good tools for test data management? Usually its about data in databases but not only. Generating, subsetting, versioning, anonymizing production data. So far ive tested Redgate's tools and i have mixed opinion about them. Anonymizer and DB Clone are good but generator and subsetter are not. There is also Delphix but i dont know it. Are there any opensource solutions?
I guess that the answer will differ depending on ones use case. In my case, primary objective is to confirm that resource expansion works. If i create s Deployment, i want to confirm that it created a replicaset which created pods which requested storage, etc.
@@DevOpsToolkit here's what I have trouble understanding: so the deployment resource expansion (replicaset, pods etc.) is k8s functionality right? aim should be testing your own cluster configuration, correct? so assertion in tests should be covering cluster specifics? e.g. a negative case would be: we have a deployment which is configured with data that is not viable to our cluster configuration (e.g. user has no rights to create resources in a particular namespace), hence it should not expand resources.
@mmihailicenko normally, i would test that the policías ir rbac that' prevent user from doing what should not be done is indeed there. You can also take it the other way around and test that the resource user tried to create was not created. P.S. Plesse let me know if i misunderstood your question.
@@DevOpsToolkit almost, (sorry for a bit misleading example) more interested in k8s test coverage concept. When you have to cover cluster with tests from scratch how do you determine what to automate?
@mmihailicenko i rarely think of a cluster as a whole but of each individual project. The examples you saw in the video are like that'. They are from a Crossplane project so I'm ensuring that all the resources that composition males are done correctly.
We don't test our manifests, but we want. But this is not enough for us. We don't want to create real database. What should we test? Only the manifest applied? I think we need to precondition step where we install our crds etc. Testing manifests is really difficult and I cannot imagine a good way here
You can apply anything, including CRDs. Whether you'll test against a real database or not depends. It's the same dillema people have with any other type of testing. Sometimes you want to mock or simulate dependencies and at other times you want the "real deal". Ideally, you'll have both and use mock or simulations while developing so that it is very fast and run a full suite through CI pipelines. I tend to use Chainsaw for testing my Crossplane compositions. While writing them, I avoid creating configurations so that Crossplane cannot create "real" resources like databases. That allows me to practive TDD since tests take only a few seconds (unless there is an assert failure). Once I'm done I push my changes to Git, that initiates GitHub Actions build during which I am testing using "real" resources in AWS, GCP, and Azure. That takes quite some time, but that's okay since those tests rarely fail so I don't need to wait for them. Most of the issues are encountered locally while writing code.
chainsaw seems nice, but to be honest the example presented is so complex and tightly coupled to CrossPlane which becomes not understandable :( it would have been better a simple but complete deployment (configmap, secret, deployment, maybe pvc, service, pdb)... CrossPlane is nice, but it cannot be used as example for everything! Especially if someone doesn't know its syntax like me... Here the focus should be how chainsaw works, not the tons of CrossPlane yaml manifests in another format...
I used Crossplane compositions as an example simply because that's one of the projects i use chainsaw. Still, you're right. I should have created a demo repo.
Do you test your Kubernetes manifests/resources? If you do, what do you use?
Thank you for the nice video! In Percona we have 4 database operators, 2 are tested with a custom framework written in Bash, and 2 are tested with Kuttl. Since Kuttl is almost dead we are currently considering either moving to Chainsaw or writing something in python or golang.
@TomislavPlavcic I would suggest giving Chsinsaw a try. It might fit your needs or it might not. If it doesn't, the question is whether it's easier to add that something to the project or start from scratch. More often than not, it is the former.
Thanks for another great video! I don't particularly like the approach of writing more and more yaml but it's definitively good news having a new player in the scene for kubernetes testing
An awesome video like usual ❤
It seems Chainsaw could be very useful for rapidly writing operator tests... Declare things that you want to create (or patch, delete etc...) in the cluster, assert the operator creates the CRDs that it manages...
Unfortunately for what I'm working on right now, it does not seem to fulfill my needs... I'm working on a Controller that watches over Services with some specifics Annotations and makes requests to an external API... So the Controller does not make any changes to the cluster... I was looking for something that would ease the pain of having to manually create mocks everywhere for me to test the Reconcile loop but chainsaw does not seem to solve that
Chainsaw does not solve all that. You still need to test that your code (doesn't matter whether it is running in a controller or elsewhere) makes correct API calls. You probably do not need even to run it in kubernetes for that.
Anther Awesome one !!!
I just started to study Kubernetes, and I just stuck with my on site cluster. I need to make a network among few pcs across different places. On all pcs instances are running on multipass instances. How to connect those instances and make 1 cluster form multiple nodes?
I can see that control plane receives node IP’s that are locally assigned by multipass instances? Thank you
Unfortunately, I work mostly with managed-Kubernetes (e.g., GKE, EKS, AKS, etc.) so I might not be the right person to help with that one.
Do you know any good tools for test data management? Usually its about data in databases but not only. Generating, subsetting, versioning, anonymizing production data. So far ive tested Redgate's tools and i have mixed opinion about them. Anonymizer and DB Clone are good but generator and subsetter are not. There is also Delphix but i dont know it. Are there any opensource solutions?
Unfortunately, that's one of the areas I never got into deep so I can't recommend anything :(
How do you determine testing scope? How to test your k8s cluster specifics and not k8s functionality?
I guess that the answer will differ depending on ones use case. In my case, primary objective is to confirm that resource expansion works. If i create s Deployment, i want to confirm that it created a replicaset which created pods which requested storage, etc.
@@DevOpsToolkit here's what I have trouble understanding: so the deployment resource expansion (replicaset, pods etc.) is k8s functionality right? aim should be testing your own cluster configuration, correct? so assertion in tests should be covering cluster specifics? e.g. a negative case would be: we have a deployment which is configured with data that is not viable to our cluster configuration (e.g. user has no rights to create resources in a particular namespace), hence it should not expand resources.
@mmihailicenko normally, i would test that the policías ir rbac that' prevent user from doing what should not be done is indeed there. You can also take it the other way around and test that the resource user tried to create was not created.
P.S. Plesse let me know if i misunderstood your question.
@@DevOpsToolkit almost, (sorry for a bit misleading example) more interested in k8s test coverage concept. When you have to cover cluster with tests from scratch how do you determine what to automate?
@mmihailicenko i rarely think of a cluster as a whole but of each individual project. The examples you saw in the video are like that'. They are from a Crossplane project so I'm ensuring that all the resources that composition males are done correctly.
We don't test our manifests, but we want. But this is not enough for us. We don't want to create real database. What should we test? Only the manifest applied? I think we need to precondition step where we install our crds etc. Testing manifests is really difficult and I cannot imagine a good way here
You can apply anything, including CRDs.
Whether you'll test against a real database or not depends. It's the same dillema people have with any other type of testing. Sometimes you want to mock or simulate dependencies and at other times you want the "real deal". Ideally, you'll have both and use mock or simulations while developing so that it is very fast and run a full suite through CI pipelines.
I tend to use Chainsaw for testing my Crossplane compositions. While writing them, I avoid creating configurations so that Crossplane cannot create "real" resources like databases. That allows me to practive TDD since tests take only a few seconds (unless there is an assert failure). Once I'm done I push my changes to Git, that initiates GitHub Actions build during which I am testing using "real" resources in AWS, GCP, and Azure. That takes quite some time, but that's okay since those tests rarely fail so I don't need to wait for them. Most of the issues are encountered locally while writing code.
chainsaw seems nice, but to be honest the example presented is so complex and tightly coupled to CrossPlane which becomes not understandable :( it would have been better a simple but complete deployment (configmap, secret, deployment, maybe pvc, service, pdb)...
CrossPlane is nice, but it cannot be used as example for everything! Especially if someone doesn't know its syntax like me... Here the focus should be how chainsaw works, not the tons of CrossPlane yaml manifests in another format...
I used Crossplane compositions as an example simply because that's one of the projects i use chainsaw. Still, you're right. I should have created a demo repo.