Hello Rahul, I am getting 'No suitable test runtime provider was found for any source in this run' error in vs test and The process 'C:\AzureAgent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.215.0\Modules\DTAExecutionHost.exe' failed with exit code 1
Hey Rahul - thanks for the video. Very useful. I'm new to the Azure DevOps environment and was looking around for how to report the Code Coverage test results. Learnt so much more than just that from your video.
Hi Rahul! Nice video. It's very helpful. Could you give me some advice about running automated tests against REST API in a build pipeline? Thanks in advance.
Glad it helped Victor. You can follow different method to test API endpoints. Here is an example where I use Specflow to test API endpoints. www.rahulpnath.com/blog/specflow-and-autofixture/ You can also write execute xunit tests against a deployed environment or even use postman scripts.www.rahulpnath.com/blog/automated-api-testing-using-postman-collection-runner/ I will create a video on this sometime soon. Feel free to ask if you have more questions.
haha fair point Chetan! I am stilling learning the art of talking to a camera, think the more recent one's are slightly better. But I am planning to spend some time to learn this art better. Glad you are liking the content though.
@@RahulNath Yea, I am no expert at talking to a camera, but perhaps you need to relax a little bit, keep it light! :) Yes, I am watching some of your vids... they are good!
@@sandeepnegi4939 Thank you for the idea and suggestion Sandeep. I will try and do this. Between I have a blog where I some times write about my goals, processes, etc as well if that is of any help. rahulpnath.com/
Hello Rahul amazing video but had one doubt like In Azure DevOps how do i extract the pipeline status for each stage or task and forward each pipeline stage status to google chat hangout ?
Really help this content. try to use Dark themes for you content like Vscode, Azure devops, Azure portal, Visual studio etc. Generally IT professionals used watch video at night.
Glad you like it and thank you for the suggestion. I haven't heard this from anyone else. Maybe I'll put a poll on the channel to see what others think. I don't have a strong preference with it anyways. (And I record videos early in the morning, when it's dark as well 😀)
Hi Rahul, Your Course Content was very good and really O understood your concept. But in VS test report I found an error "Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again." So in the Build Console I can found the "Test" tab, found but the test cases was Zero. Please help me on this
Can you confirm in the logs if all the tests are run successfully? Hard to tell what the issue would be without seeing, but if it's 0 tests mostly it's because it's looking into a wrong folder or convention to run the tests. Hope you sort out the issue!
Hi @Rahul, Thanks for a helpful video. I am using Selenium c# to develop my scripts. Is there any way to pass execution domain/ url as variable/ parameter in Azure pipeline rather than committing my code( change execution domain) everytime I wanted to run automation tests in staging/ testing enviornment?
Glad it helped Yedidi. Yes you can have them as part of Release Variables and replace the test configuration file. More details on Release Variables here ruclips.net/video/OFr6Vkw0mq8/видео.html Let me know if you have additional questions.
Hi @Rahul Nath, can u help me for this issue. For my test data excel i can keep in my local and that excal path i given in code. It is possible to keep test data excel in external storage like azure and how we give that path in our code.
@@RahulNath thank you so much for hands on explanation. Can you please explain sample end to end devops CICD from build and deployment to different environments and how we integrate sonar or any tools in CI. Is there any particular yml format we need to use for different types of applications like react js, .net, python etc. Please make a detailed video on explaining yml syntax, why we use only that particular job or step etc. I saw which you explained on syntax already but I am confused in few steps. I never found any videos like yours it is really helping me alot. All the best and I will share it to all my friends.
Hi Rahul, Thanks for the nice video. We are adding our c# tests on Azure pipeline. Could you pls suggest few reports that we can use? Thanks in advance.
Thank you - What kind of information are you looking to get from the reports? Mostly depends on that and looking around for the report formatters available. Most common ones do give the info on pass/failed etc.
Hi Rahul, Thank you for your amazing videos. We use Azure Dev Ops for running automation tests. Recently some of the tests are throwing 'This task has timed out' error before it execute all tests in that suite. Do you know why it is happening? I would appreciate if you response. Thanks in advance.
@@RahulNath Thanks for your response. I really appreciate it. Timeout is set for 90 mins and it is running for 1 hour and 30 mins. Also when I looked at the logs it is skipping some of the tests in between or not executing all of the tests under the suite.
@@RahulNath I tried with increasing timeout other day but still same result. I will try one more time tomorrow for different test suite and let you know. Also I have noticed that the same tests pass on my laptop but they fail in pipeline. What are your thoughts on this? I am so sorry for all these questions. I really appreciate your help 🙂 Thanks in advance
Let me know how that goes. I would look at the logs and make sure if there are test data set up issues etc. Hard to tell why it would fail on server but pass on local. Hope you are able to resolve the issue.
Hi Rahul, My Junit test codes are available inside the package of services/src/test/java/org/ssp/tod/mol/controllers. So can u pls guide me how to create testAssemblyVer2 for this path
Glad you like it Rajkumar. Thank you for the suggestion. What kind of integration tests are you running? connecting to a database? Let me know more details and I will try and do a follow up video.
Hi Rahul. Actually I am assigned to automate dynamics 365 crm online application. I m planning to do it with selenium with JAVA. Pls give some valuable suggestions
I've mostly been using Cypress for UI tests www.rahulpnath.com/blog/category/cypress/. Haven't been doing much of Selenium nor JAVA, so can't be of much help here.
Hey Rahul, Nice video. Can you please let me know if we can publish Cucumber reports to Azure Pipelines? If Yes, please let me know for procedure to publish Cucumber reports
Glad you liked the video, Ravikanth. You can publish cucumber reports to pipeline. (I haven't done this myself though). A quick google search shows a plugin for that (marketplace.visualstudio.com/items?itemName=MaciejMaciejewski.azure-pipelines-cucumber). Let know if you are facing any specific issues while doing that.
Hi Rahul, thanks for the video, it's very helpful ,can you help me to solve a problem , how can I configure the pipeline agent to find the path of my test project, when I try to create the pull request it throws the error MSB3202 ( Error MSB3202: The project file "E:\agent\_work\31\myapp.Tests\myapp.Tests.csproj" was not found)
Possibly you have the project file in a different path. If you check the checkout step, you can see the full path where the repository is getting checked out. You can update the scripts to make sure it uses the correct path to find the csproj file
Thank you for the suggestion. I will add it to my list. I have a post on SpecFlow on generating data here if that interests you. www.rahulpnath.com/blog/specflow-and-autofixture/
@@RahulNath Thank you, i looked at your page but can you please help me to understand how TestContextFixture was used in the stepdef class . i added Bodi from nuget package but still it is undetected .Is there any way without using TestContextFixure i can use specflow?
I am using Jasmine standalone package i.e. not installed it via npm (because of proxy i am unable to install it) Is there any way by which we can run the tests in azure devops pipeline? I tried installing karma but having same proxy issue Please guide
Easiest way I can think of is to include those packages as part of your source code and use it from there. Between what proxy issue are you facing? Is the build agent self hosted?
@@RahulNath I will try give that a try and get back to you, if I get stuck. Right now on my local machine at work we are directly using the browser to show execute and check the test results (as its all client side). Is karma required to run these tests in azure devops pipeline? Sorry for sounding so dumb
@@adiety512 To run them automatically I guess you would need karma. You could use an export plugin for karma that will output the test results in the way you want. Check out this article (from a quick google, you should be able to find similar ones) tharaka-madhusanka.medium.com/publish-execute-angular-unit-tests-in-azure-devops-build-pipeline-with-code-test-unit-test-c32288d59e8d Let me know how that goes and if you have further questions (no such thing as a dumb question!).
@@RahulNath hello sir sorry for late reply I find error it's actually bug from api side azure devops so developers fixed that issue and solved . Many thanks for your support ☺️
Hi rahul, i have a issue, i wanted to run my automation selenium regression test through azure pipeline and when i am triggered pipeline every prerequisite is pass but at regression test its fails, so total 2025 test cases were there, and non of the test cases is pass, but issue is it is not saving test result in c drive and also not sending any test report in email, i am stuck in this situation since 1 month but getting the solution can you help me out
Hi Rahul, thanks for the video. When I run the test cases in the Azure devops there are duplicate test cases are running i.e same test case running twice and one test case is failing from that. Please let me know if you have any idea. Thanks.
what script/command are you using to run the tests? does this happen on your local machine as well? Why would the test fail when run twice? Can you set up a code repro/gist and I can take a look
@@lakkampellisaikrishna6384 Hard to tell without seeing. Do you have a sample repo that you can share? What is the test command that runs? Is it picking up multiple similar dll's? You can try restricting it to specific dlls in specific bin/release folder.
There is an another issue causing test cases to fail. We have stored the connection string in the key vault as secret. In the local machine I have installed certificate so it was able to retrieve the connection string. But in the build pipeline the test cases are failing due to the no connection string is retrieved from the key vault. Do I need to install the certificate in the build server also? Could you please let me know how to handle this?
Hi Rahul -- thank you so much for your wonderful video. I 'm a newbie in Visual studio 2019 , I just added a new test case to the my test suite, but Test Explorer do not discover it locally and even in my azure pipeline. any idea from you willbe very appreciated !
It is most likely missing NuGet packages for your test framework. For xUnit I have added 'xunit.runner.visualstudio' and 'Microsoft.NET.Test.Sdk' (rahulpnath.visualstudio.com/RUclips%20Samples/_git/MyApp?path=%2FMyApp.Tests%2FMyApp.Tests.csproj) This will be different based on the test framework used and you will need to check their documentation. For xUnit it is available here - xunit.net/docs/getting-started/netfx/visual-studio#run-tests-visualstudio Let me know if that helps.
@@RahulNath Thanks for replying to my question. I'm using Mstest Framework, It seems It seems that I have the right Item group. for now I have only 1 test case which test Explorer discover. However , if I write a new test case inside my project , it would be not the case. thanks Here is my packages reference :
all runtime; build; native; contentfiles; analyzers; buildtransitive
@@qa_engineer2522 Google might be able to help here better and faster. Tests not appearing in Test Explorer is usually a config/missing nuget issue. Not sure what exactly is causing the issue here. I have not used much of Mstest Framework. if you have a public repo for this, I can take a look if you share the link.
You could use variables and get it from there . When you run the tests from command line you can specific the browser based on the tool you are using for the tests.
hi Rahul! do you know, is it possible to have any output in "Result details" if test passed? If test passed- its empty even if I put any "prints" in the middle of the test
Hi my vstest are not passing. My doubts 1)In the build artifact's...will the testing files should be present 2)I am getting an error as:Execute login failure 3)when I am trying to check the test results (which are not passing)it is open as an attachment in visual studio says: couldn't find the file version: culture:...... Can you please help me @Rahul nath
Hey Sai, The artifacts will have the testing files if you have packaged it. You don't need to if you are running the tests in the build pipeline. You need a visual studio premium version to view the files from what I last remember. There is also an associated issue here on it not getting displayed in devops github.com/Microsoft/azure-pipelines-tasks/issues/8749 You could try opening the file in a text editor. It should be in an XML format. Let me know if you have additional questions.
@@RahulNathThank you so much Rahul..I am stuck with this work since two days...can you please help My project is .net project where test were later pushed into repo Yes...In the build pipeline no test are being artifacted and when i am using test assemblies in vs test task ... files .*DLL are not getting passed 2)tests are not published 3)I should try to connect visual studio from local machine into azure pipeline (if so how can I connect )
@@susus999 I dont think you can connect to DevOps agent from Visual Studio and you wouldn't need it. Make sure you are building the test project and using the correct filter to run the test. If you watch the video I use the pattern with Test in the name to filter out only test assemblies. If you look at the build log you should be able to see these dlls generated as well. What error are you facing when you run the test task? Is it that it is unable to find the dll's - then make sure you are using the correct pattern
I don't have anything specific. This should be a good start marketplace.visualstudio.com/items?itemName=MaciejMaciejewski.azure-pipelines-protractor You should be able to configure the test reports similar to the ones in the video here. Let know if you have any additional questions.
I have written jest automated tests. I am not sure what I am doing wrong. The paths are correct, the commands are the same. I really have no idea what have I missed
@@RahulNath Consider it like under my tests I am navigating to a URL. Can I configure that URL into azure variables such that if I change URL over devops variable my tests follows that changes URL. If yes HOW?
@@pankajsainani8226 You can define the variables as part of the build pipeline and add a file transform task to replace your config file, after checking out the source code. Here is a similar video on using variables in the release pipeline ruclips.net/video/OFr6Vkw0mq8/видео.html If these are not sensitive information you can keep them as part of the test config in source control itself. If you are facing issues, set up a public repo/build and share me the link and I can take a look. Let me know if that helps.
@@RahulNath Hi Rahul, actually it is resolved now. pipeline was failing because it was taking wrong path for the pictures. So I changed the path of pictures according to the path of my repo.
Hi Rahul,When I downloaded the code coverage file from Azure Devops Board.I am unable to view that file.Getting error while opening that file.any suggestions?
You need Visual Studio Premium to view the reports. ruclips.net/video/FFxww1-M25E/видео.html Also an existing issue related to it not showing up on the UI in DevOps - github.com/Microsoft/azure-pipelines-tasks/issues/8749
[16:33:12] E/runner - Unable to start a WebDriver session. [16:33:12] E/launcher - Error: Error: Server terminated early with status 127 at /azp/agent/_work/1/s/node_modules/selenium-webdriver/remote/index.js:252:52 at processTicksAndRejections (internal/process/task_queues.js:97:5) [16:33:12] E/launcher - Process exited with error code 100 Agent machine is ubuntu,Igetting this error while running protractor test through azure pipeline
Hello Rahul, I am getting 'No suitable test runtime provider was found for any source in this run' error in vs test and The process 'C:\AzureAgent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.215.0\Modules\DTAExecutionHost.exe' failed with exit code 1. Any solution to this above error
@@RahulNath no when test case failed build should be successful but in realase it's should ask for approval to deploy if test case gets passed it's should not ask for approval
@@SpiritualAdda00 You will need to write some custom scripts to achieve this. Have you seen the available approval flows in devops? learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass&WT.mc_id=AZ-MVP-5003875
Thanks Paras - Hard to tell what the issue will be looking at the yml file. Do you have a public url of the build pipeline where this issue is happening, that way I can take a look at the logs.
@@RahulNath Hey thanks for replying and sorry for late reply from my side. As public url is not enabled but as you mentioned about logs,I have seen and observed that npm test is not running inside 'workingDir' i.e client folder(package.json of front-end) so finally decide to run that command from server end (write it inside the package.json of nodejs) which solves my problem. At last once again thanks for creating such types of videos....
You might need to use script to read the percentage etc. But I wouldn't. How can you tell the other 5% is not the most important parts of your application. Why would you do that? Anything I am missing?
OfCourse and good to hear from you! It's the Windows Terminal. Check out the link here for details www.rahulpnath.com/blog/setting_up_windows_terminal/
You need Visual Studio Premium to view the reports. (18:24) Also this issue might help get more details - github.com/Microsoft/azure-pipelines-tasks/issues/8749 Let know if that helps.
@@kishoreyogeshwaran224 are you looking to view the results? Not sure why that’s related with the classic editor. Sorry if I misunderstood the question. CN you clarify?
@@RahulNath i am generating html from command line in devops and i am looking to view that html .i used classic editor instead of yaml for generating this html.
I couldn't focus because of the excessive editing. In the beginning, it is ok to show your self, but throughout the video I got destructed because you paused yourself taking too many times. Also, You sounded like a robot to me because of your editings. I'm giving feedback so that you can correct your future videos.
Hey Sunny, Thank you for your feedback. Based on other similar feedback I have reduced my video pip size in my newer videos. (Let know what you feel about them). Regarding the editing, yes I am still learning the tricks of the trade and slowly trying to improve on it with each video. If there are any specific areas of the video that is unclear or you need help with, feel free to reach out. Thanks again for the feedback!.
Sir you going to fast and do not show exactly what are you doing, you can also reduce the picture of yourself, it is not good to see you all the time, plus we can not see what you doing in the screen. We like to see more screen and less you, do not take it personally. Now the content is good but go slow and pointing out what you are doing in the screen.
Thank you for the valuable feedback Cesar and agree with you. I have reduced my picture size in the future videos. This was early when I started and was still learning how to make videos. Hope you find the newer videos better. Are there any places that found too hard to follow? I can try making other videos to explain those areas separately
Source Code - rahulpnath.visualstudio.com/RUclips%20Samples/_git/MyApp
Build Pipeline - rahulpnath.visualstudio.com/RUclips%20Samples/_build?definitionId=23
Can you make a tutorial how to add attachment for fail tests here?
@@gitakirshan5818 sorry Gita, just seeing this now. Can you provide more details on what exactly you are looking for (if you still are)
Hi Rahul , Can you please create a video on automate test cases azure devops with postman
Hello Rahul, I am getting 'No suitable test runtime provider was found for any source in this run' error in vs test
and The process 'C:\AzureAgent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.215.0\Modules\DTAExecutionHost.exe' failed with exit code 1
Very rarely do content creators be so comprehensive in their videos. Do keep up the good work!
Glad you think so Sojan!
Really helped me a lot! Thanks man. React tests were being a pain until I saw this video. Thank you.
Glad to hear it!
Hey Rahul - thanks for the video. Very useful. I'm new to the Azure DevOps environment and was looking around for how to report the Code Coverage test results. Learnt so much more than just that from your video.
Thank you Nigel and happy that you found it useful!
Great Job! This is expained very nicely. Exactly what I was looking for👍🏻
Glad it was helpful Ritu!
Thank you Rahul. This video helped me to quickly set up testing.
Glad you were able to Vikas!
Hi Rahul, I would like to thank you for doing this tutorial. This is a very rare and useful content.
Glad you liked it!
Thanks for the informative and good explanation video
Glad you liked it!
Excellent video, it gives me a good idea of what we can implement in our team
Glad it was helpful! Feel free to reach out if you have specific questions.
@@RahulNath Will do, thank you sir
@@mzamomahaeng268 And you can call me Rahul 😀
This is a really good video on Azure DevOps & Testing. Thank you very much.
Glad it was helpful Burak!
This is good. You covered lot of useful and practical scenarios. Helped a lot. Thank you
Glad to hear that! Thank you
One of the best video
Glad it helped Bhanu!
Add video of rerun failed test cases alone using vst failed rerun or using any option
Great, comprehensive video. Have you managed to populate automated test in azure test plan? Is it possible at all?
This is a great video, thank you for the content.
Glad you enjoyed it Matt!
Hi Rahul! Nice video. It's very helpful. Could you give me some advice about running automated tests against REST API in a build pipeline? Thanks in advance.
Glad it helped Victor. You can follow different method to test API endpoints. Here is an example where I use Specflow to test API endpoints.
www.rahulpnath.com/blog/specflow-and-autofixture/
You can also write execute xunit tests against a deployed environment or even use postman scripts.www.rahulpnath.com/blog/automated-api-testing-using-postman-collection-runner/
I will create a video on this sometime soon. Feel free to ask if you have more questions.
Cant look at you looking back! When you meet people, do you look into their face the same way...?
Anyways, the content is good/useful!
haha fair point Chetan! I am stilling learning the art of talking to a camera, think the more recent one's are slightly better. But I am planning to spend some time to learn this art better. Glad you are liking the content though.
@@RahulNath Yea, I am no expert at talking to a camera, but perhaps you need to relax a little bit, keep it light! :)
Yes, I am watching some of your vids... they are good!
@@chetanesque158 Thank you - yes will try and relax more 👍
your video are so informative thank you so much for such videos
Thank you Sandeep and happy that they are helpful!
@@RahulNath hey man please create some videos about yourself what you do and what are the challenges you face as a dev in day to day life
@@sandeepnegi4939 Thank you for the idea and suggestion Sandeep. I will try and do this. Between I have a blog where I some times write about my goals, processes, etc as well if that is of any help. rahulpnath.com/
Thank you so much...this helped me a lot
Thank you for the comment Namrata and happy it helped!
Hello Rahul amazing video but had one doubt like
In Azure DevOps how do i extract the pipeline status for each stage or task and forward each pipeline stage status to google chat hangout ?
Really help this content. try to use Dark themes for you content like Vscode, Azure devops, Azure portal, Visual studio etc. Generally IT professionals used watch video at night.
Glad you like it and thank you for the suggestion. I haven't heard this from anyone else. Maybe I'll put a poll on the channel to see what others think. I don't have a strong preference with it anyways. (And I record videos early in the morning, when it's dark as well 😀)
Hi Rahul, Your Course Content was very good and really O understood your concept.
But in VS test report I found an error "Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again." So in the Build Console I can found the "Test" tab, found but the test cases was Zero. Please help me on this
Can you confirm in the logs if all the tests are run successfully? Hard to tell what the issue would be without seeing, but if it's 0 tests mostly it's because it's looking into a wrong folder or convention to run the tests. Hope you sort out the issue!
Superb video bro. Keep posting.
Glad you liked it Venkatesh!
how we can do same thing if the build repo is in github and pipeline in github action
Hi @Rahul, Thanks for a helpful video. I am using Selenium c# to develop my scripts. Is there any way to pass execution domain/ url as variable/ parameter in Azure pipeline rather than committing my code( change execution domain) everytime I wanted to run automation tests in staging/ testing enviornment?
Glad it helped Yedidi. Yes you can have them as part of Release Variables and replace the test configuration file. More details on Release Variables here ruclips.net/video/OFr6Vkw0mq8/видео.html
Let me know if you have additional questions.
@@RahulNath thankyou Rahul..! Will try this
@@yedidimounika9824 Do let know how it goes and if you have any further questions
Hi @Rahul Nath, can u help me for this issue. For my test data excel i can keep in my local and that excal path i given in code. It is possible to keep test data excel in external storage like azure and how we give that path in our code.
Please do more such videos
Thank you! Any thing in particular you are looking for?
@@RahulNath thank you so much for hands on explanation. Can you please explain sample end to end devops CICD from build and deployment to different environments and how we integrate sonar or any tools in CI. Is there any particular yml format we need to use for different types of applications like react js, .net, python etc. Please make a detailed video on explaining yml syntax, why we use only that particular job or step etc. I saw which you explained on syntax already but I am confused in few steps. I never found any videos like yours it is really helping me alot. All the best and I will share it to all my friends.
Hi Rahul, Thanks for the nice video. We are adding our c# tests on Azure pipeline. Could you pls suggest few reports that we can use? Thanks in advance.
Thank you - What kind of information are you looking to get from the reports? Mostly depends on that and looking around for the report formatters available. Most common ones do give the info on pass/failed etc.
Hi, does VSTest work for azure legacy pipelines. Also can VSTest make network calls/Http endpoint calls, as I wanted to use them for Integration test.
Hi Rahul, Thank you for your amazing videos. We use Azure Dev Ops for running automation tests. Recently some of the tests are throwing 'This task has timed out' error before it execute all tests in that suite. Do you know why it is happening? I would appreciate if you response. Thanks in advance.
Thank you Swati. How long after is it timing out? Is is crosing the task/jobs allowed timeout duration?
@@RahulNath Thanks for your response. I really appreciate it.
Timeout is set for 90 mins and it is running for 1 hour and 30 mins. Also when I looked at the logs it is skipping some of the tests in between or not executing all of the tests under the suite.
So looks like it is exceeding the timeout then ? Might be worth looking at increasing the time out or run smaller batches of tests if possible.
@@RahulNath I tried with increasing timeout other day but still same result. I will try one more time tomorrow for different test suite and let you know.
Also I have noticed that the same tests pass on my laptop but they fail in pipeline. What are your thoughts on this? I am so sorry for all these questions. I really appreciate your help 🙂
Thanks in advance
Let me know how that goes. I would look at the logs and make sure if there are test data set up issues etc. Hard to tell why it would fail on server but pass on local. Hope you are able to resolve the issue.
Hi Rahul, My Junit test codes are available inside the package of services/src/test/java/org/ssp/tod/mol/controllers. So can u pls guide me how to create testAssemblyVer2 for this path
Hey Rahul, Great Quality Content.
Can you help, how to Configure Integration Testing & Regression testing in Azure pipelines
Glad you like it Rajkumar. Thank you for the suggestion. What kind of integration tests are you running? connecting to a database? Let me know more details and I will try and do a follow up video.
@@RahulNath I only need basic hands-on, how to do, what is this regression, integration,sanity testing, how to co-ordinate with QA team
Hi Rahul. Actually I am assigned to automate dynamics 365 crm online application. I m planning to do it with selenium with JAVA. Pls give some valuable suggestions
I've mostly been using Cypress for UI tests www.rahulpnath.com/blog/category/cypress/. Haven't been doing much of Selenium nor JAVA, so can't be of much help here.
Hey Rahul, Nice video. Can you please let me know if we can publish Cucumber reports to Azure Pipelines? If Yes, please let me know for procedure to publish Cucumber reports
Glad you liked the video, Ravikanth. You can publish cucumber reports to pipeline. (I haven't done this myself though). A quick google search shows a plugin for that (marketplace.visualstudio.com/items?itemName=MaciejMaciejewski.azure-pipelines-cucumber). Let know if you are facing any specific issues while doing that.
Hi Rahul, thanks for the video, it's very helpful ,can you help me to solve a problem , how can I configure the pipeline agent to find the path of my test project, when I try to create the pull request it throws the error MSB3202 ( Error MSB3202: The project file "E:\agent\_work\31\myapp.Tests\myapp.Tests.csproj" was not found)
Possibly you have the project file in a different path. If you check the checkout step, you can see the full path where the repository is getting checked out. You can update the scripts to make sure it uses the correct path to find the csproj file
Hi Rahul, can you make a video on xunit specflow Integration test and how we manage that in azure pipeline?
Thank you for the suggestion. I will add it to my list. I have a post on SpecFlow on generating data here if that interests you. www.rahulpnath.com/blog/specflow-and-autofixture/
@@RahulNath Thank you, i looked at your page but can you please help me to understand how TestContextFixture was used in the stepdef class . i added Bodi from nuget package but still it is undetected .Is there any way without using TestContextFixure i can use specflow?
How to run unit test for react and node js application
Really great
Thank you Sneha, happy you liked it!
I am using Jasmine standalone package i.e. not installed it via npm (because of proxy i am unable to install it) Is there any way by which we can run the tests in azure devops pipeline? I tried installing karma but having same proxy issue
Please guide
Easiest way I can think of is to include those packages as part of your source code and use it from there. Between what proxy issue are you facing? Is the build agent self hosted?
@@RahulNath I will try give that a try and get back to you, if I get stuck. Right now on my local machine at work we are directly using the browser to show execute and check the test results (as its all client side). Is karma required to run these tests in azure devops pipeline? Sorry for sounding so dumb
@@adiety512 To run them automatically I guess you would need karma. You could use an export plugin for karma that will output the test results in the way you want. Check out this article (from a quick google, you should be able to find similar ones) tharaka-madhusanka.medium.com/publish-execute-angular-unit-tests-in-azure-devops-build-pipeline-with-code-test-unit-test-c32288d59e8d
Let me know how that goes and if you have further questions (no such thing as a dumb question!).
@@RahulNath thank you so much, I will try it out
Could you also please explain how to send custom test results via email
Hi @Rahul Nath I want to add videos of test runs in each test attachments ... How to do it???
You will have to write a script if all it's possible. I haven't done this. Cant be of much help, Shahid.
Hello sir,
In azure devops i face error 500 internal server when build pipeline.
Hey Riten, Very hard to tell what the issue might be - You will need to check the logs to dig more into it and find what the 500 error is about.
@@RahulNath hello sir sorry for late reply
I find error it's actually bug from api side azure devops so developers fixed that issue and solved .
Many thanks for your support ☺️
Hi rahul, i have a issue, i wanted to run my automation selenium regression test through azure pipeline and when i am triggered pipeline every prerequisite is pass but at regression test its fails, so total 2025 test cases were there, and non of the test cases is pass, but issue is it is not saving test result in c drive and also not sending any test report in email, i am stuck in this situation since 1 month but getting the solution can you help me out
Hi Rahul, thanks for the video. When I run the test cases in the Azure devops there are duplicate test cases are running i.e same test case running twice and one test case is failing from that. Please let me know if you have any idea. Thanks.
what script/command are you using to run the tests? does this happen on your local machine as well? Why would the test fail when run twice? Can you set up a code repro/gist and I can take a look
This is not happening in VS test explorer but only happening in azure devops. We using VsTest and tests are running automatically after the build.
@@lakkampellisaikrishna6384 Hard to tell without seeing. Do you have a sample repo that you can share? What is the test command that runs? Is it picking up multiple similar dll's? You can try restricting it to specific dlls in specific bin/release folder.
I was able to fix the issue by adding !**\obj\**. Before it was only **\*test*.dll. So it was running the tests for dll in obj folder also.
There is an another issue causing test cases to fail. We have stored the connection string in the key vault as secret. In the local machine I have installed certificate so it was able to retrieve the connection string. But in the build pipeline the test cases are failing due to the no connection string is retrieved from the key vault. Do I need to install the certificate in the build server also?
Could you please let me know how to handle this?
Hi Rahul -- thank you so much for your wonderful video. I 'm a newbie in Visual studio 2019 , I just added a new test case to the my test suite, but Test Explorer do not discover it locally and even in my azure pipeline. any idea from you willbe very appreciated !
It is most likely missing NuGet packages for your test framework. For xUnit I have added 'xunit.runner.visualstudio' and 'Microsoft.NET.Test.Sdk' (rahulpnath.visualstudio.com/RUclips%20Samples/_git/MyApp?path=%2FMyApp.Tests%2FMyApp.Tests.csproj)
This will be different based on the test framework used and you will need to check their documentation. For xUnit it is available here - xunit.net/docs/getting-started/netfx/visual-studio#run-tests-visualstudio
Let me know if that helps.
@@RahulNath Thanks for replying to my question. I'm using Mstest Framework, It seems It seems that I have the right Item group. for now I have only 1 test case which test Explorer discover. However , if I write a new test case inside my project , it would be not the case. thanks
Here is my packages reference :
all
runtime; build; native; contentfiles; analyzers; buildtransitive
@@qa_engineer2522 Google might be able to help here better and faster. Tests not appearing in Test Explorer is usually a config/missing nuget issue. Not sure what exactly is causing the issue here. I have not used much of Mstest Framework. if you have a public repo for this, I can take a look if you share the link.
I want to pass on the browser name property from ADO to run my test cases on that browser, how can I achieve that?
You could use variables and get it from there . When you run the tests from command line you can specific the browser based on the tool you are using for the tests.
hi Rahul! do you know, is it possible to have any output in "Result details" if test passed? If test passed- its empty even if I put any "prints" in the middle of the test
Can you provide more details? Not sure where you are looking to get output printed to?
hi rahul can u tell me how to do automation testing after loading the data into azure gen2
Hey Sana, Unfortunately this is not an area I've worked on, so can't be of much help there. Happy New Year!
Hi Rahul great content.. can you make a video on React Native app deployment using azure pipelines to APP Center
Ty
Glad you like it - I have a few videos on React deployment here ruclips.net/p/PL59L9XrzUa-npmS1p7AIPOH7eZqzgMqV0 . Not specific to App Center though
@@RahulNath im trying to deploy React Natie Mobile app (android and iOS ) to App center using Azure Pipelines
@@saitejareddygurujala2534 Cool - don’t have one specific to that. Not sure if I’ll be doing that soon (since that’s new to me too 😀)
is possible to use the same Publish Tests Results task but for a Release pipeline?
I guess you can, but why would you want to run the tests in the Release Pipeline?
Hi my vstest are not passing.
My doubts
1)In the build artifact's...will the testing files should be present
2)I am getting an error as:Execute login failure
3)when I am trying to check the test results (which are not passing)it is open as an attachment in visual studio says: couldn't find the file version: culture:......
Can you please help me @Rahul nath
Hey Sai, The artifacts will have the testing files if you have packaged it. You don't need to if you are running the tests in the build pipeline.
You need a visual studio premium version to view the files from what I last remember. There is also an associated issue here on it not getting displayed in devops github.com/Microsoft/azure-pipelines-tasks/issues/8749
You could try opening the file in a text editor. It should be in an XML format.
Let me know if you have additional questions.
@@RahulNathThank you so much Rahul..I am stuck with this work since two days...can you please help
My project is .net project where test were later pushed into repo
Yes...In the build pipeline no test are being artifacted and when i am using test assemblies in vs test task ... files .*DLL are not getting passed
2)tests are not published
3)I should try to connect visual studio from local machine into azure pipeline (if so how can I connect )
@@susus999 I dont think you can connect to DevOps agent from Visual Studio and you wouldn't need it. Make sure you are building the test project and using the correct filter to run the test. If you watch the video I use the pattern with Test in the name to filter out only test assemblies. If you look at the build log you should be able to see these dlls generated as well. What error are you facing when you run the test task? Is it that it is unable to find the dll's - then make sure you are using the correct pattern
Do you have any documentation on how to run protractor tests on azure pipeline
I don't have anything specific. This should be a good start marketplace.visualstudio.com/items?itemName=MaciejMaciejewski.azure-pipelines-protractor
You should be able to configure the test reports similar to the ones in the video here. Let know if you have any additional questions.
I have written jest automated tests. I am not sure what I am doing wrong. The paths are correct, the commands are the same. I really have no idea what have I missed
Hope you sorted out the issue Nazneen. Hard to tell without seeing what it would be.
@@RahulNath nop...but as a team we didnt need it...so i didnt spend much time on it
Hello Rahul, can we put environment variables that are used in script and define those variables in azure? If yes how it is done?
Hey Pankaj, Can you clarify which script/environment variable you are looking to replace?
@@RahulNath Consider it like under my tests I am navigating to a URL. Can I configure that URL into azure variables such that if I change URL over devops variable my tests follows that changes URL. If yes HOW?
@@pankajsainani8226 You can define the variables as part of the build pipeline and add a file transform task to replace your config file, after checking out the source code. Here is a similar video on using variables in the release pipeline ruclips.net/video/OFr6Vkw0mq8/видео.html
If these are not sensitive information you can keep them as part of the test config in source control itself.
If you are facing issues, set up a public repo/build and share me the link and I can take a look.
Let me know if that helps.
How can I run newman tests on butbucket pipeline, given that collection contains multipart forma data of images?
How are you running this locally at the moment? I would use the same scripts. What issues are you facing?
@@RahulNath Hi Rahul, actually it is resolved now. pipeline was failing because it was taking wrong path for the pictures. So I changed the path of pictures according to the path of my repo.
@@divyankakumari303 thank you for letting me know and glad you sorted it out 👏
Hi Rahul,When I downloaded the code coverage file from Azure Devops Board.I am unable to view that file.Getting error while opening that file.any suggestions?
You need Visual Studio Premium to view the reports.
ruclips.net/video/FFxww1-M25E/видео.html
Also an existing issue related to it not showing up on the UI in DevOps - github.com/Microsoft/azure-pipelines-tasks/issues/8749
[16:33:12] E/runner - Unable to start a WebDriver session.
[16:33:12] E/launcher - Error: Error: Server terminated early with status 127
at /azp/agent/_work/1/s/node_modules/selenium-webdriver/remote/index.js:252:52
at processTicksAndRejections (internal/process/task_queues.js:97:5)
[16:33:12] E/launcher - Process exited with error code 100
Agent machine is ubuntu,Igetting this error while running protractor test through azure pipeline
Sorry not sure if I can be of much help here.
Hello Rahul, I am getting 'No suitable test runtime provider was found for any source in this run' error in vs test
and The process 'C:\AzureAgent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.215.0\Modules\DTAExecutionHost.exe' failed with exit code 1.
Any solution to this above error
Not sure what that would be - Google might be the best help
When test case gets failed it should ask for approval in release pipeline is this possible in azure devops
when test case fails it would fail the build. Isn't that what you would want?
@@RahulNath no when test case failed build should be successful but in realase it's should ask for approval to deploy if test case gets passed it's should not ask for approval
@@SpiritualAdda00 You will need to write some custom scripts to achieve this. Have you seen the available approval flows in devops? learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass&WT.mc_id=AZ-MVP-5003875
Hey Rahul - Thanks for creating a video but in my case the pipeline stuck at @Npm@1 task:
variables:
uiSource: 'client'
uiBuild: $(uiSource)/build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool@0
inputs:
versionSpec: '12.x'
displayName: 'Install Nodejs...'
- script: |
npm install
npm run client-install
npm run build-client
displayName: 'Installing Dependencies...'
- task: Npm@1
displayName: "Front-End Unit tests"
inputs:
command: 'custom'
workingDir: '$(uiSource)'
customCommand: 'run test'
condition: succeededOrFailed()
- task: PublishTestResults@2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/test-report.xml'
failTaskOnFailedTests: true
condition: succeededOrFailed()
- task: CopyFiles@2
inputs:
sourceFolder:
contents: '**'
targetFolder: '$(uiBuild)'
#cleanTargetFolder: false # Optional
overWrite: true
#flattenFolders: false # Optional
#preserveTimestamp: false # Optional
displayName: 'Copying Server-Side files...'
- task: ArchiveFiles@2
inputs:
rootFolderOrFile: '$(uiBuild)'
includeRootFolder: false
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
In this pipeline, I am using NodeJS as Back-end and ReactJs as Front-End.So right now I am testing a Jest tests for Front-End only.
Thanks Paras - Hard to tell what the issue will be looking at the yml file. Do you have a public url of the build pipeline where this issue is happening, that way I can take a look at the logs.
@@RahulNath Hey thanks for replying and sorry for late reply from my side.
As public url is not enabled but as you mentioned about logs,I have seen and observed that npm test is not running inside 'workingDir' i.e client folder(package.json of front-end) so finally decide to run that command from server end (write it inside the package.json of nodejs) which solves my problem.
At last once again thanks for creating such types of videos....
@@parasTal Awesome thanks and glad you sorted it out!
How to fail the pipeline only if the unit test pass percentage is less than 95%
You might need to use script to read the percentage etc. But I wouldn't. How can you tell the other 5% is not the most important parts of your application. Why would you do that? Anything I am missing?
I want that fancy console.. which one is that? thanks in advance. :)
OfCourse and good to hear from you! It's the Windows Terminal. Check out the link here for details www.rahulpnath.com/blog/setting_up_windows_terminal/
I have the similar terminal setup on my mac; it's called 'oh my zsh'...
hai ,i generated html from cmd line script ,how to view it from tests tab ??
You need Visual Studio Premium to view the reports. (18:24)
Also this issue might help get more details - github.com/Microsoft/azure-pipelines-tasks/issues/8749
Let know if that helps.
@@RahulNath thank you rahul ,but i am used classic editor instead of yaml, will you tell me how to do it with classic editor. Thanks
@@kishoreyogeshwaran224 are you looking to view the results? Not sure why that’s related with the classic editor. Sorry if I misunderstood the question. CN you clarify?
@@RahulNath i am generating html from command line in devops and i am looking to view that html .i used classic editor instead of yaml for generating this html.
I couldn't focus because of the excessive editing. In the beginning, it is ok to show your self, but throughout the video I got destructed because you paused yourself taking too many times. Also, You sounded like a robot to me because of your editings. I'm giving feedback so that you can correct your future videos.
Hey Sunny,
Thank you for your feedback. Based on other similar feedback I have reduced my video pip size in my newer videos. (Let know what you feel about them).
Regarding the editing, yes I am still learning the tricks of the trade and slowly trying to improve on it with each video.
If there are any specific areas of the video that is unclear or you need help with, feel free to reach out.
Thanks again for the feedback!.
cyborg?
Sorry I haven't used that
add some facial expression will be much appreciated
Thank you Amar for your suggestion. Yes I’m trying to improve on my camera presence. Hope you like the content barring my face 😀
Sir you going to fast and do not show exactly what are you doing, you can also reduce the picture of yourself, it is not good to see you all the time, plus we can not see what you doing in the screen. We like to see more screen and less you, do not take it personally. Now the content is good but go slow and pointing out what you are doing in the screen.
Thank you for the valuable feedback Cesar and agree with you. I have reduced my picture size in the future videos. This was early when I started and was still learning how to make videos. Hope you find the newer videos better. Are there any places that found too hard to follow? I can try making other videos to explain those areas separately