⏱️ Timestamps: 00:00 | Intro 00:22 | Not storing the pipeline code in the Jenkinsfile | - code.visualstudio.com | - Install Jenkins Pipeline Linter Connector 03:47 | No unit tests for the pipeline code | - github.com/jenkinsci/JenkinsP... | - github.com/jenkinsci/jenkins-... | - github.com/jenkinsci/jenkinsf... 07:13 | Ignoring Jenkins Pipeline Best Practices | - www.jenkins.io/doc/book/pipel... | - www.jenkins.io/doc/book/pipel... 10:00 | Not using timeouts at the stage level 13:17 | Blocking executor and waiting for the input 17:16 | End screen
Hey Szymon! Just wanted to thank you for your time doing videos like this. It is very helpful for Juniors and not only :) Learned a lot from you, Keep going!
Hey Szymon! First of all - you are great and thank you for doing that! Not sure if this counts as a pipeline or logic mistake but what I see often is not splitting build and deploy jobs in big/huge pipelines. For me this is a big no-no. You should be able to redeploy an artifact as many times as you want without rebuilding it and waste this X time to do so. Also, this helps and improves eventual rollback process.
I agree with you 100%, Rosen! Thanks for bringing this up. Yes, pipeline should utilize as many stages as needed, and each one of them should do one thing and do it well. I've seen it many times how useful it is. Imagine the following workflow - the pipeline builds the artifact, releases a new version, pushes docker image of the new service, runs it in UTA-like environment to run some automated QA tests, and this stage fails because the runtime environment was not ready. It does not affect the artifact we just created and it has to be fixed outside of our Jenkins pipeline. After fixing it, we can restart from the UTA stage to do the deployment again, without a need to restart whole CI part from scratch. Alternatively, we would need to waste resources to rebuild the same artifact, just as you mentioned in your comment. Thank you so much for the valuable comment, I'm sure other people will learn from it. Have a good day!
Excellent video, I'm learning a lot with your videos. I like how you take advantage of the timestamps to divide the video, very nice feature. I'm looking forward for your Test Jenkinsfile video. Keep it up!
Thank you so much for the kind words! I'm super happy to hear you learn from my videos - that's my main goal. Stay tuned for the Testing Jenkinsfile video and have a good day!
Thank you so much for your kind words! The plugin I use for the syntax highlighting is code-groovy (ver 0.1.2). I don't remember if it autodetects Jenkinsfile as a Groovy file, but if not then adding #!groovy in the first line of the Jenkinsfile may help. Take care and have a good day!
Excellent video, Please add more topics like this. This agent unblocking is a new thing I learnt. Thanks. Want to know more about what is executed in master context and what is executed in agent node context from jenkinsfile. And please do one video on Jenkins best practise, especially heavily loading the master because of sh block
Hi Vignesh! Thank you so much for your kind words. I will create more videos covering best practices and some other common Jenkins pipeline misuses. Stay tuned and have a good day! 👍
Hi, GREAT videos!!! I'm running the Jenkins/jenkins docker image in a docker container, I'd love to get some input on how to get the agent { docker{...}} to work when running jenkins in a dockercontainer.
Hi Jakob! You might want to take a look at this blog post that covers issues with docker-in-docker and how to overcome them - jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ Good luck, and have a good day!
Many thanks for advising Visual Code, it makes my life really easy, and Im starting to remember the syntax at last :D ! I think a lot of us would like to see more videos about jenkins and would be thankful Thanks for your work.
Thanks (again) for your kind words, Narek! 👍 I can't promise anything. My Jenkins videos collection was recorded when I was actively working with Jenkins on a daily basis. And when I left my previous job, I stopped using Jenkins, even for my pet projects. And it's almost 2 years now since I departured from Jenkins. I know there is not that many high-quality content around Jenkins, so with my established audience it would make sense to fill this gap, but it's hard to find motivation to cover topics about the technology you stopped using some time ago. I don't say definitive "no", but I can't also say "hell yes", so let's see what the future brings. Take care, and have a good day!
@@szymonstepniak Thanks for your reply! I think we (your audience) would be glad to learn from you anything possible related to our current needs. Take care!
Thanks for your kind words! Check out the JenkinsPipelineUnit library mentioned in the video - it has some examples that should help you start. Good luck, and have a good day!
Excellent, a good question I came across was how to checkout a repo if you're using a pipeline script directly on Jenkins instead of using pipeline script from VSC way. That made me question myself about the belief of considering the Pipeline script from VSC doing all of that by default. Something I Took for granted without even knowing it for sure XD
Thanks for the comment, Brian! Yes, Jenkins in Kubernetes utilizes nodes more efficiently, but it won't solve all problems. It's like adding more memory to run a program that does not manage available memory well. In this video, I wanted to focus on solutions we can apply independently from the Jenkins infrastructure, so everyone can benefit from watching the video. Thanks again, and have a good day!
Hi Akash! As far as I remember, agents should have env variables configured from the higher scope, e.g. env variable defined inside the "environment" block at the "pipeline" block level should be exposed to all agents. But if there is an env variable defined in the "stage" block for a specific agent (or defined programatically inside the "script" block by one agent), then such a variable is not shared. Good luck and have a good day!
I've learned more following your videos then by reading the docs (although obviously they should be used as reference). Just one question tho: when was "options" supported inside stages? Seems like I'm not able to run them.
Thanks for your kind words, Edoardo! I checked my version of Jenkins and the options with timeout under the stage works when I run it on Jenkins 2.277.4, as well as on the latest 2.346.2 with all plugins up to date. I put a single Jenkinsfile + the list of all installed plugins + the screenshot of the pipeline's console output (taken from the latest Jenkins version) - gist.github.com/wololock/34715421920d87ca1e1f8accba40b717 This might help you investigate what is the difference between this and your Jenkins instance. I don't know starting from which version of Jenkins (or the declarative pipeline plugin) the "options" block is supported under the stages - the documentation is quite misleading on that and it says you can put only a single "options" block under the "pipeline" level, which is definitely not true. But I'm guessing that not every option works under the "stage" level. I used things like timeout or timestamps successfully, but retry on timeout does not work (as shown in the example attached.) I hope it help. Good luck, and have a good day!
Szymon, thanks your videos. Y will follow you. I started to work with a jenkins cloud version with kubernetes pod agents , and I would like to know what the meaning of not to blocking and executor with an input without an agent asigned. What happen with this input processing in jenkins? Thank you so much.
You really are amazing. This really helped me a lot while building my own pipeline. I just need a bit of advice. I created a pipeline that executes in the master, mostly consists of the groovy code. This pipeline is mainly for executing tests, and these tests were distributed along with my nodes (e.g N number of tests = N number of nodes) and executed in parallel. My pipeline consists of groovy codes because I have to manually get the raw data for the builds that executed the tests for email notification. Is this a good approach or there is a much better way? It would be a great help, thanks in advance!
Thanks for your kind words, Jett! I'm super happy to hear this video helped you with your pipeline. Regarding your question, create a gist file (gist.github.com) and share an example (either here, or shoot me an email if you don't want to share it in a public comment.) If you use Groovy code to distribute parallel tests execution, then it's fine, and this is exactly what you can use some Groovy for. What matters is what do you use to execute those tests - if you execute them with a pipeline step (like "sh" step for example), then you execute them on dedicated nodes and this is OK. Let me know if it helps, and if you still struggle - send me a pipeline example (without sensitive data ofc) so I can help you more. Have a good day!
@@szymonstepniak I created a separate freestyle job that I am executing inside my pipeline using the "build" step. I already shoot you an email. Hoping you can help me with this. Big thanks!
Thanks for your Pipeline videos, pretty nice. I love to use the "replay" option on Jenkins to quickly test on some changes would apply. Its a pitty its not in blueocean though. I'm not sure If you'll read this but have you ever had the need to set certain options conditional? (I mean declarative pipeline options) I know I can use "when" with beforeOptions , but I have a case where I have "office365ConnectorWebhooks" and also "buildDiscarder" in option, and let say I want to use "office365ConnectorWebhooks" only for the master branch, I couldn't find any way to get that to work, perhaps its simply not possible especially with Declarative... keep up the good work
Hi Miguel! I'm afraid there's no good way to achieve the pipeline behavior you described with the declarative pipeline. The "options" block does not accept anything but the options one by one, so you can't put any if-statement to declare options conditionally. Maybe one way to overcome this limitation would be to store values you use to configure specific options in variables that are defined outside of the pipeline block, and then use them (either directly or inside interpolated string). But I can't guarantee that it will work. Another option is to use scripted pipeline where you can set properties conditionally. Or if you want to stick to the declarative pipeline, another option is to maintain two Jenkinsfiles - one for the master, and one for all other branches. It's not perfect and it adds code repetition, but if your pipeline behavior for the master branch has a few more exceptions compared to all other branches, having dedicated Jenkinsfile for that branch is not the bad idea imho. Good luck, and have a good day!
Hi Andrei! I have nothing I could share with you out of my mind. I would check CloudBees RUclips channel and see if they have any video like that. Good luck, and have a good day!
Thanks for your kind words, Alan. Could you elaborate on the problem with escaping characters you're facing? I will see if there is anything I could help you with. Have a good day!
thank you Szymon. I really enjoy the gift of teaching and presenting that you have. I dislike some features of groovy. I was wondering if Groovy might evolve to be more like java 16? also I would like to know about any product roadmap for Jenkins? thank you
Thanks for your kind words, Andres! Regarding Jenkins product roadmap, you can check this page on the official Jenkins website - www.jenkins.io/project/roadmap/ Also, there is @jenkinscicd official RUclips channel where you can find videos from e.g. the product teams meetings, so I guess this could be right place to ask some questions about the topics they discussed during those meetings. Regarding Groovy evolving to something like Java 16 - I don't know the answer. The upcoming Groovy 4.0 will be integrated with some (if not most) of the new things introduced in Java 11 and 16, but the syntax of the language will remain the same. We keep to forget that Groovy was started back in 2003, even before Java 1.5. Its syntax evolved since that time, but before Java 8, it was a real game changer for the developer's productivity. However, the landscape changed and now Java is one of the most rapidly evolving languages on the JVM, which wasn't the case when languages like Groovy, Scala, or Kotlin were invented. On the other hand, it's hard to look at Groovy evolution from the Jenkins pipeline perspective. As far as I remember, Jenkins pipeline still uses Groovy 2.4.12 which is already 4 years old. I hope this answer casts some light and you find it helpful :) Take care, and have a good day!
Great video... one question i wanted to do something similar to free my executor when i am waiting for the appoval but i have parallel stages in my pipeline. I would be a great help.. Thanks in advance
Hi, Saurav! Thanks for the kind words! I haven't done it in the parallel stages, but I'm guessing it should work exactly the same, meaning: you use "agent none" at the pipeline level, and then you define agents on each parallel stage, and you don't define one on the stage you want to run without blocking an executor. Parallel stages should not require different approach in my opinion. Try it out and let me know if it worked, please. Have a good day!
Hello Brain..thanks for the valuable info.. My actual question is when i have completed the build step u want to upload the artifacts to jfrog..so in pipeline syntax i have used "rtupload" block to move artifacts to jfrog instead of using "curl" command,but rtupload is not working..could you please check on this..!!
Support for the declarative pipeline was added in version 1.3 (github.com/jenkinsci/JenkinsPipelineUnit/blob/master/src/main/groovy/com/lesfurets/jenkins/unit/declarative/DeclarativePipelineTest.groovy)
Are there any VSCode extensions allow you to validate scripted pipeline files? The one you show in this video at 1:30 is for declarative pipelines only
I haven't use anything like that with the scripted pipeline, but you may want to give npm-groovy-lint a try - www.npmjs.com/package/npm-groovy-lint I don't know how accurate it is in validating scripted Jenkins pipeline, but this may help you catch some Groovy issues.
Hi, Mohammad! Yes, you can get GDSL file from you Jenkins server and set it up in the IntelliJ IDEA to get the syntax completion. Here is a blog post that may help you - vgaidarji.me/blog/2018/07/30/working-with-jenkinsfile-in-intellij-idea/ Let me know if it worked for you, please. Have a good day!
I'm guessing you are referring to the Blue Ocean UI plugin - www.jenkins.io/projects/blueocean/ It can be found with the Jenkins plugin manager and installed with just one click :)
Can you tell how we can access a file from a shell ( file passed as prameter ) in the stage in declarative pipeline . checking this everywhere . pls help
Hi Sat! I'm guessing you are asking about using the "sh" step in the Jenkins pipeline, correct? It is used to execute any available shell command. There is no other way to run those commands. Java and Groovy offers executing processes, but this is not correct in the Jenkins environment - any such code gets executed on your main Jenkins node, and not on the node that executes given stage of the pipeline. So the answer is "no, you can't execute commands in a different way". I hope it helps. Have a good day!
Is more memory spent when from an upstream jenkins job we monitor a downstream jenkins job using: def jb: build wait: false, job: xyz, propagate: false ? Or we trigger a downstream jenkins job and sleep for the time it completes and check memory usage. In which case is more memory spent ? @Szymon
Hi Akash! I don't know the answer, and I could only recommend testing it out on the real example. What comes to my mind, however, is to recommend you experimenting with the unblocking an agent as explained in the waiting for the input example in the video. I would expect that blocking an agent and waiting for the downstream job result is not the most effective for your Jenkins infrastructure - the agent could handle another job in the meantime. And I think this might be even a bigger concern, but it depends on your specific problems. Good luck, and have a good day!
Hi, I am trying to add Jenkins build tag value as a parameter and I want this to be auto populate with assessing order like time stamp or version number etc.. any advises please?
Hi, Mahesh! Could you elaborate on the problem with details so I can better understand what you are trying to achieve and what attempts you made that didn't work for you? Look forward, have a good day!
End of Executing .bat file in Jenkins pipeline showing "The batch file is not found" and hanging there itself. But the batchfile worked as expected. How to handle this situation? Trying timeout will end as aborted, so what is the suited method?
I don't know how to help you. There must be a reason the pipeline hangs there, and I'm guessing the .bat file itself probably contains a command that does not terminate on error. I would recommend starting investigation here. Good luck!
Hi @Szymon Stepniak, I appreciate your attention for me. I found the cause to the issue, the issue is with the batch file itself, so I removed that step, now it's fine. Thank you 🙂
Hey nice video, Could you me explain, very fast, how take the log at runtime jenkins pipeline and save the data on local variable? I can´t do this ;( i tried with several optiones with no results. Thanks again for video!
Hi, Alejandro! Can you elaborate on what you are trying to achieve? Can you show an example of what you tried so far? I don't understand what "take the log at runtime" means to you specifically. Thanks and have a good day!
I could control it using "throw" on my ".js" file and in "pipeline" I used "Try{}Catch{}" to get the error, I don't know if it's the best way... Do you think otherwise?
@@alejandrolozanorodriguez8381 According to the screenshot attached, you use sh step to execute the script. You can use its parameter called "returnStdout" to redirect output from a command and capture it in the variable. (You need to executed sh step in the "script" block in such a case.) Take a look at this example - gist.github.com/wololock/89a5813a815a25d97a9e223747f8c4f7 It stores the output under the "result" variable.
Thanks for your comment, Satyajit! I wouldn't think of this music as spiritual :) I just like its smooth ambient sound. Hope it doesn't distract you from consuming the content. Have a good day!
i have 1 question if a job fails how can we see errors for specific line ..so that we can make correction eg> if Buildimage stages get error due to docker or any syntax error then how can we see in build console the specific line has the issue
You should see in the console log the number of line pointing to the step that caused pipeline to fail. It should be the first (or one of the first) item on the stacktrace.
@@szymonstepniak But it does not point to specific line number as: [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (test) Stage "test" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (deploy) Stage "deploy" skipped due to earlier failure(s)
I guess you can't validate scripted jenkins pipeline like the declarative one. The scripted pipeline comes with no constraints, while the declarative pipeline has predefined syntax you have to follow.
using too much shell script, instead of creating a dedicated lets say python library for those tasks, having all the data in the code and not outside in XML or JSON struct.
⏱️ Timestamps:
00:00 | Intro
00:22 | Not storing the pipeline code in the Jenkinsfile
| - code.visualstudio.com
| - Install Jenkins Pipeline Linter Connector
03:47 | No unit tests for the pipeline code
| - github.com/jenkinsci/JenkinsP...
| - github.com/jenkinsci/jenkins-...
| - github.com/jenkinsci/jenkinsf...
07:13 | Ignoring Jenkins Pipeline Best Practices
| - www.jenkins.io/doc/book/pipel...
| - www.jenkins.io/doc/book/pipel...
10:00 | Not using timeouts at the stage level
13:17 | Blocking executor and waiting for the input
17:16 | End screen
Great tips! I will definitely use these and never knew about the ability to unit test a Jenkinsfile
I'm glad to hear you've found this video useful, Don! Take care, and have a good day!
one of the best Jenkins learning channels. It was really helpful
Thanks for your kind words, Vishu! I'm glad to hear you've found this video helpful. Take care, and have a good day!
Hey Szymon! Just wanted to thank you for your time doing videos like this. It is very helpful for Juniors and not only :) Learned a lot from you, Keep going!
Thanks a lot for your kind words, Piotr! Take care, and have a good day!
Most useful tips, It saves a lot of time. Looking forward to unit/integration test videos. You are simply Awesome :)
Thanks for the kind words, Anand! Testing Jenkinsfile video coming soon :) Stay tuned and have a good day!
Great video. Would love to see a testing Jenkinsfile video
Thanks for the kind words, Mike! I'm happy to hear you find it useful. Testing Jenkinsfile video coming soon! :D Have a good day!
Great videos! I would be very interested in a video on using shared libraries to add features to your pipelines.
Thank you for the kind words, Dylan! Video on Jenkins shared libraries coming soon :) Have a good day!
Hey Szymon! First of all - you are great and thank you for doing that!
Not sure if this counts as a pipeline or logic mistake but what I see often is not splitting build and deploy jobs in big/huge pipelines. For me this is a big no-no. You should be able to redeploy an artifact as many times as you want without rebuilding it and waste this X time to do so. Also, this helps and improves eventual rollback process.
I agree with you 100%, Rosen! Thanks for bringing this up. Yes, pipeline should utilize as many stages as needed, and each one of them should do one thing and do it well. I've seen it many times how useful it is. Imagine the following workflow - the pipeline builds the artifact, releases a new version, pushes docker image of the new service, runs it in UTA-like environment to run some automated QA tests, and this stage fails because the runtime environment was not ready. It does not affect the artifact we just created and it has to be fixed outside of our Jenkins pipeline. After fixing it, we can restart from the UTA stage to do the deployment again, without a need to restart whole CI part from scratch. Alternatively, we would need to waste resources to rebuild the same artifact, just as you mentioned in your comment. Thank you so much for the valuable comment, I'm sure other people will learn from it. Have a good day!
While this idea is very common and appropriate to building apps, If using a gitops approach, this might not always be valid.
Excellent video, I'm learning a lot with your videos. I like how you take advantage of the timestamps to divide the video, very nice feature. I'm looking forward for your Test Jenkinsfile video. Keep it up!
Thank you so much for the kind words! I'm super happy to hear you learn from my videos - that's my main goal. Stay tuned for the Testing Jenkinsfile video and have a good day!
Crisp and clear and efficient ,
Thanks for your kind words, Datta! Take care, and have a good day!
Thanks a lot for this wonderful video. Your explanation & presentation is excellent and praiseworthy. Love from India. :)
Thank you so much for your kind and very motivating words, Rohit! Take care and have a good day! :)
This channel is a gem with so many helpful tips about Jenkins. What vscode extensions are you using for the nice Jenkinsfile syntax highlighting?
Thank you so much for your kind words! The plugin I use for the syntax highlighting is code-groovy (ver 0.1.2). I don't remember if it autodetects Jenkinsfile as a Groovy file, but if not then adding #!groovy in the first line of the Jenkinsfile may help. Take care and have a good day!
Very informative videos. Some stages can be built in parallel if they are independent from others.
Absolutely! Thanks for the comment, Kashyap! I'm happy to hear you find the video informative. Have a good day!
Great video, thank you!
Thanks, Jean! Take care, and have a good day!
I had freaking hard time in dealing with jenkinsfile 🥵
I finally ended up learning basics of groovy language and then my life became normal.
Your video on environment variables helped a lot during that learning phase 💐
Thanks for the kind words. Have a good day!
Excellent video, Please add more topics like this. This agent unblocking is a new thing I learnt. Thanks. Want to know more about what is executed in master context and what is executed in agent node context from jenkinsfile. And please do one video on Jenkins best practise, especially heavily loading the master because of sh block
Hi Vignesh! Thank you so much for your kind words. I will create more videos covering best practices and some other common Jenkins pipeline misuses. Stay tuned and have a good day! 👍
Hello Szymon,
Very useful video, Thanks much :)
Thank you so much for the kind words, Ramarao! I'm happy to hear you find it useful. Have a good day!
Hi, GREAT videos!!! I'm running the Jenkins/jenkins docker image in a docker container, I'd love to get some input on how to get the agent { docker{...}} to work when running jenkins in a dockercontainer.
Hi Jakob! You might want to take a look at this blog post that covers issues with docker-in-docker and how to overcome them - jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ Good luck, and have a good day!
Many thanks for advising Visual Code, it makes my life really easy, and Im starting to remember the syntax at last :D !
I think a lot of us would like to see more videos about jenkins and would be thankful
Thanks for your work.
Thanks (again) for your kind words, Narek! 👍 I can't promise anything. My Jenkins videos collection was recorded when I was actively working with Jenkins on a daily basis. And when I left my previous job, I stopped using Jenkins, even for my pet projects. And it's almost 2 years now since I departured from Jenkins. I know there is not that many high-quality content around Jenkins, so with my established audience it would make sense to fill this gap, but it's hard to find motivation to cover topics about the technology you stopped using some time ago. I don't say definitive "no", but I can't also say "hell yes", so let's see what the future brings. Take care, and have a good day!
@@szymonstepniak Thanks for your reply! I think we (your audience) would be glad to learn from you anything possible related to our current needs. Take care!
@@narekhambardzumyan9665 Thanks, it means a lot to me! 👍
Hey it was a nice video getting introduced in to
Validation jenkinsfile. Interested to know how to unit test Jenkins file
Thanks for your kind words! Check out the JenkinsPipelineUnit library mentioned in the video - it has some examples that should help you start. Good luck, and have a good day!
Nice..video. Need more videos on this.
Thanks for your kind words, Pernankil! Have a good day! 👍
Excellent, a good question I came across was how to checkout a repo if you're using a pipeline script directly on Jenkins instead of using pipeline script from VSC way. That made me question myself about the belief of considering the Pipeline script from VSC doing all of that by default. Something I Took for granted without even knowing it for sure XD
Thank you so much, Artgento! I'm so happy to hear you find it useful :) Have a good day!
If you run jenkins in kubernetes you avoid much off the issues with timeouts, and executor blocking.
Thanks for the comment, Brian! Yes, Jenkins in Kubernetes utilizes nodes more efficiently, but it won't solve all problems. It's like adding more memory to run a program that does not manage available memory well. In this video, I wanted to focus on solutions we can apply independently from the Jenkins infrastructure, so everyone can benefit from watching the video. Thanks again, and have a good day!
Thanks!
Welcome!
Please do a testing pipeline using the jenkinsPipelineUnit video, too. Thanks!
Thanks, I hope I will finally find the time to do it :)
In multi agent pipeline how to define environment variables/global variables?
Hi Akash! As far as I remember, agents should have env variables configured from the higher scope, e.g. env variable defined inside the "environment" block at the "pipeline" block level should be exposed to all agents. But if there is an env variable defined in the "stage" block for a specific agent (or defined programatically inside the "script" block by one agent), then such a variable is not shared. Good luck and have a good day!
Great Video Sir, Please make some more videos on Jenkins YAML pipeline
Thanks for your kind words and suggesting a topic! Have a good day!
I've learned more following your videos then by reading the docs (although obviously they should be used as reference).
Just one question tho: when was "options" supported inside stages? Seems like I'm not able to run them.
Thanks for your kind words, Edoardo! I checked my version of Jenkins and the options with timeout under the stage works when I run it on Jenkins 2.277.4, as well as on the latest 2.346.2 with all plugins up to date. I put a single Jenkinsfile + the list of all installed plugins + the screenshot of the pipeline's console output (taken from the latest Jenkins version) - gist.github.com/wololock/34715421920d87ca1e1f8accba40b717 This might help you investigate what is the difference between this and your Jenkins instance. I don't know starting from which version of Jenkins (or the declarative pipeline plugin) the "options" block is supported under the stages - the documentation is quite misleading on that and it says you can put only a single "options" block under the "pipeline" level, which is definitely not true. But I'm guessing that not every option works under the "stage" level. I used things like timeout or timestamps successfully, but retry on timeout does not work (as shown in the example attached.) I hope it help. Good luck, and have a good day!
Szymon, thanks your videos. Y will follow you.
I started to work with a jenkins cloud version with kubernetes pod agents , and I would like to know what the meaning of not to blocking and executor with an input without an agent asigned. What happen with this input processing in jenkins? Thank you so much.
Hi Javier! I never used Jenkins Cloud, so I don't know the answer to your question. Take care, and have a good day!
You really are amazing. This really helped me a lot while building my own pipeline.
I just need a bit of advice. I created a pipeline that executes in the master, mostly consists of the groovy code. This pipeline is mainly for executing tests, and these tests were distributed along with my nodes (e.g N number of tests = N number of nodes) and executed in parallel. My pipeline consists of groovy codes because I have to manually get the raw data for the builds that executed the tests for email notification. Is this a good approach or there is a much better way? It would be a great help, thanks in advance!
Thanks for your kind words, Jett! I'm super happy to hear this video helped you with your pipeline.
Regarding your question, create a gist file (gist.github.com) and share an example (either here, or shoot me an email if you don't want to share it in a public comment.) If you use Groovy code to distribute parallel tests execution, then it's fine, and this is exactly what you can use some Groovy for. What matters is what do you use to execute those tests - if you execute them with a pipeline step (like "sh" step for example), then you execute them on dedicated nodes and this is OK. Let me know if it helps, and if you still struggle - send me a pipeline example (without sensitive data ofc) so I can help you more. Have a good day!
@@szymonstepniak I created a separate freestyle job that I am executing inside my pipeline using the "build" step. I already shoot you an email. Hoping you can help me with this. Big thanks!
@@jettmarcella1762 Hi, Jett! I've seen your email but didn't have time to dig into it yet. I will reply to it this week. Have a good day!
Thanks for your Pipeline videos, pretty nice. I love to use the "replay" option on Jenkins to quickly test on some changes would apply. Its a pitty its not in blueocean though.
I'm not sure If you'll read this but have you ever had the need to set certain options conditional? (I mean declarative pipeline options)
I know I can use "when" with beforeOptions , but I have a case where I have "office365ConnectorWebhooks" and also "buildDiscarder" in option, and let say I want to use "office365ConnectorWebhooks" only for the master branch, I couldn't find any way to get that to work, perhaps its simply not possible especially with Declarative...
keep up the good work
Hi Miguel! I'm afraid there's no good way to achieve the pipeline behavior you described with the declarative pipeline. The "options" block does not accept anything but the options one by one, so you can't put any if-statement to declare options conditionally. Maybe one way to overcome this limitation would be to store values you use to configure specific options in variables that are defined outside of the pipeline block, and then use them (either directly or inside interpolated string). But I can't guarantee that it will work. Another option is to use scripted pipeline where you can set properties conditionally. Or if you want to stick to the declarative pipeline, another option is to maintain two Jenkinsfiles - one for the master, and one for all other branches. It's not perfect and it adds code repetition, but if your pipeline behavior for the master branch has a few more exceptions compared to all other branches, having dedicated Jenkinsfile for that branch is not the bad idea imho. Good luck, and have a good day!
Cześć! Is there a good public example of a complex Jenkins pipeline, including Groovy shared library, asking user for input, etc. ?
Hi Andrei! I have nothing I could share with you out of my mind. I would check CloudBees RUclips channel and see if they have any video like that. Good luck, and have a good day!
Thank you!
Hey @Szymon, Your videos of jenkinsfile are too good. Can you post a video of how and where to use escape characters while using groovy syntax.
Thanks for your kind words, Alan. Could you elaborate on the problem with escaping characters you're facing? I will see if there is anything I could help you with. Have a good day!
thank you Szymon. I really enjoy the gift of teaching and presenting that you have.
I dislike some features of groovy. I was wondering if Groovy might evolve to be more like java 16?
also I would like to know about any product roadmap for Jenkins?
thank you
Thanks for your kind words, Andres! Regarding Jenkins product roadmap, you can check this page on the official Jenkins website - www.jenkins.io/project/roadmap/ Also, there is @jenkinscicd official RUclips channel where you can find videos from e.g. the product teams meetings, so I guess this could be right place to ask some questions about the topics they discussed during those meetings.
Regarding Groovy evolving to something like Java 16 - I don't know the answer. The upcoming Groovy 4.0 will be integrated with some (if not most) of the new things introduced in Java 11 and 16, but the syntax of the language will remain the same. We keep to forget that Groovy was started back in 2003, even before Java 1.5. Its syntax evolved since that time, but before Java 8, it was a real game changer for the developer's productivity. However, the landscape changed and now Java is one of the most rapidly evolving languages on the JVM, which wasn't the case when languages like Groovy, Scala, or Kotlin were invented.
On the other hand, it's hard to look at Groovy evolution from the Jenkins pipeline perspective. As far as I remember, Jenkins pipeline still uses Groovy 2.4.12 which is already 4 years old. I hope this answer casts some light and you find it helpful :) Take care, and have a good day!
Great video... one question i wanted to do something similar to free my executor when i am waiting for the appoval but i have parallel stages in my pipeline. I would be a great help.. Thanks in advance
Hi, Saurav! Thanks for the kind words! I haven't done it in the parallel stages, but I'm guessing it should work exactly the same, meaning: you use "agent none" at the pipeline level, and then you define agents on each parallel stage, and you don't define one on the stage you want to run without blocking an executor. Parallel stages should not require different approach in my opinion. Try it out and let me know if it worked, please. Have a good day!
Excellent !!🙂💥
Can you please do video on how to upload artifactory to Jfrog using rtupload ...i have tried it but it's not working
Jenkins pipeline unittest does not officially support declarative pipelines.
If running jenkins with kubernetes agents you can replace groovy functions with sidecar containers
Hello Brain..thanks for the valuable info..
My actual question is when i have completed the build step u want to upload the artifacts to jfrog..so in pipeline syntax i have used "rtupload" block to move artifacts to jfrog instead of using "curl" command,but rtupload is not working..could you please check on this..!!
Support for the declarative pipeline was added in version 1.3 (github.com/jenkinsci/JenkinsPipelineUnit/blob/master/src/main/groovy/com/lesfurets/jenkins/unit/declarative/DeclarativePipelineTest.groovy)
Hello would u post Jenkins shared libraries with example
Hi, Dhiyan! Yes, I will make one in the near future. Stay tuned, and have a good day!
Are there any VSCode extensions allow you to validate scripted pipeline files? The one you show in this video at 1:30 is for declarative pipelines only
I haven't use anything like that with the scripted pipeline, but you may want to give npm-groovy-lint a try - www.npmjs.com/package/npm-groovy-lint I don't know how accurate it is in validating scripted Jenkins pipeline, but this may help you catch some Groovy issues.
Is there a way to support Jenkinsfile syntax on Intellij IDEA?
Hi, Mohammad! Yes, you can get GDSL file from you Jenkins server and set it up in the IntelliJ IDEA to get the syntax completion. Here is a blog post that may help you - vgaidarji.me/blog/2018/07/30/working-with-jenkinsfile-in-intellij-idea/ Let me know if it worked for you, please. Have a good day!
Not sure why my UI (the pipeline run and stage UI) looks very different with yours and yours have better UI.
I'm guessing you are referring to the Blue Ocean UI plugin - www.jenkins.io/projects/blueocean/ It can be found with the Jenkins plugin manager and installed with just one click :)
@@szymonstepniak thanks just installed and it looks awesome!
Can you tell how we can access a file from a shell ( file passed as prameter ) in the stage in declarative pipeline . checking this everywhere . pls help
I don't know if something like this is possible, I have never done anything like that. Good luck, and have a good day!
While we using Shell inside Jenkins? Can we use it without shell in Jenkins file?
Hi Sat! I'm guessing you are asking about using the "sh" step in the Jenkins pipeline, correct? It is used to execute any available shell command. There is no other way to run those commands. Java and Groovy offers executing processes, but this is not correct in the Jenkins environment - any such code gets executed on your main Jenkins node, and not on the node that executes given stage of the pipeline. So the answer is "no, you can't execute commands in a different way". I hope it helps. Have a good day!
Is more memory spent when from an upstream jenkins job we monitor a downstream jenkins job using:
def jb: build wait: false, job: xyz, propagate: false ?
Or we trigger a downstream jenkins job and sleep for the time it completes and check memory usage.
In which case is more memory spent ?
@Szymon
Hi Akash! I don't know the answer, and I could only recommend testing it out on the real example. What comes to my mind, however, is to recommend you experimenting with the unblocking an agent as explained in the waiting for the input example in the video. I would expect that blocking an agent and waiting for the downstream job result is not the most effective for your Jenkins infrastructure - the agent could handle another job in the meantime. And I think this might be even a bigger concern, but it depends on your specific problems. Good luck, and have a good day!
@@szymonstepniak Thank you for the response
Hi, I am trying to add Jenkins build tag value as a parameter and I want this to be auto populate with assessing order like time stamp or version number etc.. any advises please?
Hi, Mahesh! Could you elaborate on the problem with details so I can better understand what you are trying to achieve and what attempts you made that didn't work for you? Look forward, have a good day!
End of Executing .bat file in Jenkins pipeline showing "The batch file is not found" and hanging there itself. But the batchfile worked as expected. How to handle this situation?
Trying timeout will end as aborted, so what is the suited method?
I don't know how to help you. There must be a reason the pipeline hangs there, and I'm guessing the .bat file itself probably contains a command that does not terminate on error. I would recommend starting investigation here. Good luck!
Hi @Szymon Stepniak, I appreciate your attention for me. I found the cause to the issue, the issue is with the batch file itself, so I removed that step, now it's fine. Thank you 🙂
Hi Stepniak, can we insert a sh command after the timeout directive inside the options directive?
Nope. The "options" block in the declarative pipeline has restricted syntax and running any sh step inside of it is prohibited.
you are a crack!!!! Thanks
Thanks, Carlos! Have a good day!
Hey nice video, Could you me explain, very fast, how take the log at runtime jenkins pipeline and save the data on local variable? I can´t do this ;( i tried with several optiones with no results. Thanks again for video!
Hi, Alejandro! Can you elaborate on what you are trying to achieve? Can you show an example of what you tried so far? I don't understand what "take the log at runtime" means to you specifically. Thanks and have a good day!
@@szymonstepniak Thanks for the fast answer
I could control it using "throw" on my ".js" file and in "pipeline" I used "Try{}Catch{}" to get the error, I don't know if it's the best way... Do you think otherwise?
@@alejandrolozanorodriguez8381 According to the screenshot attached, you use sh step to execute the script. You can use its parameter called "returnStdout" to redirect output from a command and capture it in the variable. (You need to executed sh step in the "script" block in such a case.)
Take a look at this example - gist.github.com/wololock/89a5813a815a25d97a9e223747f8c4f7 It stores the output under the "result" variable.
@@szymonstepniak Thanks a lot!!
What is so spiritual in Jenkins? I mean the background music.
Thanks for your comment, Satyajit! I wouldn't think of this music as spiritual :) I just like its smooth ambient sound. Hope it doesn't distract you from consuming the content. Have a good day!
No branch available in github While build pipeline job then jenkins which error comes?
I am getting element click intercepted nad test is failing
i have 1 question if a job fails how can we see errors for specific line ..so that we can make correction eg> if Buildimage stages get error due to docker or any syntax error then how can we see in build console the specific line has the issue
You should see in the console log the number of line pointing to the step that caused pipeline to fail. It should be the first (or one of the first) item on the stacktrace.
@@szymonstepniak But it does not point to specific line number as:
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
Stage "test" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy)
Stage "deploy" skipped due to earlier failure(s)
how to validate the scripted jenkins pipeline in visual studio code?
I guess you can't validate scripted jenkins pipeline like the declarative one. The scripted pipeline comes with no constraints, while the declarative pipeline has predefined syntax you have to follow.
using too much shell script, instead of creating a dedicated lets say python library for those tasks, having all the data in the code and not outside in XML or JSON struct.
Great Videos. Quality and presentation of the content ist perfect done! Thanks a lot!! 🫡
Thank you for your kind words, Andreas! Take care, and have a good day!
Thank you for outstanding information it's really help us, @
Szymon Stepniak
Thanks for your kind words, Mahesh! Take care, and have a good day!