Terraform vs Pulumi. Who's best?
HTML-код
- Опубликовано: 10 сен 2024
- Join as I discuss the pro's and cons of Pulumi and Terraform. How do they compare to each other? Is one better and why?
Share you own opinions below! Love to hear what you think of each or if you've found a even better service.
Subscribe: bit.ly/3tjY9cy
Our community: chat.learndevo...
End to End DevOps: learndevops.co...
About Michael Crilly:
Michael Crilly is a leading expert in DevOps and automation. Mike has automated everything from Cisco hardware to AWS resources using various tools like Python, Ruby, Go, Terraform, Packer, Ansible, CloudFormation, CI/CD, JenkinsCI, GitLab, Docker, Kubernetes, GitOps, and much more. Mike also loves to teach everyone and anyone about the joy of automating complex Cloud resources to deliver a faster, better infrastructure that's cheaper to operate and more stable to manage.
Checkout my FREE courses!
What are IAC & CAC?: bit.ly/3BT6OG9
Terraform Crash Course: bit.ly/3nawGZO
Ansible Crash Course: bit.ly/3l4nKT0
Packer Crash Course: bit.ly/3kZkY1E
Terraform has its advantages and its inconvenients, like Pulumi. I prefer Pulumi but I understand that people can prefer Terraform because it is an interesting and well know product with a great community. However personally I don't think the reasons you gave are good reasons to choose one over the other. You said people may not know C# because it is not very common, python is not well known by DevOps engineers, javascript is too complicated .... but there are many development teams that know these languages, when using Pulumi people don't have to learn a completely new language like HCL that is only used in Terraform, they can reuse their existing development skills. As previously said in another comment, switching language is not a problem you use the same SDK whatever the language, moreover you can work with Pulumi components written in another language, it's not a problem. Now concerning the SAS argument, Terraform has also its own SAS Product (Terraform Cloud, that is how the company earns money), it's just that by default in Terraform you only store you state locally whereas the default in Pulumi is Pulumi Service but it is completely configurable to choose the backend to store your state. However one big difference is that by default Terraform does not encrypt the state whereas whatever the backend you choose for Pulumi (local, azure storage account, AWS S3) you get encryption by default and you can configure the encryption provider. Another thing you don't mention is that Pulumi has native provider for major cloud providers which means are always up-to-date (because automatically built against providers APIs) whereas new resources have to be manually implemented in Terraform providers which can take time. Again, I am not saying Pulumi is perfect or that Terraform is bad, but in my opinion if someone should choose Terraform it is not for the reasons you gave.
I wrote an article about the reasons why I would choose Pulumi over Terraform for my next project for those interested: www.techwatching.dev/posts/pulumi-vs-terraform
To add to this It's not just the language, it's the entire ecosystem. I can leverage nuget or nunit testing with c# and jasmine etc with JS/TS. The point of this is to be able to work with the tools you're close to already to cut the gap in learning a limited declarative language from scratch and also having to DIY the management of packages and libraries you want to support.
I've seen horrible implementations of Terraform too, nothing is stopping a bad developer from doing an awful job. If you have an engineer that does not know Python / JavaScript, is unwilling to learn and/or cannot write good code, what are the chances of that person writing good HCL code?
I've spent way too much time working around tools that impose limitations "for my own good". Give me a tool that lets me do what I want to do.
`totally agree. bottom line. the only tool ive worked with that provisioned and destroyed assets perfectly was CDK/Cloudformation. But most devops i know just find it too challenging
Sorry to say that, but your argumentation seems not valid to me. It seems as if you already worked a lot with terraform and never really tried pulumi.
Don’t be silly.
Totally agree with Schrom
Also agree with Schrom, the first argument against pulumi also applies to terraform also. If someone in an organization who used to do infra with terraform leaves and someone else comes along who used to do his infra with cloud formation template as he does, will he agree to learn HCL? And with regard to secrets, pulumi encrypts and manages them better, even though terraform doesn't have a build in concept for that and we're forced to use an external SAAS like aws Secret Manager or terraform vault, which is a SAAS offered by terraform. Don't denigrate technologies with bogus arguments if you don't like them, let them coexist.
Nice and straightforward 😊
Terraform is IaaC for devops. Pulumi is IaaC for developers. If you're not a so good devop, the next devop may fix your errors. If you're not a so good dev, the next dev will try to commit seppuku. That's always a case for the next dev when dealing with complex systems, and the code for Pulumi can get as large as anyone wants, so dev good practices needs to be applied as in any other complex codebase.
Developers should be kept away from infrastructure for a whole host of reasons.
@@MichaelCrilly If that was the case no devops had a job today
Not quite. Infrastructure teams brought development methods and tech to their side of the fence and then, eventually, DevOps became a holistic approach to the entire business cycle around DevOps.
@@MichaelCrilly Why? I see lots of people saying they're in DevOps but don't want to touch the Dev part, that's just Ops which is fine but DevOps imo is being able to leverage dev skills to automate the things.
@@markxavior this is just insulting. Software Engineers and Systems Engineers are two separate sides of the industry and both have their place. Focus on getting goods at one of them whilst understanding the other.
Besides, no one in Ops is afraid developers are going to take their jobs because we've all seen how developers "do" ops... very poorly in most cases.
It's also impossible for a developer to focus on software features, bug fixes, and more, whilst also looking after backups and recovery, observability, security, and more. If you want to try and do that, however, then be my guest.
Agreed. I haven't tried Pulumi, but for me, the slant toward declarativeness and standardization of Terraform is more of a vital feature than a setback when managing something as vital as infrastructure where you really can't afford to make a mistake: you make a mistake in frontend code and the user experience is not optimal, you make a mistake in your infra code and maybe your on-prem databases are deleted and you need to restore your data from a backup. Codifying a stable infra is doable, but being a little conservative and favouring simple correctness over complex design patterns is not a bad thing when you can't afford to make mistakes.
You can code regular languages in a more dependable declarative way using things like functional paradigms, but its not what I've seen most of the industry do. For a while, it seemed like functional js would be a thing, but increasingly, what I'm seeing are people sticking complicated imperative class patterns all over creation and additional layers of transpilation too, making the final deliverable hard to reason about. I definitely don't want my infra code to be like that.
Spot on.
Absolutely agree. Very surprised so many positive comments for Pulumi. When you have such unrestricted choices you end up thinking a lot less about how to simplify your infrastructure, and end up overengineering things, which is very typical in the cloud/k8s world (as opposed to Hashicorp's approaches), wasting resources, increasing chatter, instability, and ultimately decreasing maintainability and quality of service (money is spent putting out fires rather than improving the end product). Hardware/infra and software cannot be developed in the same way.
Thanks for the great succinct videos.
Exactly. Keep it simple. In fact I live by this motto: Keep it so simple a junior could maintain it for years to come.
As long as costs don’t go too high, I’d rather pay a little extra for compute versus a lot extra for more specialist, highly skilled, raw human talent - which is way more expensive than just keeping it simple.
Sounds like Pulumi with a functional programming language could provide a nice combination of flexibility, DRY, and determinism. I guess right now, the only option for that is F#, but I've heard it's a very nice language.
That’s an interesting take. Could work nicely.
I worked with Terraform for a few years. Depending on the organisation type May be a good option. Specially if you have a dedicated dev ops team . However after trying and using pulumi in production I must say that, with certain rules and control in place and in an organisation where dev ops is dealt by the engineers/devs, it could be a great option.
I actually totally agree. It’s not my first choice, but it is a good choice given the circumstances.
I see your arguments for standardization and default state mgmt at Pulumi. Still, for both I think its crucial that users understand the underlying abstractions; which are actually very similar. Also, managing state locally or at your own state mgmt instance is super easy to configure and practice: so this argument is imho really only about the default, not about the capability of the tool. Pulumi additionally makes it easier to build your own abstractions, which can be very useful, e.g., for reoccurring infrastructure patterns that are specific to your org. This also makes Pulumi a great tool for growing organizations that go from super-self-managed teams gradually to more standardization across the organization (towards an internal developer platform). On that path, various kinds and granularity of Pulumi-built abstractions can be very handy.
I think Pulumi and infrastructure SDKs are a bad idea. Imperative, and thus endless complexity, languages don’t belong in the declarative field. You want a language that let’s you define facts about the state you want. You don’t want a language that allows you to define those same facts in literally thousands of different ways.
We need standardised approaches that scale, meet business needs, and make it easier for the labour market to move their skills around (and businesses to hire from that labour market.)
Consider this: can a developer walk into a Python shop and understand a complex application within a few days? Not a chance. There’s simply too much variation in how things can be done that the learning curve is steep.
The same doesn’t apply to HCL and Terraform - there is only one way to define a resource object, or use a data call.
@@MichaelCrilly Again, it totally see your point regarding standardization; that's a big plus of Terraform and one idiomatic input language.
However, imho HCL is not complete enough; eventually, it is a configuration language. More and more complexity moves from inside applications into their composition, e.g., what used to be an easy-to-deploy monolith now may be tens or hundreds smaller components. And this trend may continue: Having powerful abstractions is a need to combat that complexity - perhaps not for everyone, but for some of us.
I totally agree, that imperative code opens room for many unnecessary mistakes, which declarative code prevents. That's a clear case for functional programming; Most probably, we would like to have an IaC Haskell :D And at the moment, I see using mainstream languages (most if them meanwhile implement at least some basic FP concepts, allowing to write functional code; even Java gets pattern matching!) as being way closer to IaC Haskell than HCL.
Just write a Terraform provider for any additional functionality you need. It’ll be first class, declarative, and cross platform.
You don’t mention that Pulumi has a paid version with really useful stuff that an organization an leverage. You also don’t mention that TF has serious concurrency issues. I have run into inconsistent state in TF a few times, where I have to manually tap into the state file. Editing the state file directly is awful practice. This video is very biased and you have no clue what Pulumi really is. 7 out of 10 I ask, they prefer Pulumi
Cool. Then use Pulumi.
@@MichaelCrilly yep. POCing to leave TF behind
Are you coming from the same open source world that I am?
When is choice a bad thing?
Can you mess things up with too many options? Absolutely YES! But it's all on you.
You mess things up, you can fix them as well. Freedom is NEVER a bad thing.
Bending yourself backwards to fit yourself to what your tools can do is not a good thing.
Your tools should be flexible enough for you express your desires.
If freedom and choices were the reason why open-source prevailed what makes you think a solution that forces you to express yourself via a pretty restrictive and clunky DSL that will drive you to madness trying to share variables between modules?
You're making the same case that programmers made for Java so many years ago.
"Java is good because it's restrictive and you can only do things one way"
Look at them now. Beating a dead horse, trying to add features that have existed on other programming languages, not so much more modern (actually, python is older than Java).
No one wants to write Java anymore unless they have to.
The restrictive nature of terraform will drive people away and soon enough, they'll understand that, taking a stance on staff like global variables and stupidass decisions like the order of evaluation of blocks which was probably chosen by fair dice roll are things that are driving people away, and the next thing you know, they'll incorporate terragrunt (which is absolute dogshit btw) but it still won't be enough.
Moral of the story, having options is ALWAYS better than "My way or the highway".
*But Terraform has its limits too, how does TF compare to SDKs like Boto3 and Azure pip and Bicep 😐 ?*
Far better. Boto3 is just a library.
A lot of your problems with pulumi are organizational issues not problems between the tools. People going rogue and implementing one-off abstractions and processes without support is not a pulumi problem. "The fact that you can use an imperative language to do anything you want is not a good thing" 100% opinion based again around your views of an organization that doesn't support the tool and people can just do whatever they want. I'd like to see you put actual facts behind the capabilities of the tools on display here, not your opinions on how organizations work.
The tool is a tool. Use it if you like. People are people, however, and Pulumi deploys an imperative language, something that isn’t new, thus people will continue to apply the same crazy patterns to their Pulumi code as they do when writing software.
Have you moved between five organisations and thought, “This is code is so well written, maintained, and easy to work with?” I doubt it.
That’s because programming languages allows the developer a lot of freedom - which is good in the software world, but bad in the infrastructure world.
I’ve removed your other two responses as they’re needlessly aggressive. If you’re going to act like a man-child I’ll just ban you from commenting in future.
@@MichaelCrilly no need won't be watching any more of your worthless videos
Pulumi is a game changer especially with the AutomaionApi.
“Use the Pulumi engine as a strongly typed SDK in your application code, enabling it to call functions that can provision and configure infrastructure on any cloud.” - what, like the API of the provider itself?
@@MichaelCrilly Automation API lets you control the engine via code. So things like creating stacks, pulumi up, preview, destroy, etc are all available via an API. It allows users to do more advanced things outside of the pulumi command line. One example, we use envkey as a source of truth for config so I'm able to integrate that into pulumi's config system. I've done it with 1password but the config source can literally be anything. Another example, we're able to build stack dependencies so if I stand up a stack, our custom IaC tool is able to see if that stack depends on any others if so, it can stand those up or warn the user.
And for those that don't want to learn a language or HCL, Pulumi supports yaml now.
But you’ve been able to do all of that with Terraform since 0.1? Writing a provider isn’t difficult, and you can write and use any external executable to feed information into Terraform, again, since day one.
That also sounds like it’s taking core out of the GitOps workflow, with all its advantages, and allowing developers to skip code reviews and simply deploy whatever they like, whenever they like.
Am I misunderstanding?
@@MichaelCrilly Not a provider, I'm not talking about a provider, its hard to explain because terraform doesn't have an equivilent to automation api. It would be like wiring code to call terraform preview or apply without using the terraform binary and in doing so, you can programaticlly feed it config however you want.
Like calling a CI/CD pipeline? Or having one trigger on an event? Or Terraform Cloud? Or Atlas? I do get what you mean. These are all solved problems. They’ve been done over and over, Derek. Time and time again. Very little is new anymore.
I’m glad Pulumi is working for you, truly. It sounds like it’s rocking your world, and I hope it continues to serve you well. Just know that nothing you’ve told me is new or currently unavailable in Terraform.
(Also, you’re aware you can use Terraform native with Python now? Using its CDK?)
@MichaelCrilly - great video, and super discussion in the comments below. I like how you defended your points. People did make interesting and insightful comments, learnt a lot. At this early stage in my DevOps journey, TF seems the way to go as its stable and reduces complexity. Base layers(infra) must always be stable and easy to maintain. Even though High Level languages provide a common codebase, their use in IaC appears to add unrequired complexity - new user training, dev discipline, code standardization, large org with different teams and app codebases.
That said, it appears to me that Pulumi might be a better tool for smaller teams with good ops and documentation skills who need to be able to manipulate their infrastructure quickly using a single codebase. building proof of concepts maybe.
Thanks again for the video, you've got yourself a new subscriber 🙂
I’m super glad you got value from this. I’ll do more content on this as I think it’s important.
*Terraform docs are terrible.* No examples. It encourages copy and paste from old code.
You encourage you to copy paste code. That’s your choice.
*In TF if your state file gets deleted or becomes corrupt then your screwed 😐*
Back it up. S3 has versions you can enable and replication to another Bucket, even in another region. You have no reason or excuse for losing a terraform state file.
Wouldn’t that file be in version control?
@terry.chootiyaa How's that different from Pulumi though?
In my experience, cloud brought infrastructure closer to developers. Pulumi takes this further by allowing developers to describe the desired infrastructure state in the programming language that they already know. I defer with the programming language challenges quoted in this video because, all of these would be true for the applications built on that programming language, and Pulumi being polyglot, most companies would choose the language that their company already uses. Regarding vendor dependency, pulumi is atleast opensource, compared to Terraform. At the end of this video, it felt like there were some biases in favor of Terraform
why is your video so dark
I’m a Death Metal kind of guy.
Terraform is always boasted for being declarative. But I dont think I've been in an organization that has terraform that doesnt have conditionals and loops 😅
Is that just me? 😅
No, that’s not just you. That’s very common. Looping in HCL is merely a means of structuring data in a way that lets you add or subtract from the data set whilst only having a single resource definition do the work for you (by looping over the data set.) It’s simply a convenience.
Sorry but i totally disagree with you. Javascript is "cpmplex"? and for devops? a devoops guy that cant write javascript most likely wont be able to code in Python. thats the honest truth. Plus-the current trend in deveops is to give as much responsebility to developers when it comes to pipelines and integrating with apps. you keep forgeting that not everything is about provisionong an infra, which is basically what terraform dose better(but destroying? oh boy what a mess!). we are far beyond the stage that IaaS tools are needed just for constant reuse for infra. Pipelines is the deal. and if you work in a dynamic hotzone you need the dev team to do some hands-on. this is the way. of course nothing is 100%. but this is our job as devops to refine the and make sure it would work before it passes to prod. you just miss the bigg advantage of saving time both bridging the gaps of minor details us devops dont see and other collaborators can
This video is biased. It’s clear many teams are being converted to Pulumi and it’s not their marketing. To viewers, the growth of Pulumi and conversion many teams from TF to Pulumi is the only valid indication of their value. It’s at least worth playing with it in your own time and see for yourself if it’s worth switching.
Just like any tool, really.
Starting to come around to Pulumi because I can just use my fav language, Python. Terraform is not as declarative as it needs to be, that's why we end up with all these damn wrappers: Terragrunt, Terramate, Terraspace, etc. Might as well use a full blown general purpose language. The organization and style does end up being your decision but you can create functional declarative Python and yeah you have to document what you've done, but I'm not sure Terraform doesn't have the documentation problem at all. You run into terraform formatting, setup, structure things, etc. I've seen people do the whole recursive Terragrunt thing of death and it's just as bad as any other straight up "dev' project. Don't take the 'dev' out of devops, it makes no sense. You still have to follow best practices as any other "dev" project tbh.
Terraform makes people write obscure scripts. Looping is ᵃ horrible business and TF forces you to hack scripts. I really dislike TF. The dsl is poorly designed
Your use of it is poor, as is your understanding of how it’s designed to be used. You’re overthinking IAC and the fact IAC isn’t generalised programming - it’s state declaration.
@@MichaelCrilly sure
I think you can debate a lot of the things Michael said. I however tried Pulumi today and the first thing it wanted to do after doing pulumi new is setup pushing my stuff to their service. I think I may just fall back to terraform.
I tried pulumi back when it was still very early days of the tool. There was only javascript at the time and I don't recall it being that hard to figure out setting up a local only setup. I'm guessing that doesn't pay the bills however.
I hate it’s defaults behaviour. So insecure. Overall it’s a valid tool, but there are simply better tools.
@@MichaelCrilly How is it insecure?
It’s pushing your state to someone else’s network without asking or telling you upfront. It just does it. That’s terrible, insecure behaviour.
There are other backends. I just used the pulumi cloud thing when going through the Getting Started guide.
Perhaps a bit of a "hot take", however, in my experience, Pulumi is for people who cannot be bothered to learn Terraform. With all due respect, a technical professional who cannot be bothered with continuous professional development is, perhaps, in the wrong field. One should always aim to use the right tool for the job, we should never just use the tool we know as Maslow's hammer.
Please don't create nested class hierarchies. Especially don't do it for just a configuration software. That's not about pulumi at all.
Pulumi is IaC for actual engineers and not “infrastructure” people 😂
testing in TF is almost non existent besides looking at the plan. That’s not testing. Pulumi will allow to really unit test it. Devops people barely test anything. That’s why they like TF while devs like Pulumi
You don’t test your Pulumi neither. Also, what are you testing? The code or the result? If you’re testing the code, then you REALLY don’t get it.
@@MichaelCrilly I think you are the one that doesn’t get it. This is a biased video (and I mostly do TF by far). Look at the comments around. Why is it so hard to admit you are being biased I know little about Pulumi? I don’t think you’re qualified to review a comparison when you know little about one of them
So the choice of languages in Pulumi is a shortcoming? This is like saying that having ABS in your car is bad because when you learn to drive with ABS, and if it malfunctions, you have no skill to recover from sliding in slippery roads.
No? The wide range of choices means there will be no consistency between organisations, even departments inside of the same org, and there is too much room for “creative thinking” that leads to a fragmented, hard to maintain code base. HCL helps reduce this a lot.
@@MichaelCrilly You do not have to repeat the same you said in the video. Your point is pretty much clear, as well as my point - "choice is not a real shortcoming". It is not the job of technology(or tooling) to safeguard you from bad decisions in a dysfunctional organization.
The only argument I agree - little to no DevOps know typescript(or any other language). But Pulumi is probably not intended for DevOps, it tries to bring infrastructure closer to pure dev teams. So typescript kiddies could spin their own AWS EKS cluster. Weather it succeeds in that or not IS a far more interesting topic. If that is even possible to remove the need for low level knowledge from running your infrastructure? And I am using "low level" loosely, but I think you know what I mean.
@sarunas8002 infrastructure is not simple. You’re overly simplifying it and making it out to be “a few lines of TypeScript and you’re done”. You’ve done serious infrastructure work: I can tell. Writing software is hard, but your job as an SWE is very focused on converting business logic to code. As a DevOps Engineer I have to focus on so much more than simply running your code - monitoring, security, networking, backups, recovery, scaling, zero-downtime, automation… and I have to do all of that as code.
You’re a reductionist.
yes it was helpful and I agree. Pulumi is the best way to keep DevOps away, only developers enjoy adding unnecessary complexity
Terraform CDK, it's the best of both worlds and terraform's answer to Pulumi. It gives you full access to the terraform ecosystem and a real programming language (including that languages entire ecosystem) to manage configuration.
The only reason I can see to continue using terraform is non-developers who are already entrenched into terraform, but even then they're better off upskilling to whatever programming language your company primarily uses and bringing over their existing terraform knowledge.
I would never willingly ask a team who hasn't used terraform before to start with HCL these days.
Then you’ve never worked with large amount of teams, all using different languages to manage IAC, and all doing it so differently that there’s no interoperability at all.
Good luck with that.
So... Pulumi is worst because of... organization changes?
No. CDKs don’t provide a unified, consistent approach across organisations, so you end up with a labour force that cannot move between businesses and fully adapt their skills unless the exact same stack is being used by the two organisations they’re moving between. If one uses Python and the other JavaScript, the labourer has to retrain entirely.
We need consistency and stability in this field, not yet more distractions and abstractions.