Devops is Terrible
HTML-код
- Опубликовано: 14 июн 2023
- Recorded live on twitch, GET IN
/ theprimeagen
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact 
Наука
As the head of DevOps at some of the biggest companies in the world, all I can say is...
"Thanks for reaching out, Please raise a ticket through the correct channels for CICD support."
I need this at my work
@@replikvltyoutube3727 just build an auto response bot for the ticket system. It's fine
Just to get ignored
Oh god, I'm so glad I don't have to deal with this
I am in a fortune 200 tier company and its hard to find DevOps to help us with basic OPS shit. I am Dev and we always struggle to get help to lubricate the devops the philosophy of devops was supposed to be to connect dev and ops, but they proceeded with disconnecting dev team from ops team like its nothing
As others have mentioned, pls do the courtesy of linking to the article whenever you read it. It helps us viewers, as well as the author.
100% this. But maybe the sole point of this channel is to entertain and not to help.
@@changoviejo9575 truth.
You can solve this yourself. It’s not hard to search the title on google. Takes an extra 10 seconds of your time. Spoon-feeding is not owed.
You know what its called. Use your initiative
@@russellsanders4535 true, but also takes the reactor 10 seconds to link the article in their description. Helping boost the articles click through and adding text based credit
I come from a sys admin, and data center management background, and I have noticed a savage divide between developers and operations. As a DevOps eng, I'm constantly battling dev's requests to manually run code in production; individual user accounts with write access to DBs, or "just give me X resource I need publically available." My background forces me to look at costs, security, and performance, and developers DO NOT care. It is the same thing as it was with on-prem resources, bad code? add more CPU and or mem, app needs to communicate with X, just open the firewall, need a new service no documentation so round and round with requests they didn't know they needed. My solution has been self-service tools but I'm CONSTANTLY catching dev's miss using these tools and putting the company at risk! So I have to pull access and limit what they can do to protect the company. For my WoW players, DevOps is the healer of the raid no one cares until it affects them. Our job is to keep you from destroying yourself and the company.
I really like your take. Thank you for sharing.
This is an interesting point. If devs have the tools to provision their own resources, how do you avoid them throwing more resources at a problem without oversight?
How do you prevent opening a resource to the entire world “but it works” situations? Someone needs to be auditing these things somehow.
Maybe having a security auditing team and a performance auditing team rolling between all internal projects?
I've been in your situation and what I ended up doing was to sit down with management (eventually even the CTO) and explain that there need to be standard hard and fast security practices that nobody deviates from without approval (security team will be your ally on this), approved services with bulk buys to keep costs down and a team budget for deploying anything else, all enforced via the self-service tooling. We then revoked dev write access to all cloud APIs and ran workshops to help the dev teams onboard. Took us 2 years in a company of about 100 engineers.
The thing is, we needed a catalyst event (most companies will because you'll be working against entrenched culture).
In our case, we had an R&D project that pulled retail customer PII into ElasticSearch but the devs on that project went quick and easy and just deployed the AWS service. The problem was that this was new at the time and not supported in-VPC which meant open to the Internet. Project wound down and they didn't clean up. I come in, look at AWS billing, notice the line item and decide to investigate. Single document on cluster read something to the effect of "we have your data hostage, pay us BTC or we publish it". Well, there goes the quarter, call InfoSec, they hurry over and there's jaws on the floor. Eventually, the GDPR fine hit, CEO and CTO walk onto the engineering floor and declare "the cowboy dev era is over, everything goes through DevOps and InfoSec".
Good luck.
You are the gatekeeper at a petting zoo basically, and the kids will always be kids. It will never change
I also come as a sysadmin role and the only memorable experience I've had from our devops tooling was that it was constantly deleting or misconfiguring my configuration files and when I spoke to the devops team they tried to explain to me that they have a policy. Their policy was ad-hoc and went both against the documentation and the empirical evidence of how things actually work for that specific system. I showed them both the documentation and the evidence, they didn't fix it. I fixed it with a workaround so it will break again on the next reboot, because I literally had no other option. I had to explain to the customer why the goddamn app randomly stops working as if it's my fault, since I'm responsible for it. And yet I don't have access to the devops tooling that directly affects it and the devops team doesn't respect my input. Devops to me is more like that healer in raids that keeps crying how bad the tank and DPS are, crying how the healer's should never DPS and then misses all the healing timings and gets you killed. I'm not saying all devops is like that, but I feel like rule #1 of big companies is that nobody knows wtf they are doing.
If copying and pasting previous work was a sign that a job was bullshit, then I have grave news…
Golang in a nuttshell lol
@tablettablete186 Hello bootcamp student.
I'm fully on board that a part of my job is bullshit.
Now imagine moving this bullshit from one type of system to another. and then expect anything to work at all.
Name a job that isn’t repeating the same thing over and over, or at least having one part that you repeat over and over.
@@danielthompson2561damn, I have not had a job like that in a decade, maybe time for something more interesting?
When publishing videos like this, reacting to articles I mean, if you can share the link of the article in the description section that would be great... just saying... Great videos by the way
classic react video not crediting anything
It takes a shorter to type author, article name and google it, than it took you to write this comment, lol.
@@gercius Soy take, it takes like 5 clicks to copy the link and post it in desc
@@ThatOpinionIsWrong Yeah boy, those few seconds are too precious for your waifu pillow cuddling sigma male grindset.
You can look up what a p tag is in stack overflow but you cant google the article title? You're too virgin
The fact that we even need DevOps is a testament to the amount of completely unnecessary tech debt 99% of tech companies have.
the question of "how do we deliver code to customers?" is not one to avoid or 'pawn off'
I think it is more a byproduct of how at many companies the backlog is owned by product. Product rarely makes time for the types of work that are often considered "devops", but the works needs doing. Eventually there is a break-off group of devs that get to operate a backlog without the oversight of product. They are going to try to get on top of all the "devops" issues that have piled up and therefore turn into a _DevOPS team_.
Companies that give engineering more influence over team's backlogs may be more likely to evolve towards platforms, because there is less of a build up of toilsome "devops" work.
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence.
At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence.
Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) with in their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign is in in the works, well. Devs are now business on the re-arch. Not Devops! That's where we come in.
my guy, all internal politics aside:
undelivered product is UNFINISHED product. period. no way you can tell a customer "works on my machine" and call it a day.
@@davidthomson3575 I'd consider "devops" tech debt created by devs to be things like:
- our deployment process is someone manually running a script
- we are using a dev's IAM keys to give a server access to another service
And so many more. It is the brittle and insecure things that happen when corners are being cut that work until they don't.
The word 'velocity' in the dev space drives me nuts
I hate these terms
The espeed is alot mate
we gotta synergize to provide value through elevated velocity in the agile environment to disrupt the industry maaaaaaan
ok, but what happened to 'acceleration'?
jerk?
position?
@@jonathanalonso6492 I think we need to increase the 'jerk' on the 'acceleration' of our 'velocity' otherwise we just won't reach the 'position' we aspire to 😂
“One man’s golden path is another man’s golden shower” fixed that for ya
As a DevOps practitioner I obviously have some opinions on this article - mostly positive, tbh. I totally agree that too many businesses treat roles like SRE/so called 'DevOps teams' as an excuse to palm off responsibilities that should be shared. Also fully support the call for more platform engineering in place of a bajillion teams with a bajillion different cloud environments, all managed with whatever hand-rolled solution that team came up with. The reality is we've known about, and tried to steer away from, these problems within the DevOps community for a while now. A big part of the problem is the fact that, if you get into DevOps through a bootcamp, the only experience you get of implementing these engineering practices comes in the form of individual/small group projects, where the inherent problem in trying to coordinate these practices between tens or hundreds of development teams remains hidden. We really need to do more to expose trainee practitioners to problems of scale early on - DevOps is, after all, a philosophy for improving organisational performance, and a bootcamp simply can't do that justice.
This is also an issue in software engineering. There is no way to simulate scale-like work without actually working in an environment that has big data.
And there isn’t really anyone in CS trying to get an internship in DevOps because from what I’ve seen, it’s treated as a pivot for an engineer who doesn’t want to code as much.
So the only options are:
Hire more juniors and train them accordingly to take over once someone leaves
Continue as we are now
Someone (boogibugs) asked a great question in the Twitch chat: "Why is the operations team not classed as an engineering team? That's one of the issues right there."
Boogibugs is correct. Just look at Star Trek. Engineering and operations always wore the same division colors. #justsayin
Who cares abou the title
@@jordixboy The title decides how much pay and respect you are given.
DevOps was never intended to be a role. It's a concept. Have your Ops integrated into the development process.. influence the product architecture from the design stage. Not a place where you show up on Friday saying "I need this whole new thing deployed on Monday." Then go to the nearest member of management and cry you didn't deliver on time because those "damn devOps are blocking you".
DevOps is a concept of enablement, not a role of blockage.
Only in the most traditional and monolithic environments I’ve worked in have I seen ops not classified as engineering.
@@jonathanalonso6492 not at all bro
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence. Granted this can be a group of product devs or a separate team, eventually the former won't scale very long.
At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a product developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence.
Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) within their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign/re-arch is in in the works, well. Devs are now busy on the re-arch, not Devops! That's where we come in.
Thanks for sharing your gripes. I finally feel like I'm not the only one who has devops issues (mine was Docker compose, which I fixed w/ Swarm) - heck, you've shared issues I didn't even know were such a big thing (Terraform and autoscaler).
When you have a "DevOps" team you need to file a ticket with to get anything done, then, guess what, thats just the ops team with a buzzwordy name.
Yeah I agree, worked at a place where everything on infra was repo driven, you just sent a PR and post it on a channel to get a review, the checked if everything was fine, merge, wait a min and done
It was pretty good in contrast with the old ticket system I am used to work
"Devops does not work because the hired devops guy cuts corners and engineers should not work with ops. The solution? Build your own PaaS platform."
??????
Yeah, exactly. Bob, when do you think we can make that change to the frontend I was asking you about. Hey Joe, that will take me about 45 more days because I am busy building out the whole Platform. If you are a 1 man shop or a few dudes doing a startup, I understand. But when you have a scaling organization what you are referring to is not ideal.
@@davidthomson3575 Not only that. Even if you have the resources, you are still just rebuilding an inferior version of already existing AWS Kubernetes/Terraform style hosting services just to virtue signal that you are "doing deployment right".
@@adamih96 this is true, they're not in the PaaS business, are they?
Maybe if you build some quick macros to facade above those PaaS accounts and wrap common cookie cutter process in happy buttons and forms?
@@Fernando-ry5qt If easy enough to put behind dashboard buttons then there are likely already tools to automate that process within the same service or with something like github actions. The best button is no button, the best app is no app.
I'm "doing devops" in a small company and I like it. Important part is the end-to-end-responsibility, devs learning from ops problems. But it does require management understands "OPS > DEV" and you can't have 100% schedule management. We plan for 2w sprints (~scrumban) but it's just setting priority, what tracks is everyone focusing on. Tickets are done when they are done. Most important part of "my devops" is that once I hit the same thing Nth time, I will either do a bug fix (if I think I know the code cause) or add logging or setup monitoring. In traditional Development this can't happen because the guy looking at logs doesn't understand the source line behind the log line.
Compare a java dev w/1-2 years of experience to one with 10+. Pretty drastic difference, no? (sql injection? idor? session fixation? txn boundaries?) Same thing applies to 'devops'; k8s isn't just some 'configs', there's a whole ecosystem of best practices, frameworks rising/falling, architecture and so on. That equally applies to tracing/logging, etc
The reality is you end up with devs doing stuff in application code that should really be infra, but that's not the teams expertise. They are just copy/pasting the top SO results and effectively `chmod 777`ing your cloud account to get the server to work. Generally, theres no checks/balances (that's a silo, yo!) until you're "hacked".
I don't disagree, but I work with people who got 20 years of experience and still manage to produce the most ridiculous vulnerabilities known to man.
@@freesoftwareextremist8119 humans are imperfect, it's not about writing code with 0 vulnerabilities, it expecting them to be there and to put a practice in place to catch them and to not trust the human.
@@freesoftwareextremist8119 100%. There are old fools as well. Still, there's generally a correlation with experience and specialization.
This article shows me how much I have yet to learn. I understood very little of this article.
@ThePrimeTimeagen can you PLEASE post the links to the articles you talk about in the description?
Hey! Missing reference link... What article was that?
Can you add the articles into the description of videos?
I would find it very helpful if the read article was in the video description. I often find myself the need to re read this or share it and it's kind of tough right now. I also think it's fair for the writer of course.
google can help 😂 🕹
I WAS MLOPS AND NOW I’M PUSHING MOPS GIPPITY TOOK MY JERB
When will you become MFLOPS
Legend
We should all more aspire to be jacks in all trades than masters in just one.
It is very important to have overlapping skill sets and at least know what other roles are doing to enable communication and a mindset of shared responsibility and belonging.
Unfortunately big corps are usually driven by automated processes and use business units, which separate people into well defined boxes.
To say it with the words of Melvin Conway
`Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.`
The biggest problem no one thinks about is that there is a time gradient across the software you create and deploy.
DevOps only assumes that all 1000 different services are always up to date and deployed on a regular basis.
Good working services will rarely be modified and updated or deployed, so you create a rift between your latest deployed services and the oldest deployed services that may contain technology that does not even exist anymore.
When you have a shitty dev team ( which is 99% of the time) your services will be modified weekly ( if not daily)
@@houssem009 I am in currently in the 1% team. I hope it lasts long time.
Article just sounded like the guy was complaining about everything. “DevOps done by engineers is bad because the engineers hate it and suck at it.” Then he goes “DevOps done by the DevOps engineers is bad because they are not good at their jobs and just copy paste things” Then his solution is to rename the DevOps engineers to platform engineers and magically everything will be better.
The reality is that hard things are hard and random hat swapping is not gonna fix it. This is fundamentally a technology problem. When the tech changes then the situation will change.
In trying to solve waterfall development's "problems" we have created a hydra of problems. Waterfall was simple and could apply to anything. Software was less brittle that way. "Real Agile" is also good, but the common way it's done and this whole DevOps stuff is just a ton of make work for very little actual work that gets done. Most of the time, I'm just left shocked what people call work. It's akin to bureaucracy and paper pushing than actually building stuff.
Based
yeah honestly the amount of filler meetings to discuss work, or the amount of "investigate blah blah blah" tickets where the scope of the ticket is just to figure out how to do the actual work - it's just too much. the team would be much more productive if we cut the filler and just built the shit. sometimes the sheer volume of meetings feels like excuses for people to avoid getting their hands dirty with code
From an Engineer to a Comedian: A journey through time
CTO: Our software is complety secure.
Dev: Shell script goes brrr.
CTO: Suprise pikachu face.
OPS, SRE, DevOPS, Platform Engineering - the drama of what to call the job of getting stuff to be easily shipped and monitored is almost comical.
Exactly
My main problem with DevOps is that I don't even know what exactly are they supposed to even do.
Where do their responsibilities start and mine end?
Why are they refusing to give me a read only prod db access, when I do really need it for analytical stuff and our app isn't even close to being released yet?
"They"? As in a separate DevOps team? That's #1 in the article: DevOps doesn't mean you have a separate DevOps team.
@@JeyPeyy idk what happen to my comment.
devops is a production methodology for agile management.
devops, right now, work very well for web ressources .
Due to tech debt (or aging workforce), the average tech company is based on ITIL management.
where each step of the devops cycle is managed by a different team.
In those company the job title "devops" can mean a lot of thing (depending on HR).
@CHE6yp is talking to an infrastructure/platform admin .
Because read only DB access to a prod database can send the sucker crashing with a single query. This is not a best practice, because we learned long ago no one should do this.
You need a data pipelines or atleast a read replica.
You write code, you should worry only about writing code. If you want a shit ton of distractions that keep you from writing code, start worrying about all of the non code writing responsibilities that fall under the guise of devops.
@@davidthomson3575 Oh I am so glad you said this! Illustrates my main PITA in modern development perfectly.
In general, I would say giving any access to prod is highly dubious and not a "best practice". However, "best practice" is often just a massive distraction in itself when applied to a concrete example.
In my case, we have 2 backend devs and our project basically just agregates a bunch of data from a third party. All of that data is completely recoverable, and the amount of users is currently zero. The reason I needed a prod db is because prod is the only place that does have actual data, and my task was to basically mutate some field values to make them all confirm to the same standart. Nobody has a full list of actual values that need mutation, so basically I needed to make a bunch of SELECT DISTINCT queries just to find out what exactly is needed.
So, whats the path of least resistance and the best practice? Waiting for our DevOps/Platform Admin to make a replica, going to our prod frontend and manually cataloging all 2 million records, or, getting a prod (not even read-only necessarily) db access?
@@CHE6ypprobably asking for a data dump haha
could you link the article in the description ? it's more convenient when trying to get to the article
the company i work for has outsourced our "devops" team, so i really felt this 1000x
My company writes applications in PHP with a very simple stack.
The concept of 'devops' is foreign to us because we don't need to compile, transpile, restart services, etc. The dev hits save, then refreshes the browser to test.
Deploying to production is a single line command.
DevOps sounds like a nightmare, glad we aren't needing any of that (yet). :)
"we don't need to ... restart services" - what does this mean? You clearly don't understand the stack you're running on, how can you understand an issue of a higher order paradigm?
@@cloudboogie It means our application doesn't have services to restart during development. PHP doesn't work that way.
Been writing PHP for 15 years, i think i know what i'm talking about by now.
What is his name again? I didn't get it in the end
So the real problem is people don't value the support people...
Exactly, every DevOps person (yeah yeah or whatever title you want) has to not only shoulder the bitch work the Devs were too LAZY to do but then catch a bunch of shit about how “useless” they are at the same time, fun fun
@@itsdylac sadly, yes. Of course the ops people can be just as lazy.
@@disguysn they totally can, and incompetent too. But the competent Ops people get treated like trash, and the incompetent devs get a pass
@@itsdylac I wish I could contradict you. I moved from dev to devops at my last job and it's exactly as you stated.
This is unrelatable to me because our team develops, deploys end to end, writes the CDK infrastructure, handles oncall, etc. I think separate dev ops teams just encourage engineers to be lazy tbh
I used to do that. The problem with generalization of that sort is that you’ll keep tripping over amateur problems as it takes forever to develop expertise in any one area. I love the concept, but in practice it can lead to burnout.
@@samgould8567Yes. I too believe it’s better for every organization, with capacity, to engage individuals who have specialized in solving specific problems, like devops as a practice especially in the sense of ensuring reliability. You can’t have everybody doing everything, it will end in chaos. 😅
So who's managing the cloud then? Presuming you're on AWS, who's managing organization itself? Security, compliance, fin ops? Who's going to be waken up if CloudTrail logs have something suspicions - on-call from full-stack Team A or Team B? Who's gonna call you out on unused resources?
"FAST FUCKING AUTOSCALLERRRRR" has became a meme that is said so often in my work place it's basically echoing off the walls and in Teams chats 24/7.
Guys stop commenting below and open a ticket
Can we get an epic?
opposite of white glove is bare hands.
I remember years ago when I was a little spud starting out no software developer would touch anything related to devops or continuous integration or continuous delivery. They wouldn't even do any SCCM work if pushed. Then Google came along and created the site reliability engineer role.....Suddenly the software developers who swore against anything relating to chef, maven or ansible suddenly wanted to be knee deep in this shite. It was the best marketing ploy in tech in the last twenty years and tonnes of people fell hook line and sink for it.
Does anyone know if there is some Chrome extension to autoskip his squeaking seizures?
DevOps is fine tbh. Kubernetes is a great tool but sometimes overkill for small startups but if you are willing to create k8s manifest files and create helm charts, it makes things so much easier to deploy applications to your clusters. There is a lot to handle though and once you bring devsecops in, it immediately fucks up your CI/CD pipeline
"But companies pay me to FORGET TO TURN OFF ALERTS!" 😂
You should develop in the way you run on production either by running automated tests or just running in the containers - that’s no slower really if you set up your Docker. And your tests already helped you so when your tests secured your Docker should also test fine.
It’s a mortal sin to develop in a different environment and not automatically build your stuff! That’s when you forget to merge ImageMagick (uggghhh slow use OpenCV).
tom, we’re gonna need to double - nay, triple - the amount of DevFLOPS this quarter or WE ARE TOAST
where can i find this article he is reacting
I'm MLOps. It's just software engineering good practices around ML. Introducing CI/CD for ML. Introducing model monitoring.
Just like DevOps, it's mostly about the philosophy and requires "organization buy-in".
It's just that instead of having unit tests and monitoring for (just) code, we do it for the model performance and data.
And you don't make that work with a vague MLOps / DevOps title. You do that by writing code that enables that. Therefore, it's closer to the concept of SWE imo. (provided I'm not mixing terms)
I was stuck waiting for DevOps to set up a Jenkins slave for over a year. Turns out he just had to fill out a 5 minute form. Management tried escalating this repeatedly during that year.
To quote Mike Judge's magnum opus, "Office Space" (1999):
"What would you say you DO here?"
DevOps is a stupid buzzword name, but at least the point of taking the best practices of software development and applying to infrastructure delivery makes some sense. The challenge is mostly a people thing, nobody wants to learn how to do their job over again of let others take over their kingdom, and they resist in the most stubborn ways possible. Technology usually fails at the hands of an inflexible mind.
I'm planning to change career at 40 years old, which IT career should I go for? (need help, thanks)
devops
Hey Bro,
Love From India.
I'm learning devops , Pelase let me know is devops good career down the line say 10 years ?
Really need your help , I not a programmer just learning ops now .
A lot of developers just cannot be bothered to learn about cloud services or containers.
NGL - this one flew over my head a little bit.
An example can go a long way in explaining things, but this article is only an abstract wall of text.
And I still don't know what ADP means.
Abstract Data Ptype
^ Payroll!
IDP = Internal Developer Platform
Platform engineering is just applying DevOps principles to all of the tasks that get roped into DevOps enablement (the stuff DevOps teams tend to get farmed out to do). Make a clear boundry define your interface, everything in that is your baby, craddle to grave, everything else is a customer. This is true for your k8s/serverless/vm platforms, your CI/CD pipeline infrastructure, as well as the actual user facing applications.
Finally, someone finally said it.
And the very disturbing things is that bad to mediocre DevOps earn on average MORE than good software developer. I didn't believe it until a guy I know, who doesn't even know how to write a for-loop, who worked at RedHat told me his salary and showed me their repos... He was writing yaml files and introducing very lil amount of "code", I think that three of them full time for about a year still didn't produce as much code as I alone do in just a week. Another thing is that most of them are clowns and liked to be called engineers while they can't solve the easiest equation ever.
If they can't solve easiest equation ever, why don't you do it, perhaps you don't know how to write a yaml file either
@@abdulhamid.mmousa2329 because it seems that deploy is not a developer's concern.
@FredoCorleone is right, it's common that newbies are hired for DevOps on high salaries. Managers are impressed by the DevOps term... it's all a big crap and the results are shitty.
For me sounds it all like organizational problem, the employees were not prepared well and there was like no real plan for implementation step by step. Normally companies should start from docker and the slowly try out kubernetes, but yes some software vendors can sell Kubernetes very fast, me and my company we always ask our clients what are theirs real goal of achievement , when they have particular tech in mind.
Question: How many of you are doing devops without a college degree? I have no education beyond high school diploma, but my job history is sort of all over the place. I bounced around between C++ developer, IT Support, System Administrator, Python developer, PLC programmer, electronics manufacturing. Always used Linux and have been spinning up VPS to run my own git, mail, websites, etc.
Where do I go, what do I do? Certifications? Should I go to college? Both ideally obviously?
If there is a special team named as "DevOps team", and if other teams don't care about how to filter the online status etc. - As there is no feedback with a proper Closed-Loop, it is possible that it is already on the way to a failure.
Most important, DevOps or DevSecOps is a culture for engineering at first. Tools & process are only the method to implement that culture, to make each software engineer be a better one.
From my perspective working with DC infrastructure, DevOps seems like crap that either a) breaks my configuration files or b) deletes my configuration files... automatically. or c) doesn't do anything useful anyway because it can't handle servers or highly unpredictible new releases of enterprise software.
I agree in some sense as devs have to rely on devops personnel to get access to some resources
Are you Dr. Disrespects’ brother ? Man you both have almost the same voice, mustache, reaction and humor !!
Please create a ticket and we will get to it in our monday morning intake... code for leave me alone I am doing something that was dropped on my lap this morning that was previously techdebt and now is a fire. No one gives the ops people high fives unless they are on late getting the release fixed.
DevOps internal tooling is the way to go, even though it has downsides in terms of job flexibility for devops engineers and developers in general.
Thing is, if you want a naming scheme - enforce it!
Create an internal tool that validates those inputs to ad nauseum to leave 0 flexibility in naming to devs, otherwise, they will just NAH-prod_2-eu-west-2_test-2_July_momLookAtMeNoHands all over the place...
Ops work sucks and I don’t want to do it. For that reason I’m happy they are there when I have them.
The black ink-like texture at the bottom of the screen scared the shit out of me. LMAO :D
While this author isn't doing a great job acknowledging their biases (you know, the whole "I'm selling a product in this space" thing) there are some really good points there.
I've worked as a member of the DevOps team in a company with about 70 devs and what we'd have to do was assign each devops guy to a few dev teams based on how impactful their work was on production. In the beginning we did little more than Terraform all AWS shit and build CI/CD but I don't wanna be repeating the same task over and over so I built something to let the devs self-service. The biggest limitation for me was always my deep-seated hatred for the web frontend stuff so I built them a Python CLI tool to do the whole and published a Docker container on the internal registry. That tool and the automation around quickly became my job.
Fast forward to big tech now and we are gradually moving towards platform-building; I am liking the trend in my current employer of building internal tools to solve problems by enabling self-service for the different engineers. Our problem is we're not really building a cohesive platform, instead we're building a bunch of disjointed internal self-service tools that don't talk to each other and still need to be orchestrated by hand or another layer of tooling on top of them.
What I'd suggest is to start with dedicated platform teams that can own the whole platform stack consisting of, at least, some kind of workflow engine, generic compute, a modular/pluggable architecture, network and a consistent UI. I don't care if you're building a big monolith CLI, a massive web app console thing or just some tooling that consumes our beloved YAML, just have a consistent one-stop shop for people to launch and manage what they need and then get out of their way.
The problem is that "dev" is creation, and "ops" is maintaining. Completely opposite perspectives (and cultures). Very few people can do both, let alone well (with the right mindset).
We had a DevOps team. After a while, they renamed themselves to SRE. After we got used to the name, they renamed themselves to Production. Their lead even gave a presentation explaining why we should call them Production, not DevOps or SRE. It felt like an identity crisis :)
As someone that comes from a different sector and now entering devops that I have quickly realized that DevOps is a organizational philosophy that the entire company needs to adhere to. You can just hire two devops engineers and hope that will fix things.
1:30 12 years of data engineering, bi-dev and analytics .. you have no idea how true this line is xD
took a sip and laughed for half a minute fighting not to spill something xD
I cut so many corners that I have now found myself in a circle. Maybe its a spiral to hell, one can only hope.
Devops, scrum... all those things were created by middle management hoping to tame programmers
fr. I think each product should have an engineer making sure the infrastructure for that product is set in place. By using existing tools or makin new ones. I always found it weird that DevOps is a completely separated team.
Maybe have a team that is Product-agnostic that deals with the infra to deploy the pipelines and tools of each product, but at most that.
Each product has different needs, by having a team that deals with requirements for many projects just because "it's backend/infra stuff" makes their work extra hard, and in my opinion, breaks the organizational structure of the business
DevOps is a perfect example of how our industry fucks everything up. Development and Operations are *two different* teams. DevOps is the practice of these two teams working collaboratively instead of being at odds with each other.
If you've founded a "DevOps Team", you've failed to do even basic studies to learn what DevOps is and deserve every bit of pain and suffering you experience.
Don't get me started on Scrum and how utterly fucked up that is.
Nobody is actually doing CI/CD right. At least technically speaking.
If I’m not mistaken, CI/CD imply pushing directly to master and then building and deploying the new version on every push.
In reality this is a ridiculous approach, so some sort of change management is used. So instead we open PRs and all that.
Please learn what CI/CD is.
@@cloudboogie Isn't it exactly what they said?
@@HominisLupis
No.
Ci -> integrate changes of everyone in team often. Multiple times a day. That it. Everyone is working with the same project state without hiding anything.
CD -> always have working build at every step. Release to world or client is separate business decision.
The simple issue I have with most of the "Dev Ops" that I did was that the system I was forced to use that was described as a "Simple, opinionated declarative language" was that it forced you to copy pasta code that otherwise would have been simply handed with a variable. For example, for each deployment, I had to create several yaml files that had values embedded in them that had to be the same - stuff that was easily generated with a simple script.
Half the problem is that DevOps is introduced as a theory (we've all read The Phoenix Project) and a responsibility (Make automation to support the new thing on the new platform). Ultimately sounds like this individual is a bit jaded because they have taken on the company's miscellaneous catch-all requests and that becomes frustrating for everyone involved. I've participated in a startup where everyone had 'DevOps' in front of their title early on (and everyone considered their automation and infrastructure needs) and slowly migrated to a dedicated DevOps team. Fortunately those teams stayed connected and DevOps would constantly assess what problems Engineers would face and provide secure solutions and a few neat little observation tools along the way. 🤷♂
I don't know man, every time I read a new title introduced as the solution - I can't help but think that it stems from communication issues that leadership failed to address.
DEVOPS IS AWESOME, LOL DIDN"T WATCH AND WITH AD BLOCKER ON.
I want the opposite of white glove to be either "bare hand" or "garden glove"
For a second there I thought that you didn't forget to disable alerts. 😆
Devops IMHO is a discipline not a role, not a platform, not a team. Similar to agile software development
In theory, yes. But in practice note, this is why we have the role devops engineer now.
it's one of those useless terms that no one really knows what it means, like API
One time I looked at the dev ops team’s JIRA… never again
If you feel like you're useless as a devops engineer, you can install some new UI, like rancher or something. Or you can just limit access to devs so they can have new blockers. 😂
This is why heroku is amazing. No one wants to deal with DevOps. I just started at a job where they moved from heroku to AWS and no budget for DevOps. The moment I finish updating their old rails app to a newer version I'm moving everything back to heroku. This business wants new features and app bugs fixes for a super old app with 10% test coverage. I don't have enough time to also be dealing with DevOps crap from AWS.
isn't DevOps just make changes, push, deploy?
The most recent Stack Overflow Survey results would disagree with you.
@@ea_naseer /sarcasm ?
To get started! Sure, it's great. But so many have blazed that trail before you. Why do you think cloud exists because there's a market. Why don't all services run on Heroku?
It's great to get your ideas off the ground and you can run it for a longs as you want, at some point with real growth, you won't be able to.
Heroku is only good for small apps, as soon as you reach a certain threshold, I guarantee you gonna run out of memory or CPU regardless of the node scaling being used
I agree. The same goes for full stack garbage. Civilization is built on specialization and the key to succeeding is the collaboration of many specialists. Imagine this in any other industry: “we’re looking for someone who can work on every single aspect of the building: electrical, heating, plumbing, concrete, etc. We will assign each full stack trades person a room, and they are responsible for the whole thing.” You’d end up with inferior, inconsistent garbage, just like today’s software!
NX doesn’t do this. There needs to be another layer of abstraction that perhaps uses NX commands. I’m thinking along the lines of a platform orchestration tool that analyzes existing K8s configs and exposes a drag and drop editor so that you can visually create the new service and define the relationships in the service stack.
"No one high fives the autoscaler guy" somebody put this on a t-shirt!
When you realize you're somehow doing both dev ops team by yourself, and it's shit 😂
I'm currently studying AWS Developer Cert. Just completed the CCP foundational cert thru an AWS re/start program. And passed the exam. What is wrong with Operation work? I'm excited about this opportunity
This author could have saved us all some time by saying "Good engineering is hard, time consuming and expensive."
Overall he contradicts himself. He says "DevOps bad", but then says you need people with skills in both areas. 👍🤦♂️
Here's the "secret" of DevOps: it's just good Systems Engineering. We just call it DevOps for the benefit of management types who don't know DNS from a runaway chain reaction.
So much of what he complains about, is a sign that there are architectural issues or that the system has outgrown the staffing levels.
Also, his advocating for "engineers should be able to do whatever the hell they like" is about the most retarded thing I've heard in a loooong time.
Holy sweet Jesus! Have you ever had to support a heterogeneous environment? Ever had to take one over?
Yes, you need to be using the most appropriate tools for a job, but what tools you have *must* be used consistently, so that others can unpick your mess when you leave.
Does this mean that larger organisations become more stiltified? YES, OF COURSE IT DOES, because that's the fucking DEFINITION of a "big business".
Frankly, this guy comes across as "just" a programmer (not least of all because of his "hobbyist" comment!!!), who doesn't like doing serious engineering.
I'm a serious professional, fuck-you-very-much. I'm not a "hobbyist". This shit is HARD and made harder by tools like this that refuse to accept that engineering is about compromise.
Further, devs who have "no interest in the platform" are, by definition, amateurs; F1 drivers know almost as much about the cars as the people that designed and built them. This is the attitude that software engineers *need* to have about platform.
I love DevOps but you need to go 100% full in. And that costs time which you earn back in ops. At a client we are 100% DevOps we just provide the devs with a web menu, they select what they will be building and we create the yaml IaC, we create the dockerfile or telde serverless stub. We setup all the firewall rules, something non-standard they just can add to the yaml. We create the certificates and slipstream them during deployment. So a new certificate will be generated automatically when yours expires either by the next deploy or our resource manager pipelines.
The thing is my customer base are financial and energy, we can’t deal with outages. Especially the energy client when we do not deliver the energy we promised we could cause power outages.
The financial client is we do not send the reports to the ECB and or the National Bank before a set date, we can incur incredible fines or in worse case if that happens frequently loose a banking license.
So they need to be more mature than most other software companies in dev.
ride a cowboy, save a horse: the alternate title to brokeback Mountain
devops named the prod and the QA system PROD and non-PROD. this is terror inducing when you get an alert or are e.g. rebooting QA and read PROD in your peripheral visions
I'm not devops but I do this with some homologated terraform modules. I wouldn't ever want to do that full-time.
Welcome to Costco, I love you
Canaries is something most companies that are really serious about DevOps and CI/Cd (for that reason) cannot do. We as people that generate energy and deliver it to the grid only have 1 nuclear power plant and 1 coal power plant and 3 gas tribune plants. We can’t phase it out. We role it out and it need to work. If we fail, we disrupt the grid, causing power outages and when we are fully inlicht a blackout.
Banking is my other customer we deliver required reports to the ECB and Dutch Bank those are the only two and it needs to be correct! And it needs to be there in time.
And for accounts you can’t have a new feature running on X customers because you can’t role back financial transactions! It’s not allowed by law! This is why you can’t book back a transaction you did. Payments are final. And even according to SOX law you need to be able to prove how any transaction worked at any time. So your audit would and reproduction or evidence will be a hell!
A “fingerless glove” approach.
what's the name though?
"I wrote some of the trashiest PHP" - "No you just wrote some PHP"
You can talk about any subject without screaming! You woke everyone up