I come from a sys admin, and data center management background, and I have noticed a savage divide between developers and operations. As a DevOps eng, I'm constantly battling dev's requests to manually run code in production; individual user accounts with write access to DBs, or "just give me X resource I need publically available." My background forces me to look at costs, security, and performance, and developers DO NOT care. It is the same thing as it was with on-prem resources, bad code? add more CPU and or mem, app needs to communicate with X, just open the firewall, need a new service no documentation so round and round with requests they didn't know they needed. My solution has been self-service tools but I'm CONSTANTLY catching dev's miss using these tools and putting the company at risk! So I have to pull access and limit what they can do to protect the company. For my WoW players, DevOps is the healer of the raid no one cares until it affects them. Our job is to keep you from destroying yourself and the company.
This is an interesting point. If devs have the tools to provision their own resources, how do you avoid them throwing more resources at a problem without oversight? How do you prevent opening a resource to the entire world “but it works” situations? Someone needs to be auditing these things somehow. Maybe having a security auditing team and a performance auditing team rolling between all internal projects?
I've been in your situation and what I ended up doing was to sit down with management (eventually even the CTO) and explain that there need to be standard hard and fast security practices that nobody deviates from without approval (security team will be your ally on this), approved services with bulk buys to keep costs down and a team budget for deploying anything else, all enforced via the self-service tooling. We then revoked dev write access to all cloud APIs and ran workshops to help the dev teams onboard. Took us 2 years in a company of about 100 engineers. The thing is, we needed a catalyst event (most companies will because you'll be working against entrenched culture). In our case, we had an R&D project that pulled retail customer PII into ElasticSearch but the devs on that project went quick and easy and just deployed the AWS service. The problem was that this was new at the time and not supported in-VPC which meant open to the Internet. Project wound down and they didn't clean up. I come in, look at AWS billing, notice the line item and decide to investigate. Single document on cluster read something to the effect of "we have your data hostage, pay us BTC or we publish it". Well, there goes the quarter, call InfoSec, they hurry over and there's jaws on the floor. Eventually, the GDPR fine hit, CEO and CTO walk onto the engineering floor and declare "the cowboy dev era is over, everything goes through DevOps and InfoSec". Good luck.
I also come as a sysadmin role and the only memorable experience I've had from our devops tooling was that it was constantly deleting or misconfiguring my configuration files and when I spoke to the devops team they tried to explain to me that they have a policy. Their policy was ad-hoc and went both against the documentation and the empirical evidence of how things actually work for that specific system. I showed them both the documentation and the evidence, they didn't fix it. I fixed it with a workaround so it will break again on the next reboot, because I literally had no other option. I had to explain to the customer why the goddamn app randomly stops working as if it's my fault, since I'm responsible for it. And yet I don't have access to the devops tooling that directly affects it and the devops team doesn't respect my input. Devops to me is more like that healer in raids that keeps crying how bad the tank and DPS are, crying how the healer's should never DPS and then misses all the healing timings and gets you killed. I'm not saying all devops is like that, but I feel like rule #1 of big companies is that nobody knows wtf they are doing.
As the head of DevOps at some of the biggest companies in the world, all I can say is... "Thanks for reaching out, Please raise a ticket through the correct channels for CICD support."
I am in a fortune 200 tier company and its hard to find DevOps to help us with basic OPS shit. I am Dev and we always struggle to get help to lubricate the devops the philosophy of devops was supposed to be to connect dev and ops, but they proceeded with disconnecting dev team from ops team like its nothing
@@russellsanders4535 true, but also takes the reactor 10 seconds to link the article in their description. Helping boost the articles click through and adding text based credit
I'm fully on board that a part of my job is bullshit. Now imagine moving this bullshit from one type of system to another. and then expect anything to work at all.
When publishing videos like this, reacting to articles I mean, if you can share the link of the article in the description section that would be great... just saying... Great videos by the way
I think it is more a byproduct of how at many companies the backlog is owned by product. Product rarely makes time for the types of work that are often considered "devops", but the works needs doing. Eventually there is a break-off group of devs that get to operate a backlog without the oversight of product. They are going to try to get on top of all the "devops" issues that have piled up and therefore turn into a _DevOPS team_. Companies that give engineering more influence over team's backlogs may be more likely to evolve towards platforms, because there is less of a build up of toilsome "devops" work.
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence. At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence. Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) with in their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign is in in the works, well. Devs are now business on the re-arch. Not Devops! That's where we come in.
my guy, all internal politics aside: undelivered product is UNFINISHED product. period. no way you can tell a customer "works on my machine" and call it a day.
@@davidthomson3575 I'd consider "devops" tech debt created by devs to be things like: - our deployment process is someone manually running a script - we are using a dev's IAM keys to give a server access to another service And so many more. It is the brittle and insecure things that happen when corners are being cut that work until they don't.
@@jonathanalonso6492 I think we need to increase the 'jerk' on the 'acceleration' of our 'velocity' otherwise we just won't reach the 'position' we aspire to 😂
Someone (boogibugs) asked a great question in the Twitch chat: "Why is the operations team not classed as an engineering team? That's one of the issues right there." Boogibugs is correct. Just look at Star Trek. Engineering and operations always wore the same division colors. #justsayin
DevOps was never intended to be a role. It's a concept. Have your Ops integrated into the development process.. influence the product architecture from the design stage. Not a place where you show up on Friday saying "I need this whole new thing deployed on Monday." Then go to the nearest member of management and cry you didn't deliver on time because those "damn devOps are blocking you". DevOps is a concept of enablement, not a role of blockage.
"Devops does not work because the hired devops guy cuts corners and engineers should not work with ops. The solution? Build your own PaaS platform." ??????
Yeah, exactly. Bob, when do you think we can make that change to the frontend I was asking you about. Hey Joe, that will take me about 45 more days because I am busy building out the whole Platform. If you are a 1 man shop or a few dudes doing a startup, I understand. But when you have a scaling organization what you are referring to is not ideal.
@@davidthomson3575 Not only that. Even if you have the resources, you are still just rebuilding an inferior version of already existing AWS Kubernetes/Terraform style hosting services just to virtue signal that you are "doing deployment right".
@@Fernando-ry5qt If easy enough to put behind dashboard buttons then there are likely already tools to automate that process within the same service or with something like github actions. The best button is no button, the best app is no app.
As a DevOps practitioner I obviously have some opinions on this article - mostly positive, tbh. I totally agree that too many businesses treat roles like SRE/so called 'DevOps teams' as an excuse to palm off responsibilities that should be shared. Also fully support the call for more platform engineering in place of a bajillion teams with a bajillion different cloud environments, all managed with whatever hand-rolled solution that team came up with. The reality is we've known about, and tried to steer away from, these problems within the DevOps community for a while now. A big part of the problem is the fact that, if you get into DevOps through a bootcamp, the only experience you get of implementing these engineering practices comes in the form of individual/small group projects, where the inherent problem in trying to coordinate these practices between tens or hundreds of development teams remains hidden. We really need to do more to expose trainee practitioners to problems of scale early on - DevOps is, after all, a philosophy for improving organisational performance, and a bootcamp simply can't do that justice.
This is also an issue in software engineering. There is no way to simulate scale-like work without actually working in an environment that has big data. And there isn’t really anyone in CS trying to get an internship in DevOps because from what I’ve seen, it’s treated as a pivot for an engineer who doesn’t want to code as much. So the only options are: Hire more juniors and train them accordingly to take over once someone leaves Continue as we are now
Compare a java dev w/1-2 years of experience to one with 10+. Pretty drastic difference, no? (sql injection? idor? session fixation? txn boundaries?) Same thing applies to 'devops'; k8s isn't just some 'configs', there's a whole ecosystem of best practices, frameworks rising/falling, architecture and so on. That equally applies to tracing/logging, etc The reality is you end up with devs doing stuff in application code that should really be infra, but that's not the teams expertise. They are just copy/pasting the top SO results and effectively `chmod 777`ing your cloud account to get the server to work. Generally, theres no checks/balances (that's a silo, yo!) until you're "hacked".
@@freesoftwareextremist8119 humans are imperfect, it's not about writing code with 0 vulnerabilities, it expecting them to be there and to put a practice in place to catch them and to not trust the human.
Yeah I agree, worked at a place where everything on infra was repo driven, you just sent a PR and post it on a channel to get a review, the checked if everything was fine, merge, wait a min and done It was pretty good in contrast with the old ticket system I am used to work
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence. Granted this can be a group of product devs or a separate team, eventually the former won't scale very long. At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a product developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence. Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) within their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign/re-arch is in in the works, well. Devs are now busy on the re-arch, not Devops! That's where we come in.
I'm "doing devops" in a small company and I like it. Important part is the end-to-end-responsibility, devs learning from ops problems. But it does require management understands "OPS > DEV" and you can't have 100% schedule management. We plan for 2w sprints (~scrumban) but it's just setting priority, what tracks is everyone focusing on. Tickets are done when they are done. Most important part of "my devops" is that once I hit the same thing Nth time, I will either do a bug fix (if I think I know the code cause) or add logging or setup monitoring. In traditional Development this can't happen because the guy looking at logs doesn't understand the source line behind the log line.
My main problem with DevOps is that I don't even know what exactly are they supposed to even do. Where do their responsibilities start and mine end? Why are they refusing to give me a read only prod db access, when I do really need it for analytical stuff and our app isn't even close to being released yet?
@@JeyPeyy idk what happen to my comment. devops is a production methodology for agile management. devops, right now, work very well for web ressources . Due to tech debt (or aging workforce), the average tech company is based on ITIL management. where each step of the devops cycle is managed by a different team. In those company the job title "devops" can mean a lot of thing (depending on HR). @CHE6yp is talking to an infrastructure/platform admin .
Because read only DB access to a prod database can send the sucker crashing with a single query. This is not a best practice, because we learned long ago no one should do this. You need a data pipelines or atleast a read replica. You write code, you should worry only about writing code. If you want a shit ton of distractions that keep you from writing code, start worrying about all of the non code writing responsibilities that fall under the guise of devops.
@@davidthomson3575 Oh I am so glad you said this! Illustrates my main PITA in modern development perfectly. In general, I would say giving any access to prod is highly dubious and not a "best practice". However, "best practice" is often just a massive distraction in itself when applied to a concrete example. In my case, we have 2 backend devs and our project basically just agregates a bunch of data from a third party. All of that data is completely recoverable, and the amount of users is currently zero. The reason I needed a prod db is because prod is the only place that does have actual data, and my task was to basically mutate some field values to make them all confirm to the same standart. Nobody has a full list of actual values that need mutation, so basically I needed to make a bunch of SELECT DISTINCT queries just to find out what exactly is needed. So, whats the path of least resistance and the best practice? Waiting for our DevOps/Platform Admin to make a replica, going to our prod frontend and manually cataloging all 2 million records, or, getting a prod (not even read-only necessarily) db access?
DevOps was developed with the four key metrics in mind and explicitly defines its success through: - Deployment Frequency - Lead Time for Changes - Mean Time to Recovery (MTTR) - Change Failure Rate which I find great! Since I came to know DevOps in around the 2015s or so, I found no company, no Project, nothing, where these four metrics are computed and made available, neither confidential nor public. I mean, besides the companies supplying to the "Devops Report". I never worked for a company in the report, afaik. It seems this is the main issue: If you introduce something that relies on measurability to function but fail to measure, the result is likely to mutate into something undesirable. Makes sense, right?
Exactly, every DevOps person (yeah yeah or whatever title you want) has to not only shoulder the bitch work the Devs were too LAZY to do but then catch a bunch of shit about how “useless” they are at the same time, fun fun
This is unrelatable to me because our team develops, deploys end to end, writes the CDK infrastructure, handles oncall, etc. I think separate dev ops teams just encourage engineers to be lazy tbh
I used to do that. The problem with generalization of that sort is that you’ll keep tripping over amateur problems as it takes forever to develop expertise in any one area. I love the concept, but in practice it can lead to burnout.
@@samgould8567Yes. I too believe it’s better for every organization, with capacity, to engage individuals who have specialized in solving specific problems, like devops as a practice especially in the sense of ensuring reliability. You can’t have everybody doing everything, it will end in chaos. 😅
So who's managing the cloud then? Presuming you're on AWS, who's managing organization itself? Security, compliance, fin ops? Who's going to be waken up if CloudTrail logs have something suspicions - on-call from full-stack Team A or Team B? Who's gonna call you out on unused resources?
Article just sounded like the guy was complaining about everything. “DevOps done by engineers is bad because the engineers hate it and suck at it.” Then he goes “DevOps done by the DevOps engineers is bad because they are not good at their jobs and just copy paste things” Then his solution is to rename the DevOps engineers to platform engineers and magically everything will be better. The reality is that hard things are hard and random hat swapping is not gonna fix it. This is fundamentally a technology problem. When the tech changes then the situation will change.
I agree. The same goes for full stack garbage. Civilization is built on specialization and the key to succeeding is the collaboration of many specialists. Imagine this in any other industry: “we’re looking for someone who can work on every single aspect of the building: electrical, heating, plumbing, concrete, etc. We will assign each full stack trades person a room, and they are responsible for the whole thing.” You’d end up with inferior, inconsistent garbage, just like today’s software!
My company writes applications in PHP with a very simple stack. The concept of 'devops' is foreign to us because we don't need to compile, transpile, restart services, etc. The dev hits save, then refreshes the browser to test. Deploying to production is a single line command. DevOps sounds like a nightmare, glad we aren't needing any of that (yet). :)
"we don't need to ... restart services" - what does this mean? You clearly don't understand the stack you're running on, how can you understand an issue of a higher order paradigm?
@@cloudboogie It means our application doesn't have services to restart during development. PHP doesn't work that way. Been writing PHP for 15 years, i think i know what i'm talking about by now.
In trying to solve waterfall development's "problems" we have created a hydra of problems. Waterfall was simple and could apply to anything. Software was less brittle that way. "Real Agile" is also good, but the common way it's done and this whole DevOps stuff is just a ton of make work for very little actual work that gets done. Most of the time, I'm just left shocked what people call work. It's akin to bureaucracy and paper pushing than actually building stuff.
yeah honestly the amount of filler meetings to discuss work, or the amount of "investigate blah blah blah" tickets where the scope of the ticket is just to figure out how to do the actual work - it's just too much. the team would be much more productive if we cut the filler and just built the shit. sometimes the sheer volume of meetings feels like excuses for people to avoid getting their hands dirty with code
mate that is corporate in a nutshell (beyond a small sized company everything becomes mostly beaurocracy and politics and keeping up appearances through sheer company size
And the very disturbing things is that bad to mediocre DevOps earn on average MORE than good software developer. I didn't believe it until a guy I know, who doesn't even know how to write a for-loop, who worked at RedHat told me his salary and showed me their repos... He was writing yaml files and introducing very lil amount of "code", I think that three of them full time for about a year still didn't produce as much code as I alone do in just a week. Another thing is that most of them are clowns and liked to be called engineers while they can't solve the easiest equation ever.
@FredoCorleone is right, it's common that newbies are hired for DevOps on high salaries. Managers are impressed by the DevOps term... it's all a big crap and the results are shitty.
DevOps is a stupid buzzword name, but at least the point of taking the best practices of software development and applying to infrastructure delivery makes some sense. The challenge is mostly a people thing, nobody wants to learn how to do their job over again of let others take over their kingdom, and they resist in the most stubborn ways possible. Technology usually fails at the hands of an inflexible mind.
I would find it very helpful if the read article was in the video description. I often find myself the need to re read this or share it and it's kind of tough right now. I also think it's fair for the writer of course.
The biggest problem no one thinks about is that there is a time gradient across the software you create and deploy. DevOps only assumes that all 1000 different services are always up to date and deployed on a regular basis. Good working services will rarely be modified and updated or deployed, so you create a rift between your latest deployed services and the oldest deployed services that may contain technology that does not even exist anymore.
This author could have saved us all some time by saying "Good engineering is hard, time consuming and expensive." Overall he contradicts himself. He says "DevOps bad", but then says you need people with skills in both areas. 👍🤦♂️ Here's the "secret" of DevOps: it's just good Systems Engineering. We just call it DevOps for the benefit of management types who don't know DNS from a runaway chain reaction. So much of what he complains about, is a sign that there are architectural issues or that the system has outgrown the staffing levels. Also, his advocating for "engineers should be able to do whatever the hell they like" is about the most retarded thing I've heard in a loooong time. Holy sweet Jesus! Have you ever had to support a heterogeneous environment? Ever had to take one over? Yes, you need to be using the most appropriate tools for a job, but what tools you have *must* be used consistently, so that others can unpick your mess when you leave. Does this mean that larger organisations become more stiltified? YES, OF COURSE IT DOES, because that's the fucking DEFINITION of a "big business". Frankly, this guy comes across as "just" a programmer (not least of all because of his "hobbyist" comment!!!), who doesn't like doing serious engineering. I'm a serious professional, fuck-you-very-much. I'm not a "hobbyist". This shit is HARD and made harder by tools like this that refuse to accept that engineering is about compromise. Further, devs who have "no interest in the platform" are, by definition, amateurs; F1 drivers know almost as much about the cars as the people that designed and built them. This is the attitude that software engineers *need* to have about platform.
Thank you, well said and makes far better sense. I couldn't see the sense in how the entirety of software was bundled in as inherently hobbyist in nature when there's nothing about what we do that's different to... well, any other profession that takes time to learn and has specializations. Just sounds like someone who thinks that what we do is holy rather than it being a highly technically rigorous profession.
I remember years ago when I was a little spud starting out no software developer would touch anything related to devops or continuous integration or continuous delivery. They wouldn't even do any SCCM work if pushed. Then Google came along and created the site reliability engineer role.....Suddenly the software developers who swore against anything relating to chef, maven or ansible suddenly wanted to be knee deep in this shite. It was the best marketing ploy in tech in the last twenty years and tonnes of people fell hook line and sink for it.
Thanks for sharing your gripes. I finally feel like I'm not the only one who has devops issues (mine was Docker compose, which I fixed w/ Swarm) - heck, you've shared issues I didn't even know were such a big thing (Terraform and autoscaler).
1:30 12 years of data engineering, bi-dev and analytics .. you have no idea how true this line is xD took a sip and laughed for half a minute fighting not to spill something xD
The problem is that "dev" is creation, and "ops" is maintaining. Completely opposite perspectives (and cultures). Very few people can do both, let alone well (with the right mindset).
I'm MLOps. It's just software engineering good practices around ML. Introducing CI/CD for ML. Introducing model monitoring. Just like DevOps, it's mostly about the philosophy and requires "organization buy-in". It's just that instead of having unit tests and monitoring for (just) code, we do it for the model performance and data. And you don't make that work with a vague MLOps / DevOps title. You do that by writing code that enables that. Therefore, it's closer to the concept of SWE imo. (provided I'm not mixing terms)
We had a DevOps team. After a while, they renamed themselves to SRE. After we got used to the name, they renamed themselves to Production. Their lead even gave a presentation explaining why we should call them Production, not DevOps or SRE. It felt like an identity crisis :)
As someone that comes from a different sector and now entering devops that I have quickly realized that DevOps is a organizational philosophy that the entire company needs to adhere to. You can just hire two devops engineers and hope that will fix things.
Dev Ops is a little annoying in my company but mainly because there's just too many layers to it and my company isn't even that big. The current structure kinda goes like this: Me: Managing the actual Dev Op tools like CI/CD, distributed compilation. Co-founding programmer: Working with the Docker stuff and scalability for the Dev Op tools. IT: Dealing with permissions, tokens, cost, etc. Parent company IT: I don't even know what they do. Server proxy and email stuff? I just know our IT department complains a lot about it. For me it's the IT part that just sucks. We can end up waiting days for them to sort out permissions or token issues and there's so much back and forth with them attempting to resolve the issue. It's really just like ''does it work yet?'', "does it work yet?", "does it work yet?", "does it work yet?".
This is why heroku is amazing. No one wants to deal with DevOps. I just started at a job where they moved from heroku to AWS and no budget for DevOps. The moment I finish updating their old rails app to a newer version I'm moving everything back to heroku. This business wants new features and app bugs fixes for a super old app with 10% test coverage. I don't have enough time to also be dealing with DevOps crap from AWS.
To get started! Sure, it's great. But so many have blazed that trail before you. Why do you think cloud exists because there's a market. Why don't all services run on Heroku? It's great to get your ideas off the ground and you can run it for a longs as you want, at some point with real growth, you won't be able to.
Heroku is only good for small apps, as soon as you reach a certain threshold, I guarantee you gonna run out of memory or CPU regardless of the node scaling being used
My market analyis shows Devops & Kubernetes are actually growing 20% per year, while sometimes an ungrateful position there's a huge positive outlook for people in DevOps & Kubernetes.
If there is a special team named as "DevOps team", and if other teams don't care about how to filter the online status etc. - As there is no feedback with a proper Closed-Loop, it is possible that it is already on the way to a failure. Most important, DevOps or DevSecOps is a culture for engineering at first. Tools & process are only the method to implement that culture, to make each software engineer be a better one.
NGL - this one flew over my head a little bit. An example can go a long way in explaining things, but this article is only an abstract wall of text. And I still don't know what ADP means.
As someone who is a 'security oriented devops' person, I used to build platforms that help speed up the devs, and help automate things for the SoC, and other stuff, I can say that with where my org has gone in the last two years: the people driving us have no idea how to use us, or even understand what we do. Half my team has quit, they're not backfilling, and it feels like drowning. Where we once had a beautiful self service portal for engineering resources, today, I have to ask ops for a vmz and wait ten days. RIP
From my perspective working with DC infrastructure, DevOps seems like crap that either a) breaks my configuration files or b) deletes my configuration files... automatically. or c) doesn't do anything useful anyway because it can't handle servers or highly unpredictible new releases of enterprise software.
Half the problem is that DevOps is introduced as a theory (we've all read The Phoenix Project) and a responsibility (Make automation to support the new thing on the new platform). Ultimately sounds like this individual is a bit jaded because they have taken on the company's miscellaneous catch-all requests and that becomes frustrating for everyone involved. I've participated in a startup where everyone had 'DevOps' in front of their title early on (and everyone considered their automation and infrastructure needs) and slowly migrated to a dedicated DevOps team. Fortunately those teams stayed connected and DevOps would constantly assess what problems Engineers would face and provide secure solutions and a few neat little observation tools along the way. 🤷♂ I don't know man, every time I read a new title introduced as the solution - I can't help but think that it stems from communication issues that leadership failed to address.
Platform engineering is just applying DevOps principles to all of the tasks that get roped into DevOps enablement (the stuff DevOps teams tend to get farmed out to do). Make a clear boundry define your interface, everything in that is your baby, craddle to grave, everything else is a customer. This is true for your k8s/serverless/vm platforms, your CI/CD pipeline infrastructure, as well as the actual user facing applications.
DevOps internal tooling is the way to go, even though it has downsides in terms of job flexibility for devops engineers and developers in general. Thing is, if you want a naming scheme - enforce it! Create an internal tool that validates those inputs to ad nauseum to leave 0 flexibility in naming to devs, otherwise, they will just NAH-prod_2-eu-west-2_test-2_July_momLookAtMeNoHands all over the place...
If you feel like you're useless as a devops engineer, you can install some new UI, like rancher or something. Or you can just limit access to devs so they can have new blockers. 😂
DevOps is fine tbh. Kubernetes is a great tool but sometimes overkill for small startups but if you are willing to create k8s manifest files and create helm charts, it makes things so much easier to deploy applications to your clusters. There is a lot to handle though and once you bring devsecops in, it immediately fucks up your CI/CD pipeline
Question: How many of you are doing devops without a college degree? I have no education beyond high school diploma, but my job history is sort of all over the place. I bounced around between C++ developer, IT Support, System Administrator, Python developer, PLC programmer, electronics manufacturing. Always used Linux and have been spinning up VPS to run my own git, mail, websites, etc. Where do I go, what do I do? Certifications? Should I go to college? Both ideally obviously?
While this author isn't doing a great job acknowledging their biases (you know, the whole "I'm selling a product in this space" thing) there are some really good points there. I've worked as a member of the DevOps team in a company with about 70 devs and what we'd have to do was assign each devops guy to a few dev teams based on how impactful their work was on production. In the beginning we did little more than Terraform all AWS shit and build CI/CD but I don't wanna be repeating the same task over and over so I built something to let the devs self-service. The biggest limitation for me was always my deep-seated hatred for the web frontend stuff so I built them a Python CLI tool to do the whole and published a Docker container on the internal registry. That tool and the automation around quickly became my job. Fast forward to big tech now and we are gradually moving towards platform-building; I am liking the trend in my current employer of building internal tools to solve problems by enabling self-service for the different engineers. Our problem is we're not really building a cohesive platform, instead we're building a bunch of disjointed internal self-service tools that don't talk to each other and still need to be orchestrated by hand or another layer of tooling on top of them. What I'd suggest is to start with dedicated platform teams that can own the whole platform stack consisting of, at least, some kind of workflow engine, generic compute, a modular/pluggable architecture, network and a consistent UI. I don't care if you're building a big monolith CLI, a massive web app console thing or just some tooling that consumes our beloved YAML, just have a consistent one-stop shop for people to launch and manage what they need and then get out of their way.
The simple issue I have with most of the "Dev Ops" that I did was that the system I was forced to use that was described as a "Simple, opinionated declarative language" was that it forced you to copy pasta code that otherwise would have been simply handed with a variable. For example, for each deployment, I had to create several yaml files that had values embedded in them that had to be the same - stuff that was easily generated with a simple script.
Please create a ticket and we will get to it in our monday morning intake... code for leave me alone I am doing something that was dropped on my lap this morning that was previously techdebt and now is a fire. No one gives the ops people high fives unless they are on late getting the release fixed.
jajajaja, perdió impacto cuando grito "fast fu...ing autoscaler" y al tratar de enfocar se le perdió el párrafo, como me reí de ese error. quedo épico!
I have to work occasionally with CI/CD pipelines that take 10 hours to complete. And it’s constantly failing because of some dependencies. And all I want to do is just add a new region to the deployment config and deploy that stupid services there (don’t ask why we deploy services that aren’t ours, our processes are… weird). Oh when you’re doing this you can block devs working on their stuff, fucking beautiful. Every time I have to do this, I really want to meet mfs who designed and developed that bullshit. Oh, the pain, I’m getting mad even thinking about it. CI/CD my ass.
I was stuck waiting for DevOps to set up a Jenkins slave for over a year. Turns out he just had to fill out a 5 minute form. Management tried escalating this repeatedly during that year.
This is the type of Devops that gives a bad image to the actual good ones, sorry to hear it. Plus there is nothing conplicated about Jenkins, wtf are these people on
Every techie person I know who is secretly working 2 or 3 jobs at a time are in dev ops, sys admin, data science or a tech middle manager. And 3 x 150k, they are making bank!
devops named the prod and the QA system PROD and non-PROD. this is terror inducing when you get an alert or are e.g. rebooting QA and read PROD in your peripheral visions
For me sounds it all like organizational problem, the employees were not prepared well and there was like no real plan for implementation step by step. Normally companies should start from docker and the slowly try out kubernetes, but yes some software vendors can sell Kubernetes very fast, me and my company we always ask our clients what are theirs real goal of achievement , when they have particular tech in mind.
My problem with Devops is that the line responsibly is not clear at my company. I end up having to learn just enough to get the thing done every time I need anything done, but never get enough reps to feel comfortable doing it
So, so many people saying “X is bad” and they are right because their interpretation of X makes it bad. DevOps bad? No. The idea op DevOps is that the developer is also responsible for operating it. No one in their right mind says that the developer must design and build the operations infrastructure, tools, procedures. There is no contradiction in doing DevOps AND have engineering teams/IT4IT/IT4DEV/SREs. Developers in DevOps must be given a comfortable platform to use to Operate their software on in development and in production.
fr. I think each product should have an engineer making sure the infrastructure for that product is set in place. By using existing tools or makin new ones. I always found it weird that DevOps is a completely separated team. Maybe have a team that is Product-agnostic that deals with the infra to deploy the pipelines and tools of each product, but at most that. Each product has different needs, by having a team that deals with requirements for many projects just because "it's backend/infra stuff" makes their work extra hard, and in my opinion, breaks the organizational structure of the business
DevOps is a perfect example of how our industry fucks everything up. Development and Operations are *two different* teams. DevOps is the practice of these two teams working collaboratively instead of being at odds with each other. If you've founded a "DevOps Team", you've failed to do even basic studies to learn what DevOps is and deserve every bit of pain and suffering you experience. Don't get me started on Scrum and how utterly fucked up that is.
@@thewiirocks As a Devops that started without even knowing what scrub i have no idea what is all of this hate. A devops is just a software engineer that works to tie up some complicated steps in the development, automating those process that could take too much time from developers, how can that be wrong i have no idea. Ofc when all you do is making tiny and minimalistic applications you don't need a devops, but when things get big and there is so much work to do from developers, would it be great to have an actually skilled engineer that help you out? That is the actual role of a Devops. If you had bad experiences i think it's fully based on some rotten apples if you know what i mean..
Wasn't the idea of DevOps to make the Devs not only responsible for building new stuff, but also for continuity of the application while Ops would not only be responsible for the stability, but also to ease changing it. In our company, it sort of meant that Ops people learned how to code and developers learned responsibility (of course they already knew how to maintain a platform, just had to learn it at scale) I still like it.
This is sort of what our DevOps team does. They build a platform and set the templates up to have proper security infra so that the dev teams can easily create what they need in a scalable and secure way. We control our github actions based on the runners that devops provides with the appropriate iam roles. We can even spin up ephemeral environments for testing including spin up and tear down of databases in an RDS postgres cluster, all automated using strong security and vetted templates.
"DevOps" is just another desperate label for "Quality Assurance", AkA "QA". As we all who suffered as "QA engineers" for decades before we finally wised up know, no software company currently in existence gives a flying f---- about whether their product is any good, much less even works at all. They only care that they get their customer's money. Accordingly, no software company currently in existence wants to do QA or fund it in any way at all. So, given that QA is a totally dead-end affair because of the foregoing total apathy about SW "quality", and DevOps is just another name for QA, well...... you know what they say about 2 + 2 equaling 4........
IDP is defined in the title of the section where it starts to appear. "What's in a Great Internal Developer Platform". I like the concept he proposes, and his description of its definition. See 17:25 or so.
DevOps here, people would call DevOps anything but DevOps :))). Platform Engineering is still just DevOps. What dude says works well if you start project from scratch, but let see how well this applies to real world scenario where you have legacy apps, you have to do lift and shift and also have to do training for devs. At the end of the day this dude only wants to sell his platform, which is fine, but yeah DevOps is not a job is a a set of practices. Also to me seems like a business idea: why pay a 'DevOps' guy x-amount of money when you can force a developer to do the work and increase his salary with 10% so he doesnt hate you. What works best in large companies is to have teams that are made up from devs, qa, devops etc. and each team member is embedded into the team. It's still some kind of silo but not in the traditional way.
How does a platform solve anything? You are just building limits on top of the cloud platforms? I'm sorry but I didn't see this article add any value other than pointing out that no one, not even the author, has a good solution
Sorry but the fact the writer actually wrote a product in the area of Platform engineering and thinks DevOps should be thrown out of the window shows he has ZERO understanding what DevOps is and instead thinks that creating silos / titles changes the role. He has a VERY good reason to push A over B as that is his living. He chooses to wrap his ideas with "nice-words" and promotes his pink glasses. Platform Engineering uses DevOps practices', DevOps methodologies, DevOps tools. Every single "Platform engineering" topic he mentioned is basically DevOps. The term "platform engineering" has been around for almost 2 decade, and as you mentioned PE is just a new marketing-fused new term to present same ideas we have been having for 2 decades but because of BAD implementations, our indesutry things that it needs to "reinvent" itself, get new titles, new budgets, new silos. If you really want to have a discussion about DevOps @Primeagen, feel free. Our industry really needs to start talking and have good leadership to stop this nonsense of new names every 3 years. it started with cyber, then DevOps, now PE and in 3 years well have something new as AI is shaping up the landscape.
I got offered a job as a DevOps for a reaaally good pay. Thing is, I’ve been studying for being a Java/Spring Backend developer. Now I’m not sure if to take this job or now
I come from a sys admin, and data center management background, and I have noticed a savage divide between developers and operations. As a DevOps eng, I'm constantly battling dev's requests to manually run code in production; individual user accounts with write access to DBs, or "just give me X resource I need publically available." My background forces me to look at costs, security, and performance, and developers DO NOT care. It is the same thing as it was with on-prem resources, bad code? add more CPU and or mem, app needs to communicate with X, just open the firewall, need a new service no documentation so round and round with requests they didn't know they needed. My solution has been self-service tools but I'm CONSTANTLY catching dev's miss using these tools and putting the company at risk! So I have to pull access and limit what they can do to protect the company. For my WoW players, DevOps is the healer of the raid no one cares until it affects them. Our job is to keep you from destroying yourself and the company.
I really like your take. Thank you for sharing.
This is an interesting point. If devs have the tools to provision their own resources, how do you avoid them throwing more resources at a problem without oversight?
How do you prevent opening a resource to the entire world “but it works” situations? Someone needs to be auditing these things somehow.
Maybe having a security auditing team and a performance auditing team rolling between all internal projects?
I've been in your situation and what I ended up doing was to sit down with management (eventually even the CTO) and explain that there need to be standard hard and fast security practices that nobody deviates from without approval (security team will be your ally on this), approved services with bulk buys to keep costs down and a team budget for deploying anything else, all enforced via the self-service tooling. We then revoked dev write access to all cloud APIs and ran workshops to help the dev teams onboard. Took us 2 years in a company of about 100 engineers.
The thing is, we needed a catalyst event (most companies will because you'll be working against entrenched culture).
In our case, we had an R&D project that pulled retail customer PII into ElasticSearch but the devs on that project went quick and easy and just deployed the AWS service. The problem was that this was new at the time and not supported in-VPC which meant open to the Internet. Project wound down and they didn't clean up. I come in, look at AWS billing, notice the line item and decide to investigate. Single document on cluster read something to the effect of "we have your data hostage, pay us BTC or we publish it". Well, there goes the quarter, call InfoSec, they hurry over and there's jaws on the floor. Eventually, the GDPR fine hit, CEO and CTO walk onto the engineering floor and declare "the cowboy dev era is over, everything goes through DevOps and InfoSec".
Good luck.
You are the gatekeeper at a petting zoo basically, and the kids will always be kids. It will never change
I also come as a sysadmin role and the only memorable experience I've had from our devops tooling was that it was constantly deleting or misconfiguring my configuration files and when I spoke to the devops team they tried to explain to me that they have a policy. Their policy was ad-hoc and went both against the documentation and the empirical evidence of how things actually work for that specific system. I showed them both the documentation and the evidence, they didn't fix it. I fixed it with a workaround so it will break again on the next reboot, because I literally had no other option. I had to explain to the customer why the goddamn app randomly stops working as if it's my fault, since I'm responsible for it. And yet I don't have access to the devops tooling that directly affects it and the devops team doesn't respect my input. Devops to me is more like that healer in raids that keeps crying how bad the tank and DPS are, crying how the healer's should never DPS and then misses all the healing timings and gets you killed. I'm not saying all devops is like that, but I feel like rule #1 of big companies is that nobody knows wtf they are doing.
As the head of DevOps at some of the biggest companies in the world, all I can say is...
"Thanks for reaching out, Please raise a ticket through the correct channels for CICD support."
I need this at my work
@@replikvltyoutube3727 just build an auto response bot for the ticket system. It's fine
Just to get ignored
Oh god, I'm so glad I don't have to deal with this
I am in a fortune 200 tier company and its hard to find DevOps to help us with basic OPS shit. I am Dev and we always struggle to get help to lubricate the devops the philosophy of devops was supposed to be to connect dev and ops, but they proceeded with disconnecting dev team from ops team like its nothing
As others have mentioned, pls do the courtesy of linking to the article whenever you read it. It helps us viewers, as well as the author.
100% this. But maybe the sole point of this channel is to entertain and not to help.
You can solve this yourself. It’s not hard to search the title on google. Takes an extra 10 seconds of your time. Spoon-feeding is not owed.
You know what its called. Use your initiative
@@russellsanders4535 true, but also takes the reactor 10 seconds to link the article in their description. Helping boost the articles click through and adding text based credit
Yea he is getting free content, he only reacts to it lol
If copying and pasting previous work was a sign that a job was bullshit, then I have grave news…
Golang in a nuttshell lol
@tablettablete186 Hello bootcamp student.
I'm fully on board that a part of my job is bullshit.
Now imagine moving this bullshit from one type of system to another. and then expect anything to work at all.
Name a job that isn’t repeating the same thing over and over, or at least having one part that you repeat over and over.
@@danielthompson2561damn, I have not had a job like that in a decade, maybe time for something more interesting?
When publishing videos like this, reacting to articles I mean, if you can share the link of the article in the description section that would be great... just saying... Great videos by the way
classic react video not crediting anything
It takes a shorter to type author, article name and google it, than it took you to write this comment, lol.
@@gercius Soy take, it takes like 5 clicks to copy the link and post it in desc
@@ThatOpinionIsWrong Yeah boy, those few seconds are too precious for your waifu pillow cuddling sigma male grindset.
You can look up what a p tag is in stack overflow but you cant google the article title? You're too virgin
The fact that we even need DevOps is a testament to the amount of completely unnecessary tech debt 99% of tech companies have.
the question of "how do we deliver code to customers?" is not one to avoid or 'pawn off'
I think it is more a byproduct of how at many companies the backlog is owned by product. Product rarely makes time for the types of work that are often considered "devops", but the works needs doing. Eventually there is a break-off group of devs that get to operate a backlog without the oversight of product. They are going to try to get on top of all the "devops" issues that have piled up and therefore turn into a _DevOPS team_.
Companies that give engineering more influence over team's backlogs may be more likely to evolve towards platforms, because there is less of a build up of toilsome "devops" work.
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence.
At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence.
Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) with in their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign is in in the works, well. Devs are now business on the re-arch. Not Devops! That's where we come in.
my guy, all internal politics aside:
undelivered product is UNFINISHED product. period. no way you can tell a customer "works on my machine" and call it a day.
@@davidthomson3575 I'd consider "devops" tech debt created by devs to be things like:
- our deployment process is someone manually running a script
- we are using a dev's IAM keys to give a server access to another service
And so many more. It is the brittle and insecure things that happen when corners are being cut that work until they don't.
The word 'velocity' in the dev space drives me nuts
I hate these terms
The espeed is alot mate
we gotta synergize to provide value through elevated velocity in the agile environment to disrupt the industry maaaaaaan
ok, but what happened to 'acceleration'?
jerk?
position?
@@jonathanalonso6492 I think we need to increase the 'jerk' on the 'acceleration' of our 'velocity' otherwise we just won't reach the 'position' we aspire to 😂
Someone (boogibugs) asked a great question in the Twitch chat: "Why is the operations team not classed as an engineering team? That's one of the issues right there."
Boogibugs is correct. Just look at Star Trek. Engineering and operations always wore the same division colors. #justsayin
Who cares abou the title
@@jordixboy The title decides how much pay and respect you are given.
DevOps was never intended to be a role. It's a concept. Have your Ops integrated into the development process.. influence the product architecture from the design stage. Not a place where you show up on Friday saying "I need this whole new thing deployed on Monday." Then go to the nearest member of management and cry you didn't deliver on time because those "damn devOps are blocking you".
DevOps is a concept of enablement, not a role of blockage.
Only in the most traditional and monolithic environments I’ve worked in have I seen ops not classified as engineering.
@@jonathanalonso6492 not at all bro
"Devops does not work because the hired devops guy cuts corners and engineers should not work with ops. The solution? Build your own PaaS platform."
??????
Yeah, exactly. Bob, when do you think we can make that change to the frontend I was asking you about. Hey Joe, that will take me about 45 more days because I am busy building out the whole Platform. If you are a 1 man shop or a few dudes doing a startup, I understand. But when you have a scaling organization what you are referring to is not ideal.
@@davidthomson3575 Not only that. Even if you have the resources, you are still just rebuilding an inferior version of already existing AWS Kubernetes/Terraform style hosting services just to virtue signal that you are "doing deployment right".
@@adamih96 this is true, they're not in the PaaS business, are they?
Maybe if you build some quick macros to facade above those PaaS accounts and wrap common cookie cutter process in happy buttons and forms?
@@Fernando-ry5qt If easy enough to put behind dashboard buttons then there are likely already tools to automate that process within the same service or with something like github actions. The best button is no button, the best app is no app.
“One man’s golden path is another man’s golden shower” fixed that for ya
As a DevOps practitioner I obviously have some opinions on this article - mostly positive, tbh. I totally agree that too many businesses treat roles like SRE/so called 'DevOps teams' as an excuse to palm off responsibilities that should be shared. Also fully support the call for more platform engineering in place of a bajillion teams with a bajillion different cloud environments, all managed with whatever hand-rolled solution that team came up with. The reality is we've known about, and tried to steer away from, these problems within the DevOps community for a while now. A big part of the problem is the fact that, if you get into DevOps through a bootcamp, the only experience you get of implementing these engineering practices comes in the form of individual/small group projects, where the inherent problem in trying to coordinate these practices between tens or hundreds of development teams remains hidden. We really need to do more to expose trainee practitioners to problems of scale early on - DevOps is, after all, a philosophy for improving organisational performance, and a bootcamp simply can't do that justice.
This is also an issue in software engineering. There is no way to simulate scale-like work without actually working in an environment that has big data.
And there isn’t really anyone in CS trying to get an internship in DevOps because from what I’ve seen, it’s treated as a pivot for an engineer who doesn’t want to code as much.
So the only options are:
Hire more juniors and train them accordingly to take over once someone leaves
Continue as we are now
Compare a java dev w/1-2 years of experience to one with 10+. Pretty drastic difference, no? (sql injection? idor? session fixation? txn boundaries?) Same thing applies to 'devops'; k8s isn't just some 'configs', there's a whole ecosystem of best practices, frameworks rising/falling, architecture and so on. That equally applies to tracing/logging, etc
The reality is you end up with devs doing stuff in application code that should really be infra, but that's not the teams expertise. They are just copy/pasting the top SO results and effectively `chmod 777`ing your cloud account to get the server to work. Generally, theres no checks/balances (that's a silo, yo!) until you're "hacked".
I don't disagree, but I work with people who got 20 years of experience and still manage to produce the most ridiculous vulnerabilities known to man.
@@freesoftwareextremist8119 humans are imperfect, it's not about writing code with 0 vulnerabilities, it expecting them to be there and to put a practice in place to catch them and to not trust the human.
@@freesoftwareextremist8119 100%. There are old fools as well. Still, there's generally a correlation with experience and specialization.
When you have a "DevOps" team you need to file a ticket with to get anything done, then, guess what, thats just the ops team with a buzzwordy name.
Yeah I agree, worked at a place where everything on infra was repo driven, you just sent a PR and post it on a channel to get a review, the checked if everything was fine, merge, wait a min and done
It was pretty good in contrast with the old ticket system I am used to work
Devops != Tech Debt, nor is it a result of tech debt. Devops have their own tech debt, development teams or products have their own. In some cases there is convergence. Granted this can be a group of product devs or a separate team, eventually the former won't scale very long.
At the end of the day, someone has to handle (build, maintain, support, scale, secure, cost optimize, observability) infrastructure, build and maintain pipelines, ensure some form of security best practices are being implemented along the way and lastly - I've rarely seen a product developer answer compliance questions that didn't only apply to their product, let a lone gather and submit evidence.
Lastly, most devs in a large number of orgs are paid to code and deliver on something(s) within their allotted time (sprint) and it's usually a feature. Rarely they're afforded the time to worry about anything outside of the features they're working on because its not in the business best interest when they want to crank out features. Think v1. V2 is typically when companies start to scale and a redesign/re-arch is in in the works, well. Devs are now busy on the re-arch, not Devops! That's where we come in.
Guys stop commenting below and open a ticket
Can we get an epic?
I'm "doing devops" in a small company and I like it. Important part is the end-to-end-responsibility, devs learning from ops problems. But it does require management understands "OPS > DEV" and you can't have 100% schedule management. We plan for 2w sprints (~scrumban) but it's just setting priority, what tracks is everyone focusing on. Tickets are done when they are done. Most important part of "my devops" is that once I hit the same thing Nth time, I will either do a bug fix (if I think I know the code cause) or add logging or setup monitoring. In traditional Development this can't happen because the guy looking at logs doesn't understand the source line behind the log line.
This article shows me how much I have yet to learn. I understood very little of this article.
My main problem with DevOps is that I don't even know what exactly are they supposed to even do.
Where do their responsibilities start and mine end?
Why are they refusing to give me a read only prod db access, when I do really need it for analytical stuff and our app isn't even close to being released yet?
"They"? As in a separate DevOps team? That's #1 in the article: DevOps doesn't mean you have a separate DevOps team.
@@JeyPeyy idk what happen to my comment.
devops is a production methodology for agile management.
devops, right now, work very well for web ressources .
Due to tech debt (or aging workforce), the average tech company is based on ITIL management.
where each step of the devops cycle is managed by a different team.
In those company the job title "devops" can mean a lot of thing (depending on HR).
@CHE6yp is talking to an infrastructure/platform admin .
Because read only DB access to a prod database can send the sucker crashing with a single query. This is not a best practice, because we learned long ago no one should do this.
You need a data pipelines or atleast a read replica.
You write code, you should worry only about writing code. If you want a shit ton of distractions that keep you from writing code, start worrying about all of the non code writing responsibilities that fall under the guise of devops.
@@davidthomson3575 Oh I am so glad you said this! Illustrates my main PITA in modern development perfectly.
In general, I would say giving any access to prod is highly dubious and not a "best practice". However, "best practice" is often just a massive distraction in itself when applied to a concrete example.
In my case, we have 2 backend devs and our project basically just agregates a bunch of data from a third party. All of that data is completely recoverable, and the amount of users is currently zero. The reason I needed a prod db is because prod is the only place that does have actual data, and my task was to basically mutate some field values to make them all confirm to the same standart. Nobody has a full list of actual values that need mutation, so basically I needed to make a bunch of SELECT DISTINCT queries just to find out what exactly is needed.
So, whats the path of least resistance and the best practice? Waiting for our DevOps/Platform Admin to make a replica, going to our prod frontend and manually cataloging all 2 million records, or, getting a prod (not even read-only necessarily) db access?
@@CHE6ypprobably asking for a data dump haha
DevOps was developed with the four key metrics in mind and explicitly defines its success through:
- Deployment Frequency
- Lead Time for Changes
- Mean Time to Recovery (MTTR)
- Change Failure Rate
which I find great!
Since I came to know DevOps in around the 2015s or so, I found no company, no Project, nothing, where these four metrics are computed and made available, neither confidential nor public. I mean, besides the companies supplying to the "Devops Report". I never worked for a company in the report, afaik.
It seems this is the main issue: If you introduce something that relies on measurability to function but fail to measure, the result is likely to mutate into something undesirable. Makes sense, right?
So the real problem is people don't value the support people...
Exactly, every DevOps person (yeah yeah or whatever title you want) has to not only shoulder the bitch work the Devs were too LAZY to do but then catch a bunch of shit about how “useless” they are at the same time, fun fun
@@itsdylac sadly, yes. Of course the ops people can be just as lazy.
@@disguysn they totally can, and incompetent too. But the competent Ops people get treated like trash, and the incompetent devs get a pass
@@itsdylac I wish I could contradict you. I moved from dev to devops at my last job and it's exactly as you stated.
This is unrelatable to me because our team develops, deploys end to end, writes the CDK infrastructure, handles oncall, etc. I think separate dev ops teams just encourage engineers to be lazy tbh
I used to do that. The problem with generalization of that sort is that you’ll keep tripping over amateur problems as it takes forever to develop expertise in any one area. I love the concept, but in practice it can lead to burnout.
@@samgould8567Yes. I too believe it’s better for every organization, with capacity, to engage individuals who have specialized in solving specific problems, like devops as a practice especially in the sense of ensuring reliability. You can’t have everybody doing everything, it will end in chaos. 😅
So who's managing the cloud then? Presuming you're on AWS, who's managing organization itself? Security, compliance, fin ops? Who's going to be waken up if CloudTrail logs have something suspicions - on-call from full-stack Team A or Team B? Who's gonna call you out on unused resources?
Can you add the articles into the description of videos?
Article just sounded like the guy was complaining about everything. “DevOps done by engineers is bad because the engineers hate it and suck at it.” Then he goes “DevOps done by the DevOps engineers is bad because they are not good at their jobs and just copy paste things” Then his solution is to rename the DevOps engineers to platform engineers and magically everything will be better.
The reality is that hard things are hard and random hat swapping is not gonna fix it. This is fundamentally a technology problem. When the tech changes then the situation will change.
I agree. The same goes for full stack garbage. Civilization is built on specialization and the key to succeeding is the collaboration of many specialists. Imagine this in any other industry: “we’re looking for someone who can work on every single aspect of the building: electrical, heating, plumbing, concrete, etc. We will assign each full stack trades person a room, and they are responsible for the whole thing.” You’d end up with inferior, inconsistent garbage, just like today’s software!
My company writes applications in PHP with a very simple stack.
The concept of 'devops' is foreign to us because we don't need to compile, transpile, restart services, etc. The dev hits save, then refreshes the browser to test.
Deploying to production is a single line command.
DevOps sounds like a nightmare, glad we aren't needing any of that (yet). :)
"we don't need to ... restart services" - what does this mean? You clearly don't understand the stack you're running on, how can you understand an issue of a higher order paradigm?
@@cloudboogie It means our application doesn't have services to restart during development. PHP doesn't work that way.
Been writing PHP for 15 years, i think i know what i'm talking about by now.
u clearly don‘t. Give it maybe another 15 years.
Why the hate? I actually do not know why.. someone explain@@thunderstein5041
@ThePrimeTimeagen can you PLEASE post the links to the articles you talk about in the description?
In trying to solve waterfall development's "problems" we have created a hydra of problems. Waterfall was simple and could apply to anything. Software was less brittle that way. "Real Agile" is also good, but the common way it's done and this whole DevOps stuff is just a ton of make work for very little actual work that gets done. Most of the time, I'm just left shocked what people call work. It's akin to bureaucracy and paper pushing than actually building stuff.
Based
yeah honestly the amount of filler meetings to discuss work, or the amount of "investigate blah blah blah" tickets where the scope of the ticket is just to figure out how to do the actual work - it's just too much. the team would be much more productive if we cut the filler and just built the shit. sometimes the sheer volume of meetings feels like excuses for people to avoid getting their hands dirty with code
mate that is corporate in a nutshell (beyond a small sized company everything becomes mostly beaurocracy and politics and keeping up appearances through sheer company size
And the very disturbing things is that bad to mediocre DevOps earn on average MORE than good software developer. I didn't believe it until a guy I know, who doesn't even know how to write a for-loop, who worked at RedHat told me his salary and showed me their repos... He was writing yaml files and introducing very lil amount of "code", I think that three of them full time for about a year still didn't produce as much code as I alone do in just a week. Another thing is that most of them are clowns and liked to be called engineers while they can't solve the easiest equation ever.
If they can't solve easiest equation ever, why don't you do it, perhaps you don't know how to write a yaml file either
@@abdulhamid.mmousa2329 because it seems that deploy is not a developer's concern.
@FredoCorleone is right, it's common that newbies are hired for DevOps on high salaries. Managers are impressed by the DevOps term... it's all a big crap and the results are shitty.
DevOps is a stupid buzzword name, but at least the point of taking the best practices of software development and applying to infrastructure delivery makes some sense. The challenge is mostly a people thing, nobody wants to learn how to do their job over again of let others take over their kingdom, and they resist in the most stubborn ways possible. Technology usually fails at the hands of an inflexible mind.
I would find it very helpful if the read article was in the video description. I often find myself the need to re read this or share it and it's kind of tough right now. I also think it's fair for the writer of course.
google can help 😂 🕹
Hey! Missing reference link... What article was that?
The biggest problem no one thinks about is that there is a time gradient across the software you create and deploy.
DevOps only assumes that all 1000 different services are always up to date and deployed on a regular basis.
Good working services will rarely be modified and updated or deployed, so you create a rift between your latest deployed services and the oldest deployed services that may contain technology that does not even exist anymore.
When you have a shitty dev team ( which is 99% of the time) your services will be modified weekly ( if not daily)
@@houssem009 I am in currently in the 1% team. I hope it lasts long time.
the company i work for has outsourced our "devops" team, so i really felt this 1000x
From an Engineer to a Comedian: A journey through time
This author could have saved us all some time by saying "Good engineering is hard, time consuming and expensive."
Overall he contradicts himself. He says "DevOps bad", but then says you need people with skills in both areas. 👍🤦♂️
Here's the "secret" of DevOps: it's just good Systems Engineering. We just call it DevOps for the benefit of management types who don't know DNS from a runaway chain reaction.
So much of what he complains about, is a sign that there are architectural issues or that the system has outgrown the staffing levels.
Also, his advocating for "engineers should be able to do whatever the hell they like" is about the most retarded thing I've heard in a loooong time.
Holy sweet Jesus! Have you ever had to support a heterogeneous environment? Ever had to take one over?
Yes, you need to be using the most appropriate tools for a job, but what tools you have *must* be used consistently, so that others can unpick your mess when you leave.
Does this mean that larger organisations become more stiltified? YES, OF COURSE IT DOES, because that's the fucking DEFINITION of a "big business".
Frankly, this guy comes across as "just" a programmer (not least of all because of his "hobbyist" comment!!!), who doesn't like doing serious engineering.
I'm a serious professional, fuck-you-very-much. I'm not a "hobbyist". This shit is HARD and made harder by tools like this that refuse to accept that engineering is about compromise.
Further, devs who have "no interest in the platform" are, by definition, amateurs; F1 drivers know almost as much about the cars as the people that designed and built them. This is the attitude that software engineers *need* to have about platform.
Thank you, well said and makes far better sense. I couldn't see the sense in how the entirety of software was bundled in as inherently hobbyist in nature when there's nothing about what we do that's different to... well, any other profession that takes time to learn and has specializations. Just sounds like someone who thinks that what we do is holy rather than it being a highly technically rigorous profession.
@@HirokuDev Thanks. Exactly.
I was clearly very angry when I wrote this. 🫣😂 I am passionate about my job; no apologies offered. 🤷♂️
I remember years ago when I was a little spud starting out no software developer would touch anything related to devops or continuous integration or continuous delivery. They wouldn't even do any SCCM work if pushed. Then Google came along and created the site reliability engineer role.....Suddenly the software developers who swore against anything relating to chef, maven or ansible suddenly wanted to be knee deep in this shite. It was the best marketing ploy in tech in the last twenty years and tonnes of people fell hook line and sink for it.
Thanks for sharing your gripes. I finally feel like I'm not the only one who has devops issues (mine was Docker compose, which I fixed w/ Swarm) - heck, you've shared issues I didn't even know were such a big thing (Terraform and autoscaler).
CTO: Our software is complety secure.
Dev: Shell script goes brrr.
CTO: Suprise pikachu face.
1:30 12 years of data engineering, bi-dev and analytics .. you have no idea how true this line is xD
took a sip and laughed for half a minute fighting not to spill something xD
The problem is that "dev" is creation, and "ops" is maintaining. Completely opposite perspectives (and cultures). Very few people can do both, let alone well (with the right mindset).
I'm MLOps. It's just software engineering good practices around ML. Introducing CI/CD for ML. Introducing model monitoring.
Just like DevOps, it's mostly about the philosophy and requires "organization buy-in".
It's just that instead of having unit tests and monitoring for (just) code, we do it for the model performance and data.
And you don't make that work with a vague MLOps / DevOps title. You do that by writing code that enables that. Therefore, it's closer to the concept of SWE imo. (provided I'm not mixing terms)
We had a DevOps team. After a while, they renamed themselves to SRE. After we got used to the name, they renamed themselves to Production. Their lead even gave a presentation explaining why we should call them Production, not DevOps or SRE. It felt like an identity crisis :)
As someone that comes from a different sector and now entering devops that I have quickly realized that DevOps is a organizational philosophy that the entire company needs to adhere to. You can just hire two devops engineers and hope that will fix things.
Dev Ops is a little annoying in my company but mainly because there's just too many layers to it and my company isn't even that big.
The current structure kinda goes like this:
Me: Managing the actual Dev Op tools like CI/CD, distributed compilation.
Co-founding programmer: Working with the Docker stuff and scalability for the Dev Op tools.
IT: Dealing with permissions, tokens, cost, etc.
Parent company IT: I don't even know what they do. Server proxy and email stuff? I just know our IT department complains a lot about it.
For me it's the IT part that just sucks. We can end up waiting days for them to sort out permissions or token issues and there's so much back and forth with them attempting to resolve the issue. It's really just like ''does it work yet?'', "does it work yet?", "does it work yet?", "does it work yet?".
This is why heroku is amazing. No one wants to deal with DevOps. I just started at a job where they moved from heroku to AWS and no budget for DevOps. The moment I finish updating their old rails app to a newer version I'm moving everything back to heroku. This business wants new features and app bugs fixes for a super old app with 10% test coverage. I don't have enough time to also be dealing with DevOps crap from AWS.
isn't DevOps just make changes, push, deploy?
The most recent Stack Overflow Survey results would disagree with you.
@@ea_naseer /sarcasm ?
To get started! Sure, it's great. But so many have blazed that trail before you. Why do you think cloud exists because there's a market. Why don't all services run on Heroku?
It's great to get your ideas off the ground and you can run it for a longs as you want, at some point with real growth, you won't be able to.
Heroku is only good for small apps, as soon as you reach a certain threshold, I guarantee you gonna run out of memory or CPU regardless of the node scaling being used
My market analyis shows Devops & Kubernetes are actually growing 20% per year, while sometimes an ungrateful position there's a huge positive outlook for people in DevOps & Kubernetes.
All of these “devops is over” articles are sales ads for some deployment templating system or some new “platform”
If there is a special team named as "DevOps team", and if other teams don't care about how to filter the online status etc. - As there is no feedback with a proper Closed-Loop, it is possible that it is already on the way to a failure.
Most important, DevOps or DevSecOps is a culture for engineering at first. Tools & process are only the method to implement that culture, to make each software engineer be a better one.
"FAST FUCKING AUTOSCALLERRRRR" has became a meme that is said so often in my work place it's basically echoing off the walls and in Teams chats 24/7.
NGL - this one flew over my head a little bit.
An example can go a long way in explaining things, but this article is only an abstract wall of text.
And I still don't know what ADP means.
Abstract Data Ptype
^ Payroll!
IDP = Internal Developer Platform
As someone who is a 'security oriented devops' person, I used to build platforms that help speed up the devs, and help automate things for the SoC, and other stuff, I can say that with where my org has gone in the last two years: the people driving us have no idea how to use us, or even understand what we do. Half my team has quit, they're not backfilling, and it feels like drowning. Where we once had a beautiful self service portal for engineering resources, today, I have to ask ops for a vmz and wait ten days. RIP
From my perspective working with DC infrastructure, DevOps seems like crap that either a) breaks my configuration files or b) deletes my configuration files... automatically. or c) doesn't do anything useful anyway because it can't handle servers or highly unpredictible new releases of enterprise software.
Half the problem is that DevOps is introduced as a theory (we've all read The Phoenix Project) and a responsibility (Make automation to support the new thing on the new platform). Ultimately sounds like this individual is a bit jaded because they have taken on the company's miscellaneous catch-all requests and that becomes frustrating for everyone involved. I've participated in a startup where everyone had 'DevOps' in front of their title early on (and everyone considered their automation and infrastructure needs) and slowly migrated to a dedicated DevOps team. Fortunately those teams stayed connected and DevOps would constantly assess what problems Engineers would face and provide secure solutions and a few neat little observation tools along the way. 🤷♂
I don't know man, every time I read a new title introduced as the solution - I can't help but think that it stems from communication issues that leadership failed to address.
Platform engineering is just applying DevOps principles to all of the tasks that get roped into DevOps enablement (the stuff DevOps teams tend to get farmed out to do). Make a clear boundry define your interface, everything in that is your baby, craddle to grave, everything else is a customer. This is true for your k8s/serverless/vm platforms, your CI/CD pipeline infrastructure, as well as the actual user facing applications.
Finally, someone finally said it.
Devops IMHO is a discipline not a role, not a platform, not a team. Similar to agile software development
In theory, yes. But in practice note, this is why we have the role devops engineer now.
it's one of those useless terms that no one really knows what it means, like API
DEVOPS IS AWESOME, LOL DIDN"T WATCH AND WITH AD BLOCKER ON.
DevOps internal tooling is the way to go, even though it has downsides in terms of job flexibility for devops engineers and developers in general.
Thing is, if you want a naming scheme - enforce it!
Create an internal tool that validates those inputs to ad nauseum to leave 0 flexibility in naming to devs, otherwise, they will just NAH-prod_2-eu-west-2_test-2_July_momLookAtMeNoHands all over the place...
could you link the article in the description ? it's more convenient when trying to get to the article
If you feel like you're useless as a devops engineer, you can install some new UI, like rancher or something. Or you can just limit access to devs so they can have new blockers. 😂
opposite of white glove is bare hands.
"Increasing velocity and tearing down silos" - in other words, we're accelerating straight into the barn!
DevOps is fine tbh. Kubernetes is a great tool but sometimes overkill for small startups but if you are willing to create k8s manifest files and create helm charts, it makes things so much easier to deploy applications to your clusters. There is a lot to handle though and once you bring devsecops in, it immediately fucks up your CI/CD pipeline
Ops work sucks and I don’t want to do it. For that reason I’m happy they are there when I have them.
Question: How many of you are doing devops without a college degree? I have no education beyond high school diploma, but my job history is sort of all over the place. I bounced around between C++ developer, IT Support, System Administrator, Python developer, PLC programmer, electronics manufacturing. Always used Linux and have been spinning up VPS to run my own git, mail, websites, etc.
Where do I go, what do I do? Certifications? Should I go to college? Both ideally obviously?
A lot of developers just cannot be bothered to learn about cloud services or containers.
While this author isn't doing a great job acknowledging their biases (you know, the whole "I'm selling a product in this space" thing) there are some really good points there.
I've worked as a member of the DevOps team in a company with about 70 devs and what we'd have to do was assign each devops guy to a few dev teams based on how impactful their work was on production. In the beginning we did little more than Terraform all AWS shit and build CI/CD but I don't wanna be repeating the same task over and over so I built something to let the devs self-service. The biggest limitation for me was always my deep-seated hatred for the web frontend stuff so I built them a Python CLI tool to do the whole and published a Docker container on the internal registry. That tool and the automation around quickly became my job.
Fast forward to big tech now and we are gradually moving towards platform-building; I am liking the trend in my current employer of building internal tools to solve problems by enabling self-service for the different engineers. Our problem is we're not really building a cohesive platform, instead we're building a bunch of disjointed internal self-service tools that don't talk to each other and still need to be orchestrated by hand or another layer of tooling on top of them.
What I'd suggest is to start with dedicated platform teams that can own the whole platform stack consisting of, at least, some kind of workflow engine, generic compute, a modular/pluggable architecture, network and a consistent UI. I don't care if you're building a big monolith CLI, a massive web app console thing or just some tooling that consumes our beloved YAML, just have a consistent one-stop shop for people to launch and manage what they need and then get out of their way.
What is his name again? I didn't get it in the end
Is there a bug on the camera???
bottom middle
I never get time estimates close, unless it's something monotonous. So, is it even possible to measure your own efficiency in coding?
The simple issue I have with most of the "Dev Ops" that I did was that the system I was forced to use that was described as a "Simple, opinionated declarative language" was that it forced you to copy pasta code that otherwise would have been simply handed with a variable. For example, for each deployment, I had to create several yaml files that had values embedded in them that had to be the same - stuff that was easily generated with a simple script.
Please create a ticket and we will get to it in our monday morning intake... code for leave me alone I am doing something that was dropped on my lap this morning that was previously techdebt and now is a fire. No one gives the ops people high fives unless they are on late getting the release fixed.
jajajaja, perdió impacto cuando grito "fast fu...ing autoscaler" y al tratar de enfocar se le perdió el párrafo, como me reí de ese error. quedo épico!
I have to work occasionally with CI/CD pipelines that take 10 hours to complete. And it’s constantly failing because of some dependencies. And all I want to do is just add a new region to the deployment config and deploy that stupid services there (don’t ask why we deploy services that aren’t ours, our processes are… weird). Oh when you’re doing this you can block devs working on their stuff, fucking beautiful.
Every time I have to do this, I really want to meet mfs who designed and developed that bullshit.
Oh, the pain, I’m getting mad even thinking about it. CI/CD my ass.
Are you Dr. Disrespects’ brother ? Man you both have almost the same voice, mustache, reaction and humor !!
I was stuck waiting for DevOps to set up a Jenkins slave for over a year. Turns out he just had to fill out a 5 minute form. Management tried escalating this repeatedly during that year.
This is the type of Devops that gives a bad image to the actual good ones, sorry to hear it. Plus there is nothing conplicated about Jenkins, wtf are these people on
"But companies pay me to FORGET TO TURN OFF ALERTS!" 😂
Every techie person I know who is secretly working 2 or 3 jobs at a time are in dev ops, sys admin, data science or a tech middle manager. And 3 x 150k, they are making bank!
Does anyone know if there is some Chrome extension to autoskip his squeaking seizures?
devops named the prod and the QA system PROD and non-PROD. this is terror inducing when you get an alert or are e.g. rebooting QA and read PROD in your peripheral visions
As a devops engineer i like to revist this video / article every couple of months. keeps me humble
When you realize you're somehow doing both dev ops team by yourself, and it's shit 😂
For me sounds it all like organizational problem, the employees were not prepared well and there was like no real plan for implementation step by step. Normally companies should start from docker and the slowly try out kubernetes, but yes some software vendors can sell Kubernetes very fast, me and my company we always ask our clients what are theirs real goal of achievement , when they have particular tech in mind.
My problem with Devops is that the line responsibly is not clear at my company. I end up having to learn just enough to get the thing done every time I need anything done, but never get enough reps to feel comfortable doing it
So, so many people saying “X is bad” and they are right because their interpretation of X makes it bad.
DevOps bad? No. The idea op DevOps is that the developer is also responsible for operating it. No one in their right mind says that the developer must design and build the operations infrastructure, tools, procedures. There is no contradiction in doing DevOps AND have engineering teams/IT4IT/IT4DEV/SREs. Developers in DevOps must be given a comfortable platform to use to Operate their software on in development and in production.
where can i find this article he is reacting
Devops, scrum... all those things were created by middle management hoping to tame programmers
fr. I think each product should have an engineer making sure the infrastructure for that product is set in place. By using existing tools or makin new ones. I always found it weird that DevOps is a completely separated team.
Maybe have a team that is Product-agnostic that deals with the infra to deploy the pipelines and tools of each product, but at most that.
Each product has different needs, by having a team that deals with requirements for many projects just because "it's backend/infra stuff" makes their work extra hard, and in my opinion, breaks the organizational structure of the business
DevOps is a perfect example of how our industry fucks everything up. Development and Operations are *two different* teams. DevOps is the practice of these two teams working collaboratively instead of being at odds with each other.
If you've founded a "DevOps Team", you've failed to do even basic studies to learn what DevOps is and deserve every bit of pain and suffering you experience.
Don't get me started on Scrum and how utterly fucked up that is.
@@thewiirocks As a Devops that started without even knowing what scrub i have no idea what is all of this hate. A devops is just a software engineer that works to tie up some complicated steps in the development, automating those process that could take too much time from developers, how can that be wrong i have no idea. Ofc when all you do is making tiny and minimalistic applications you don't need a devops, but when things get big and there is so much work to do from developers, would it be great to have an actually skilled engineer that help you out? That is the actual role of a Devops. If you had bad experiences i think it's fully based on some rotten apples if you know what i mean..
Wasn't the idea of DevOps to make the Devs not only responsible for building new stuff, but also for continuity of the application while Ops would not only be responsible for the stability, but also to ease changing it. In our company, it sort of meant that Ops people learned how to code and developers learned responsibility (of course they already knew how to maintain a platform, just had to learn it at scale) I still like it.
This is sort of what our DevOps team does. They build a platform and set the templates up to have proper security infra so that the dev teams can easily create what they need in a scalable and secure way.
We control our github actions based on the runners that devops provides with the appropriate iam roles. We can even spin up ephemeral environments for testing including spin up and tear down of databases in an RDS postgres cluster, all automated using strong security and vetted templates.
To quote Mike Judge's magnum opus, "Office Space" (1999):
"What would you say you DO here?"
what's the name though?
"DevOps" is just another desperate label for "Quality Assurance", AkA "QA". As we all who suffered as "QA engineers" for decades before we finally wised up know, no software company currently in existence gives a flying f---- about whether their product is any good, much less even works at all. They only care that they get their customer's money. Accordingly, no software company currently in existence wants to do QA or fund it in any way at all. So, given that QA is a totally dead-end affair because of the foregoing total apathy about SW "quality", and DevOps is just another name for QA, well...... you know what they say about 2 + 2 equaling 4........
As a DevOps engineer I do what I gotta do. Then get off work, spool up some go cobra and scratch my coding itch. Always code after work, it's nice.
IDP is defined in the title of the section where it starts to appear. "What's in a Great Internal Developer Platform". I like the concept he proposes, and his description of its definition. See 17:25 or so.
DevOps here, people would call DevOps anything but DevOps :))). Platform Engineering is still just DevOps. What dude says works well if you start project from scratch, but let see how well this applies to real world scenario where you have legacy apps, you have to do lift and shift and also have to do training for devs. At the end of the day this dude only wants to sell his platform, which is fine, but yeah DevOps is not a job is a a set of practices. Also to me seems like a business idea: why pay a 'DevOps' guy x-amount of money when you can force a developer to do the work and increase his salary with 10% so he doesnt hate you.
What works best in large companies is to have teams that are made up from devs, qa, devops etc. and each team member is embedded into the team. It's still some kind of silo but not in the traditional way.
How does a platform solve anything?
You are just building limits on top of the cloud platforms?
I'm sorry but I didn't see this article add any value other than pointing out that no one, not even the author, has a good solution
Sorry but the fact the writer actually wrote a product in the area of Platform engineering and thinks DevOps should be thrown out of the window shows he has ZERO understanding what DevOps is and instead thinks that creating silos / titles changes the role. He has a VERY good reason to push A over B as that is his living. He chooses to wrap his ideas with "nice-words" and promotes his pink glasses.
Platform Engineering uses DevOps practices', DevOps methodologies, DevOps tools. Every single "Platform engineering" topic he mentioned is basically DevOps. The term "platform engineering" has been around for almost 2 decade, and as you mentioned PE is just a new marketing-fused new term to present same ideas we have been having for 2 decades but because of BAD implementations, our indesutry things that it needs to "reinvent" itself, get new titles, new budgets, new silos. If you really want to have a discussion about DevOps @Primeagen, feel free. Our industry really needs to start talking and have good leadership to stop this nonsense of new names every 3 years. it started with cyber, then DevOps, now PE and in 3 years well have something new as AI is shaping up the landscape.
Yo wtf, what's the name?
I got offered a job as a DevOps for a reaaally good pay.
Thing is, I’ve been studying for being a Java/Spring Backend developer.
Now I’m not sure if to take this job or now
That devOps diagram with the cow and the turd is gold