3:46 most important thing in computer science: *problem decomposition - How do you take a complicated problem or system and chop it up into pieces that you can build relatively independently* 10:10 what are the secrets 12:36 classes should be deep 15:25 typical shallow method 17:50 one of the biggest mistakes people make: 18:46 *20:16* 20:32 example of a deep interface 21:55 defines errors out of existence 23:00 try to minimize the number of places we have to handle exceptions 26:49 3rd example: substring 34:13 tactical VS strategic programming 34:24 tactical 37:29 strategic programming approach
Great lecture, with great points. But apart from the topic, what really impressed me was how honest he was. He was not afraid to say "I will have to think on that" or "I have no idea" multiple times.
The book is well worth it. Thanks Prof Ousterhout. Edit: and I would add that it shines most at providing the rationales behind certain "good practices" which are more used as dogma than as heuristics to produce strong systems. Some of these rationales are not trivial (at least not for me!) and understanding them will shape your intuition for future work. Nicely done.
Good explanation. It reminds me of my life at the second part of the seventies, when I used the ideas of Parnas (information hiding) and Dijkstra (semaphores and layering) to design an OS for 16-bits Philips mini-computers for our Air Traffic Control systems. We used the OS from 1976 to 1990, when that mini-computer range was discontinued. One system would consist of 1 to 100 of these mini computers (2 or 3 per radar; 1 per radar display and 2 to 10 centrally).
I wish there might be an open access to his software design studio class for public. I'm really curious about how they teach the student about tackling design problems.
2 cents: 1. A deep class is a borderline monolith. A module should be deep with classes that have high cohesion and purpose. 2. The substring debate was meaningless. It does not have to be either-or. First function: string Substring(int, int) throws IndexOutOfBoundsException Second function: string GetSubstringOrNull(int, int) // return null if the index is out of bounds/ mismatch etc Third function: string GetSubstringOrEmpty(int, int) // return an empty string if the index is out of bounds/ mismatch etc Second and third save this code from being written repeated by hundreds of developers and further give them a choice of whether they care about the bounds (check for null on return if they do) or don't (empty).
At the beginning, "Talent is overrated - the only thing that really differentiates the top performers from the average performers is how much they've practiced." but at the end, " I think you want to hire the person who's the fastest learner, who has the fastest slope. I look for people who are really smart, fast learners"
Yes, there does seem to be a tension between those statements. Perhaps his point about talent was that designing is something that can be learned rather than purely down to innate ability. That's kind of orthogonal to being a fast learner: fast learners merely get there faster.
Problem decomposition ... this is what Descartes talked about in his Dicourse On The Method ... I have some notes on this, if I find them I paste a link here later ... btw, I wouldnt agree that we dont teach problem decomposition ... for example the applied category theory community talks about nothing but "compositionality" which is all about problem decomposition and solution synthesis.
About teaching people to be better developers (the 10x thingy), the issue goes way deeper than just sheer practice. It does involve talent in a limited sense and many other aspects but, much more importantly, it involves your work ethics and ultimately your moral values. To begin with, people do have different inherent abilities. I'm not saying he's arguing the contrary but some people do have the idea that we're all a blank slate and equal in all aspects. This is self-evidently wrong. Some people develop muscles better than others, some are naturally good with numbers, some thrive in dealing with people while others shy away from such a thing. If you ever lived in or close to a large family, you'll be able to easily correlate how wildly different each child can be in terms of interests and capacity to do things. However, I agree those inherent abilities and capacities are not necessarily the determining factors of productivity. They can become a hindrance if the person is forcing himself or herself to do something diametrically contrary to their skills but even then self determination can get one at least above the average. The ethics and moral values, however, are what drives people to put their minds and hearts into whatever they are doing. I have taught countless developers how to perform tasks from the simplest to the most complex in the work environment and outside it, and in different cultures as well. What I have observed is that very few of those developers were actually invested in their work - either trying to solve a problem or deliver a feature in the best way they could - as opposed to just doing their job and satisfying their manager or whatever external factor was pressuring them. The good developer feels bad when they deliver code they know it's less than ideal. This sentiment drives them to do better and sometimes make contributions that are not asked by anyone else. They set a higher bar to themselves to the best of their abilities. They care about such things because they have an internal value and a conscience that bothers them otherwise. The average developer feels good if they deliver what they were asked to deliver. They only care if they are doing a good job for external reasons, such as fear from what other people will say about their code, fear of not getting a promotion, etc. Those external factors can, to some extent, drive them to be more productive temporarily, with bursts of focus and effort. But it fails miserably in making them more productive over time and really grow as developers because the external pressure soon will burn them out and they will retreat. If possible, good developers will spend countless hours doing and re-doing stuff until it satisfies them, thus they practice without even realizing it. With each effort, their minds are engaged and are looking for opportunities to improve and satisfy their own high standards. The average developer can repeat the same task over and over again but their mind is not as engaged and focused because they are not looking for an improvement but just to finish the job at hand. Repetition can make them a little faster as they get used to perform the same tasks. But that's not the same as improving how they perform the tasks, not by a long shot. The conclusion is that making developers more productive is not about providing them with more training or more time to practice, it's about changing their hearts, giving them passion for what they are doing, making them care about the code and the features. I don't have an answer for how to make such fundamental changes in people's mind and so far I don't think anybody can do that effectively and at scale. So I guess we will have just to continue pretending it's all about opportunity and knowledge.
I'm a beginner developer, and this talk was incredibly relatable even to me as a novice. Some of the things he's objecting to/suggesting I thought of as well, and just make sense (though I'm still stuck on the substring point). I agree with the commenter that I wish there was access to his course.
The substring point is to make the function "total". The fundamental problem with the methods is that there are parameters that you can pass to the function that it can't handle. Because of that, it's forced to throw an exception. Instead, "define the error out of existence". The caller should not be able to send bogus values into the method in the first place. Then there would be no need to throw an exception.
@@danielt63 Just to add to this; The redefinition is that if the caller asks for a substring outside of the string, of course that is an empty string. So just return that. That's what the caller asked for. You don't let the caller inherit any unwanted complexity.
One thing though about a shallow class is, even if the signature is technically a bit more, it helps you abstract business logic so that if tgat piece ever needs to change it's only a single spot. Also it helps guard against mis-writing that small logical bit in multiple places in the code. A compiler can verify correct usage (mostly) of the function but not of the logic.
[23/01, 11:18] Bhuvanesh Nadella: In the video, John Ousterhout discusses his views on software design and how it can be improved. He talks about his new book on the subject, and how he hopes it will start a discussion on how to improve software design. He also talks about how he thinks testing considerations like unit testing can influence interfaces and abstractions. 00:00:00 In this talk, John Ousterhout argues that software design is still a black art, and that there is no agreed-upon definition of what a good piece of software looks like. He goes on to discuss his new course on software design, and how he hopes it will start a conversation about how to improve the design of software. 00:05:00 In this talk, John Ousterhout discusses his philosophy of software design, which is based on the idea that anyone can learn to be a great software developer with the right amount of practice. He describes his course at Stanford, which uses an iterative approach to teaching students how to design software. [23/01, 11:18] Bhuvanesh Nadella: 00:20:00 In the video, John Ousterhout argues that striving for shorter methods is not as important as striving for deeper abstractions, and that exceptions are a huge source of complexity. He gives three examples of how redefining the semantics of a function can eliminate the need for exceptions altogether. 00:25:00 John Ousterhout argues that exceptions should be avoided whenever possible, as they are often the result of programmer error. He gives the example of the Java substring method, which he believes should be more forgiving in terms of indexing. Ousterhout believes that making it easier for programmers to do the right thing will ultimately lead to fewer mistakes being made. 00:30:00 The speaker argues that exceptions are best used when they are thrown farthest up the stack, as this allows for better error handling by the caller. He also argues that crashing is sometimes the best option, as it can avoid complexity and data corruption. 00:35:00 John Ousterhout talks about the importance of good design in software development, and how it can help developers work faster in the future. He argues that working code is not enough, and that developers need to sweat the small stuff in order to create a great design. [23/01, 11:18] Bhuvanesh Nadella: 00:05:00 In this talk, John Ousterhout discusses his philosophy of software design, which is based on the idea that anyone can learn to be a great software developer with the right amount of practice. He describes his course at Stanford, which uses an iterative approach to teaching students how to design software. 00:10:00 John Ousterhout discusses the importance of deep classes in software design. He argues that classes should have a simple interface with a large amount of functionality underneath. He also discusses the importance of mindset in software design, and how red flags can help beginners identify problems in their designs. 00:15:00 In the RUclips video "A Philosophy of Software Design | John Ousterhout | Talks at Google", John Ousterhout discusses the importance of design in software development and gives examples of shallow and deep design. He argues that shallow design is often more complex than it needs to be and that classes should be designed with future expansion in mind. [23/01, 11:18] Bhuvanesh Nadella: 00:40:00 John Ousterhout argues that a good software design culture is important for attracting top talent, and that a company should invest 10-20% of their resources into design. He also advocates for taking small steps, designing interfaces and documentation, and always looking for ways to improve the codebase. 00:45:00 In this video, John Ousterhout talks about his philosophy of software design. He emphasizes the importance of constantly asking oneself whether one is doing the best they possibly can, and of investing time and resources into making designs better. He also talks about how teaching the course has helped him to become a better designer himself. 00:50:00 In the "A Philosophy of Software Design | John Ousterhout | Talks at Google" video, John Ousterhout discusses his views on software design and how he thinks it can be improved. He talks about his new book on the subject, and how he hopes it will start a discussion on how to improve software design. He also talks about how he thinks testing considerations like unit testing can influence interfaces and abstractions. [23/01, 11:18] Bhuvanesh Nadella: 00:55:00 In the RUclips video "A Philosophy of Software Design | John Ousterhout | Talks at Google", John Ousterhout discusses his views on software design, specifically in relation to the use of threads and the Tk programming language. He states that threads are a necessary evil in some cases, but that they can be difficult to program. He also shares his opinion that the Tk language is not as well suited for web applications as it could be. 01:00:00 - 01:00:00 John Ousterhout argues that the best way to hire software designers is to find people that you enjoy talking to during the interview process. He admits that this may not lead to the most diverse team, but says that it has worked well for him in the past. 01:00:00 In this talk, John Ousterhout discusses his philosophy on software design. He argues that the best way to hire software designers is to find people that you enjoy talking to during the interview process. He admits that this may not lead to the most diverse team, but says that it has worked well for him in the past.
On tight schedules, my experience is that most managers actually listen if you have sound rationale behind your claims. What I've noticed that we devs often complain about such things among ourselves only, and happily accept any new stories in next Sprint planning -- none of which involve major refactoring or paying tech debt. If you need time for improving design, ensure you have a good plan and communicate it, don't just grumble. Even it it won't work, at least you did your best if you communicated it to management as well.
Around 43:00 John mentions the tenets of CI/CD, Continuous Integration, Continuous Design, mentioning that we never get the entire system correct at the first go-around. I wish a bit more emphasize was presented on CI/CD, developing software in uncertain circumstance/context: Small changes and done very very frequently, very strong (automated) testing, immediate feedback, adjust/refactor, rinse-and-repeat... Quality, in this way, does not decrease speed, it's the opposite: In the end it will increase it.
I think lot of small classes/functions approach is good if you have a smart type system, similar to those of rust and typescript. Unique and expressive types as input and outputs to a function eliminates need for lot of unit cases and compiler can play a more active role. All of these little functions could be wrapped in some sort of shallow interface before exposing them for use outside module.
On Exceptions, Good advice to try to get rid of them. As John described, maybe you can try to change the behavior a bit that still satisfies the contract/api/interface. Another way is to *model* the error-value in your domain. Eg defined a `no_char` value being returned by string.charAt(x) when x is either negative or too large. Or take a page from functional programming and have your method return a sum/union type, returning either an expected value or an error: `Either`, like "fun String.charAt(x:Integer): Either { ... }" Throwing exceptions up the callstack runs the risk of higher abstractions needing to deal with lower abstraction errors.
Te: 17:55, trying to keep “methods” small doesn’t lead to shallow classes, because they may just be calling many private functions. The advice to keep methods short is still bad because it’s quantitative rather than qualitative. A better advice is keep your methods “readable” and “understandable”. It’s about choosing words/abstractions.
John Ousterhout, please consider learning an Actor language like Elixir and OTP patterns. It will expose temporal coupling and other fundamental problems with isolation (the single most important property of any system) that you're not considering right now.
I like the concept of approaching software design as philosophy instead of prescriptive practices. I agree with the goals here and some of the thoughts, but I'm not yet convinced I agree on all points. For instance, it seems as if he disagrees with the notion that functions can be overly long, though it doesn't sound like he quite believes that an extremely long function is likely to be good design. I think it's important in this case to come back to the term "code smell". A function that takes up more than a full computer screen (in my experience, that's about 50-ish lines) is not automatically poorly written, but it is a _smell_ that the function may be a poor abstraction that could be improved. However, this is actually in pursuit of John's same goal of good interfaces and good abstractions. However, it's not a guarantee that a longer function is badly abstracted, nor does it mean a short function is automatically a good abstraction.
I think it means he might just value the interface to the class over the implementation. He did make points about that the interface can introduce complexity for the entire codebase. Meanwhile, if you have a 100 lines of code function in your class it´s more localized complexity. You inherently always come back to the famous "well it depends" with making statements about this. When you have an interface with a single "RunTheThing" function and hundreds of lines of code behind it you just wrote an extraordinarily deep class that is also extraordinarily horrible. But say for example you´ve got a CLI application going. It happens to have 10 arguments. All with default values and some documentation. I would not be offended by the resulting code if you put all that in a single "ParseArguments" function. Sure it will probably be like 5 lines give or take some per CLI argument, then spaces, creating the parser or whatever and creating the result object. You easiely end up with a 70 lines code function. Still, it´s not complex code and if you have a block for every variable and use good variable names, its still I would say just as easy to read as if you created a a function for every argument. So, I would argue wether or not you want to pull that one apart becomes personal preference at that size in this case. You can always pull it apart later. And that is a case where it is at the same time very easy to pull the method apart if you decided to. There are cases where it´s really challenging to pull methods apart below 50 lines and cases where no questions asked the method does too much and should be pulled apart. I think obsessively long methods are an indication of bad code. So in that sence I believe it to be a good rule of thumb to set yourself some limit. But I would say also that it´s one of those rules you can break. Just be careful when you do and re-evaluate wether or not this method is getting out of hand or not.
@@shioli3927 No disagreement. That's why I said it's a "code smell", not automatically bad. In fact, I'm doing a lot of hardware testing right now, and it's just very low-level, highly procedural/structural code. Several of our test are a couple of hundred lines long, even with helper functions to split out some of the work. A lot of those are assignments of 1-10 bits that set a single parameter on the FPGA, so it's wordy. That said, the functions have gotten a _lot_ better and easier to read since I've gotten hold of them. During the improvements, they have naturally become shorter as well. This is why long functions are a code smell -- better functions are usually short-ish and easy to read. On the other side of it, on my last project, I came across a bit of simulation code where a single function was over **6K** lines long. It was completely unreadable and undecipherable.
In fact, I think the Prof's proposed idea of throwing less exceptions is flawed. In his example about Java substrings being exception-happy ("many runtime errors that we get in software are because somebody forgot to wrap their substring indeces incorrectly") is perfect for my argument. If substring indeces are wrong, why return an empty string as he is proposing instead of trhowing an error? If you return an empty string, the engineer would think: Good, the substring call is correct. And then the software would be running without runtime errors but giving incorrect results. Throwing an exception on the other hand would let the engineer know there's something wrong. I'm I wrong on this?....
@@andrewskangah3710 I have one thing to say : Unit testing. Handling an exception that is your fault is kind of useless. I believe exceptions are not for debugging your code. But more to accomodate it to the imperfection of the world we live in.
@@andrewskangah3710 throwing the right or the right amount of exceptions is part of good api design, if almost all the clients have the same assumption about returning empty string and the use case are almost the same (checking boundaries) then java api prob should adjust accordingly a good example regarding to your concern is the Set operation, when you add an existed value to a set, add operation will return false, that way the api accommodated the client assumption (set should never have duplicate value, in this case it's something very obvious) and notified client without throwing any error/exception (should the client catch an exception in such case? API can make them do that but they didn't) substring is definitely a bit controversial since not all client assumptions are the same and it's a fundamental use case unlike some other complicated API interfaces where clients should read the API definition carefully (though I think clients should never assume anything about an API) from the book TIJ, throwing exception is less about letting engineers know something is wrong, it's more of providing a path to let program recover (returning the control to the original code path), I think this is what the professor was talking about, why creating so many paths to complicate the programming? To make things worse, there're paths API didn't cover... This is the major difference between some of the functional programming language (throwing less or non errors) and Java (more easy going I should say) in the end it's all about the contract, it's a shared responsibility between clients and api developers
Objection on the jasa Substring solution: if we try to get more than in the string, we might require an exception, especially if the expected outcome is a string of size 6, which the clipping and no exception solution might not fulfill. But otherwise, I like the approach, it's just about what we have to expect...and code accordingly (if possible without exceptions).
28:28 Imo thats exactly what tests are for. If the language/framework was responsible for protecting from making the programming mistake by throwing and exception, how would it protect from an the actual runtime exception that will be raised if programmer does not catch it. I strongly agree with the Professor Ousterhout's opinion on that matter.
More things to think about: 1. Every line should do one and only one thing. Nested methods/functions are typically harder to read and debug. "One line of code" is the enemy if it hides multiple branches of logic, aside from the standard one-line if-else statements. These should either be broken out for readability, or refactored to better handle the parameters available. 2. Teach some basic ideas for reducing cyclomatic complexity... shorter methods/functions, eliminating if-else-statments (and particularly else-statements), etc. 3. Use good tools that point out errors in your code automatically, like Jetbrains products or the various Linting tools. Also use more modern/advanced IDEs. The downside is that each tool comes with a learning curve (and some cost $$$).
The slower approach, where you take care to do it properly, is also the one where you learn how to do it well on the first attempt. With the fast approach you will go into management after 5 years max, because you found that you reached your peak after one year and then no your skill level just stagnates.
Slow is very underrated, unfortunately in current software industry it's all about speed, minimum viable product, etc. (I'm not blaming, because I understand there are market pressures etc.)
@@pyepye-io4vu And after 12 months max the project dies because all devs left because they can't develop fast in this mess and rather start something new somewhere else
I agree, but if you search for "divide and conquer", you'll find the phrase seems to have been taken over to describe a fairly narrow technique for writing recursive functions. So I would use that phrase sparingly to avoid misunderstandings.
The talk starts off with the assumption that "good programming" means breaking every problem into independent parts. However, anyone who designs any program of sufficient scope and complexity soon recognizes just how interdependent the parts are. People may wish they could break up the program into smaller units they can more easily grasp, but in practice some ways of breaking up the program are more natural to the problem space and hardware design than others. Oftentimes, a decision not to break a program into independent parts is the better one because it keeps the holistic vision of how the problem is solved, from end to end. OOP assumes the problem space of the program can, and should, be broken up into smaller "independent" pieces, as if there is no cost to doing so. There is a cost. It can make your program more complicated and harder to understand than the more straightforward alternative of just writing simple procedural code. As you create your program, you will recognize the things it needs to do over and over again, and at that point, you can lift those things up into higher abstractions like functions to make it easier to maintain. Assuming the program has a design up-front, or should have a design that we can figure out before we begin working on the program, is the greatest mistake we currently make in our industry. We need to stop teaching people to do that. I would have taken this more seriously if the presenter were willing to make a strong argument in favor of OOP instead of jumping in and assuming OOP is automatically proven to be the right choice for all of software engineering. If OOP is a good philosophy of software design, why? There are some very productive programmers who do not agree with this premise, do not take it for granted, and in fact prefer not to use OOP to make software.
The MAJOR exception to what the speaker talks about in 33:00 is when you're dealing with a Real-Time System for a Critical device for either medicine, aerospace and defense industries where when you allow the software to put up its hands and say "I give up" and crash very BAD things happen. in these industries, you not only have to make sure your system is resilient to crashing but WHEN it does crash, how do we recover from the crash? You usually need your system to do this in VERY quick succession. For soft systems like consumer and server systems applications an occasional crash is fine and won't necessarily have a BIG impact on the overall functionality of the system as a whole. Thankfully in systems OUTSIDE of medical, aerospace and defense, systems engineers have a bit more "breathing room" in that regard.
The techniques that he is saying used to be the way that pupils were taught in India by "guru shishya parampara" where the guru would give constant feedback + make the pupils do many other things. We threw this entire system and now we have to hear that certain parts of our old technique do work and work well. :)
Which paper from David Parnas from 1970s that was referenced at 12:47? Was it "A technique for software module specification with examples" or "On the Criteria to Be Used in Decomposing Systems into Modules" or some other one?
We're 5 years on and we don't have these principle of software design nailed down yet. There is currently a lot of push back on "Clean Code". So there is a shift happening. Maybe another 5 years is required.
I completely agree with the philosophy, but I was rather surprised when at the end professor Ousterhout says that maybe one day we will be able to agree to principles of good software design, I would rather say we already do ;) this is well-known stuff, e.g. most, if not all of the design principles in the talk are (interesting) corollaries of SOLID. Another thing is how to introduce strategic programming in real-world projects, there we have work to do ;)
Not sure if at 12:00 "Deep vs Shallow classes" is a great metaphor. Instead, a class should minimize coupling (to external code) and maximize cohesion (talk to 'near-by' code). Ideally, a class should do *one* thing *well*. Same for a function: Ideally, it should do one thing well.
When he talks about methods can be big, I think that goes directly against some ideas Clean Code by uncle Bob. I am also having a hard time to reconcile the idea of having classes with dept and some of the advices in his book.
I can tell you from personal experience, Uncle Bob recommends 5 lines per method if possible. That just makes you jump ALL OVER THE CLASS (and other classes) just to find out what is happening, and at the end of all that, you find out only a couple things are happening. Would've been clearer to just have a method with 10-100 lines.
Deep class: a very simple interface with a very large amount of functionality underneath it. The idea of abstraction is we're trying to provide a simple way of thinking about something that's actually quite complicated underneath. In fact, the whole idea behind software design is we're doing things for the future. It's we're doing things today to make it easier for us to develop in the future. BUT the classic problem with software is we CAN NOT visualize the future very well. software design is we're doing things for the future, but CAN NOT visualize the future very well. You want to make the common case really, really simple.
Great talk! I also see it my way ... that the medal of 'software design' has two sides: - statics: structure, hierarchy, components & interfaces, pattern, libraries, databases, encryption etc. ... - dynamics: latency, speed, asychnronicity, redundancies, fault-tolerance(s) targeting distributed systems etc. ... I would start bigger projects with an executable spec ... (procurement?) ... written in the highest possible layer of abstractions using a dynamic programming language like phyton ... for rapid-prototyping & conceptual proofs ... and as MVPs ... Than the implementation phase follows in industrial hardened programming languages like GO & C++ probably combined with Web-Assembly for an Internet-GUI ... including a growing set of unit-testing components talking to the executable spec (MVP) ... keeping the dynamics in developers view from the begining on ... The flexibility for products should always stay with and above the components level, which enables compositions of components in order to address solutions for customer challenges ...
read the article "stop recommending clean code" ;> in the comments secitons lots of people recommend John Ousterhout's book on the philosophy of system design :>
Additionally, and perhaps implied, code should be self-commenting. That is, method names should describe their function, variables names should describe what they store, etc. A rule I use is, "if you have to leave a comment, should you rename or slightly alter the logic, instead?" This rule will greatly reduce comments by forcing you to do a sanity check on your code and reanalyze your approach. Ultimately, the re-readability skyrockets and you get in the habit of better design.
Disagree, what's obvious to the programmer at the time he or she is writing the code may not seem obvious to another person, or at a much later date. That has been my experience. When I write a program every line of code is obvious to me. That is not always the case at a later date, or to other people. There is a huge range of talent and style among programmers, expecting things to be obvious to all of them is a mistake, imo.
The point of making the code "self-documenting" is, I think, exactly to try to make your code more obvious. If I had a function called prepCust(list) and I comment it with "sorts the list of customers by age", maybe I should rather rename the function to "sortByAge(customers)" or something. Suddenly it's a lot more obvious what it's doing. Now I might have no idea *why* I sorted by age. Maybe it's to make it possible to check for senior citizen and children's discounts. This might be obvious if the function calling "sortByAge" is itself called "findDiscountGroups" or something (obviously my naming conventions aren't perfect, but you get the idea). A modern IDE should allow you to search on where your sortByAge function is called and allow you to figure out where or why it's being used the way it is. This might look like: findDiscountedGroups(customers) { sortedByAge = sortByAge(customers) seniorDiscounted = findSeniors(sortedByAge) juniorDiscounted = findJuniors(sortedByAge) return ... } Now, even an untalented or inexperienced programmer should be able to look at that and get the basic idea of what's happening. They can go into the "findSeniors" and "findJuniors" functions and determine what qualifies someone as senior or junior (or maybe those could be additional parameters that are passed in as well). Again, you might be tempted to put a comment on the "findSeniors" function saying it should be passed a sorted list of customers, but why not call its input parameter "customersSortedByAge"? But, of course, one should not make the opposite mistake of thinking one should not use comments. There's no harm in putting a comment on "sortedByAge" saying "this is used to calculate discounts...", or specifying that the input list for findSeniors should be sorted. And, of course, if you are making an external-facing library or API, you cannot expect people to be able to figure out what your functions are doing by going into them, so in that case having good commenting is essential (and should actually be given as much focus in code reviews etc. as the code itself). Also, where comments really shine is explaining the *why* of a given function. E.g. "we used [sort X] (instead of the more popular [sort Y] or [sort Z]) for this function because in practice the list of customers tends to [have property A] and profiling found [sort X] worked the best" or "we sort the list once at the beginning and then assume the list is sorted everywhere else" or "due to a bug in [library] we pass the input list in batches of 1000 instead of all in one go" (and bonus points if you can link to the issue number of a reported bug on the library's issue tracker - but... you might be using an old, unmaintained library where that's not possible).
That sounds great. But with Deep classes where there is a lot of complexity that may not be enough, so comments can be very useful for people who have to maintain the code. If you liked the video you'll probably love the book, it's really good. Your library might have it, or purchase it if you request it, if you don't want to buy it.
Ideally your code and your comments and your spec should be the same thing... although its hard to do this in most languages, although Lisp macros, and Smalltalk eDSL classes (maybe TCL upvalues, but TCL lost mwhahaha) can help you do this. (Lisp :is (just English) :with parenthesis :for grouping).
29:00 if you orde program a mistake extracting a sub string and you get the wrong chunk, how far will this chunk go before it results in a business logic error? Wouldn’t you have tests for that business logic?
Outstanding talk - really sound philosophies like more speed does not equal to greater haste, and strategic over tactical development every day of the week! Some really sound principles for good software development that should always be borne in mind. Technical debt which inevitably results from tactical, and "fast" development is always extremely expensive to fix, and thus almost certainly never is. Cheaper to wait for the next iteration of the application - i.e.: tear down the building and start again... not very smart. My thoughts on the matter... 🙂
Really interesting talk! I think it would be great to add this kind of topics in university subjects, besides being able to access online to the subject he teaches for those who don't live there. It would also be useful to include in the university libraries in other countries, books like the one he wrote and to apply some kind of discount in the purchase for academic purposes or according to the country or region.
49:17 Does he mean in other words: Making class 1 lvl of abstraction deep make class simple and deeper? Like class PostgreDbConnector extends DbConnector {...}
seems to me there are lots of connections to the topic of Technical Debt - esp at the Architecture/Design level. There exists quite some literature on that which is not mentioned (yet)...
@29:55 Don't use NULLs or, in this case, an exception -- Return an empty. So the correct thing to return would be and empty string ("") or character (''). That is effectively your sentinel return value. It very often requires no additional logic and it fits nicely within the principles in this talk.
On the hiring question, I think the correlation of good hires being ones enjoyable to talk to comes a lot from a candidates confidence in themselves and their ability to do the role as presented, as well as the interviewer having a well-honed bullshit detector for anyone trying to fake their way through.
I think agreement on basic rules for software design would be an indicator of the progress having already happened: an increased understanding of how to do software design would naturally bring about an agreement to follow/embody that understanding-but agreeing to write short methods does not make short methods better, nor does engaging in the behavior increase our understanding of why it makes sense. (IOW I think you have the causality arrow pointing in a backwards direction)
Yes. They are also good if they replace a commonly occurring mistake with a plainly-named function that does it correctly. That's my counter-argument to his example of list vs pointer manipulation; the named methods for lists are clearer and less error-prone than pointer manipulation, even the one-liners.
90% software engineers are egoistic monsters who just like to consume stuff and dont know how to talk..they talk like aliens talking geeky stuff because they dont understand it well....but this professor is so well organised and speaks so well
Many database libraries in many languages returns an error when queries did not found a record instead of returning empty list, this makes coding hard since empty query result for me is a non error issue
You can have the public function do the deep thing with less than 20 lines of code, just separate the levels of abstraction into different functions. code in function must not cross levels of abstraction
30:40 AUDIENCE: So one question-- so when is it a good idea to actually throw exceptions? So the intuitive case is you want to eliminate exceptions as much as possible, then how do you decide, no, there's nothing that I can do? My changing semantics doesn't work. And I really have to throw an exception now. JOHN OUTSERHOUT: Well, if you fundamentally can't carry out your contract with your caller and if you can't implement your interface, then you sort of have to throw an exception.
Examples for "defining errors out of existence": SQL NULL propagation in expressions, Lua's nil and Lua's reluctance to throw in general (unlike Python). Little things that separate men from boys and ladies from little tiny girls.
3:46 most important thing in computer science: *problem decomposition - How do you take a complicated problem or system and chop it up into pieces that you can build relatively independently*
10:10 what are the secrets
12:36 classes should be deep
15:25 typical shallow method
17:50 one of the biggest mistakes people make:
18:46
*20:16*
20:32 example of a deep interface
21:55 defines errors out of existence
23:00 try to minimize the number of places we have to handle exceptions
26:49 3rd example: substring
34:13 tactical VS strategic programming
34:24 tactical
37:29 strategic programming approach
Oh hey it's you again! Nice indenting
thanks for the summary
@@riansyahtohamba8215 :)
@@ruixue6955 thanks!
Thank you bro
Great lecture, with great points.
But apart from the topic, what really impressed me was how honest he was. He was not afraid to say "I will have to think on that" or "I have no idea" multiple times.
"Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat." --Sun Tzu, "The Art of War"
"If fighting is sure to result in victory then you must fight!" -Sun Tzu prolly
@@commandertaco1762 ccpcp??
@@commandertaco1762 lol tautology
@@boggianluzecriacao
Vo x yy
Strategy vs tactic?
His dedication to teach is inspiring
The book is well worth it. Thanks Prof Ousterhout.
Edit: and I would add that it shines most at providing the rationales behind certain "good practices" which are more used as dogma than as heuristics to produce strong systems. Some of these rationales are not trivial (at least not for me!) and understanding them will shape your intuition for future work. Nicely done.
Good explanation. It reminds me of my life at the second part of the seventies, when I used the ideas of Parnas (information hiding) and Dijkstra (semaphores and layering) to design an OS for 16-bits Philips mini-computers for our Air Traffic Control systems. We used the OS from 1976 to 1990, when that mini-computer range was discontinued. One system would consist of 1 to 100 of these mini computers (2 or 3 per radar; 1 per radar display and 2 to 10 centrally).
I wish there might be an open access to his software design studio class for public.
I'm really curious about how they teach the student about tackling design problems.
I second that
His book is in the description
this is the most abstract talk i've ever seen
One of best lectures I have seen. Thank you, John Ousterhout. Thank you, Google.
Wow, imagine the teacher actually providing feedback on your code. No wonder Stanford has a good reputation.
2 cents:
1. A deep class is a borderline monolith. A module should be deep with classes that have high cohesion and purpose.
2. The substring debate was meaningless. It does not have to be either-or.
First function:
string Substring(int, int) throws IndexOutOfBoundsException
Second function:
string GetSubstringOrNull(int, int) // return null if the index is out of bounds/ mismatch etc
Third function:
string GetSubstringOrEmpty(int, int) // return an empty string if the index is out of bounds/ mismatch etc
Second and third save this code from being written repeated by hundreds of developers and further give them a choice of whether they care about the bounds (check for null on return if they do) or don't (empty).
At the beginning, "Talent is overrated - the only thing that really differentiates the top performers from the average performers is how much they've practiced."
but at the end, "
I think you want to hire the person who's the fastest learner, who has the fastest slope.
I look for people who are really smart, fast learners"
Yes, there does seem to be a tension between those statements. Perhaps his point about talent was that designing is something that can be learned rather than purely down to innate ability. That's kind of orthogonal to being a fast learner: fast learners merely get there faster.
*Practice at learning fast.
this guy speaks from experience, very practical advice
Problem decomposition ... this is what Descartes talked about in his Dicourse On The Method ... I have some notes on this, if I find them I paste a link here later ... btw, I wouldnt agree that we dont teach problem decomposition ... for example the applied category theory community talks about nothing but "compositionality" which is all about problem decomposition and solution synthesis.
About teaching people to be better developers (the 10x thingy), the issue goes way deeper than just sheer practice. It does involve talent in a limited sense and many other aspects but, much more importantly, it involves your work ethics and ultimately your moral values.
To begin with, people do have different inherent abilities. I'm not saying he's arguing the contrary but some people do have the idea that we're all a blank slate and equal in all aspects. This is self-evidently wrong. Some people develop muscles better than others, some are naturally good with numbers, some thrive in dealing with people while others shy away from such a thing. If you ever lived in or close to a large family, you'll be able to easily correlate how wildly different each child can be in terms of interests and capacity to do things.
However, I agree those inherent abilities and capacities are not necessarily the determining factors of productivity. They can become a hindrance if the person is forcing himself or herself to do something diametrically contrary to their skills but even then self determination can get one at least above the average.
The ethics and moral values, however, are what drives people to put their minds and hearts into whatever they are doing. I have taught countless developers how to perform tasks from the simplest to the most complex in the work environment and outside it, and in different cultures as well. What I have observed is that very few of those developers were actually invested in their work - either trying to solve a problem or deliver a feature in the best way they could - as opposed to just doing their job and satisfying their manager or whatever external factor was pressuring them.
The good developer feels bad when they deliver code they know it's less than ideal. This sentiment drives them to do better and sometimes make contributions that are not asked by anyone else. They set a higher bar to themselves to the best of their abilities. They care about such things because they have an internal value and a conscience that bothers them otherwise.
The average developer feels good if they deliver what they were asked to deliver. They only care if they are doing a good job for external reasons, such as fear from what other people will say about their code, fear of not getting a promotion, etc. Those external factors can, to some extent, drive them to be more productive temporarily, with bursts of focus and effort. But it fails miserably in making them more productive over time and really grow as developers because the external pressure soon will burn them out and they will retreat.
If possible, good developers will spend countless hours doing and re-doing stuff until it satisfies them, thus they practice without even realizing it. With each effort, their minds are engaged and are looking for opportunities to improve and satisfy their own high standards.
The average developer can repeat the same task over and over again but their mind is not as engaged and focused because they are not looking for an improvement but just to finish the job at hand. Repetition can make them a little faster as they get used to perform the same tasks. But that's not the same as improving how they perform the tasks, not by a long shot.
The conclusion is that making developers more productive is not about providing them with more training or more time to practice, it's about changing their hearts, giving them passion for what they are doing, making them care about the code and the features. I don't have an answer for how to make such fundamental changes in people's mind and so far I don't think anybody can do that effectively and at scale. So I guess we will have just to continue pretending it's all about opportunity and knowledge.
I'm a beginner developer, and this talk was incredibly relatable even to me as a novice. Some of the things he's objecting to/suggesting I thought of as well, and just make sense (though I'm still stuck on the substring point). I agree with the commenter that I wish there was access to his course.
The substring point is to make the function "total". The fundamental problem with the methods is that there are parameters that you can pass to the function that it can't handle. Because of that, it's forced to throw an exception. Instead, "define the error out of existence". The caller should not be able to send bogus values into the method in the first place. Then there would be no need to throw an exception.
@@danielt63 Just to add to this; The redefinition is that if the caller asks for a substring outside of the string, of course that is an empty string. So just return that. That's what the caller asked for. You don't let the caller inherit any unwanted complexity.
One thing though about a shallow class is, even if the signature is technically a bit more, it helps you abstract business logic so that if tgat piece ever needs to change it's only a single spot. Also it helps guard against mis-writing that small logical bit in multiple places in the code. A compiler can verify correct usage (mostly) of the function but not of the logic.
the exception handling part is really helps, thank you, john
Great talk. Great presenter. Great developer. Great ideas!
Thank you so much for sharing this!
❤🧡💛💙
[23/01, 11:18] Bhuvanesh Nadella: In the video, John Ousterhout discusses his views on software design and how it can be improved. He talks about his new book on the subject, and how he hopes it will start a discussion on how to improve software design. He also talks about how he thinks testing considerations like unit testing can influence interfaces and abstractions.
00:00:00 In this talk, John Ousterhout argues that software design is still a black art, and that there is no agreed-upon definition of what a good piece of software looks like. He goes on to discuss his new course on software design, and how he hopes it will start a conversation about how to improve the design of software.
00:05:00 In this talk, John Ousterhout discusses his philosophy of software design, which is based on the idea that anyone can learn to be a great software developer with the right amount of practice. He describes his course at Stanford, which uses an iterative approach to teaching students how to design software.
[23/01, 11:18] Bhuvanesh Nadella: 00:20:00 In the video, John Ousterhout argues that striving for shorter methods is not as important as striving for deeper abstractions, and that exceptions are a huge source of complexity. He gives three examples of how redefining the semantics of a function can eliminate the need for exceptions altogether.
00:25:00 John Ousterhout argues that exceptions should be avoided whenever possible, as they are often the result of programmer error. He gives the example of the Java substring method, which he believes should be more forgiving in terms of indexing. Ousterhout believes that making it easier for programmers to do the right thing will ultimately lead to fewer mistakes being made.
00:30:00 The speaker argues that exceptions are best used when they are thrown farthest up the stack, as this allows for better error handling by the caller. He also argues that crashing is sometimes the best option, as it can avoid complexity and data corruption.
00:35:00 John Ousterhout talks about the importance of good design in software development, and how it can help developers work faster in the future. He argues that working code is not enough, and that developers need to sweat the small stuff in order to create a great design.
[23/01, 11:18] Bhuvanesh Nadella: 00:05:00 In this talk, John Ousterhout discusses his philosophy of software design, which is based on the idea that anyone can learn to be a great software developer with the right amount of practice. He describes his course at Stanford, which uses an iterative approach to teaching students how to design software.
00:10:00 John Ousterhout discusses the importance of deep classes in software design. He argues that classes should have a simple interface with a large amount of functionality underneath. He also discusses the importance of mindset in software design, and how red flags can help beginners identify problems in their designs.
00:15:00 In the RUclips video "A Philosophy of Software Design | John Ousterhout | Talks at Google", John Ousterhout discusses the importance of design in software development and gives examples of shallow and deep design. He argues that shallow design is often more complex than it needs to be and that classes should be designed with future expansion in mind.
[23/01, 11:18] Bhuvanesh Nadella: 00:40:00 John Ousterhout argues that a good software design culture is important for attracting top talent, and that a company should invest 10-20% of their resources into design. He also advocates for taking small steps, designing interfaces and documentation, and always looking for ways to improve the codebase.
00:45:00 In this video, John Ousterhout talks about his philosophy of software design. He emphasizes the importance of constantly asking oneself whether one is doing the best they possibly can, and of investing time and resources into making designs better. He also talks about how teaching the course has helped him to become a better designer himself.
00:50:00 In the "A Philosophy of Software Design | John Ousterhout | Talks at Google" video, John Ousterhout discusses his views on software design and how he thinks it can be improved. He talks about his new book on the subject, and how he hopes it will start a discussion on how to improve software design. He also talks about how he thinks testing considerations like unit testing can influence interfaces and abstractions.
[23/01, 11:18] Bhuvanesh Nadella: 00:55:00 In the RUclips video "A Philosophy of Software Design | John Ousterhout | Talks at Google", John Ousterhout discusses his views on software design, specifically in relation to the use of threads and the Tk programming language. He states that threads are a necessary evil in some cases, but that they can be difficult to program. He also shares his opinion that the Tk language is not as well suited for web applications as it could be.
01:00:00 - 01:00:00
John Ousterhout argues that the best way to hire software designers is to find people that you enjoy talking to during the interview process. He admits that this may not lead to the most diverse team, but says that it has worked well for him in the past.
01:00:00 In this talk, John Ousterhout discusses his philosophy on software design. He argues that the best way to hire software designers is to find people that you enjoy talking to during the interview process. He admits that this may not lead to the most diverse team, but says that it has worked well for him in the past.
The first lecture on programming I ever see! That was awesome :)
On tight schedules, my experience is that most managers actually listen if you have sound rationale behind your claims. What I've noticed that we devs often complain about such things among ourselves only, and happily accept any new stories in next Sprint planning -- none of which involve major refactoring or paying tech debt. If you need time for improving design, ensure you have a good plan and communicate it, don't just grumble. Even it it won't work, at least you did your best if you communicated it to management as well.
His video on the thought process that went in while designing etcd is insightful!
Around 43:00 John mentions the tenets of CI/CD, Continuous Integration, Continuous Design, mentioning that we never get the entire system correct at the first go-around.
I wish a bit more emphasize was presented on CI/CD, developing software in uncertain circumstance/context:
Small changes and done very very frequently, very strong (automated) testing, immediate feedback, adjust/refactor, rinse-and-repeat...
Quality, in this way, does not decrease speed, it's the opposite: In the end it will increase it.
I think lot of small classes/functions approach is good if you have a smart type system, similar to those of rust and typescript. Unique and expressive types as input and outputs to a function eliminates need for lot of unit cases and compiler can play a more active role. All of these little functions could be wrapped in some sort of shallow interface before exposing them for use outside module.
On Exceptions,
Good advice to try to get rid of them.
As John described, maybe you can try to change the behavior a bit that still satisfies the contract/api/interface.
Another way is to *model* the error-value in your domain. Eg defined a `no_char` value being returned by string.charAt(x) when x is either negative or too large. Or take a page from functional programming and have your method return a sum/union type, returning either an expected value or an error: `Either`, like "fun String.charAt(x:Integer): Either { ... }"
Throwing exceptions up the callstack runs the risk of higher abstractions needing to deal with lower abstraction errors.
Te: 17:55, trying to keep “methods” small doesn’t lead to shallow classes, because they may just be calling many private functions.
The advice to keep methods short is still bad because it’s quantitative rather than qualitative. A better advice is keep your methods “readable” and “understandable”. It’s about choosing words/abstractions.
John Ousterhout, please consider learning an Actor language like Elixir and OTP patterns. It will expose temporal coupling and other fundamental problems with isolation (the single most important property of any system) that you're not considering right now.
I like the concept of approaching software design as philosophy instead of prescriptive practices. I agree with the goals here and some of the thoughts, but I'm not yet convinced I agree on all points.
For instance, it seems as if he disagrees with the notion that functions can be overly long, though it doesn't sound like he quite believes that an extremely long function is likely to be good design. I think it's important in this case to come back to the term "code smell". A function that takes up more than a full computer screen (in my experience, that's about 50-ish lines) is not automatically poorly written, but it is a _smell_ that the function may be a poor abstraction that could be improved. However, this is actually in pursuit of John's same goal of good interfaces and good abstractions. However, it's not a guarantee that a longer function is badly abstracted, nor does it mean a short function is automatically a good abstraction.
I think it means he might just value the interface to the class over the implementation. He did make points about that the interface can introduce complexity for the entire codebase. Meanwhile, if you have a 100 lines of code function in your class it´s more localized complexity. You inherently always come back to the famous "well it depends" with making statements about this. When you have an interface with a single "RunTheThing" function and hundreds of lines of code behind it you just wrote an extraordinarily deep class that is also extraordinarily horrible.
But say for example you´ve got a CLI application going. It happens to have 10 arguments. All with default values and some documentation. I would not be offended by the resulting code if you put all that in a single "ParseArguments" function. Sure it will probably be like 5 lines give or take some per CLI argument, then spaces, creating the parser or whatever and creating the result object. You easiely end up with a 70 lines code function. Still, it´s not complex code and if you have a block for every variable and use good variable names, its still I would say just as easy to read as if you created a a function for every argument. So, I would argue wether or not you want to pull that one apart becomes personal preference at that size in this case. You can always pull it apart later. And that is a case where it is at the same time very easy to pull the method apart if you decided to. There are cases where it´s really challenging to pull methods apart below 50 lines and cases where no questions asked the method does too much and should be pulled apart. I think obsessively long methods are an indication of bad code. So in that sence I believe it to be a good rule of thumb to set yourself some limit. But I would say also that it´s one of those rules you can break. Just be careful when you do and re-evaluate wether or not this method is getting out of hand or not.
@@shioli3927 No disagreement. That's why I said it's a "code smell", not automatically bad.
In fact, I'm doing a lot of hardware testing right now, and it's just very low-level, highly procedural/structural code. Several of our test are a couple of hundred lines long, even with helper functions to split out some of the work. A lot of those are assignments of 1-10 bits that set a single parameter on the FPGA, so it's wordy. That said, the functions have gotten a _lot_ better and easier to read since I've gotten hold of them. During the improvements, they have naturally become shorter as well. This is why long functions are a code smell -- better functions are usually short-ish and easy to read.
On the other side of it, on my last project, I came across a bit of simulation code where a single function was over **6K** lines long. It was completely unreadable and undecipherable.
We need bliss technologies for blissful generations
Great talk. Looking forward to read the book though it is not available on amazon India.
31:54 "exceptions provide the most value when you throw them the farthest"
yes, that's a good one, I think that's why now we have Promise or Optional to do the happy path all the way along, and only handle error at the end.
In fact, I think the Prof's proposed idea of throwing less exceptions is flawed.
In his example about Java substrings being exception-happy ("many runtime errors that we get in software are because somebody forgot to wrap their substring indeces incorrectly") is perfect for my argument.
If substring indeces are wrong, why return an empty string as he is proposing instead of trhowing an error? If you return an empty string, the engineer would think: Good, the substring call is correct. And then the software would be running without runtime errors but giving incorrect results.
Throwing an exception on the other hand would let the engineer know there's something wrong.
I'm I wrong on this?....
@@andrewskangah3710 I have one thing to say : Unit testing.
Handling an exception that is your fault is kind of useless.
I believe exceptions are not for debugging your code. But more to accomodate it to the imperfection of the world we live in.
@@andrewskangah3710 throwing the right or the right amount of exceptions is part of good api design, if almost all the clients have the same assumption about returning empty string and the use case are almost the same (checking boundaries) then java api prob should adjust accordingly
a good example regarding to your concern is the Set operation, when you add an existed value to a set, add operation will return false, that way the api accommodated the client assumption (set should never have duplicate value, in this case it's something very obvious) and notified client without throwing any error/exception (should the client catch an exception in such case? API can make them do that but they didn't)
substring is definitely a bit controversial since not all client assumptions are the same and it's a fundamental use case unlike some other complicated API interfaces where clients should read the API definition carefully (though I think clients should never assume anything about an API)
from the book TIJ, throwing exception is less about letting engineers know something is wrong, it's more of providing a path to let program recover (returning the control to the original code path), I think this is what the professor was talking about, why creating so many paths to complicate the programming? To make things worse, there're paths API didn't cover... This is the major difference between some of the functional programming language (throwing less or non errors) and Java (more easy going I should say)
in the end it's all about the contract, it's a shared responsibility between clients and api developers
this is especially true for layered architecture
Impressive speaker. Really good lecture. Is it possible to find slides somewhere ?
platformlab.stanford.edu/Seminar%20Talks/retreat-2017/John%20Ousterhout.pdf
@@EnderWiking thanks
Objection on the jasa Substring solution: if we try to get more than in the string, we might require an exception, especially if the expected outcome is a string of size 6, which the clipping and no exception solution might not fulfill. But otherwise, I like the approach, it's just about what we have to expect...and code accordingly (if possible without exceptions).
28:28 Imo thats exactly what tests are for. If the language/framework was responsible for protecting from making the programming mistake by throwing and exception, how would it protect from an the actual runtime exception that will be raised if programmer does not catch it. I strongly agree with the Professor Ousterhout's opinion on that matter.
One of the greats.
Great talk! Only thing I can’t agree is crashing is fine. In embedded world, crashing is definitely not acceptable.
Thanks Google for sharing this with all of us.
More things to think about:
1. Every line should do one and only one thing. Nested methods/functions are typically harder to read and debug. "One line of code" is the enemy if it hides multiple branches of logic, aside from the standard one-line if-else statements. These should either be broken out for readability, or refactored to better handle the parameters available.
2. Teach some basic ideas for reducing cyclomatic complexity... shorter methods/functions, eliminating if-else-statments (and particularly else-statements), etc.
3. Use good tools that point out errors in your code automatically, like Jetbrains products or the various Linting tools. Also use more modern/advanced IDEs. The downside is that each tool comes with a learning curve (and some cost $$$).
Having just watched some vids on 'Clean Code' I am now a bit confused, but I enjoyed the common sense approach. 🙃
'Clean Code' is overrated.
Opened my mind. Thank you very much John.
The slower approach, where you take care to do it properly, is also the one where you learn how to do it well on the first attempt.
With the fast approach you will go into management after 5 years max, because you found that you reached your peak after one year and then no your skill level just stagnates.
Slow is very underrated, unfortunately in current software industry it's all about speed, minimum viable product, etc. (I'm not blaming, because I understand there are market pressures etc.)
@@pyepye-io4vu And after 12 months max the project dies because all devs left because they can't develop fast in this mess and rather start something new somewhere else
When he posed the first question I immediately thought: divide and conquer, seems pretty close( to "problem decomposition")
That's a nice perspective to it.
I agree, but if you search for "divide and conquer", you'll find the phrase seems to have been taken over to describe a fairly narrow technique for writing recursive functions. So I would use that phrase sparingly to avoid misunderstandings.
Everyone in computer science needs to watch this
The talk starts off with the assumption that "good programming" means breaking every problem into independent parts. However, anyone who designs any program of sufficient scope and complexity soon recognizes just how interdependent the parts are.
People may wish they could break up the program into smaller units they can more easily grasp, but in practice some ways of breaking up the program are more natural to the problem space and hardware design than others. Oftentimes, a decision not to break a program into independent parts is the better one because it keeps the holistic vision of how the problem is solved, from end to end.
OOP assumes the problem space of the program can, and should, be broken up into smaller "independent" pieces, as if there is no cost to doing so. There is a cost. It can make your program more complicated and harder to understand than the more straightforward alternative of just writing simple procedural code. As you create your program, you will recognize the things it needs to do over and over again, and at that point, you can lift those things up into higher abstractions like functions to make it easier to maintain.
Assuming the program has a design up-front, or should have a design that we can figure out before we begin working on the program, is the greatest mistake we currently make in our industry. We need to stop teaching people to do that.
I would have taken this more seriously if the presenter were willing to make a strong argument in favor of OOP instead of jumping in and assuming OOP is automatically proven to be the right choice for all of software engineering. If OOP is a good philosophy of software design, why? There are some very productive programmers who do not agree with this premise, do not take it for granted, and in fact prefer not to use OOP to make software.
The MAJOR exception to what the speaker talks about in 33:00 is when you're dealing with a Real-Time System for a Critical device for either medicine, aerospace and defense industries where when you allow the software to put up its hands and say "I give up" and crash very BAD things happen. in these industries, you not only have to make sure your system is resilient to crashing but WHEN it does crash, how do we recover from the crash? You usually need your system to do this in VERY quick succession.
For soft systems like consumer and server systems applications an occasional crash is fine and won't necessarily have a BIG impact on the overall functionality of the system as a whole. Thankfully in systems OUTSIDE of medical, aerospace and defense, systems engineers have a bit more "breathing room" in that regard.
I wish, he's my professor. Faculty in our university don't really teach that important stuffs, they just give us activities to do.
The techniques that he is saying used to be the way that pupils were taught in India by "guru shishya parampara" where the guru would give constant feedback + make the pupils do many other things. We threw this entire system and now we have to hear that certain parts of our old technique do work and work well. :)
Which paper from David Parnas from 1970s that was referenced at 12:47? Was it "A technique for software module specification with examples" or "On the Criteria to Be Used in Decomposing Systems into Modules" or some other one?
. Which mentioned in the preface of his book.
@@hsldymq1315 Thank you so much! I figured it out just the other day, but really appreciate you letting me know too!
We're 5 years on and we don't have these principle of software design nailed down yet. There is currently a lot of push back on "Clean Code". So there is a shift happening. Maybe another 5 years is required.
I completely agree with the philosophy, but I was rather surprised when at the end professor Ousterhout says that maybe one day we will be able to agree to principles of good software design, I would rather say we already do ;) this is well-known stuff, e.g. most, if not all of the design principles in the talk are (interesting) corollaries of SOLID. Another thing is how to introduce strategic programming in real-world projects, there we have work to do ;)
Slides for an earlier version of the talk: platformlab.stanford.edu/Seminar%20Talks/retreat-2017/John%20Ousterhout.pdf
Thank you!
Give a talk on how to ask questions....I think the audience needs it.
42
Re: the question at 16:30, we’re talking about the implementation of the business domain, not of all classes in your app.
I.e. the production code.
Not sure if at 12:00 "Deep vs Shallow classes" is a great metaphor.
Instead, a class should minimize coupling (to external code) and maximize cohesion (talk to 'near-by' code).
Ideally, a class should do *one* thing *well*.
Same for a function: Ideally, it should do one thing well.
Thank you so much for this informative video. You explained SEO so well and I am no longer confused. Thanks for making SEO so easy to understand.
I live for programming !!!! It's all I want to do for the rest of my life :-)))))
looks like you love parens too, welcome to Lisp, we need you!
Finally someone who echoes my thoughts! I love it so much!
C# lacks the VB6 "ON ERROR RESUME NEXT".
Methods/Functions should fit screen size, IMHO.
Great Lecture!, Tnx!
We still use TCL for VHDL verification in electronics! :)
When he talks about methods can be big, I think that goes directly against some ideas Clean Code by uncle Bob. I am also having a hard time to reconcile the idea of having classes with dept and some of the advices in his book.
I can tell you from personal experience, Uncle Bob recommends 5 lines per method if possible. That just makes you jump ALL OVER THE CLASS (and other classes) just to find out what is happening, and at the end of all that, you find out only a couple things are happening. Would've been clearer to just have a method with 10-100 lines.
Problem decomposition, excellent answer
Refreshingly relatable and practical
I liked his speach from first 10 seconds.
Very solid talk. Very solid ideas!
Deep class: a very simple interface with a very large amount of functionality underneath it.
The idea of abstraction is we're trying to provide a simple way of thinking about something that's actually quite complicated underneath.
In fact, the whole idea behind software design is we're doing things for the future. It's we're doing things today to make it easier for us to develop in the future. BUT the classic problem with software is we CAN NOT visualize the future very well.
software design is we're doing things for the future, but CAN NOT visualize the future very well.
You want to make the common case really, really simple.
Great talk! I also see it my way ... that the medal of 'software design' has two sides:
- statics: structure, hierarchy, components & interfaces, pattern, libraries, databases, encryption etc. ...
- dynamics: latency, speed, asychnronicity, redundancies, fault-tolerance(s) targeting distributed systems etc. ...
I would start bigger projects with an executable spec ... (procurement?) ... written in the highest possible layer of abstractions using a dynamic programming language like phyton ... for rapid-prototyping & conceptual proofs ... and as MVPs ...
Than the implementation phase follows in industrial hardened programming languages like GO & C++ probably combined with Web-Assembly for an Internet-GUI ... including a growing set of unit-testing components talking to the executable spec (MVP) ... keeping the dynamics in developers view from the begining on ...
The flexibility for products should always stay with and above the components level, which enables compositions of components in order to address solutions for customer challenges ...
why do hordes of programmers flock to uncle bob martin and not this guy is a mystery to me
read the article "stop recommending clean code" ;> in the comments secitons lots of people recommend John Ousterhout's book on the philosophy of system design :>
Thanks, good video! I learned a lot of useful and necessary information and also broadened my horizons.
10:34 "Comments should describe things that are not obvious from the code"
Additionally, and perhaps implied, code should be self-commenting. That is, method names should describe their function, variables names should describe what they store, etc. A rule I use is, "if you have to leave a comment, should you rename or slightly alter the logic, instead?" This rule will greatly reduce comments by forcing you to do a sanity check on your code and reanalyze your approach. Ultimately, the re-readability skyrockets and you get in the habit of better design.
Disagree, what's obvious to the programmer at the time he or she is writing the code may not seem obvious to another person, or at a much later date. That has been my experience. When I write a program every line of code is obvious to me. That is not always the case at a later date, or to other people. There is a huge range of talent and style among programmers, expecting things to be obvious to all of them is a mistake, imo.
The point of making the code "self-documenting" is, I think, exactly to try to make your code more obvious. If I had a function called prepCust(list) and I comment it with "sorts the list of customers by age", maybe I should rather rename the function to "sortByAge(customers)" or something. Suddenly it's a lot more obvious what it's doing.
Now I might have no idea *why* I sorted by age. Maybe it's to make it possible to check for senior citizen and children's discounts. This might be obvious if the function calling "sortByAge" is itself called "findDiscountGroups" or something (obviously my naming conventions aren't perfect, but you get the idea). A modern IDE should allow you to search on where your sortByAge function is called and allow you to figure out where or why it's being used the way it is. This might look like:
findDiscountedGroups(customers) {
sortedByAge = sortByAge(customers)
seniorDiscounted = findSeniors(sortedByAge)
juniorDiscounted = findJuniors(sortedByAge)
return ...
}
Now, even an untalented or inexperienced programmer should be able to look at that and get the basic idea of what's happening. They can go into the "findSeniors" and "findJuniors" functions and determine what qualifies someone as senior or junior (or maybe those could be additional parameters that are passed in as well).
Again, you might be tempted to put a comment on the "findSeniors" function saying it should be passed a sorted list of customers, but why not call its input parameter "customersSortedByAge"?
But, of course, one should not make the opposite mistake of thinking one should not use comments. There's no harm in putting a comment on "sortedByAge" saying "this is used to calculate discounts...", or specifying that the input list for findSeniors should be sorted. And, of course, if you are making an external-facing library or API, you cannot expect people to be able to figure out what your functions are doing by going into them, so in that case having good commenting is essential (and should actually be given as much focus in code reviews etc. as the code itself).
Also, where comments really shine is explaining the *why* of a given function. E.g. "we used [sort X] (instead of the more popular [sort Y] or [sort Z]) for this function because in practice the list of customers tends to [have property A] and profiling found [sort X] worked the best" or "we sort the list once at the beginning and then assume the list is sorted everywhere else" or "due to a bug in [library] we pass the input list in batches of 1000 instead of all in one go" (and bonus points if you can link to the issue number of a reported bug on the library's issue tracker - but... you might be using an old, unmaintained library where that's not possible).
That sounds great. But with Deep classes where there is a lot of complexity that may not be enough, so comments can be very useful for people who have to maintain the code. If you liked the video you'll probably love the book, it's really good. Your library might have it, or purchase it if you request it, if you don't want to buy it.
Ideally your code and your comments and your spec should be the same thing... although its hard to do this in most languages, although Lisp macros, and Smalltalk eDSL classes (maybe TCL upvalues, but TCL lost mwhahaha) can help you do this. (Lisp :is (just English) :with parenthesis :for grouping).
22:48 very rarely I write my own end to end exception system, but I will cover and handle i/o exceptions as I should.
Thank you, John, for this awesome talk! :)
woww this is my favorite video, thanks for sharing .....
Software design is make our software meet current needs while also meet feature needs holpfully
29:00 if you orde program a mistake extracting a sub string and you get the wrong chunk, how far will this chunk go before it results in a business logic error? Wouldn’t you have tests for that business logic?
"Working code is not enough. That can't be the only goal" - Unfortunately true for many companies and self-proclaimed "engineers".
Outstanding talk - really sound philosophies like more speed does not equal to greater haste, and strategic over tactical development every day of the week! Some really sound principles for good software development that should always be borne in mind. Technical debt which inevitably results from tactical, and "fast" development is always extremely expensive to fix, and thus almost certainly never is. Cheaper to wait for the next iteration of the application - i.e.: tear down the building and start again... not very smart. My thoughts on the matter... 🙂
Really interesting talk!
I think it would be great to add this kind of topics in university subjects, besides being able to access online to the subject he teaches for those who don't live there.
It would also be useful to include in the university libraries in other countries, books like the one he wrote and to apply some kind of discount in the purchase for academic purposes or according to the country or region.
Great ideas, thanks a lot. Added to my favorites.
Are the slides available?
49:17 Does he mean in other words: Making class 1 lvl of abstraction deep make class simple and deeper? Like class PostgreDbConnector extends DbConnector {...}
seems to me there are lots of connections to the topic of Technical Debt - esp at the Architecture/Design level. There exists quite some literature on that which is not mentioned (yet)...
@29:55 Don't use NULLs or, in this case, an exception -- Return an empty. So the correct thing to return would be and empty string ("") or character (''). That is effectively your sentinel return value. It very often requires no additional logic and it fits nicely within the principles in this talk.
Check out the talk “nothing is something” by sandi Metz. You may like it.
For sure @@SyphriX ! I love anything by Sandi Metz -- she's one of the top voices in OOP. I appreciate the recommendation.
Fantastic talk, thank you so much.
Even more valuable than code review is test review.
On the hiring question, I think the correlation of good hires being ones enjoyable to talk to comes a lot from a candidates confidence in themselves and their ability to do the role as presented, as well as the interviewer having a well-honed bullshit detector for anyone trying to fake their way through.
This has been very enlightening and reconfirming my beliefs. Thanks John for giving this presentation.
Don't see the forest for the trees. That's what it all boils down to here. Any ideas how to teach students that class?
i like the idea of getting some basic rules and agreements for software design . i think only this way progress can be made
I think agreement on basic rules for software design would be an indicator of the progress having already happened: an increased understanding of how to do software design would naturally bring about an agreement to follow/embody that understanding-but agreeing to write short methods does not make short methods better, nor does engaging in the behavior increase our understanding of why it makes sense.
(IOW I think you have the causality arrow pointing in a backwards direction)
I really wish we could only see the powerpoint...
When he said favourite deep interface and mentioned unix io I was scared that he'd talk about ioctl(). That's bottomless pit deep!
But without a break update, exceptions maybe the only way to update the library side when the return type needs to be modified somehow.
I think shallow methods can be good at times when they make it more clear what they’re doing.
Yes. They are also good if they replace a commonly occurring mistake with a plainly-named function that does it correctly. That's my counter-argument to his example of list vs pointer manipulation; the named methods for lists are clearer and less error-prone than pointer manipulation, even the one-liners.
90% software engineers are egoistic monsters who just like to consume stuff and dont know how to talk..they talk like aliens talking geeky stuff because they dont understand it well....but this professor is so well organised and speaks so well
He could switch around only some members of a given group/project. Would make for good observations.
Many database libraries in many languages returns an error when queries did not found a record instead of returning empty list, this makes coding hard since empty query result for me is a non error issue
You can have the public function do the deep thing with less than 20 lines of code, just separate the levels of abstraction into different functions. code in function must not cross levels of abstraction
30:40
AUDIENCE: So one question-- so when is it a good idea to actually throw exceptions? So the intuitive case is you want to eliminate exceptions as much as possible, then how do you decide, no, there's nothing that I can do? My changing semantics doesn't work. And I really have to throw an exception now.
JOHN OUTSERHOUT: Well, if you fundamentally can't carry out your contract with your caller and if you can't implement your interface, then you sort of have to throw an exception.
What's the other most important paper in Computer Science?
Examples for "defining errors out of existence": SQL NULL propagation in expressions, Lua's nil and Lua's reluctance to throw in general (unlike Python). Little things that separate men from boys and ladies from little tiny girls.
Great talk, bought the book (hope i'll read it soon ;)