# Summary Re-reading *Joel on Software* in 2023. This book is published in 2004, but many essays were written years before, so this book is effectively 20 years old. Realizing that this title is a total joke on 1800s book names. (Well, the *More on Joel on Software* reused the famous Leviathan book cover) > JOEL ON SOFTWARE: And on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those Who, Whether by Good Fortune or Ill Luck, Work with Them in Some Capacity # 1. Choosing a Language > It seems like .NET gives us a "choice" of languages precisely where we couldn't care less about it—in the syntax. Joel was onto something right. The dot-NET experiment with the whole *common Language Infrastructure* concluded that most people should be programming in C#. We were so obsessed with the choice of [[Programming Languages]]s. This theme continues in [[More Joel on Software]]. # 2. Back to Basics * Think about bytes. * Some basic tutorial on `O(N) vs O(N^2)` on repeated applications of `strcat`. * Why XML is not performant; though this turned out to be real in many ways. Topic on [[Performance]] changed over time. # 3. The Joel Test > [!important] > 1. Do you use source control? > 2. Can you make a build in one step? > 3. Do you make daily builds? > 4. Do you have a bug database? > 5. Do you fix bugs before writing new code? > 6. Do you have an up-to-date schedule? > 7. Do you have a spec? > 8. Do programmers have quiet working conditions? > 9. Do you use the best tools money can buy? > 10. Do you have testers? > 11. Do new candidates write code during their interview? > 12. Do you do hallway usability testing? ## Revisiting The Joel Test in 2023 (1; source control) is "common sense" now. In fact, some workflows (inspired by Heroku *requires* you to use git) (3; daily builds) we have more rigorous version of it via [[Continuous Integration]]. (2; one-step build) is a prerequisite for this. However, a different articulation of this - say, "can you create a reproducible environment in a single step" is still difficult difficult. See [[Software Development UX]]. (4; bug database), is a prerequisite for (5; fix bugs before new code) is perhaps the most controversial one as we seem to be happy with bug-ridden software. Perhaps this is Plaid-ism, but the general trend of the industry is, prioritize bugs according to their severity and only focus on bugs if they're worth it. > zero defects meant that at any given time, the highest priority is to eliminate bugs _before_ writing any new code. (6; up to date schedule) OKR and Agile-ism made this better, but software engineers are still quite nonchalant about lax time estimation and slipping schedules unless there's a deadline crunch. (7; spec) the design doc is widespread throughout the industry. The word *Spec* is out of fashion now. Perhaps too dry and non-agile? (8; quiet working conditions) 🪦 [[Open Office]]. This section talks extensively about "the flow" and "in the zone". I wonder how relevant this is in 2023. Staying in focus and eliminating distractions is more important than ever, but we're living in a different era. Software Engineers (especially more senior ones) are expected to perform more organizational duties. Moreover, this book was written *before* smartphones, the ultimate distraction devices and push notifications (Slack, etc). Even if an engineer is in a quiet working condition, it's very easy to distract themselves. (9; best tools money can buy) Selling to software engineers / organization is harder than ever ([[Selling to Developers]]). This is an economical logical fallacy, but en mass. "The Market Can Remain Irrational Longer Than You Can Remain Solvent"? The industry moved away from this and many developer tools are free. Either; * Open Source * Loss leader for other products (Kubernetes) * Many tools are "enterprise software" requiring a 6-figure contract. These have corporate bureaucratic limitations (Though, Joel should be *proud* of this as he's the one who advocated for not pricing things in the 4~5 figure range) SV doesn't skimp on hardware (either computer or workspace setup). (10; testers) I'm not sure we do. However, UI/UX programming isn't the major form of programming we do anymore. and there's a trend towards automated testing now. (Should write about this under [[Continuous Integration#Software Testers]]). Also, this may be another symptom of gen-swe-maximalism. (11; coding during the interview) this is an industry best practice now. In fact, there's a strong backlash against the "leetcode interview" now. (12; hallway usability testing) too specific to the UI/UX programming. Also, that discipline has matured exponentially since the book. # 4. Unicode I forgot about endian-ness for a long time. Does this matter for (most) programming anymore? Most encoding formats are high level (XML, YAML, etc) or handle endianness natively (protocol buffer). > There Ain't No Such Thing As Plain Text Joel did a good job of separating out encoding and code point representation. However, this got more complicated over time with [combining characters](https://en.wikipedia.org/wiki/Combining_character) and [precomposed characters](https://en.wikipedia.org/wiki/Precomposed_character). Remember that UTF-16 is an abomination - the endian-ness, BOM, and the fact that it can't even do what it was designed to do (representing all code points in 2 bytes) makes it the worst representation format. # 9. Software Delivery Schedule Some components are timeless. 1 and 2 pushes for simplicity (use spreadsheet!) 3 and 5 pushes for breaking things into smaller chunks. This aligns with the [[Agile]] philosophy too. 4 - Only the programmer who is going to write the code can schedule it This is bad in many ways [[Parkinson's Law]]; and a bit of programmer-maximalism. 6, 7, 8 pushes for continuous and accurate tracking of time. 9, 10, 11 pushes for over-estimation, tracking for the full software development time (debugging and integration). 11 pushes for the explicit buffer time. 12 - Never, ever let managers tell programmers to reduce an estimate. 13 - A schedule is like wood blocks. They should be interpreted together. 13 pushes for [[Agile]] / MVP philosophy where one needs to cut features. 12 is push to reduce estimate without scope reduction. However, the real world rhetoric isn't this simple; most cut for estimate comes with a scope cut that is less adequate to justify the estimate cut. > As it turns out, we had no choice, and we cut what we thought was "to the bone" to make the schedule. Everybody felt unhappy about the cuts. To assuage our feelings, we simply told ourselves that we weren't cutting the features, we were simply deferring them to Excel 6, since they were less important. > ... > ... We sat down to go through the list of "Excel 6" features that had been cut from the Excel 5 schedule. We were absolutely _shocked_ to see that the list of cut features was the shoddiest list of features you could imagine. Not _one_ of those features was worth doing I would've presented 13 first then 12. 12, if read alone, has a similar vibe as 4. The term MVP didn't exist back then but Joel would've loved this idea also. # 10. Daily Builds > [!quote] > Turbo [[Pascal]] was sort of shocking, since it basically did everything that IBM Pascal did, only it ran in about 33K of memory _including the text editor_ Turbo Pascal is indeed shocking and it represents a local maxima - see [[Software Development UX#Productivity Peaks]]. Read Eval Print - loop, Edit Compile Test - loop. "REPL" has a different ring then "REP-loop" (Which is how Joel writes it). REPL refers to a program providing this functionality, REP-loop sounds like a *process*. > Visual Basic does it by parsing and lexing each line as you type it, so that the final compile can be superquick. Visual C++ does it by providing incremental compiles, precompiled headers, and incremental linking. > And it needs to build every file from scratch, not relying on the compiler's possibly imperfect incremental rebuild capabilities. This one is interesting; The problem stated here is true. How many times did we need to "blow away the cache" to either fix or reproduce the issue? However, the performance hit of "fresh build" is too big that modern build systems (Nix, Bazel) tries to formalize the notion of cache to unnecessary cache clearing. (See [[💻 Programming/Cache Invalidation]]) Otherwise, this is a primordial form of modern [[Continuous Integration]] / DevOps culture - championing rapid iteration and codifying processes. # 11. Bug Fixing > Fixing bugs is only important when the value of having the bug fixed exceeds the cost of fixing it. This is a slight revisionism to [[#3. The Joel Test]]. # 12. Five Worlds He's attempting to split the world of software engineering into 5 worlds. It aged reasonably well. The framework makes sense, the world is very different. 1. Shrinkwrap. with its sub-variants 1. Open Source 2. Web 3. Consultingware 2. Internal 3. Embedded 4. Games 5. Throwaway > [!note] 💭 Thoughts > "Web" took over everything. Mobile is a new world that resembles the shrinkwrap and web. > > Joel didn't know what he was talking about for *Embedded* and *Games*, but included them for the completeness' sake. Game became more ... gamified, continuously delivered. Embedded still exists but its relative market size is smaller than ever. > > Large amount of software are faceless. Entire class of "developing backend services" nor "data engineering" aren't represented at all; These are mostly internal but they can be external too (developing APIs, for example). These software always existed (Operating Systems and Databases) but Joel thought this represented *even smaller* share than embedded programing to not include it at all. What would be the *worlds* of software development, in 2022? * Enterprise - "Consultingware" may fall into this too, but the equation * SaaS * Open Source * Mobile (I may include Game development here too, but I am as clueless about Games as Joel) * Backend * DevOps * Data * --- a cutline --- * Embedded - it still exists, but in an even smaller share. ## Memorable Quotes > [!quote] On Internal Software > To get a reasonable ROI, internal developers can't spend as much as someone would on shrinkwrap. > [!quote] internal development > Whenever you read one of those books about programming methodologies written by a full-time software development guru/consultant, chances are they are talking about internal, corporate software development. This is a strong case for [[nocode retool-like|retool]] - bringing the economy of scale to internal tools. > [!quote] On GUI testing > For example, nobody has really been able to get test-driven development to work for GUI development. This is before Selenium. And even with Selenium, it took us *forever* to get automated testing to be good. # 14. Architecture Astronauts > When you go too far up, abstraction-wise, you run out of oxygen. Sometimes, smart thinkers just don't know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don't actually mean anything at all. Ah, that's why this is *Architecture Astronauts*, because they abstract way too high up. > Another common thing Architecture Astronauts like to do is invent some new architecture and claim it solves something. Java, XML, Soap, XML-RPC, HailStorm, .NET, Jini, oh lord I can't keep up. And that's just in the last 12 months! This is a nice piece of history. Some of the complaints are about marketing buzzwords, which I would associate with a different problem. Example (I didn't even know what *Hailstorm* was): > People are not in control of the technology that surrounds them... HailStorm makes the technology in your life work together on your behalf and under your control. Oh, the distributed system world was pretty screwed up back then; I wonder what these people would think *now*. Joel may have aneurism if he needed to write about Kubernetes. > SOAP + WSDL may be the Hot New Thing, but it doesn't really let you do anything you couldn't do before using other technologies—if you had a reason to. All that Distributed Services Nirvana the Architecture Astronauts are blathering about was promised to us in the past, if we used DCOM, or JavaBeans, or OSF DCE, or CORBA. --- Overall, I'm disappointed by this section, as the term *Architecture Astronaut* was so ingrained to me. I wanted to hear more examples of over-architecting and ways to avoid it, but this chapter is mostly about complaining about buzzwords and the newest hot trends. # 18. Biculturalism Unix vs Windows programmers - programming for programmers vs non-programmers. "Silence is Golden" re: CLI - another consequence of the [[Decision Problem]]? > So you get these religious arguments. UNIX is better because you can debug into libraries. Windows is better because Aunt Marge gets some confirmation that her email was actually sent. In this example, the latter is strictly better (it's like how accessible designs like sidewalk ramps benefit abled people too). Joel discounts this as a *cultural value differences* but it is a pitfall of [[Relativism]]. > Microsoft Internet Explorer itself is nothing but a tiny 89KB program that wraps together dozens of very powerful components that are freely available to sophisticated Windows programmers and that are mostly designed to be flexible and powerful. Was IE this small? Or was the web browser "built in" to the operating system, thus IE itself was merely an webview? On anti-growth mentality. Joel quotes Eric Raymond: > Classic UNIX documentation is written to be telegraphic but complete.... ==The style assumes an active reader==, one who is able to deduce obvious unsaid consequences of what is said, and who has the self-confidence to trust those deductions. Read every word carefully, because you will seldom be told anything twice. (Raymond) > ... > the Windows culture understands that end users don't like reading, (Joel) This is the anti [[Growth Engineering]] mindset. Nowadays, everything is designed with a funnel of some kind. Perhaps the cultural difference outlined here has some to do with [[Tech History and Culture]]. Ie. People who entered the tech industry because they truly loved the computer versus the others. Section on Unix programmers hating Windows programmers, but not vice versa. One sided hate (and many manifestation of it) is quite interesting, something I should think more about. Unix cultural superiority, self-righteousness. Registry versus dotfiles - why was Windows registry so hated? Was this just UNIX-ism? Discussed with WBK and some of the clash was between API-versus-Files. # 24. Things you should never do, part one One of the most timeless section, arguing against the "Rewrite from scratch". > There's a subtle reason why programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: _They are probably wrong_. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming: **It's harder to read code than to write it.** > When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work. ... and section on "bad code", and how almost all of them can be fixed incrementally instead of doing a full rewrite. > It's important to remember that when you start from scratch, there is _absolutely no reason_ to believe that you will do a better job than you did the first time. and finally > The artists formerly known as Netscape finally did ship something called Mozilla, their groundbreaking ground-up rewrite. I remember reading before, "Perhaps Joel was wrong about this particular example as Mozilla became successful". I would argue otherwise; Mozilla / Firefox is an exception, not the norm. # 26. Leaky Abstraction Collection of my examples @ [[Leaky Abstraction]] > [!quote] Law of Leaky Abstraction #adage > This is what I call a _leaky abstraction_.. TCP attempts to provide a complete abstraction of an underlying unreliable network, but sometimes the network leaks through the abstraction and you feel the things that the abstraction can't quite protect you from. This is but one example of what I've dubbed the Law of Leaky Abstractions: > > **All nontrivial abstractions, to some degree, are leaky.** Joel's examples: * TCP/IP and networking * File System * Memory traversal - *row* and *column* traversal * SQL and query planner * ==Network File Systems (NFS / SMB)== - this one is so leaky even after 20 years that applications like Dropbox (which is leaky but in a different way) appeared. * C++ strings not allowing `"foo" + "bar"` as these are both `char*`s. Or C++ in general. * *And you can't drive as fast when it's raining, even though your car has windshield wipers and headlights and a roof and a heater...* * COM programming - *it would be nice if I could just teach them how to use the Visual Studio wizards and all the code-generation features, but if anything goes wrong, they will not have the vaguest idea what happened or how to debug it and recover from it.* * ASP.net having a different handling of `<a>` and `<button/>` - *The abstraction leaks, though. If the end user has JavaScript disabled* This has the *The Night Watch* vibe to it. > Today, to work on CityDesk, I need to know Visual Basic, COM, ATL, C++, InnoSetup, Internet Explorer internals, regular expressions, DOM, HTML, CSS, and XML—all high-level tools compared to the old K&R stuff, but ==I still have to know the K&R stuff, or I'm toast==. Still surprised that Jeff Atwood and Joel worked together. > Ten years ago, we might have imagined that new programming paradigms would have made programming easier by now. Indeed, the abstractions we've created over the years _do_ allow us to deal with new orders of complexity in software development that we didn't have to deal with ten or fifteen years ago ... when you need to hire a programmer to do mostly VB programming, it's not good enough to hire a VB programmer, because they will get completely stuck in tar every time the VB abstraction leaks. The Law of Leaky Abstractions is dragging us down. # 29. Rick Chapman is in search of stupidity > In every high-tech company I've known, there's a war going on between the geeks and the suits. > > If you ask _me_, and I'm biased, _no software company can succeed_ unless a programmer is at the helm. a bit of Programmer-maximalism > According to Rick Chapman, the answer is simpler: Microsoft was the only company on the list that never made a fatal, stupid mistake. # 35. In defense of NIH syndrome > I stopped by Andrew Kwatinetz's office. He was my manager at the time and taught me everything I know. "The Excel development team will never accept it," he said. "You know their motto? 'Find the dependencies—and eliminate them.' They'll never go for something with so many dependencies." > "Find the dependencies—and eliminate them." When you're working on a really, really good team with great programmers, everybody else's code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. > Two yuppies in a living room buying an e-commerce engine from company A and selling merchandise made by company B and warehoused and shipped by company C, with customer service from company D, isn't honestly adding much value. ... but this *is* valuable and a lot of people (and companies) make a living out of this. This is the crux of the modern platform / nocode development, too. > That e-commerce engine you bought? There's no way it's going to be as flexible as what Amazon does with _obidos_, which they wrote themselves. ... yes, but are you Amazon? If you're Amazon, then you can afford to solve this. > **If it's a core business function—do it yourself, no matter what.** > > If you're a pharmaceutical company, write software for drug research, but don't write your own accounting package. However, the definition of the "core business function" changes over time. As a web tech company, "running your own datacenter" was a "core business function" for the longest time, but not anymore. Contemporary example of this is *platform teams* and companies running their own `k8s` clusters. Perhaps the biggest change of this is the rise of fabless semiconductor companies. The famous [Real Men have fabs](https://www.wired.com/2008/10/real-men-have-f/) . 20 years later, AMD's spinoff of GlobalFoundary appears to be a genius move compared to Intel, where they're stuck with subpar foundaries.