Developing Software

I am a software engineer at a mega-corp and the other day someone asked if I care about what the CEO thinks. No, of course I don't care what the CEO thinks - why would I?. I know for a fact that the CEO doesn't care about what I think. And rightly so - if the CEO cared about what tens of thousands of employees thought they would never get anything done. I have absolutely zero opportunity to affect what the CEO thinks, and they have zero practical opportunity to affect what I think. Our roles in the company have no overlap at all. If I was also running a billion dollar company I might care what they thought because I might be able to learn something from them. But I don't - I write software. I have little responsibility apart from my own moral standards. They have a lot of real responsibility (a CEO can go to jail if they mess up too bad). Hence I get paid much less than them. Perhaps they are more skilled at their job than I am at mine, perhaps not. Regardless, being higher up on the corporate ladder does not make them a better person than me (it also doesn't make them a worse person). Therefore I am not obligated to care what they think. The corporate structure is an emotionless construct that doesn't require its participants to care one way or the other - that is not its function. It's function is to produce action.

All this got me thinking: what is it exactly that I know about the act of software development? Can I claim to be good at it? After 20 years doing it, what can I say for sure I know about it? For me the answer is a little weak but I can try to give some thoughts. I would start by breaking it up into two domains: 1) what I know about writing programs and 2) what I know about the process of software development beyond a single developer and task. The first domain is the subject for a later time, so I will address the second one here.

The Process of Software Development

I give a lot of thought to the process of software development. I am immersed in it 8 hours a day and it takes up much more of my time than I have for myself (excluding sleep, which is a biological necessity and should be a basic human right). So it's natural that I think a lot about it.

It is difficult

Most software development is about changing existing code. Changing existing code is about rewriting parts of the application from scratch. Rewriting software from scratch is difficult, time consuming and it's very difficult to end up with something at the end that has better functionality than what you started with. The larger the chunk of software that you are rewriting the more this applies. The smaller the chunk of software the less this applies. Rewriting a single function is not that difficult, still takes longer than you might have expected and there is a high chance that the end result will be objectively better than the old version (assuming you know what you are doing). Rewriting an entire application is extremely difficult, takes much more time than you would expect and there is a high chance that it will take an exceptional amount of time and effort to end up with something that is objectively better. Since it's difficult you need motivation.

Motivation

If the people writing the code are not invested, the project has a high chance of failure. Adding project managers that are not invested spells doom. Adding more engineers that are not invested spells doom. Anyone added to the project at any level who is not invested will simply add more work to be done, decreasing the chance of success. Investment could be financial or emotional, but emotional will increase the chance of success. Motivation can come from investment or fear. Motivation based on fear is like driving a car in reverse without holding the steering wheel: it can go sideways easily, and if it does the going sideways is self reinforcing. Is it a surprise FAANG companies try so hard to get their employees emotionally invested? You do not get productivity without motivation.

Productivity

A single engineer with sufficient motivation and no distractions can produce astounding results. This is not like traditional engineering and construction where one person can not build a bridge without help. In software development one person can build a rocket and land on the moon, while a team of skilled engineers can't construct a bicycle. There are some good analogies between physical engineering and software engineering, but all analogies fall down at some point and the two activities have some big differences. There are some physical limits to software development but they matter so much less. Windows XP had about 40 million lines of code. A line of code contains say 4 words and fast typers can produce 40 words per minute. At 8 hours a day every day of the year you could type 40 million lines of code in around 2 years. Even if we double that figure to 4 years to account for weekends and holidays I think it's clear that it's not possible for a single developer to write Windows XP in that time. Typing speed is not a limiting factor in software development. Knowing a programming language well enough to type 40 words per minute that compile will also not get you there. You need to know what to write.

Modeling reality

Software tends to be a model of reality. However all software models are (by the laws of physics) simpler than reality. This means all software models are incomplete models of the domain they are modeling. Software development seems to be mostly about deciding which incomplete model you will choose, and as you find out more about reality deciding which parts of the model to extend around your new understanding. Product managers that have a poor understanding of the reality being modeled are a liability. Trying to model too much of reality too soon will spell doom. Modeling the less important parts of reality too early will spell doom. You must have domain expertise before you can decide how the software should work. If you want your engineers to be self driven you must educate them in the domain they are modeling. Either that or have domain expert product managers with some technical affinity. People are much more than what they do, and the ideal situation is that knowledge and skills overlap somewhat at the boundaries between job roles. If there is a gap in knowledge and skills between two layers many problems will arise.