Otter Thoughts
A Guide to the Software Industry
Menu

Total Compensation Package, Part 2: The Equipment October 30, 2017

Part 1: The Office

Good developers are rare and you can’t afford Golden Handcuffs. Good luck keeping them when the startup down the street offers a $2500 gaming rig, a $300 keyboard budget, and an adjustable memory foam chair for when their desk is not in standing position.

My favorite desk was a piece of plywood with steel IKEA legs. It was stained black with a semi-gloss top coat. The legs were adjustable. I was given a small rolling file cabinet to go under it. One of the co-founders built four of them for the new office. He wasn’t exactly Norm Abram but they turned out very nice. Your developers don’t need private offices with enormous oak desks but they do need enough space to work and a place to store things.

Generally, there should be enough space on both sides to have a notebook open or a couple stacks of papers without crowding the keyboard and mouse or overflowing into someone else’s space.  Six feet should be enough. Eight makes for a cleaner desk and more organization. Each dev will need a filing cabinet if your organization tends to accumulate paperwork, documentation, or written specifications from customers. Insufficient desk space makes it hard to work. You lose track of where things are and have to waste time flipping through your notes over and over. A small shop can get away with $100 home made IKEA desks. Companies that can afford it should just shell out the $900 for nice adjustable standing desks. (Experience only comes with time. Older devs get bad backs.)

Every developer has their own keyboard and mouse preferences. Some people have very strong preferences. These devices get used all day long; it pays to let each dev choose a keyboard and mouse they are comfortable with. Would you force a professional golfer to use clubs he didn’t like? Keyboards can get expensive. Don’t be surprised if tehy ask for a $200 keyboard and an $80 mouse. Just buy them. It’s only a couple hours of salary.

Monitors are less personal but still important to have choice in. I for one hate large monitors. I don’t want to have to turn my head to see the other side of the screen. (Nor do I need to with a capable window manager like Fluxbox.) 24″ 1440p is the max I’ll work on, 22″ 1080p is preferred. Other people want triple 4k 27″ curved screens. There is a limit to productivity increase. If the developer is adamant and you can’t easily find a replacement then you damn well better  make them happy. If the market isn’t so tight you can very reasonably draw the line at a pair of more affordable monitors.

Headphones are an extremely personal item. Developers will provide their own but you need to provide a safe place for those $1100 Fostex TH900 studio headphones to stay overnight. If your office is safe, no problem. If you share office space or if untrustworthy people may go by you need to provide a locking drawer or locker for each developer. People don’t take kindly to their expensive audiophile headphones growing legs.

Computers. More is better but you don’t have to break the bank. Never force someone to use Apple hardware if they don’t want to. Buy them a Mac if they want one and your dev environment is completely compatible. If they can’t work on Linux you don’t want them. If they won’t work on Linux then weigh how much time will be lost to making your stack work with their Mac vs finding someone else. Sometimes it’s none. Sometimes it’s months. Personally, I charge a $20,000 per year salary premium to occasionally have to use Windows or OSX. No amount can bribe me into doing phone dev again. Spec-wise for web work a mid-tier CPU and a generous amount of RAM will do. Specialized jobs will need specialized machines. Don’t skimp. Forcing your chef to cook with a dull knife just drives them away.

A good chair can be the difference between “I’m gonna need a drink after work” and “Fuck it, I quit”. I’ve had a Herman Miller Aeron chair. It was nice. I’ve also had a $300 Office Depot chair that was just as comfortable. It’s a bad sign when you see people bringing in pillows to sit on. Skinny guys especially need comfortable chairs. Not everyone realizes that chairs wear out after a few years. Throw out old ones; you’re not saving any money.

And for fuck’s sake, BUY SOME GODDAMN PENS.

No Comments on Total Compensation Package, Part 2: The Equipment
Categories: Uncategorized

Total Compensation Package, Part 1: The Office October 22, 2017

Software companies need to exercise caution when changing offices. The developer labor market is very competitive. It’s hard to find good developers and tricky to keep them. The old saying goes, “Managing developers is like herding cats.” Part of this comes from the incredible leverage developers hold over their employers. While some companies can afford to pay high market wages, small and medium size software companies are significantly bolstering their developer salaries with intangible benefits. Developers see any decrease in these benefits as a pay cut.

When a developer accepts a job offer, he is accepting a certain amount of money to perform a certain type of work in a certain environment. Changing any one of those things is tantamount to violating the employment agreement. If your typical developer costs $10,000/mo in salary, taxes, and benefits, and if it takes two months to onboard a new developer, losing one single dev costs $20,000 plus hiring costs plus lost productivity.  plus damaged morale. Are your changes worth that? Is that new slightly cheaper office worth it? You may not lose them immediately but they will definitely start quietly shopping around.

This is the first in a series of articles on the Total Compensation Package. First and foremost is the office.

Your office does not have to be hip or fancy. You don’t need local craft beer on tap or a ping pong table. You do desperately need three things: windows, personal space, and peace. A sufficiently large and decroative office can subsist without windows but it’s cheaper and easier to just get an office with a view. Doesn’t have to be of much, just more than a wall. Specifically, there needs to be something in the distance to look at. Across a street is a good minimum rule of thumb.

Next is personal space. You don’t need to give ever person their own office but you do need to give them a large enough desk and some space to move around. Cubicles are generally an insult and counteract windows. (Remember that scene in Office Space where Peter tears down a cubicle wall in order to see out the window?) If you can touch your neighbor without moving your chair, the desks are too close.  Don’t cram four dudes in some tiny closet of an office. The stereotype of programmers with poor hygiene exists for a reason; some people just don’t understand how to shower. Remember, floor space is cheaper than brainpower.

Tranquility is the final necessity. Programmers need quiet to do their job. For those on the outside looking in, software development is mentally equivalent to doing math homework. Could you do math homework for 8 hours straight in a loud “open workspace”? Didn’t think so. A room full of people is fine if they are all courteous and quiet, work related or not. This goes not only for the developers but for managers, interns, passersby and anyone who can be heard in the room. Be sure to crack down on earbud users whose tinny music blares out of the sides. Don’t play music for the whole office. You may like it but not everyone will and sometimes you just need peace and quiet to concentrate.

Growing companies eventually have to change offices. Renting a bigger place can get expensive. In my experience there is a trend to go for lower quality in order to keep costs down. It’s more than that though. Going from a modern second floor coworking space with wall to wall windows overlooking some beautiful scenery to a dingy basement office with a bad paint job and a no-dogs rule isn’t being frugal, it’s cutting your employees’ pay. You might be able to get away with that for minimum wage office drones. For developers it’s a substantial loss. Kiss your morale goodbye.

No Comments on Total Compensation Package, Part 1: The Office
Categories: Uncategorized

The Tyranny of Dependencies October 15, 2017

I once worked on a Node.js project with over 800 dependencies. Months of labor was wasted keeping up with API changes. Deployments were unexpectedly delayed by a week or more. A different project with hundreds of dependencies had its Node version pinned to an exact bugfix revision. We couldn’t upgrade even though there were known security holes and a serious SSL bug in that version. More dependencies, more problems.

Some dependencies are unavoidable. Never roll your own crypto code. Don’t write your own database or compression libraries. Where to draw the line can be hard to tell. A GIF loader? A JSON parser? It depends on your goals and what the options are. Does the potential dependency have a huge tree of its own dependencies? Is it made by some flaky group with a history of backwards-incompatible changes? Is it a glitchy piece of shit, at least moreso than your version would be? Does it impose requirements on your build chain? Does it have an acceptable license? Do you have the time and skill to implement your own? Would a home made version have sufficient features?

Will you end up having to write a custom implementation in the future anyway?

Don’t assume 3rd party libraries are more stable or higher quality. Just take a look at the monstrosity that is KDE these days. About a year ago I ran into a peculiar problem when updating my Gentoo desktop. I keep certain bloated software packages banned from my system through the package manager. One of them is Ruby. When trying to update a the kdewebkit module, portage threw an error because Ruby and a hundred or so of it’s dependencies got pulled in to the dependency graph. Some pig-headed developer had decided to use a Ruby-based automated testing framework for C++ based kdewebkit. Never mind the questionable utility, the automated tests weren’t even run during install. Effectively, the “simple” testing framework required a hundred packages, several gigabytes of space, and who knows how many lines of source code, all of which had absolutely no use on my system. Who’s going to maintain that? What happens when any of that breaks? What about when the Ruby community decides the kitchen sink is just not enough, as is their habit? (I masked that version of kdewebkit for the time being and emerged on. Some time later common sense prevailed and the Ruby dependency was dropped. Hail Gentoo.)

In 2016 a dev named Azer Koçulu inadvertently broke the internet when he removed 250 of his Node.js modukes from the npm repository after some jerkwad app startup’s lawyers started hassling him over one innocently named “kik”. One of these modules was “left-pad”, which add spaces to the left of some text. It’s a trivial task consuming only a eleven lines. No one should ever add an external dependency for that in Node. Turns out quite a few major projects were dependent on it. Companies everywhere large and small were left in the lurch, unable to deploy until they fixed it. The problem was so bad that the CEO of npm reinstated the left-pad module without it’s author’s permission.

You may notice that some of my projects brag about having few to no dependencies. I don’t have time to waste on keeping up with changes, reading bad documentation, or rooting out bugs in someone else’s code.  It’s faster to write easy things myself from scratch. Changes only happen when and how I make them happen. Users of my libraries only have to worry about me making a change, not dozens or hundreds of people in the entire dependency tree.

People today have a tendency to be shortsighted and impatient. Code written in that manner becomes unmanageable very quickly. It often only takes a few months before the continuing costs outweigh the perceived upfront benefits. Between myopic managers and deficient developers commercial projects tend to accumulate excessive numbers of dependencies. Fight against this whenever possible. The best option is to just not ask; roll your own code for simple things and pretend that left-pad library never existed. Remember, you will be the one under the gun battling a Babel version conflict on Friday night trying to deploy a patch for that critical production bug related to an unannounced API change in a dependency of one of your dependencies.

 

No Comments on The Tyranny of Dependencies
Categories: Uncategorized

Learn to work on your own hardware October 8, 2017

Today’s post is late in part because my desktop overheated (again) and refused to boot. The last time it overheated the USB ports refused to work until the next morning. This time it wouldn’t recognize the hard drives. This points to a failing chipset (part of the motherboard). After 8 years it’s time to upgrade.

For most people this would be a time consuming and expensive situation. They would have to take their computer to a repair shop, pay for diagnostics, wait a few days for the results, then get ripped off by the repair shop assuming they can even find the problem. God help you if it’s an Apple product. You have to drive to an Apple store, which may be quite a drive if you don’t live in a big city, ship the entire computer off to Timbuktu, hope and pray they don’t erase the hard drive (they will), then pay $1200 to fix a broken fan because apparently the concept of interchangeable parts is lost on Apple.

For me, this was mildly annoying. I opened the case with two thumb screws. Everything is easily accessible in custom built computers; there are no proprietary plastic shrouds or weird sheet metal puzzle assemblies. Since it gave an error on the hard drives, I tried plugging them into different SATA ports. Not being penny-pinched proprietary OEM junk, my motherboard had 6 SATA ports. Two of them happened to work. The machine booted up but froze a few minutes later. This confirmed that the hard drives were fine and the motherboard was indeed dying.

If the situation was urgent I could drive down to Fry’s, Best Buy, Microcenter, or the local computer parts store to get fully compatible replacement parts. Waiting a few days to get parts from Newegg.com provides better selection and lower prices. Best Buy would still have been cheaper than a repair shop. Pre-built computers are often designed in ways incompatible with standard computer parts. Some of this is to save money, some of it is for style reasons, and some of it is to prevent repairs so you have to buy a whole new machine. Not on a custom machine; everything is standard, everything is replaceable. For me this meant I only had to buy a new processor, motherboard, and ram to repair and upgrade the system to 2017. Everything else could be reused.

Installing the new parts will take about half an hour. Any problems can be fixed immediately without making a trip to the repair shop or waiting for them to open tomorrow morning. If you’re a business it’s worth it to build custom computers from the perspective of reduced downtime alone. Shipping a computer off for repairs (looking at you, Apple fanatics) is unacceptable when each hour of downtime costs you $75+ in labor, not to mention projects falling behind schedule.

Over the last 8 years I had to replace two power supplies, one video card, and one hard drive. All of them lived a reasonable lifetime for what they were. I also added more ram because of how bloated the web has become. Combined, it probably cost around $350. The computer itself only cost $700 to begin with. How much have you spent over the last 8 years on computers? This machine had equivalent power to a $1200 PC or a $1600 Mac when it was built. Probably a bit more, since it had a decent video card as opposed to the junk they put in OEM machines.

Some people get caught up on “warranties”. First off, Mr. Baby Boomer, no warranty is worth it anymore. Sure, they’ll pay for the parts and labor but you’re without a computer for a month while they ship it off to some central repair center. Second, many things are not covered by computer warranties and the official repair centers make every effort to classify your problem as one of them. Third, components you buy to build a computer are each warrantied themselves. You only have to send in the one bad part too, not the entire machine and all your data. Fourth, computer parts either work or they don’t. If it runs on the second week it’s going to last as long as it was designed to, which is usually just past the warranty expiration. Fifth, and most important, when you build a computer you can choose all high quality components. When you buy a computer it’s going to be full of cheap, low-quality, low-power, overstocked garbage from last generation. Notice how I got 8 years out of my machine? Try that with an HP.

With age the desire to build top end gaming rigs fades. A moderately fast CPU with a bunch of ram will do everything I need for another 8 years. Hopefully Ryzen chips last that long. Next time you are at the store, even in the future, see how much a 3.2Ghz 6-core desktop with 32GB of DDR4-2666 costs. I only paid $620 and a hour of my time.

No Comments on Learn to work on your own hardware
Categories: Uncategorized

How I Look For a Job October 1, 2017

YMMV

I hate looking for a new job. There is a certain intangible sludge associated with it, like standing on the edge of a rotting swamp you need to cross. How long can a man survive without food again? Perhaps it just brings up too many bad memories.

Go to craigslist, open the software jobs section. Don’t search or filter, even in a large city. Scroll down and middle-click any job that looks like it might vaguely be relevant. For example, if I’m looking for a Node.js job, both “Experienced Node/PHP/Ruby/HTML programmer wanted for contract work” and “Senior Developer” count. Don’t look at the new tabs yet, just scroll through the entire list and open all the jobs that might maybe possibly be of interest.

After you reach the end of the job listings, all of them, close that tab.  Start from one end of the tab bar and handle each one in turn without skipping. Be decisive. Don’t let yourself make excuses or procrastinate. If a post fails, close it immediately and move to the next one. If a post passes, respond immediately. Don’t “make a list” for later. Don’t “bookmark the good ones”. Just contact them. Write down which ones passed after responding, if you want to keep track. I usually don’t.

I’ve worked at software companies of every shape, size, and kind. The best kind, IMHO, are small startups with external funding. Profitable startups are more relaxed but usually can’t afford a high salary. Culture at startups past the A Round tends to quickly degrade into corporate Employee Handbook hell.  Not every small startup is a pleasant work environment either. Beware fad-chasing, buzzword spewing nut jobs. You will be rebuilding the entire stack every few months because the newest hot shit is newer and hotter than the new hot shit from a few months ago. Never mind it’s a pre-alpha with more bugs reports open than closed written in a language that hasn’t reached version 1.0 yet. The filtering rules are designed to eliminate 80% of the crap. Filter the last 20% by talking to them.

Generally speaking, I’m looking for a post written by one real human being to another in normal, well-formed English. Close the tab if you see any of the following:

  • “The candidate shall possess the following attributes…”
  • “You are a self starter. You have many projects under your belt. You ride your bike to work and eat kale.”
  • “We are looking for a Rock Star Developer.”
  • “Master’s Degree in Computer Science required, Ph.D. preferred.”
  • “Our client is a Fortune 5,000 company that we won’t tell you the name of.”
  • “Our interview is a 12 step process that should only take 6 weeks and cost you 3 personal days, $50 in gas money, and half your sanity.”
  • “Salary starts at $40k for well-qualified candidates.”
  • Excessive buzzword usage.
  • Any mention of an Employee Handbook.
  • Legal disclaimers about anything at all.

The best posts are short and sweet. “We’re looking for an experienced Node.js developer to join our restaurant analytics and consulting company, HouseOfPies.com. Experience with PostgreSQL, Ubuntu servers, Nginx, and Mongo is a plus. Call Marty at 555-123-4567 or email clyde@houseofpies.com if interested.” Close anything that doesn’t have direct contact information. Bonus points for a contact name. Application forms are not only an insult but a sign of something broken inside the company socially. Don’t waste your time with a company that can’t even be bothered to talk to you directly.

If there’s a phone number I call. “Hi, my name is C. Otter, I saw your post on craigslist about a Node job?” Then just let them talk. Confidence goes a long way. If there is an email address instead, I send a short and simple email. They all follow the same pattern so I don’t stress over what to say.

Hello Doug,

My name is C. Otter. Your Node developer position looks interesting. I like your idea of tracking pie consumption by filling type; we did a similar thing with icing flavor at a cake consulting firm I used to work at. My resume is attached. If you want to chat, feel free to call me any time at 555-890-4321. Otherwise, best wishes and good luck on your search.

 

Best,
C. Otter
555-890-4321
c_otter@theocean.kelp

www.craigslist.org/ocean/43jf83j29nf

Introduce yourself first. State your interest in the specific job they advertised; managers are busy and receive many emails.  In one or two sentences, point out that you appreciate some aspect of their business and tie it gracefully to a piece of your experience. Include a link to the original post for their convenience. Oh, and remember to actually attach your resume in the first email.

Learn proper spelling and grammar. Be polite and positive. Don’t use coarse words or slang of any kind. Don’t say “like” even once, even in context. Don’t bring up money. Provide your phone number, email address, and resume. Don’t brag about how you can help them or their customers. Don’t flatter them either. Don’t waste their time. “Classy” is the goal here.

After I finish filtering and responding to all the tabs, I’m done for the day. Every day I go back and see if there are any new posts. Sometimes it can take a couple weeks, or longer, for find a good job. But who cares? I have plenty of time.

 

No Comments on How I Look For a Job
Categories: Uncategorized