Categories
Uncategorized

Textmate 2 and why you shouldn't rewrite your code

If there is one lesson to learn from this article it is that old code isn’t bad. Its been tested, lots of bugs have been found and fixed, and it is a lot harder to read code than it is to write it.

If you take a look at the long list of companies that have tried to rewrite their code base, almost all of them ended in failure. Going all the way back to the early 2000’s, Netscape decided to rewrite the entire 4.0 codebase for 6.0. It took them 3 years and in that time, Microsoft swooped in with Internet explorer and took away their marketshare. Netscape ended up going out of business.

Microsoft also made this mistake with Word which they planned to replace with a secret project called “Pyramid.” After years of working on a replacement, they realized it was a total failure and scrapped the project. Luckily they were still working on Word, so they just released that instead.

More recently, Macromates an indie company who had a very successful text editor called Textmate decided to rewrite the code base for Textmate 2. It took them 6 years to get a beta release out the door which is an eternity in today’s time and they lost a lot of marketshare. When they did release a beta, it was too late and 6 months later they folded the project and pushed it on to Github as an open source project..

Quick customer-driven releases are better than big upfront plans or rewrites for several reasons. First, the market changes quickly and your product needs to be able to pivot to those changes. Second, its better to string your customers along meeting their incremental needs than abandoning them for a period of time to then try to meet all of their needs at once. Lastly, big projects always take longer than expected, so you’re likely to cause disappointment and pain for your customers on the bigger projects as you delay and try to explain away why they should continue to wait.

Code rewriting is a classic example of the Second-system effectwhich is the tendency of small elegant and successful systems to be replaced by feature-laden monstrosities.

People who have designed something only once before, try to do all the things they “did not get to do last time,” loading the project up with all the things they put off while making version one, even if most of them should be put off in version two as well.

This affects not only programmers, but anyone involved in design. With all of this negativity toward rewrites, is there ever a time that it is acceptable? Author Neil Guntonconcludes that there are 3 factors that should determine whether or not you need to do a total rewrite:

  1. Amount of accumulated wisdom
  2. How incompatible the new version is with the old version
  3. How many people used the old version and will be affected by the changes

With these in mind, you can quickly determine the cost of doing a total rewrite and hopefully decide against it.

Look at it as a living organism that can perhaps be healed, and can evolve. You can refactor, you can rewrite portions of the internals to work better, many things can be accomplished without abandoning all the experience and error correction that went into that codebase. When you rewrite you are abandoning history and condemning yourself to relive it.

Well said Neil.

By joshkerr

Josh is an 8x startup founder and angel investor.

26 replies on “Textmate 2 and why you shouldn't rewrite your code”

OS X wasn’t really a re-write, though. It’s more of a merge of classic Mac technology into the NeXTStep unix-based OS, both of which were honed over the course of years. Naturally, there was a lot of hard engineering that went into the combination, and it took years to finally stabilize.

Fortunately for Apple, desktop OS software at the time was at a slow period of evolution (mostly due to Microsoftian incompetence), and they could afford to take their time making it good.

They couldn’t do the same thing with iOS now, because the competitive landscape is completely different.

The trouble is, to make this indictment effective, you really need to define “rewrite”. To me this means, replace some codebase A implemented in language or runtime environment B with language C and runtime D. The problem is, of course, that this equation often involves a change in not just runtime or language, but OS, platform, or product. I agree that attempting to change all of these is a fool’s errand, but the road to failure is paved with people who got one of those choices wrong and didn’t correct in time. That’s the key.

Correlation is not causation. While I understand the role of rewrites in these failures, to claim that the rewrite was _principal_ in the failures is over-stating the effect in many cases, or ignoring sufficient detail. The anti-rewrite narrative is compelling, but I don’t think the data support that argument nearly as much as its adherents claim.

There were many other things that caused these companies or products to fail, not the least of which is the methodology of the rewrite. Rewrites which are done incrementally, with planned milestones and release dates, while not defined as “rewrites” by some, can and do succeed. To rewrite holistically is probably a mistake and perhaps I am defining the term too broadly, but I think you can crawl out of an early architectural mistake one step at a time, but you’ve obviously got to have a commitment toward reaching the singularity at the end, else you’ll wind up with a hodge podge of nonsense if you stop mid-stream.

To not attempt to correct course when early architectural mistakes are costing you time and money is just as large a mistake. I think the only difference is that you haven’t seen those documented in large numbers, but I’ve seen quite a few of them in my work in startups.

That is an incredibly false statement, thinly wrapped in current news.

I’ve had to deal with horrible bloated codebases before and one thing in our mind was always: is it good enough to edit, or scrap and rewrite? Sometimes it was to refactor and update as we go along, other times it was equally to scrap and start over again. We had the benefit of using a rather easy to write language like Ruby. But it all depends on the team; their skill level and the size.

When rewriting from scratch lots of questions gets asked; Do we really need this feature? Etc. Sometimes it’s absolutely for the best.

I have no idea where such a huge statement comes from because of ONE project by a small team? Unless you have background history I don’t?

Just because you say so, without any actual proper argument, it doesn’t make it true.

Couldn’t disagree more. Sometimes it’s quicker. When we’ve dealt with code from a poor software company the best thing we could have ever done for the future of the project, and our work was to rewrite.

This is a textbook example of why you need to write clean code all the time. Its called the broken window effect. If dirty code had not be written in the first version, this would never have happened, and a second version would not be necessary. This is not the first company this has happened to (or even the 10th), and probably not the last. At least it didn’t take 10+ years as some rewrites have.

http://www.cleancoders.com/

Seriously?

But they didn’t write clean code. Sure, if everybody wrote clean and secure code, then the digital world would be a safe place. No patches to install, no downtime, no security breaches.

Not everybody has the luxury of starting from scratch, while at the same time having 10 years of programming experience and having read 50 books about code design etc.

I have to agree with LonelyProgrammer on this one — especially in the start-up world, the new product is raced to production by a tiny team of developers, most of whom don’t have 20+ years of professional development experience to guide them, and all on a shoestring budget. Also, even once a start-up is stable, there’s little incentive to go back and improve the code — most such companies won’t make it to the three-year mark, and the few that do end up with a product utterly unlike what they first envisioned anyway. So while “writing clean code all the time” is a great idea, the reality is that all too often, it just doesn’t happen, and any recommendation to follow best practices in this regard will either be overruled, or else abandoned as artificial deadlines loom.

I’d have to agree with David Watson. “You shouldn’t rewrite” is a very generic and vague statement, and thus not particularly useful.

How big is the project that is being rewritten? Is it being rewritten in the same language? How experienced are the developers rewriting it? How experienced is/are the project manager(s)? Will the legacy product be maintained while doing the rewrite? The list of questions goes on.

Yes, a complete code rewrite will, in most cases, be a daunting task with a high failure rate, and one should very carefully evaluate whether they want to embark on such an adventure.
Still, sometimes a complete code rewrite is not only feasible, but necessary. I’ll share an example of a successful rewrite. We had been working on a web-based application that was originally written in ASP (the project started in 2000). I don’t know the lines of code, but it is a fairly substantial project that slowly evolved over the course of about 8 years. The natural progression would have been to switch to ASP.NET, but we chose to completely rewrite the application in Java. The major motivations where that we didn’t want to be tied to IIS anymore and wanted something cross-platform. We tried a few PHP frameworks but ultimately decided on Java. A partial, or incremental rewrite, was obviously not possible since we switched languages. To be fair, the rewrite is not yet complete, but we’re about 65% through and it’s going well for the most part.

Of course it’s a major undertaking, and there have been hick-ups along the way, and we’re definitely not on schedule (if there ever was one, and how do you create a schedule for something like this?). But, our developer is excited, it runs on 3 platforms without major modifications, and we’ve gotten good feedback from our customers.

We have another C++ – based project which is also in need of a rewrite, and here we’re taking the incremental approach. With every release we try to refactor the parts of the code which need it most, and when feasible even and entire feature.

I’m definitely not advocating complete rewrites, on the contrary. It sounds like the holy grail, but most people severely underestimate the risks, frustrations and time it will take to do the rewrite.

It also seems as if the motivation to do a rewrite is mostly based on assumptions, like:

* all new code written will be perfect (easy to maintain, testable, …)
* all legacy code is evil, outdated, error-prone
* the re-write will only take a fraction of the time, because we already know what we want
* the legacy code will maintain itself while we work on the new code
* there are sufficient developers to do the rewrite, and they are all familiar with the legacy code
* … and the list goes on …

Yes, a complete rewrite should be the last resort and planned very carefully, being aware of a high failure rate. Incremental rewrites should be preferred, if possible.

But there are cases where rewrites are not only possible but ultimately necessary and successful. There just aren’t that many probably :-).

Just my opinion.

I completely and totally disagree. But I’m coming from a different angle: I use a statically typed programming language with a very rich type system! (Haskell). My colleagues and I often completely rewrite large portions of our codebase and every time we do it it gets leaner and faster. Just recently I’ve cut down hundreds of lines of code and it just worked the first time! Bottom line: rewrite your code! But put in place a framework that enforces correctness. Bottom bottom line: use functional programming and learn to love type-systems!

i also think the statement “never rewrite your code” can’t be generalized like this.

We have written a hosting software and developed it further over the years. At the end it became very complex. Dependencies everywhere. We had big ideas, but to move this code forward was nearly impossible. A clear cut and beginning from scratch was the solution.

I couldn’t disagree with the author more. The hypothesis is so patently contradictory to engineering principles on the face of it. Software code is not some piece of art that only needs to be touched up at times. I am sure some of you have worked with these type of engineers that think in this fashion. They continually add layers over existing code, because they believe what they have written before is absolutely infallible.

Every project has historical lessons to learn from and those lessons aren’t inseparable from the code in which they were first implemented.

Unfortunately, rewrites are often required in corporate environments. When the OS, language, platform that you’ve been using for years is discontinued it’s very risky to stay on it. Sometimes rewrites become necessary. Doesn’t make them any less painful, but having been through it, I understand the conditions where you simply have to rewrite!

I think a lot of the this is the worst advice ever comments are missing the bigger picture. Successful projects tend to die when it takes years to release the new version. Textmate was the text editor for ruby/rails developers on mac. At least with the developers I see on a regular basis the last year almost all the textmaters have switched to sublime because it had a lot of the “incremental additions” people were looking for way back in 2009 see http://blog.macromates.com/2009/working-on-it/
If the time spent rebuilding already working features had been used to add new features and refactor the existing code base to reflect the lessons learned less of their user base would have migrated to other editors.

This site uses Akismet to reduce spam. Learn how your comment data is processed.