Hari's Corner

Humour, comics, tech, law, software, reviews, essays, articles and HOWTOs intermingled with random philosophy now and then

The Open Source Paradox - Is it a closed loop?

Filed under: Software and Technology by Hari
Posted on Tue, Dec 16, 2008 at 20:56 IST (last updated: Thu, Dec 18, 2008 @ 09:01 IST)

It's been a while since I've written a meaty article of this nature, but the thought has been in my mind for a while - ignoring all the legal and technical aspects of licensing and end-user benefits, is Open Source as it exists today really about improving software through quality feedback and user-contributed code? I know that the terms "Free Software" and "Open Source" have a wealth of history behind them. I also don't want to discuss the political philosophies of these movements. What I do want to discuss is whether the supposed software development model of the Open Source community is truly open to the cycle of development from the user community at large.

I've seen quite a few big FOSS projects that have simply stagnated over the years. I really haven't seen many major innovations. Yes - version numbers keep moving up and we do see a steady stream of minor updates, but as far as features are concerned, many FOSS projects seem to hit a plateau far too early in their development cycle. When a project gets a steady base of users after the initial period of development, it almost seems as though the programmers are committed only to fixed ideas and tend to become conservative and extremely cautious about innovating. They start worrying about upsetting their existing users if they make major changes.

Another side of the story is that the end users themselves tend to share this conservative thinking. I've also noticed that many of the big FOSS projects seem to have only a small base of users who are really interested in participating in the development process (by way of submitting bug reports, feature requests and testing out beta versions etc.)

This has more to do with the social phenomenon of group-think rather than actual technical issues. Once a FOSS project gains a "fan club" of core users, this core group seems to dominate the idea feedback loop which filters its way through to the project team. And in any case, most end users are either too lazy or too timid to provide feedback, let alone contribute code. Also this core fan club tends to silence any criticism or negative reactions pretty quickly in mailing lists and online forums. Whether the actual developers themselves form this core club or a small group of initial users take it upon themselves to defend their faith is open to question, but it's evident that most FOSS projects of reasonable size have these cliques who dominate the thinking pattern and lead to stagnation. Stability is one thing - but is stability the only goal of FOSS?

It's one thing to have a community of users - it's another to have active participation in the actual process of improving the software from one version to the next. It's also another thing to have the capability to fork a project from its roots and take it to another level - for most people this is simply unfeasible and unrealistic. It might be argued that the ultimate goal of FOSS is not innovation but simply freedom. But I would say that the freedom of the end-user is simply a myth if that end user cannot participate in a meaningful, practical way in the process of improvement.

In contrast, I've seen the smaller FOSS projects really explore obscure areas of programming. It would be interesting to study many of the smaller projects and analyze their levels of code quality. The larger projects tend to survive by their community power and popularity, but small projects depend solely on the sustained enthusiasm of its developers. And as small projects are often just a single programmer's leisure-time hobby, such projects tend to be limited in terms of scope and feature-richness. But within these small parameters, individual developers have the maximum freedom to innovate, be receptive to ideas and allow themselves to code for the joy of coding, without worrying too much about hurting their egos. Working on small ideas one at a time also seems to be a great way to produce results.

I've found the same to be true myself. Once I start over-generalizing ideas and look too far ahead, I start hitting road-blocks and meaningless abstractions. But when I focus purely on what I want to achieve at the moment and also think in terms of ideas, I write code faster, I focus on ideas and I get a better idea of what the result will be.

Maybe there is an optimal size for an FOSS project in terms of code-base, number of programmers and number of core users. Below that level, a project would simply not sustain itself in the long run. Far too many hobby projects are simply abandoned before completion. Above that level, the project would be probably become a victim of its own popularity and tend to stagnate for the reasons I've stated above. It is an interesting idea and probably worth exploring by those academically inclined.

4 comment(s)

  1. Tricky one. I think a lot of it depends on the nature of the project right from day 1.

    Firefox was created right from the start to be a lightweight but flexible browser which was easy to add features to via extensions. So it's less afflicted by the mindset of "Firefox works THIS way, it must not be changed!" - although it DOES still suffer from it, as some of the feedback for the Awesomebar shows..

    Python, too, was always based around a philosophy of not being tied to "This is how it's always been done" and so the latest release unapologetically broke backwards compatibility.

    So it really boils down to what the "core features" of your project are: If "flexibility" or "new ways of doing things" aren't in there, then you'll find it hard to be flexible or do things in new ways, because your user base will perceive such things as outside the "one true way" that that project should work..

    Comment by Dominic (visitor) on Tue, Dec 16, 2008 @ 21:19 IST #
  2. Dominic, I guess it's also true the other way round. Small projects tend to remain small because they focus on ideas that might not be popular with the majority of computer users.

    I think ultimately only one small group CAN control a project of any size effectively. How that group reacts to feedback and criticism will determine how a project will shape up. The question is: how many developers actually listen to their users legitimate concerns once their project becomes well established?

    One example of developers who listen: b2evolution! It's a great example of an FOSS software that has actually evolved from one version to the next thanks to the great attitude of its core team.

    You know, a few developers actually read my ideas and even commented on this blog on my review of b2evolution. One or two ideas even made it to version 2.x.

    Comment by Hari (blog owner) on Tue, Dec 16, 2008 @ 21:29 IST #
  3. I think some of it has to do with the team size - if you have, say, 5 coders you'll get a lot of "wouldn't it be cool if..." and more opportunities to test the idea. If you have 100 coders, you need to motivate 20x harder to get that strange, off-the-wall idea to be tried out.

    Add in the democracy that larger projects need and you end up with lots of bug-fixes until you start to plan the next major release.

    I think that coder weight and user weight do weigh down the project. On the other hand, many FOSS/OS projects are so modular that you can bolt your project onto theirs without joining the larger group.

    Comment by ray (visitor) on Thu, Dec 18, 2008 @ 00:49 IST #
  4. Ray, I agree with you about modularity. That seems to be an ideal solution at first glance, but then modularity itself can become a liability when spaghetti of conflicting code gets added on as modules and lead to a lot of incompatibility and version conflicts over a period of time.

    Features getting added as modules seems to be a great idea, but then you need the module-developers to work in close harmony with the core development team to maintain and keep them up-to-date and secure with the main codebase. In such cases, it would again lead to the same "closed-loop" phenomenon I was talking about above. But it is definitely an improvement in terms of user participation.

    Comment by Hari (blog owner) on Thu, Dec 18, 2008 @ 09:03 IST #

Comments closed

The blog owner has closed further commenting on this entry.