Open source adoption in commercial closed-source software development

As I discussed in my previous post, there has been a considerable progress in the quality of FOSS as well as an increase in openness & maturity of ISVs in adopting and contributing to open source. Their are no rules castes in stone, but one must be aware of the possible challenges that we can encounter in this endeavour. This is what this post is about.

When we start any large scale software application/product/system development, we have the following construction options for its building blocks:

1. Develop in-house

1. Buy COTs software from 3rd Party ISV.

1. Adopt FOSS ecosystem building blocks

1. Get into a joint development with another vendor

1. Outsource to a off-shore/near-shore/on-shore outsourced software development service provider

These are 5 different software projects, reach requiring a tailored project management and R&D management. But before we spawn these projects we need to have a rather rigorous decision making progress which can drive any architect to insanity. I have personally being involved in 1-3 as an employee of a Tier-I Telecommunication Equipment Manager. 

For now, our interest is point 3. You can use FOSS software in two ways:

(a) Use as it is i.e. without modifications

(b) Modify and tailor to your needs.

We will come back to this later.  Let me throw some light on the issues involved (in no partcular order of importance):

(1) Cost - Their is no doubt that if we are able to reuse a FOSS component, our CAPEX costs (or development & stabilisation costs) and time-to-market are cut drastically & especially if the reused building blockis reasonably large. However it is NOT ZERO and we incur OPEX costs & delays during the lifecycle of its usage, which is ignored by many project managers. 

First, we need to spend some effort of an experienced engineer to select the right FOSS component to use. You don't do that properly, you are going to have support issues, instability of software, security problems and conflict between future development directions of community (including closing the project) and your organization's interest. 

Second, we need to get our inhouse developers to thoroughly understand that FOSS Code so that they can use it properly, debug and support it and also modify (bug-fixes, enhancement, adaptation, improvment). This requires again an investment in effort. Even if you don't touch the source code [Point (a) above], you need to understnad it to debug it in case things don't work properly. Can't turn a blind eye (or a black one) to any adopted thing. The FOSS community is not going to do it for you and expects you to do it yourself. I have seen instances where teams start sweating if a core-dump or fault points to something deep inside the open source. Such teams have only understood the APIs (that to from another implementation, not the internals of the FOSS Component). As it turned out, the fault was in usage of the FOSS component, not the implementation of it. But it took quite a while and a lot of anxious faces to get things back in order (or so we think !!!).

Third, in case we need to mofdify the software, we need to have skill and effort for that. The community is not going to do it for you and expects yyou to do it and contribute back to the community (if it is agreeable with them). 

(2) Licensing - FOSS components are also copyrighted and are governed by copyright laws. Though the licensing terms is specified pretty clearly by the license method that that the open source uses. For eg., GPL would require you to open source all components that use it and also oepns-ource your business logic. But L-GPL will allow you the flexibility of dynamically linking to FOSS component and not requiring you to open your business logic components. MIT license will allow you to use and modify free of any commitments. So this is the most important criteria when selecting an opens-ource to adopt in your product. You do not want to mess with the law aand at teh same time protect your Intellectual Property & Innovations (which anyway manifest in code not design dopcuments or patents).

(3) Value First, Cost Second - Their is a big trend in the usage of FOSS software infrastructure. In future non-business logic components or basic platform infrastructure will be open source. OS, Middleware, SDKs & Frameworks. But one must also tradeoff principles with value and competitive advantage that can be generated. Sometimes the contribution of infrastructure is very high in overall product quality attribute (like performnace, scalability & reliability). You should evaluate the open source qualities very carefully and if you can do much better than open-source with a radically different engineering approach then you should go for inhouse development. Otherwise considering use open source, tweaking it for improvement and contribute back to the community. It will buy you a lot of goodwill among the community members who may return the favour by supporting (or even joining) you in future. 

For eg., SIP performance is the key in IMS equipments for per-port density. If I an develop an implementation that has radically higher performnace than what FOSS can offer me (they are anyways not designed for this but client side User Agents), then I will have my inhouse SIP stack and CSCF/AS frameworks. For VoiceXML platform, Javascript  performance is key on server side for port density and most open source are developed keeping the client in mind and can't meet high performance, scalability & reliability characteristics. Therefore their could be significant value in developing aninhouse  javascript engine tailored for server side implementations. Similarly for applications like configuration, their is no point in writing custom XML parsers with greatest parsing speeds, lowest footprint as the activity is not in critical path of device function, nor its business logic. You could consider any open-source resuse in this case as long is is reasonably stable and meets the functional needs. The substitution will also be easy if something goes wrong. Play the value game first and then the Cheap game.

(4) Modification of Open Source - Once you have decided that you will use open-source and which one you will use, you are likely to enounter shortcomings for your use-case. You will be tempted to make those changes and move ahead, but a word of caution is advised here. Some organizations are still not mature enough to allow you to commit your chnages back to community as policy forbids it. And hence if you go overboard with the changes which you cannot merge with the main branch at the community repository, then you are are constant inconvenience of having to merge (or even redo) your changes every time you upgraded to a new open source version for improved functionality, reliability or other quality attribute. The strict non-disclosure policy is archaic and hypocritical. It allows you to use other's work, but not return the favor by making your contribution to it. Its parasitic. It has to to go. Thankfully most organizations have matured to this reality. Next even if you could commit your changes, you should be doing changes in a manner that can be accepted and merged into main branch and you need proper collaboration with community for this. everybody should move in common direction, otherwise you may be left with your version to fully maintain & support. When we work with FOSS, we have to follow & live by the rules of that community not the otherway round.

(5) Perceived Quality & Security risks  of Open Source - Being open and done by people in spare time or on very less pay, it is thought that quality is poor. While true initially, things have progrssed in last 5 odd years. Quality levels are high. The number of developers working are beyond what a single organization could ever afford to put into such a project developed inhouse, in partnership ror by ousourcing. And these developers work & contribute with their passion and heart and not because they are being paid for it. The results are surprisngly good. If an open source has backing of many companies and developers, it is liely it will mature very fast if not already (this is one key point to consider when selecting the open-source to use.

Open-source licenses usually require you to disclose their usage when used commercially. So you are prone to security risks if some developer leaks your usage and hackers exploit a security flaw in open source (which is known to them and not published). This is a real risk. The community is usually very prompt in fixing such issues on an urgent basis. However you could mitigate this by treating your adopted child as your own and extending the same care to it as you would to your own blood child. Means actively review & test to find security holes, defects and commit them back to community. If everyone does this and brings his skills to the table, the software will mature faster than your inhouse developed one. Everybody benefits with this. And you get tremendous goodwill in the community.

As you can see open-source is a double edged sword. You could easily hurt yourself with it as you can the competition around you Yet adoption and open software infrastructure is the trend & way forward. Linux foundation in a recent (March 2014) survey report :

http://storage.pardot.com/6342/105392/lfwp_collabdevtrends_v3.pdf

highlighted the development towards collaborative open source software development by big companies across industries to cut costs & push innovation. And it involves upcoming technologies. We should tag along not fight this change.