During the The New Stack Analyst Podcast on OpenStack Contribution Trends, @VirtualTal of Adallom raised an interesting question about one of the contributing companies who were blacklisted by many governments. His concern was the possibility of this contributor adding backdoor entry code into OpenStack code. Though I addressed that briefly, I couldn’t explain in detail since the focus of Podcast was on contribution trends.

@ReneBuest had also had a tweet recently highlighting OpenStack contributions from NSA and his later tweets showed his possible concern of adding backdoor entry into OpenStack.

This post addresses these concerns (short answer is NO!), by explaining the OpenStack Code Submission Process. Here is a good link on OpenStack Code submission workflow which uses Git, Jenkins and Gerrit. Of course, you know where the OpenStack source code repository is!

OpenStack Code Submission Process

OpenStack has more than 2000 contributors, 100 contributing companies and more than 10 official projects, few incubated projects and more supporting projects and all such madness is self-regulated :).

Let’s say Alison, a developer, wants to submit code changes. Let us assume she’s has got her development environment setup (if not, I would recommend her to setup instructions here).

Now that she has code changes ready, here are the steps she takes to submit those changes to the project repository she is working on.

  1. Now that Alison has code changes, she runs the unit tests locallt, gets ready to ‘commit’ her code
  2. She commits her code (by running git commit)
  3. She requests for code review (by running git review).
  4. At the same time, the changes are queued to be tested against Jenkins based functional gating tests. Success or failure is provided as a ‘review’ feedback with +1 or -1 vote.

Her code changes will now show up now at the review queue, awaiting code reviews. At this point, any other developer can review this code patch, and gives a +1, 0, or -1 vote, indicating approve/ neutral/ disapprove. If the developer is also one of ‘core reviewer’ she can also give a +2 vote (explaining why +2, instead of regular +1).

  1. Once the patch passes the tests AND gets a +2 review from one or more core developers, Alison can request for merging the code upstream. The gate tests are run again, and on passing, code changes will be merged upstream.

If there are any merge conflicts at this point, or gate tests fail, Alison needs to fix them and submit again.

OpenStack Review Process

Any OpenStack developer can review and assign a +1/0/-1 score/ vote according to her approval/ neutral/ disapproval ratings. There are minimum recommendations on what to look for while reviewing the code. As you can see from there, each project has a review bar to meet. This is to ensure that spell-check reviewers don’t get much of a say during code review :)

Only when the code path has got +2 votes from one or more active core developer, it is ready to be merged. Here, the code reviews from core developers are trusted, and they have a large responsibility in ensuring high quality code gets merged.

This is why, a code patch could remain un-merged even after getting several +1 votes and passing the gate tests. This is because, each code patch needs to get a +2 vote from CORE reviewers in order to get merged.

Who is a Core Developer?

A core developer is a one who has submitted enough HIGH Quality code and done enough HIGH quality code reviews that the community trusts her code reviews. And in order to become a core developer, she has to meet requirements and has to be approved by at least 5 existing core developers (or majority in case of less than 5 core developers already in that project).

OpenStack, being developer driven, runs through collective participation. One establishes her credibility through active and high quality participation. Once established, the community trusts her capabilities.

Now back to the original question:

Can Backdoor Entry be added?

As you can see, the code to be submitted is reviewed by peers and core developers. If someone has malicious intent to push such code, it has to really escape the attention of other developers and gets approval from at least 1 core developer who needs to vote +2. Such +2 votes are scrutinized more, so it really needs approval from 2 core developers.

It is highly unlikely that 2 core developers would approve malicious code with the intentions of pushing backdoor entry in to OpenStack.

Here is why it is not likely:

  • If one has malicious intents to begin with, they wouldn’t have become core developer. They wouldn’t have passed approval process.
  • Let us assume one suddenly developed such intents (cinematic albeit) after becoming a core developer. Even in this case, there are enough checks in place , other core developers will override such code approvals.

So it is not really possible.

Can this really happen at all?

This could happen if a project has ALL core developers who has the intent to do such things. That means, it can’t happen in any of the existing OpenStack projects.

Someone has to start a new project with the sole intent of pushing such code, and be able to run that by all of their resources.

And that has better chances of happening in some other community/ project, not OpenStack!

Cloud Don Take

While it is a valid concerns for customers, they need to understand that no existing software is going to be 100% fool-proof, and OpenStack is no exception. However, you have excellent guidelines and success stories in implementing secure cloud using OpenStack. One can confidently use OpenStack to implement a secure, scalable cloud.

Customers need to look in to the complete picture — accompanying hardware, their security practices, processes etc. Analysts need to explain customers the entire picture, rather than pandering to incorrect FUDs/ incomplete questions like ‘Is OpenStack secure?’