How Google decides to open source its technology

Google has a solid reputation as an engineering organisation with an open source culture, with Googlers contributing a huge amount of code back to the community and projects like TensorFlow and Kubernetes making a mainstream impact.

Speaking to the press during the Google Cloud Next event in San Francisco last month, Sarah Novotny, head of open source strategy at Google Cloud Platform (GCP) and Melody Meckfessel, VP of engineering for GCP, spoke candidly about how the company decides to open source its technology and building an open culture.

"Google has a long history of engaging in open source communities," Novotny started out by saying. "We've had an open source programs office for more than 12 years and have worked with several other large companies to come into this space in a way that protects both the company and the projects and the culture of the projects."

Kubernetes and TensorFlow

When it comes to Google and open source there are two major projects that spring to mind: TensorFlow and Kubernetes.

Despite Novotny being keen to stress that Google has open sourced 2,400 projects over the years, from small libraries and STKs, it is these industry-changing projects that has put the company's open source credentials on the map.

In a nutshell Kubernetes is a container orchestration platform that has changed the way devops teams deploy and manage their application code. TensorFlow has made machine learning model building and deployment far more accessible to developers.

Novotny says that Kubernetes and TensorFlow have "very different strategic goals for Google", where Kubernetes has been effectively handed over to the community, TensorFlow is still "very company led".

When asked during a press Q&A session about how Google decides internally what technology to open source and what to commercialise, Novotny said: "Google has been very organic about that internally," after which Meckfessel interjected: "That's a very nice way of saying it."

"We are building out some process and structure about that now," Novotny said.

"For a long time it was a developer saying 'I'm into a cool thing' and the product and the leadership team didn't necessarily see that it would either be super high impact to the community as an open source thing, or was not necessarily something that was going to be deeply funded... so the engineers would push to make it open source".

The aim of this is to ensure that "when we sent out a project we knew what the expected return was, even if that was that we will maintain this forever and it is just to make our customers happy, because that is a strategic goal," she said.

Open source strategy

This is just one way Google contributes to the community though.

"We have more than 20 million lines of code we have contributed to open source projects on GitHub, and in 2017 Googlers contributed nearly one percent of total pull requests on GitHub," Novotny said.

This open source approach fits nicely with Google Cloud's more general policy of allowing customers to easily switch between the different cloud providers, as it looks to play catch up to the market leader Amazon Web Services (AWS) and the second place Microsoft Azure.

As Novotny put it: "I think it's a huge investment to ensure our customers have the tools they need working well on our platform and looking at the broader, long-term hybrid asks of our customers, that these tools are available and easy to use across the different clouds."

A concrete example of this is the Spinnaker continuous delivery tool that came out of Netflix as a 1.0 version in June 2017.

Google contributed a team to help support Spinnaker on GCP, meaning that it now "supports multi-cloud deployments to make it fast and easy to use. So that was the right thing to do, and the contributions on Spinnaker have continued to increase," Meckfessel said.

"It had been more focused on AWS deployments until then, so we were able to broaden it," Novotny added.

Meckfessel admits that Google is "still learning" in this space, but that Spinnaker is a good example of them being adaptable project to project.

She said this was a project "where it doesn't make sense for us to build it ourselves, let's build it with Netflix, who is supporting AWS, because it is multi-cloud and has a huge benefit to the community and that is the way to go as it benefits developers."

Broader impact of open source

This open source approach also inherently appeals to developers, which certainly helps when it comes to the tricky business of hiring.

"By changing cultures and making it a positive engagement and ensuring tools work for the developers to make their experience better, you end up building a cohort that can draw in more people as you are trying to recruit," Novotny said.

Meckfessel then added: "I have been very encouraged and inspired to see the positive impact on Google engineers of interacting more in the open, both in terms of how they consider a technical solution, and taking an idea and making it reality in a way that thinks about the world's developers.

"I have seen it lead to more collaboration happening outside in the open and have seen that influence their perspective."

When asked during the press Q&A if there is any concern that corporations will start to dominate the open source community, Meckfessel was predictably optimistic.

"I would like to think that we are encouraging our competitors and peers in the industry to make the same sort of impact," she said. "So we are drawing them in. We see Microsoft and Amazon both being substantial contributors to Kubernetes, so we are trying to bring that broader cloud perspective in."

Novotny went one step further, promoting the benefits of corporate open source contributions. "I think it is a false juxtaposition to suggest that if Google is benefitting then the community is at risk," she said.

"Large companies having a majority of open source contributions brings more diverse contributors to open source. There is an enormous amount of inherent privilege assumed if you are not payed to work on open source. Anyone who has a family or an ageing relative or a disability is less likely to be able to do that than if they are also paid for this. So I think that the shift toward corporate sponsored open source is a benefit to both open source and the corporations."

Istio

Google also launched its latest major open source project as a 'production ready' 1.0 version during the Next conference in San Francisco.

Istio is a 'service mesh' which allows developer teams to unify traffic flow management, access policy enforcement and telemetry data aggregation across microservices into a shared management console, regardless of environment.

Urs Holzle, SVP technical infrastructure at Google Cloud said from the stage during Google Cloud Next that: "Istio is another Google-developed open source project which extends Kubernetes into these higher level services. So you can discover, connect and monitor services holistically across multiple locations in one place" - without having to change code.

Google started working on what is now called Istio last year, along with IBM, Lyft, Pivotal, Cisco and Red Hat.